Elsevier

Current Opinion in Neurobiology

Volume 54, February 2019, Pages 28-36
Current Opinion in Neurobiology

Dendritic solutions to the credit assignment problem

https://doi.org/10.1016/j.conb.2018.08.003Get rights and content

Highlights

  • Learning in hierarchical neural networks requires credit assignment.

  • Credit assignment is difficult if regular inputs mix with credit signals.

  • Dendritic mechanisms provide potential means of distinguishing credit signals.

  • Evidence supports credit assignment in apical dendrites of pyramidal neurons.

Guaranteeing that synaptic plasticity leads to effective learning requires a means for assigning credit to each neuron for its contribution to behavior. The ‘credit assignment problem’ refers to the fact that credit assignment is non-trivial in hierarchical networks with multiple stages of processing. One difficulty is that if credit signals are integrated with other inputs, then it is hard for synaptic plasticity rules to distinguish credit-related activity from non-credit-related activity. A potential solution is to use the spatial layout and non-linear properties of dendrites to distinguish credit signals from other inputs. In cortical pyramidal neurons, evidence hints that top-down feedback signals are integrated in the distal apical dendrites and have a distinct impact on spike-firing and synaptic plasticity. This suggests that the distal apical dendrites of pyramidal neurons help the brain to solve the credit assignment problem.

Section snippets

Introduction: the credit assignment problem

The flexibility of learning in animals indicates that the brain possesses general purpose learning algorithms. A learning algorithm is a set of rules for translating the experiences an animal has into changes in their neural circuits (e.g. synaptic changes). The ultimate goal of a learning algorithm is to alter the behavioral phenotype of the animal, helping it to adapt to the environment. Understanding the brain's learning algorithms is key to understanding the biological basis of animal

What counts as evidence for credit assignment?

The ideal experiment for understanding credit assignment in the brain would be to measure a loss function explicitly, then demonstrate that a given synaptic plasticity mechanism was responsible for ensuring reductions in that loss function during learning. Such experiments are currently outside of our technical reach, though, because it is often unclear how we can identify a loss function in the brain and track its progress over time [3]. Furthermore, there is no reason to assume that the brain

Credit assignment in cortical pyramidal neurons

In the neocortex and hippocampus, pyramidal neurons are part of a hierarchical pathway with multiple sources of potential credit-related feedback. Thus, assigning credit in cortical pyramidal neurons may require more involved calculations than in the output layer of the cerebellum with one-to-one climbing fiber → Purkinje cell error signals. Where might these credit calculations take place? To date, direct experimental evidence for credit assignment calculations in neocortical neurons is

Conclusion

A major goal for researchers in coming years should be a better link between the theory of credit assignment in neural networks [7,9••,10••,11, 12, 13, 14], and our growing knowledge of the biophysics of dendrites and dendritic computation [28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44]. Clearly, there is much more to understand about dendritic computations in pyramidal neurons, how they may signal credit information, and how they contribute to learning, in-turn. Three

Conflict of interest statement

Nothing declared.

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

  • • of special interest

  • •• of outstanding interest

Acknowledgements

This work was supported by the Natural Sciences and Engineering Research Council of Canada (BAR: RGPIN-2014-04947) and the Canadian Institute for Advanced Research (BAR: Learning in Machine and Brains Program).

References (120)

  • E.S. Boyden et al.

    Selective engagement of plasticity mechanisms for motor memory storage

    Neuron

    (2006)
  • R. Llinas et al.

    Electrophysiological properties of in vitro Purkinje cell dendrites in mammalian cerebellar slices

    J Physiol

    (1980)
  • R. Llinas et al.

    Electrophysiological properties of in vitro Purkinje cell somata in mammalian cerebellar slices

    J Physiol

    (1980)
  • V. Lev-Ram et al.

    A new form of cerebellar long-term potentiation is postsynaptic and depends on nitric oxide but not cAMP

    Proc Natl Acad Sci U S A

    (2002)
  • V. Lev-Ram et al.

    Reversing cerebellar long-term depression

    Proc Natl Acad Sci U S A

    (2003)
  • S.R. Williams et al.

    Dependence of EPSP efficacy on synapse location in neocortical pyramidal neurons

    Science

    (2002)
  • P.J. Sjöström et al.

    Rate, timing, and cooperativity jointly determine cortical synaptic plasticity

    Neuron

    (2001)
  • B.M. Kampa et al.

    Calcium spikes in basal dendrites of layer 5 pyramidal neurons during action potential bursts

    J Neurosci

    (2006)
  • J.J. Letzkus et al.

    Learning rules for spike timing-dependent plasticity depend on dendritic synapse location

    J Neurosci

    (2006)
  • D.J. MacKay

    Information Theory, Inference and Learning Algorithms

    (2003)
  • Y. LeCun et al.

    Deep learning

    Nature

    (2015)
  • A.H. Marblestone et al.

    Toward an integration of deep learning and neuroscience

    Front Comput Neurosci

    (2016)
  • D.H. Wolpert et al.

    No free lunch theorems for optimization

    IEEE Trans Evol Comput

    (1997)
  • D.E. Rumelhart et al.

    Learning representations by back-propagating errors

    Nature

    (1986)
  • P.R. Roelfsema et al.

    Attention-gated reinforcement learning of internal representations for classification

    Neural Comput

    (2005)
  • M. Minsky

    Steps toward artificial intelligence

    Proc IRE

    (1961)
  • D.-H. Lee et al.

    This paper introduces difference target propagation, an alternative to the backpropagation-of-error algorithm that avoids the requirement for symmetric feedback weights. The authors demonstrate that the algorithm can engage in efficient credit assignment in multilayer networks.

    Difference target propagation

    (2015)
  • T.P. Lillicrap et al.

    Random synaptic feedback weights support error backpropagation for deep learning

    Nat Commun

    (2016)
  • B. Scellier et al.

    Equilibrium propagation: bridging the gap between energy-based models and backpropagation

    Front Comput Neurosci

    (2017)
  • G.E. Hinton et al.

    The ‘wake-sleep’ algorithm for unsupervised neural networks

    Science

    (1995)
  • G.E. Hinton et al.

    A fast learning algorithm for deep belief nets

    Neural Comput

    (2006)
  • J.O. Rombouts et al.

    How attention can create synaptic tags for the learning of working memories in sequential tasks

    PLOS Comput Biol

    (2015)
  • F. Zenke et al.

    SuperSpike: supervised learning in multilayer spiking neural networks

    Neural Comput

    (2018)
  • G. Buzsáki et al.

    Neuronal oscillations in cortical networks

    Science

    (2004)
  • L. Cauller et al.

    Synaptic physiology of horizontal afferents to layer i in slices of rat SI neocortex

    J Neurosci

    (1994)
  • S.-M. Lu et al.

    Thalamic afferents of the rat barrel cortex: a light- and electron-microscopic study using Phaseolus vulgaris leucoagglutinin as an anterograde tracer

    Somatosens Motor Res

    (1993)
  • S. Ohno et al.

    A morphological analysis of thalamocortical axon fibers of rat posterior thalamic nuclei: a single neuron tracing study with viral vectors

    Cereb Cortex

    (2012)
  • H.S. Meyer et al.

    Cell type-specific thalamic innervation in a column of rat vibrissal cortex

    Cereb Cortex

    (2010)
  • K.S. Rockland et al.

    Terminal arbors of individual ‘feedback axons projecting from area v2 to v1 in the macaque monkey: a study using immunohistochemistry of anterogradely transported Phaseolus vulgaris-leucoagglutinin

    J Comp Neurol

    (2004)
  • D. Feldmeyer et al.

    Synaptic connections between layer 4 spiny neurone-layer 2/3 pyramidal cell pairs in juvenile rat barrel cortex: physiology and anatomy of interlaminar signalling within a cortical column

    J Physiol

    (2004)
  • H. Markram et al.

    Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex

    J Physiol

    (1997)
  • J. Suh et al.

    Entorhinal cortex layer III input to the hippocampus is crucial for temporal association memory

    Science

    (2011)
  • M. Megías et al.

    Total number and distribution of inhibitory and excitatory synapses on hippocampal CA1 pyramidal cells

    Neuroscience

    (2001)
  • O. Steward et al.

    Cells of origin of entorhinal cortical afferents to the hippocampus and fascia dentata of the rat

    J Comp Neurol

    (2004)
  • P. Poirazi et al.

    Arithmetic of subthreshold synaptic summation in a model CA1 pyramidal cell

    Neuron

    (2003)
  • P. Poirazi et al.

    Pyramidal neuron as two-layer neural network

    Neuron

    (2003)
  • M.E. Larkum et al.

    A new cellular mechanism for coupling inputs arriving at different cortical layers

    Nature

    (1999)
  • M.E. Larkum et al.

    Calcium electrogenesis in distal apical dendrites of layer 5 pyramidal cells at a critical frequency of back-propagating action potentials

    Proc Natl Acad Sci U S A

    (1999)
  • M.E. Larkum et al.

    Dendritic spikes in apical dendrites of neocortical layer 2/3 pyramidal neurons

    J Neurosci

    (2007)
  • M.E. Larkum et al.

    Synaptic integration in tuft dendrites of layer 5 pyramidal neurons: a new unifying principle

    Science

    (2009)
  • Cited by (0)

    View full text