The basal ganglia: learning new tricks and loving it

https://doi.org/10.1016/j.conb.2005.10.006Get rights and content

The field of basal ganglia research is exploding on every level — from discoveries at the molecular level to those based on human brain imaging. A remarkable series of new findings support the view that the basal ganglia are essential for some forms of learning-related plasticity. Other new findings are challenging some of the basic tenets of the field as it now stands. Combined with the new evidence on learning-related functions of the basal ganglia, these studies suggest that the basal ganglia are parts of a brain-wide set of adaptive neural systems promoting optimal motor and cognitive control.

Introduction

There is convincing evidence that learning-related functions are central to the role of the basal ganglia in selecting which actions to perform based on updated representations of current context. Increasingly, studies are focusing on how these learning functions are implemented within the framework of circuits internal to the basal ganglia. These circuits are modulated by monoaminergic inputs from the midbrain and by cortico–basal ganglia pathways, and lead into pathways toward the brainstem or recurrently toward the neocortex. Here, we highlight new work on basal ganglia-based learning and the new challenges to current concepts of basal ganglia circuit function. We propose that one over-arching function of the basal ganglia is to promote optimal control of action.

Section snippets

Basal ganglia-based learning

A key idea emerging in the field of basal ganglia research is that cortico–basal ganglia circuits promote learning of action sequences through trial-and-error learning. Three new papers provide convincing evidence for such a concept of basal ganglia function in songbirds [1••, 2••] and mammals [3••]. During such trial-and-error learning, the animal (more generally, the agent) first explores the environment: behavior is variable, and reinforcers shape the behavior until it converges on an

Cortico–basal ganglia loop function and learning

It is reasonable, given the reward-related signaling of midbrain dopaminergic neurons, that this dopaminergic input system could ‘teach’ the striatum (and, hence, the basal ganglia). Several groups [7, 8, 9, 10] have further suggested that the basal ganglia could ‘teach’ the cortex in cognate cortico–basal ganglia loops via striato–pallido–thalamocortical pathways. To test these ideas, it is necessary to record simultaneously, or in close temporal contiguity, from striatal and cortical neurons

Reinforcement signals in the basal ganglia

The spike firing of dopamine-containing neurons of the midbrain, now famously appreciated as carrying signals related to reward, has proved to fit remarkably closely the constraints of reinforcement learning theory, including temporal difference (TD) models, even in the complex context of reward delivery. The dopamine-containing neurons code a reward-prediction error in their phasic firing and appear also to code the uncertainty of the prediction in their maintained firing levels [19] (but see

Basal ganglia circuit anatomy and function: new challenges

How do these learning functions relate to the motor control functions long attributed to the basal ganglia? Much clinical and experimental work on the basal ganglia has been inspired by the idea that the basal ganglia can release or inhibit movement by the opposing influences of two main pathways originating in the striatum and extending through the pallidum and substantia nigra: the movement-releasing ‘direct pathway’ and the movement inhibiting ‘indirect pathway’. These pathways, which have

Conclusions and future directions

Many issues remain to be resolved at the systems level if we are to understand the functions of the basal ganglia in motor control. What are the functions of any given cortico–basal ganglia circuit or cortico–basal ganglia–brainstem circuit? How are the learning-related functions of the basal ganglia integrated with their motor and cognitive control functions? How are these functions integrated with those of other brain systems? What is the function of the prominent oscillatory activity in the

References and recommended reading

Papers of particular interest, published within the annual period of review, have been highlighted as:

  • • of special interest

  • •• of outstanding interest

Acknowledgements

Work from our laboratory cited here was funded by the National Institutes of Mental Health (MH60379), the National Institute of Neurological Disorders and Stroke (NS25529 and NS38372) and the Office of Naval Research (N00014-02-1-0023).

References (63)

  • M. Desmurget et al.

    The basal ganglia network mediates the planning of movement amplitude

    Eur J Neurosci

    (2004)
  • N. Schmitzer-Torbert et al.

    Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task

    J Neurophysiol

    (2004)
  • T. Boraud et al.

    Oscillations in the basal ganglia: The good, the bad, and the unexpected

  • B.P. Olveczky et al.

    Vocal experimentation in the juvenile songbird requires a basal ganglia circuit

    PLoS Biol

    (2005)
  • M.H. Kao et al.

    Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song

    Nature

    (2005)
  • T. Barnes et al.

    Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories

    Nature

    (2005)
  • S. Ishii et al.

    Control of exploitation-exploration meta-parameter in reinforcement learning

    Neural Netw

    (2002)
  • R.S. Sutton et al.

    Reinforcement Learning: An Introduction

    (1998)
  • K. Doya et al.

    A novel reinforcement model of birdsong vocalization learning

  • M. Djurfeldt et al.

    Cortex-basal ganglia interaction and attractor states

    Neurocomputing

    (2001)
  • A.M. Graybiel

    The basal ganglia and cognitive pattern generators

    Schizophr Bull

    (1997)
  • M.J. Frank et al.

    Interactions between frontal cortex and basal ganglia in working memory: a computational model

    Cogn Affect Behav Neurosci

    (2001)
  • N. Fujii et al.

    Time-varying covariance of neural activities recorded in striatum and frontal cortex as monkeys perform sequential- saccade tasks

    Proc Natl Acad Sci USA

    (2005)
  • R.M. Costa et al.

    Differential corticostriatal plasticity during fast and slow motor skill learning in mice

    Curr Biol

    (2004)
  • P.J. Brasted et al.

    Comparison of learning-related neuronal activity in the dorsal premotor cortex and striatum

    Eur J Neurosci

    (2004)
  • L.H. Shi et al.

    Neural responses in multiple basal ganglia regions during spontaneous and treadmill locomotion tasks in rats

    Exp Brain Res

    (2004)
  • K. Watanabe et al.

    Immediate changes in anticipatory activity of caudate neurons associated with reversal of position-reward contingency

    J Neurophysiol

    (2005)
  • P.D. Nixon et al.

    Cortico-basal ganglia pathways are essential for the recall of well-established visuomotor associations

    Eur J Neurosci

    (2004)
  • C.D. Fiorillo et al.

    Discrete coding of reward probability and uncertainty by dopamine neurons

    Science

    (2003)
  • Y. Niv et al.

    Dopamine, uncertainty and TD learning

    Behav Brain Funct

    (2005)
  • H.M. Bayer et al.

    Midbrain dopamine neurons encode a quantitative reward prediction error signal

    Neuron

    (2005)
  • Cited by (0)

    View full text