The basal ganglia: learning new tricks and loving it
Introduction
There is convincing evidence that learning-related functions are central to the role of the basal ganglia in selecting which actions to perform based on updated representations of current context. Increasingly, studies are focusing on how these learning functions are implemented within the framework of circuits internal to the basal ganglia. These circuits are modulated by monoaminergic inputs from the midbrain and by cortico–basal ganglia pathways, and lead into pathways toward the brainstem or recurrently toward the neocortex. Here, we highlight new work on basal ganglia-based learning and the new challenges to current concepts of basal ganglia circuit function. We propose that one over-arching function of the basal ganglia is to promote optimal control of action.
Section snippets
Basal ganglia-based learning
A key idea emerging in the field of basal ganglia research is that cortico–basal ganglia circuits promote learning of action sequences through trial-and-error learning. Three new papers provide convincing evidence for such a concept of basal ganglia function in songbirds [1••, 2••] and mammals [3••]. During such trial-and-error learning, the animal (more generally, the agent) first explores the environment: behavior is variable, and reinforcers shape the behavior until it converges on an
Cortico–basal ganglia loop function and learning
It is reasonable, given the reward-related signaling of midbrain dopaminergic neurons, that this dopaminergic input system could ‘teach’ the striatum (and, hence, the basal ganglia). Several groups [7, 8, 9, 10] have further suggested that the basal ganglia could ‘teach’ the cortex in cognate cortico–basal ganglia loops via striato–pallido–thalamocortical pathways. To test these ideas, it is necessary to record simultaneously, or in close temporal contiguity, from striatal and cortical neurons
Reinforcement signals in the basal ganglia
The spike firing of dopamine-containing neurons of the midbrain, now famously appreciated as carrying signals related to reward, has proved to fit remarkably closely the constraints of reinforcement learning theory, including temporal difference (TD) models, even in the complex context of reward delivery. The dopamine-containing neurons code a reward-prediction error in their phasic firing and appear also to code the uncertainty of the prediction in their maintained firing levels [19] (but see
Basal ganglia circuit anatomy and function: new challenges
How do these learning functions relate to the motor control functions long attributed to the basal ganglia? Much clinical and experimental work on the basal ganglia has been inspired by the idea that the basal ganglia can release or inhibit movement by the opposing influences of two main pathways originating in the striatum and extending through the pallidum and substantia nigra: the movement-releasing ‘direct pathway’ and the movement inhibiting ‘indirect pathway’. These pathways, which have
Conclusions and future directions
Many issues remain to be resolved at the systems level if we are to understand the functions of the basal ganglia in motor control. What are the functions of any given cortico–basal ganglia circuit or cortico–basal ganglia–brainstem circuit? How are the learning-related functions of the basal ganglia integrated with their motor and cognitive control functions? How are these functions integrated with those of other brain systems? What is the function of the prominent oscillatory activity in the
References and recommended reading
Papers of particular interest, published within the annual period of review, have been highlighted as:
• of special interest
•• of outstanding interest
Acknowledgements
Work from our laboratory cited here was funded by the National Institutes of Mental Health (MH60379), the National Institute of Neurological Disorders and Stroke (NS25529 and NS38372) and the Office of Naval Research (N00014-02-1-0023).
References (63)
- et al.
Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action
Cereb Cortex
(1995) - et al.
Different time courses of learning-related activity in the prefrontal cortex and striatum
Nature
(2005) - et al.
The neural correlates of motor skill automaticity
J Neurosci
(2005) - et al.
By carrot or by stick: cognitive reinforcement learning in parkinsonism
Science
(2004) - et al.
Tonically active neurons in the primate caudate nucleus and putamen differentially encode instructed motivational outcomes of action
J Neurosci
(2004) - et al.
A network representation of response probability in the striatum
Neuron
(2002) - et al.
Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops
Nat Neurosci
(2004) - et al.
Getting to know you: reputation and trust in a two-person economic exchange
Science
(2005) - et al.
The neural basis of altruistic punishment
Science
(2004) - et al.
Evidence for differential cortical input to direct pathway versus indirect pathway striatal projection neurons in rats
J Neurosci
(2004)