 |
||||
|
||||
|
||||
|
||||
|
||||
Previous Article | Next Article
The Journal of Neuroscience, December 1, 1999, 19(23):10502-10511
How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues
Joshua Brown, Daniel Bullock, and Stephen Grossberg Department of Cognitive and Neural Systems and Center for Adaptive Systems, Boston University, Boston, Massachusetts 02215
After classically conditioned learning, dopaminergic cells in the substantia nigra pars compacta (SNc) respond immediately to unexpected conditioned stimuli (CS) but omit formerly seen responses to expected unconditioned stimuli, notably rewards. These cells play an important role in reinforcement learning. A neural model explains the key neurophysiological properties of these cells before, during, and after conditioning, as well as related anatomical and neurophysiological data about the pedunculopontine tegmental nucleus (PPTN), lateral hypothalamus, ventral striatum, and striosomes. The model proposes how two parallel learning pathways from limbic cortex to the SNc, one devoted to excitatory conditioning (through the ventral striatum, ventral pallidum, and PPTN) and the other to adaptively timed inhibitory conditioning (through the striosomes), control SNc responses. The excitatory pathway generates CS-induced excitatory SNc dopamine bursts. The inhibitory pathway prevents dopamine bursts in response to predictable reward-related signals. When expected rewards are not received, striosomal inhibition of SNc that is unopposed by excitation results in a phasic drop in dopamine cell activity. The adaptively timed inhibitory learning uses an intracellular spectrum of timed responses that is proposed to be similar to adaptively timed cellular mechanisms in the hippocampus and cerebellum. These mechanisms are proposed to include metabotropic glutamate receptor-mediated Ca2+ spikes that occur with different delays in striosomal cells. A dopaminergic burst in concert with a Ca2+ spike is proposed to potentiate inhibitory learning. The model provides a biologically predictive alternative to temporal difference conditioning models and explains substantially more data than alternative models.
Key words: dopamine; substantia nigra; reward; basal ganglia; conditioning; pedunculopontine tegmental nucleus; lateral hypothalamus; striosomes; adaptive timing
Copyright © 1999 Society for Neuroscience 0270-6474/99/192310502-10$05.00/0
This article has been cited by other articles:
S. Grossberg
Cortical and subcortical predictive dynamics and learning during perception, cognition, emotion and action
Phil Trans R Soc B, May 12, 2009; 364(1521): 1223 - 1234.
[Abstract] [Full Text] [PDF]
K.-i. Okada, K. Toyama, Y. Inoue, T. Isa, and Y. Kobayashi
Different Pedunculopontine Tegmental Neurons Signal Predicted and Actual Task Rewards
J. Neurosci., April 15, 2009; 29(15): 4858 - 4870.
[Abstract] [Full Text] [PDF]
C. O. Tan and D. Bullock
A Dopamine-Acetylcholine Cascade: Simulating Learned and Lesion-Induced Behavior of Striatal Cholinergic Interneurons
J Neurophysiol, October 1, 2008; 100(4): 2409 - 2421.
[Abstract] [Full Text] [PDF]
C. O. Tan and D. Bullock
A Local Circuit Model of Learned Striatal and Dopamine Cell Responses under Probabilistic Schedules of Reward
J. Neurosci., October 1, 2008; 28(40): 10062 - 10074.
[Abstract] [Full Text] [PDF]
K. Sato, C. Sumi-Ichinose, R. Kaji, K. Ikemoto, T. Nomura, I. Nagatsu, H. Ichinose, M. Ito, W. Sako, S. Nagahiro, et al.
Differential involvement of striosome and matrix dopamine systems in a transgenic model of dopa-responsive dystonia
PNAS, August 26, 2008; 105(34): 12551 - 12556.
[Abstract] [Full Text] [PDF]
T. E Hazy, M. J Frank, and R. C O'Reilly
Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system
Phil Trans R Soc B, September 29, 2007; 362(1485): 1601 - 1613.
[Abstract] [Full Text] [PDF]
M. Haruno and M. Kawato
Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning
J Neurophysiol, February 1, 2006; 95(2): 948 - 959.
[Abstract] [Full Text] [PDF]
W.-X. Pan and B. I. Hyland
Pedunculopontine Tegmental Nucleus Controls Conditioned Responses of Midbrain Dopamine Neurons in Behaving Rats
J. Neurosci., May 11, 2005; 25(19): 4725 - 4732.
[Abstract] [Full Text] [PDF]
P. J. Voulalas, L. Holtzclaw, J. Wolstenholme, J. T. Russell, and S. E. Hyman
Metabotropic Glutamate Receptors and Dopamine Receptors Cooperate to Enhance Extracellular Signal-Regulated Kinase Phosphorylation in Striatal Neurons
J. Neurosci., April 13, 2005; 25(15): 3763 - 3773.
[Abstract] [Full Text] [PDF]
J. O'Doherty, P. Dayan, J. Schultz, R. Deichmann, K. Friston, and R. J. Dolan
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
Science, April 16, 2004; 304(5669): 452 - 454.
[Abstract] [Full Text] [PDF]
T D Sanger
Childhood onset generalised dystonia can be modelled by increased gain in the indirect basal ganglia pathway
J. Neurol. Neurosurg. Psychiatry, November 1, 2003; 74(11): 1509 - 1515.
[Abstract] [Full Text] [PDF]
T. Satoh, S. Nakai, T. Sato, and M. Kimura
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
J. Neurosci., October 29, 2003; 23(30): 9913 - 9923.
[Abstract] [Full Text] [PDF]
S. Ito, V. Stuphorn, J. W. Brown, and J. D. Schall
Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding
Science, October 3, 2003; 302(5642): 120 - 122.
[Abstract] [Full Text] [PDF]
Y. Kobayashi, Y. Inoue, M. Yamamoto, T. Isa, and H. Aizawa
Contribution of Pedunculopontine Tegmental Nucleus Neurons to Performance of Visually Guided Saccade Tasks in Monkeys
J Neurophysiol, August 1, 2002; 88(2): 715 - 731.
[Abstract] [Full Text] [PDF]
L. L. Brown, S. M. Feldman, D. M. Smith, J. R. Cavanaugh, R. F. Ackermann, and A. M. Graybiel
Differential Metabolic Activity in the Striosome and Matrix Compartments of the Rat Striatum during Natural Behaviors
J. Neurosci., January 1, 2002; 22(1): 305 - 314.
[Abstract] [Full Text] [PDF]
A. S. Dave and D. Margoliash
Song Replay During Sleep and Computational Rules for Sensorimotor Vocal Learning
Science, October 27, 2000; 290(5492): 812 - 816.
[Abstract] [Full Text]
|
|
|
|
 |
|