 |
||||
|
||||
|
||||
|
||||
Previous Article | Next Article
The Journal of Neuroscience, December 1, 1999, 19(23):10502-10511
How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues
Joshua Brown, Daniel Bullock, and Stephen Grossberg Department of Cognitive and Neural Systems and Center for Adaptive Systems, Boston University, Boston, Massachusetts 02215
After classically conditioned learning, dopaminergic cells in the substantia nigra pars compacta (SNc) respond immediately to unexpected conditioned stimuli (CS) but omit formerly seen responses to expected unconditioned stimuli, notably rewards. These cells play an important role in reinforcement learning. A neural model explains the key neurophysiological properties of these cells before, during, and after conditioning, as well as related anatomical and neurophysiological data about the pedunculopontine tegmental nucleus (PPTN), lateral hypothalamus, ventral striatum, and striosomes. The model proposes how two parallel learning pathways from limbic cortex to the SNc, one devoted to excitatory conditioning (through the ventral striatum, ventral pallidum, and PPTN) and the other to adaptively timed inhibitory conditioning (through the striosomes), control SNc responses. The excitatory pathway generates CS-induced excitatory SNc dopamine bursts. The inhibitory pathway prevents dopamine bursts in response to predictable reward-related signals. When expected rewards are not received, striosomal inhibition of SNc that is unopposed by excitation results in a phasic drop in dopamine cell activity. The adaptively timed inhibitory learning uses an intracellular spectrum of timed responses that is proposed to be similar to adaptively timed cellular mechanisms in the hippocampus and cerebellum. These mechanisms are proposed to include metabotropic glutamate receptor-mediated Ca2+ spikes that occur with different delays in striosomal cells. A dopaminergic burst in concert with a Ca2+ spike is proposed to potentiate inhibitory learning. The model provides a biologically predictive alternative to temporal difference conditioning models and explains substantially more data than alternative models.
Key words: dopamine; substantia nigra; reward; basal ganglia; conditioning; pedunculopontine tegmental nucleus; lateral hypothalamus; striosomes; adaptive timing
Copyright © 1999 Society for Neuroscience 0270-6474/99/192310502-10$05.00/0
This article has been cited by other articles:
K. Sato, C. Sumi-Ichinose, R. Kaji, K. Ikemoto, T. Nomura, I. Nagatsu, H. Ichinose, M. Ito, W. Sako, S. Nagahiro, et al.
Differential involvement of striosome and matrix dopamine systems in a transgenic model of dopa-responsive dystonia
PNAS, August 26, 2008; 105(34): 12551 - 12556.
[Abstract] [Full Text] [PDF]
N. D. Daw, A. C. Courville, and D. S. Touretzky
Representation and Timing in Theories of the Dopamine System
Neural Comput., July 1, 2006; 18(7): 1637 - 1677.
[Abstract] [Full Text] [PDF]
M. Haruno and M. Kawato
Different Neural Correlates of Reward Expectation and Reward Expectation Error in the Putamen and Caudate Nucleus During Stimulus-Action-Reward Association Learning
J Neurophysiol, February 1, 2006; 95(2): 948 - 959.
[Abstract] [Full Text] [PDF]
M. E. Hasselmo
A Model of Prefrontal Cortical Mechanisms for Goal-directed Behavior
J. Cogn. Neurosci., July 1, 2005; 17(7): 1115 - 1129.
[Abstract] [Full Text] [PDF]
W.-X. Pan and B. I. Hyland
Pedunculopontine Tegmental Nucleus Controls Conditioned Responses of Midbrain Dopamine Neurons in Behaving Rats
J. Neurosci., May 11, 2005; 25(19): 4725 - 4732.
[Abstract] [Full Text] [PDF]
P. J. Voulalas, L. Holtzclaw, J. Wolstenholme, J. T. Russell, and S. E. Hyman
Metabotropic Glutamate Receptors and Dopamine Receptors Cooperate to Enhance Extracellular Signal-Regulated Kinase Phosphorylation in Striatal Neurons
J. Neurosci., April 13, 2005; 25(15): 3763 - 3773.
[Abstract] [Full Text] [PDF]
F. Worgotter and B. Porr
Temporal Sequence Learning, Prediction, and Control: A Review of Different Models and Their Relation to Biological Mechanisms
Neural Comput., February 1, 2005; 17(2): 245 - 319.
[Abstract] [Full Text] [PDF]
J. O'Doherty, P. Dayan, J. Schultz, R. Deichmann, K. Friston, and R. J. Dolan
Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning
Science, April 16, 2004; 304(5669): 452 - 454.
[Abstract] [Full Text] [PDF]
T D Sanger
Childhood onset generalised dystonia can be modelled by increased gain in the indirect basal ganglia pathway
J. Neurol. Neurosurg. Psychiatry, November 1, 2003; 74(11): 1509 - 1515.
[Abstract] [Full Text] [PDF]
T. Satoh, S. Nakai, T. Sato, and M. Kimura
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
J. Neurosci., October 29, 2003; 23(30): 9913 - 9923.
[Abstract] [Full Text] [PDF]
S. Ito, V. Stuphorn, J. W. Brown, and J. D. Schall
Performance Monitoring by the Anterior Cingulate Cortex During Saccade Countermanding
Science, October 3, 2003; 302(5642): 120 - 122.
[Abstract] [Full Text] [PDF]
N. D. Daw and D. S. Touretzky
Long-Term Reward Prediction in TD Models of the Dopamine System
Neural Comput., November 1, 2002; 14(11): 2567 - 2583.
[Abstract] [Full Text] [PDF]
Y. Kobayashi, Y. Inoue, M. Yamamoto, T. Isa, and H. Aizawa
Contribution of Pedunculopontine Tegmental Nucleus Neurons to Performance of Visually Guided Saccade Tasks in Monkeys
J Neurophysiol, August 1, 2002; 88(2): 715 - 731.
[Abstract] [Full Text] [PDF]
L. L. Brown, S. M. Feldman, D. M. Smith, J. R. Cavanaugh, R. F. Ackermann, and A. M. Graybiel
Differential Metabolic Activity in the Striosome and Matrix Compartments of the Rat Striatum during Natural Behaviors
J. Neurosci., January 1, 2002; 22(1): 305 - 314.
[Abstract] [Full Text] [PDF]
H. Nakahara, K. Doya, and O. Hikosaka
Parallel Cortico-Basal Ganglia Mechanisms for Acquisition and Execution of Visuomotor Sequences--A Computational Approach
J. Cogn. Neurosci., July 1, 2001; 13(5): 626 - 647.
[Abstract] [Full Text] [PDF]
R. E. Suri and W. Schultz
Temporal Difference Model Reproduces Anticipatory Neural Activity
Neural Comput., April 1, 2001; 13(4): 841 - 862.
[Abstract] [Full Text]
A. S. Dave and D. Margoliash
Song Replay During Sleep and Computational Rules for Sensorimotor Vocal Learning
Science, October 27, 2000; 290(5492): 812 - 816.
[Abstract] [Full Text]
|
|
|
|
 |
|