 |
||||
|
||||
|
||||
|
||||
The Journal of Neuroscience, October 29, 2003, 23(30):9913-9923
Previous Article | Next Article
Behavioral/Systems/Cognitive
Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons
Takemasa Satoh, Sadamu Nakai, Tatsuo Sato, andMinoru Kimura Department of Physiology, Kyoto Prefectural University of Medicine, Kawaramachi-Hirokoji, Kamigyo-ku, Kyoto 602-8566, Japan
We recorded the activity of midbrain dopamine neurons in an instrumental conditioning task in which monkeys made a series of behavioral decisions on the basis of distinct reward expectations. Dopamine neurons responded to the first visual cue that appeared in each trial [conditioned stimulus (CS)] through which monkeys initiated trial for decision while expecting trial-specific reward probability and volume. The magnitude of neuronal responses to the CS was approximately proportional to reward expectations but with considerable discrepancy. In contrast, CS responses appear to represent motivational properties, because their magnitude at trials with identical reward expectation had significant negative correlation with reaction times of the animal after the CS. Dopamine neurons also responded to reinforcers that occurred after behavioral decisions, and the responses precisely encoded positive and negative reward expectation errors (REEs). The gain of coding REEs by spike frequency increased during learning act-outcome contingencies through a few months of task training, whereas coding of motivational properties remained consistent during the learning. We found that the magnitude of CS responses was positively correlated with that of reinforcers, suggesting a modulation of the effectiveness of REEs as a teaching signal by motivation. For instance, rate of learning could be faster when animals are motivated, whereas it could be slower when less motivated, even at identical REEs. Therefore, the dual correlated coding of motivation and REEs suggested the involvement of the dopamine system, both in reinforcement in more elaborate ways than currently proposed and in motivational function in reward-based decision-making and learning.
Key words: dopamine neuron; learning; teaching signal; reward prediction error; trial and error; decision-making
Received June 16, 2003; revised September 8, 2003; accepted September 9, 2003.
This article has been cited by other articles:
W.-X. Pan, R. Schmidt, J. R. Wickens, and B. I. Hyland
Tripartite Mechanism of Extinction Suggested by Dopamine Neuron Activity and Temporal Difference Model
J. Neurosci., September 24, 2008; 28(39): 9619 - 9631.
[Abstract] [Full Text] [PDF]
K. Nakamura, M. Matsumoto, and O. Hikosaka
Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus
J. Neurosci., May 14, 2008; 28(20): 5331 - 5343.
[Abstract] [Full Text] [PDF]
H. Yamada, N. Matsumoto, and M. Kimura
History- and Current Instruction-Based Coding of Forthcoming Behavioral Outcomes in the Striatum
J Neurophysiol, December 1, 2007; 98(6): 3557 - 3567.
[Abstract] [Full Text] [PDF]
M. J. Frank, A. A. Moustafa, H. M. Haughey, T. Curran, and K. E. Hutchison
Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning
PNAS, October 9, 2007; 104(41): 16311 - 16316.
[Abstract] [Full Text] [PDF]
H. M. Bayer, B. Lau, and P. W. Glimcher
Statistics of Midbrain Dopamine Neuron Spike Trains in the Awake Primate
J Neurophysiol, September 1, 2007; 98(3): 1428 - 1439.
[Abstract] [Full Text] [PDF]
H. Ji and P. D. Shepard
Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism
J. Neurosci., June 27, 2007; 27(26): 6923 - 6930.
[Abstract] [Full Text] [PDF]
B. Pasquereau, A. Nadjar, D. Arkadir, E. Bezard, M. Goillandeau, B. Bioulac, C. E. Gross, and T. Boraud
Shaping of Motor Responses by Incentive Values through the Basal Ganglia
J. Neurosci., January 31, 2007; 27(5): 1176 - 1183.
[Abstract] [Full Text] [PDF]
S. N. Haber, K.-S. Kim, P. Mailly, and R. Calzavara
Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning.
J. Neurosci., August 9, 2006; 26(32): 8368 - 8376.
[Abstract] [Full Text] [PDF]
R. R. Metzger, N. T. Greene, K. K. Porter, and J. M. Groh
Effects of reward and behavioral context on neural activity in the primate inferior colliculus.
J. Neurosci., July 12, 2006; 26(28): 7468 - 7476.
[Abstract] [Full Text] [PDF]
N. D. Daw, A. C. Courville, and D. S. Touretzky
Representation and Timing in Theories of the Dopamine System
Neural Comput., July 1, 2006; 18(7): 1637 - 1677.
[Abstract] [Full Text] [PDF]
T. Mogami and K. Tanaka
Reward association affects neuronal responses to visual stimuli in macaque te and perirhinal cortices.
J. Neurosci., June 21, 2006; 26(25): 6761 - 6770.
[Abstract] [Full Text] [PDF]
K. Nakamura and O. Hikosaka
Role of dopamine in the primate caudate nucleus in reward modulation of saccades.
J. Neurosci., May 17, 2006; 26(20): 5360 - 5369.
[Abstract] [Full Text] [PDF]
J.-C. Dreher, P. Kohn, and K. F. Berman
Neural Coding of Distinct Statistical Properties of Reward Information in Humans
Cereb Cortex, April 1, 2006; 16(4): 561 - 573.
[Abstract] [Full Text] [PDF]
J. Cheng and M. G.P. Feenstra
Individual differences in dopamine efflux in nucleus accumbens shell and core during instrumental learning.
Learn. Mem., March 1, 2006; 13(2): 168 - 177.
[Abstract] [Full Text] [PDF]
K. Samejima, Y. Ueda, K. Doya, and M. Kimura
Representation of Action-Specific Reward Values in the Striatum
Science, November 25, 2005; 310(5752): 1337 - 1340.
[Abstract] [Full Text] [PDF]
M. Campos, B. Breznen, K. Bernheim, and R. A. Andersen
Supplementary Motor Area Encodes Reward Expectancy in Eye-Movement Tasks
J Neurophysiol, August 1, 2005; 94(2): 1325 - 1335.
[Abstract] [Full Text] [PDF]
T. Minamimoto, Y. Hori, and M. Kimura
Complementary Process to Response Bias in the Centromedian Nucleus of the Thalamus
Science, June 17, 2005; 308(5729): 1798 - 1801.
[Abstract] [Full Text] [PDF]
P. N. Tobler, C. D. Fiorillo, and W. Schultz
Adaptive Coding of Reward Value by Dopamine Neurons
Science, March 11, 2005; 307(5715): 1642 - 1645.
[Abstract] [Full Text] [PDF]
M. J. Frank, L. C. Seeberger, and R. C. O'Reilly
By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism
Science, December 10, 2004; 306(5703): 1940 - 1943.
[Abstract] [Full Text] [PDF]
D. Arkadir, G. Morris, E. Vaadia, and H. Bergman
Independent Coding of Movement Direction and Reward Prediction by Single Pallidal Neurons
J. Neurosci., November 10, 2004; 24(45): 10047 - 10056.
[Abstract] [Full Text] [PDF]
|
|
|
|
 |
|