 |
|||||
|
|||||
|
|||||
|
|||||
|
|||||
The Journal of Neuroscience, August 1, 2007, 27(31):8366-8377; doi:10.1523/JNEUROSCI.2369-07.2007
Previous Article | Next Article
Behavioral/Systems/Cognitive
Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game
Hyojung Seo andDaeyeol Lee Department of Neurobiology, Yale University School of Medicine, New Haven, Connecticut 06510
Correspondence should be addressed to Dr. Daeyeol Lee, Department of Neurobiology, Yale University School of Medicine, 333 Cedar Street, SHM B404, New Haven, CT 06510. Email: daeyeol.lee{at}yale.edu
The process of decision making in humans and other animals is adaptive and can be tuned through experience so as to optimize the outcomes of their choices in a dynamic environment. Previous studies have demonstrated that the anterior cingulate cortex plays an important role in updating the animal's behavioral strategies when the action outcome contingencies change. Moreover, neurons in the anterior cingulate cortex often encode the signals related to expected or actual reward. We investigated whether reward-related activity in the anterior cingulate cortex is affected by the animal's previous reward history. This was tested in rhesus monkeys trained to make binary choices in a computer-simulated competitive zero-sum game. The animal's choice behavior was relatively close to the optimal strategy but also revealed small systematic biases that are consistent with the use of a reinforcement learning algorithm. In addition, the activity of neurons in the dorsal anterior cingulate cortex that was related to the reward received by the animal in a given trial often was modulated by the rewards in the previous trials. Some of these neurons encoded the rate of rewards in previous trials, whereas others displayed activity modulations more closely related to the reward prediction errors. In contrast, signals related to the animal's choices were represented only weakly in this cortical area. These results suggest that neurons in the dorsal anterior cingulate cortex might be involved in the subjective evaluation of choice outcomes based on the animal's reward history.
Key words: reinforcement learning; game theory; neuroeconomics; decision; dopamine; reward
Received Feb. 18, 2007; revised June 23, 2007; accepted June 25, 2007.
Correspondence should be addressed to Dr. Daeyeol Lee, Department of Neurobiology, Yale University School of Medicine, 333 Cedar Street, SHM B404, New Haven, CT 06510. Email: daeyeol.lee{at}yale.edu
This article has been cited by other articles:
K. Wunderlich, A. Rangel, and J. P. O'Doherty
Neural computations underlying action-based decision making in the human brain
PNAS, October 6, 2009; 106(40): 17199 - 17204.
[Abstract] [Full Text] [PDF]
M. Ito and K. Doya
Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia
J. Neurosci., August 5, 2009; 29(31): 9861 - 9874.
[Abstract] [Full Text] [PDF]
H. Seo, D. J. Barraclough, and D. Lee
Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game
J. Neurosci., June 3, 2009; 29(22): 7278 - 7289.
[Abstract] [Full Text] [PDF]
J. Fujiwara, P. N. Tobler, M. Taira, T. Iijima, and K.-I. Tsutsui
Segregated and Integrated Coding of Reward and Punishment in the Cingulate Cortex
J Neurophysiol, June 1, 2009; 101(6): 3284 - 3293.
[Abstract] [Full Text] [PDF]
N. Huh, S. Jo, H. Kim, J. H. Sul, and M. W. Jung
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents
Learn. Mem., April 29, 2009; 16(5): 315 - 323.
[Abstract] [Full Text] [PDF]
M. Haruno and M. Kawato
Activity in the Superior Temporal Sulcus Highlights Learning Competence in an Interaction Game
J. Neurosci., April 8, 2009; 29(14): 4542 - 4547.
[Abstract] [Full Text] [PDF]
H. Seo and D. Lee
Behavioral and Neural Changes after Gains and Losses of Conditioned Reinforcers
J. Neurosci., March 18, 2009; 29(11): 3627 - 3641.
[Abstract] [Full Text] [PDF]
S. Tsujimoto, A. Genovesio, and S. P. Wise
Monkey Orbitofrontal Cortex Encodes Response Choices Near Feedback Time
J. Neurosci., February 25, 2009; 29(8): 2569 - 2574.
[Abstract] [Full Text] [PDF]
D. Thevarajah, A. Mikulic, and M. C. Dorris
Role of the Superior Colliculus in Choosing Mixed-Strategy Saccades
J. Neurosci., February 18, 2009; 29(7): 1998 - 2008.
[Abstract] [Full Text] [PDF]
J. Glascher, A. N. Hampton, and J. P. O'Doherty
Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making
Cereb Cortex, February 1, 2009; 19(2): 483 - 495.
[Abstract] [Full Text] [PDF]
K. K Watson and M. L Platt
Neuroethology of reward and decision making
Phil Trans R Soc B, December 12, 2008; 363(1511): 3825 - 3835.
[Abstract] [Full Text] [PDF]
H. Seo and D. Lee
Cortical mechanisms for reinforcement learning in competitive games
Phil Trans R Soc B, December 12, 2008; 363(1511): 3845 - 3857.
[Abstract] [Full Text] [PDF]
F. Krueger, J. Grafman, and K. McCabe
Neural correlates of economic game playing
Phil Trans R Soc B, December 12, 2008; 363(1511): 3859 - 3874.
[Abstract] [Full Text] [PDF]
J. I. Gold, C.-T. Law, P. Connolly, and S. Bennur
The Relative Influences of Priors and Sensory Evidence on an Oculomotor Decision Variable During Perceptual Learning
J Neurophysiol, November 1, 2008; 100(5): 2653 - 2668.
[Abstract] [Full Text] [PDF]
A. Mikulic and M. C. Dorris
Temporal and Spatial Allocation of Motor Preparation During a Mixed-Strategy Game
J Neurophysiol, October 1, 2008; 100(4): 2101 - 2108.
[Abstract] [Full Text] [PDF]
P. C. J. Taylor, M. F. S. Rushworth, and A. C. Nobre
Choosing Where to Attend and the Medial Frontal Cortex: An fMRI Study
J Neurophysiol, September 1, 2008; 100(3): 1397 - 1406.
[Abstract] [Full Text] [PDF]
J.-W. Sohn and D. Lee
Order-Dependent Modulation of Directional Signals in the Supplementary and Presupplementary Motor Areas
J. Neurosci., December 12, 2007; 27(50): 13655 - 13666.
[Abstract] [Full Text] [PDF]
|
|
|
|
 |
|