 |
The Journal of Neuroscience, August 9, 2006, 26(32):8360-8367; doi:10.1523/JNEUROSCI.1010-06.2006
Previous Article | Next Article 
Behavioral/Systems/Cognitive
The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans
Alan N. Hampton,1
Peter Bossaerts,1,2 and
John P. ODoherty1,2
1Computation and Neural Systems Program and 2Division of Humanities and Social Sciences, California Institute of Technology, Pasadena, California 91125
Correspondence should be addressed to Dr. John P. ODoherty, Division of Humanities and Social Sciences, California Institute of Technology, 1200 East California Boulevard, M/C 228-77, Pasadena, CA 91125. Email: jdoherty{at}hss.caltech.edu
Many real-life decision-making problems incorporate higher-order structure, involving interdependencies between different stimuli, actions, and subsequent rewards. It is not known whether brain regions implicated in decision making, such as the ventromedial prefrontal cortex (vmPFC), use a stored model of the task structure to guide choice (model-based decision making) or merely learn action or state values without assuming higher-order structure as in standard reinforcement learning. To discriminate between these possibilities, we scanned human subjects with functional magnetic resonance imaging while they performed a simple decision-making task with higher-order structure, probabilistic reversal learning. We found that neural activity in a key decision-making region, the vmPFC, was more consistent with a computational model that exploits higher-order structure than with simple reinforcement learning. These results suggest that brain regions, such as the vmPFC, use an abstract model of task structure to guide behavioral choice, computations that may underlie the human capacity for complex social interactions and abstract strategizing.
Key words: decision making; state-based inference; ventromedial prefrontal cortex; fMRI; reversal learning; Bayesian
Received March 7, 2006;
revised July 7, 2006;
accepted July 8, 2006.
Correspondence should be addressed to Dr. John P. ODoherty, Division of Humanities and Social Sciences, California Institute of Technology, 1200 East California Boulevard, M/C 228-77, Pasadena, CA 91125. Email: jdoherty{at}hss.caltech.edu
Related articles in J. Neurosci.:
- Simple Reinforcement Learning Models Are Not Always Appropriate
- Hansem Sohn and Seungyeon Kim
J. Neurosci. 2006 26: 11511-11512.
[Full Text]
This article has been cited by other articles:

|
 |

|
 |
 
D. G. Ghahremani, J. Monterosso, J. D. Jentsch, R. M. Bilder, and R. A. Poldrack
Neural Components Underlying Behavioral Flexibility in Human Reversal Learning
Cereb Cortex,
November 13, 2009;
(2009)
bhp247v1.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. J. Gershman, B. Pesaran, and N. D. Daw
Human Reinforcement Learning Subdivides Structured Action Spaces by Learning Effector-Specific Values
J. Neurosci.,
October 28, 2009;
29(43):
13524 - 13531.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
L. K. Krugel, G. Biele, P. N. C. Mohr, S.-C. Li, and H. R. Heekeren
Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions
PNAS,
October 20, 2009;
106(42):
17951 - 17956.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
K. Wunderlich, A. Rangel, and J. P. O'Doherty
Neural computations underlying action-based decision making in the human brain
PNAS,
October 6, 2009;
106(40):
17199 - 17204.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
V. S. Chib, A. Rangel, S. Shimojo, and J. P. O'Doherty
Evidence for a Common Representation of Decision Values for Dissimilar Goods in Human Ventromedial Prefrontal Cortex
J. Neurosci.,
September 30, 2009;
29(39):
12315 - 12320.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
A. Christakou, M. Brammer, V. Giampietro, and K. Rubia
Right Ventromedial and Dorsolateral Prefrontal Cortices Mediate Adaptive Decisions under Ambiguity by Integrating Choice Utility and Outcome Evaluation
J. Neurosci.,
September 2, 2009;
29(35):
11020 - 11028.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
K. Tachibana, K. Suzuki, E. Mori, N. Miura, R. Kawashima, K. Horie, S. Sato, J. Tanji, and H. Mushiake
Neural Activity in the Human Brain Signals Logical Rule Identification
J Neurophysiol,
September 1, 2009;
102(3):
1526 - 1537.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Ito and K. Doya
Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia
J. Neurosci.,
August 5, 2009;
29(31):
9861 - 9874.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
N. Huh, S. Jo, H. Kim, J. H. Sul, and M. W. Jung
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents
Learn. Mem.,
April 29, 2009;
16(5):
315 - 323.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
A. Kazama and J. Bachevalier
Selective Aspiration or Neurotoxic Lesions of Orbital Frontal Areas 11 and 13 Spared Monkeys' Performance on the Object Discrimination Reversal Task
J. Neurosci.,
March 4, 2009;
29(9):
2794 - 2804.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
J. Glascher, A. N. Hampton, and J. P. O'Doherty
Determining a Role for Ventromedial Prefrontal Cortex in Encoding Action-Based Value Signals During Reward-Related Decision Making
Cereb Cortex,
February 1, 2009;
19(2):
483 - 495.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Beckmann, H. Johansen-Berg, and M. F. S. Rushworth
Connectivity-Based Parcellation of Human Cingulate Cortex and Its Relation to Functional Specialization
J. Neurosci.,
January 28, 2009;
29(4):
1175 - 1190.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
J. D. Ragland, R. Cools, M. Frank, D. A. Pizzagalli, A. Preston, C. Ranganath, and A. D. Wagner
CNTRICS Final Task Selection: Long-Term Memory
Schizophr Bull,
January 1, 2009;
35(1):
197 - 212.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. d'Acremont and P. Bossaerts
Neurobiological studies of risk assessment: A comparison of expected utility and mean-variance approaches
Cogn Affect Behav Neurosci,
December 1, 2008;
8(4):
363 - 374.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
D. Schiller, I. Levy, Y. Niv, J. E. LeDoux, and E. A. Phelps
From Fear to Safety and Back: Reversal of Fear in the Human Brain
J. Neurosci.,
November 5, 2008;
28(45):
11517 - 11525.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
R.J.R Blair
The amygdala and ventromedial prefrontal cortex: functional contributions and dysfunction in psychopathy
Phil Trans R Soc B,
August 12, 2008;
363(1503):
2557 - 2565.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. C. Tanaka, B. W. Balleine, and J. P. O'Doherty
Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions
J. Neurosci.,
June 25, 2008;
28(26):
6750 - 6755.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. X. COHEN
Neurocomputational mechanisms of reinforcement-guided learning in humans: A review
Cogn Affect Behav Neurosci,
June 1, 2008;
8(2):
113 - 125.
[Abstract]
[PDF]
|
 |
|

|
 |

|
 |
 
E. A. Murray, J. P. O'Doherty, and G. Schoenbaum
What We Know and Do Not Know about the Functions of the Orbitofrontal Cortex after 20 Years of Cross-Species Studies
J. Neurosci.,
August 1, 2007;
27(31):
8166 - 8169.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
D. Lee, M. F. S. Rushworth, M. E. Walton, M. Watanabe, and M. Sakagami
Functional Specialization of the Primate Frontal Cortex during Decision Making
J. Neurosci.,
August 1, 2007;
27(31):
8170 - 8173.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
A. N. Hampton and J. P. O'Doherty
Decoding the neural substrates of reward-related decision making with functional MRI
PNAS,
January 23, 2007;
104(4):
1377 - 1382.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
H. Sohn and S. Kim
Simple Reinforcement Learning Models Are Not Always Appropriate
J. Neurosci.,
November 8, 2006;
26(45):
11511 - 11512.
[Full Text]
[PDF]
|
 |
|
|

|