The Journal of Neuroscience, August 1, 2007, 27(31):8170-8173; doi:10.1523/JNEUROSCI.1561-07.2007
Previous Article | Next Article 
Mini-Review
Functional Specialization of the Primate Frontal Cortex during Decision Making
Daeyeol Lee,1
Matthew F. S. Rushworth,2
Mark E. Walton,2
Masataka Watanabe,3 and
Masamichi Sakagami4
1Department of Neurobiology, Yale University School of Medicine, New Haven, Connecticut 06510, 2Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, United Kingdom, 3Department of Psychology, Tokyo Metropolitan Institute for Neuroscience, Tokyo 183-8526, Japan, and 4Brain Science Research Institute, Tamagawa University, Tokyo 194-8610, Japan
 |
Abstract
|
|---|
Economic theories of decision making are based on the principle of utility maximization, and reinforcement-learning theory provides computational algorithms that can be used to estimate the overall reward expected from alternative choices. These formal models not only account for a large range of behavioral observations in human and animal decision makers, but also provide useful tools for investigating the neural basis of decision making. Nevertheless, in reality, decision makers must combine different types of information about the costs and benefits associated with each available option, such as the quality and quantity of expected reward and required work. In this article, we put forward the hypothesis that different subdivisions of the primate frontal cortex may be specialized to focus on different aspects of dynamic decision-making processes. In this hypothesis, the lateral prefrontal cortex is primarily involved in maintaining the state representation necessary to identify optimal actions in a given environment. In contrast, the orbitofrontal cortex and the anterior cingulate cortex might be primarily involved in encoding and updating the utilities associated with different sensory stimuli and alternative actions, respectively. These cortical areas are also likely to contribute to decision making in a social context.
Key words: reinforcement learning; reward; cingulate cortex; prefrontal cortex; orbitofrontal cortex; neuroeconomics
Decision making refers to the process by which the animal chooses a particular response based on its evaluation of the potential costs and benefits associated with alternative actions. Economic theories suggest that this can be based on a set of numerical scores, referred to as utilities, associated with alternative choices (von Neumann and Morgenstern, 1944
). This process of decision making is dynamic and continually adjusted to reflect the animal's experience. As described by the reinforcement learning theory (Sutton and Barto, 1998
), any discrepancy between the outcome expected by the animal and the actual outcome from its chosen action influences the animal's future decision-making strategies. Throughout this interactive process, the animal also needs to take into consideration the possibility that the mapping between its chosen actions and their outcomes might change depending on the environmental context. Furthermore, the desirability of a particular outcome is dependent on the animal's current biological needs.
Given the complex nature of the decision-making process and the need to integrate information about the animal's external environment and internal milieu, the primate frontal cortex, with its close anatomical connections with high-order sensorimotor cortical areas (Petrides and Pandya, 1999
, 2002
) and subcortical limbic system (Öngür and Price, 2000
), is likely to be involved in various aspects of decision making (Fig. 1). Recently, fertile interactions between the disciplines of economics, reinforcement learning theory, and animal learning psychology have influenced the development of new hypotheses regarding the nature of decision making and these are increasingly being tested in neurobiological studies of the primate frontal cortex. Many findings from these studies have already begun to provide a more complete and coherent perspective on multiple functions of the frontal cortex (Fellows, 2004
; Wallis, 2007
). Although this review focuses on the role of primate frontal cortex in decision making, it should be noted that decision making is a process distributed in numerous brain areas (Sugrue et al., 2005
; Daw and Doya, 2006
), and that frontal cortex is likely to be involved in functions other than decision making (Miller and Cohen, 2001
; Amodio and Frith, 2006
).

View larger version (77K):
[in this window]
[in a new window]
|
Figure 1. A, B, Medial (A) and lateral (B) view of the rhesus monkey's brain showing the lateral prefrontal cortex, the orbitofrontal cortex, and the anterior cingulate cortex.
|
|
 |
Lateral prefrontal cortex and state representation
|
|---|
An optimal action, namely an action that is most likely to yield the most desirable outcome, often changes according to the state of the animal's environment. Therefore, if a sensory stimulus informs the animal of a change in the state of its environment, the animal needs to store such information until it produces an appropriate action or until this information can be combined with another stimulus to determine the new state of the environment. For example, a classic working memory task requires the animal to remember a particular state of the environment, as often indicated by a brief sensory stimulus. The animal is rewarded only when it produces an action corresponding to this state after some delay. Therefore, neural activity related to the animal's working memory, as commonly seen in the lateral prefrontal cortex (Fuster and Alexander, 1971
; Funahashi et al., 1989
), can be characterized as representing the state of the environment. Similarly, many other types of signals found in the prefrontal cortex, such as the abstract properties of a sensory stimulus (Freedman et al., 2001
; Nieder et al., 2002
) and the rules of the task that specify how to determine a correct action based on sensory stimuli (Sakagami and Niki, 1994
; Wise and Murray, 2000
; Hoshi et al., 2000
; Wallis et al., 2001
) can be considered as encoding the state of the environment. Neurons in the lateral prefrontal cortex are also involved in accumulating sensory evidence (Kim and Shadlen, 1999
) or transforming perceptual information to motor outputs (Takeda and Funahashi, 2004
). In short, neurons in the lateral prefrontal cortex encode specific states of the environment necessary to determine an optimal action unequivocally.
More recently, many investigators have manipulated the motivational context during working memory tasks. In these tasks, the animal is informed of the correct action and its immediate outcome. The results from these experiments have shown that so-called delay activity is often modulated by the outcomes expected from the correct and incorrect actions (Watanabe, 1996
; Leon and Shadlen, 1999
; Amemori and Sawaguchi, 2006
; Kobayashi et al., 2006
), indicating that the lateral prefrontal cortex might encode the state of the environment and the properties of the expected reward conjunctively (Watanabe and Sakagami, 2007
). In the lateral prefrontal cortex, increasing the value of the expected reward often increased the reliability of the state representation (Kobayashi et al. 2002
). Therefore, in addition to encoding the state of the environment, the lateral prefrontal cortex might represent the utilities or values associated with various states of the environment.
 |
Orbitofrontal cortex and value representation
|
|---|
Lesions which encompass the orbitofrontal cortex in human patients often impair their ability to adjust decision making strategies when the previously successful choices are no longer advantageous (Bechara et al., 2000
; Rolls, 2000
) and to make preference judgments consistently (Fellows and Farah, 2007
). Impaired decision making abilities in such psychiatric conditions as substance abuse and frontotemporal dementia may also result from the dysfunction of the orbitofrontal cortex (Rahman et al., 2001
). Similarly, orbitofrontal lesions in monkeys impair the ability to modify behavior when the outcomes of decisions change dynamically (Izquierdo et al., 2004
) and to assign appropriate values to different objects in the environment (Izquierdo et al., 2005
). According to one novel proposal derived from reinforcement learning, this faculty may depend on the ability to represent the statistical interdependence between the values of alternative options (Hampton et al., 2006
). Knowledge of such interdependence might allow the decision makers to switch to a better choice immediately, as soon as they experience the reduction of values from a particular choice.
Signals related to the expected reward have been also identified in the primate orbitofrontal cortex. However, in contrast to the neurons in the lateral prefrontal cortex, neurons in the orbitofrontal cortex seldom encode different states of the environment and optimal actions associated with them. Instead, their activity is largely determined by the expected outcome (Tremblay and Schultz, 2000
; Wallis and Miller, 2003
; Roesch and Olson, 2004
), even when the outcome is determined by the animal's own choice (Padoa-Schioppa and Assad, 2006
). These results demonstrate that it is possible to design experiments that determine how frontal areas differ in their processing of signals related to the states of the environment and the outcomes that can be expected from those states. Although it is widely assumed that various frontal areas make distinct contributions to decision making and cognition, few studies have directly tested this assumption and compared the contributions of different frontal areas in the same task. Formal accounts of decision making suggest a number of critical axes along which to compare frontal areas in future experiments.
 |
Anterior cingulate cortex and outcome evaluation
|
|---|
Several lines of evidence suggest that the primate anterior cingulate cortex might play a key role in choosing appropriate actions when the environment is uncertain or dynamic. First, many single-neuron recording studies have found that the neurons in the anterior cingulate cortex modulate their activity according to the outcome of the animal's action (Niki and Watanabe, 1979
; Ito et al., 2003
; Matsumoto et al., 2003
, 2007
). Second, this outcome-related activity might be required for the animals to update its decision-making strategies after committing an error (Shima and Tanji, 1998
; Procyk et al., 2000
). Third, a lesion in the anterior cingulate cortex impairs the animal's ability to integrate signals related to the outcomes of the animal's previous choices to make optimal decisions (Kennerley et al., 2006
). Finally, the results from rodent studies suggested that the anterior cingulate cortex may also be involved in combining information about the costs and benefits associated with alternative actions (Rudebeck et al., 2006b
). Combined with the anatomical finding that much of the primate anterior cingulate cortex projects to cortical areas with motor functions (Dum and Strick, 1991
), including the presupplementary motor area and supplementary eye field (Wang et al., 2001
; Luppino et al., 2003
), these results suggest that the utilities associated with different actions, referred to as action value functions in reinforcement learning theory, may be encoded and updated in the anterior cingulate cortex (Rushworth et al., 2007
).
 |
Frontal cortex and social interactions
|
|---|
During social interaction, the outcome of an action can change dynamically depending on the actions of other decision makers in a group. Although choice behaviors of humans and animal in social settings often deviate from the optimal strategies described by game theory, such deviations can be often accounted for by reinforcement learning algorithms (Camerer, 2003
; Lee et al., 2004
). Given that the different areas of primate frontal cortex are intimately involved in reinforcement learning, this suggests that they might also play an important role in socially interactive decision making. For example, lesions in the orbitofrontal cortex induce a loss of social dominance with increased aversion and reduced aggression in threatening situations (Butter and Snyder, 1972
).
During a computer-simulated zero-sum game, the activity in the lateral prefrontal cortex encode the signals related to the animal's previous choices and their outcomes in multiple trials (Barraclough et al., 2004
; Seo et al., 2007
), and therefore might provide an appropriate context in which the animal's decision-making strategies can be updated during social interaction. For more complex social interactions, such as cooperation, the process of identifying successful decision-making strategies might depend on some cortical areas specialized for processing socially meaningful stimuli and thereby inferring actions expected from other animals. For example, the direction of gaze in other animals might provide information about their probable actions during social interactions (Deaner et al., 2005
; Flombaum and Santos, 2005
). In fact, a lesion in the primate anterior cingulate gyrus causes the animals to become less interested in gathering information from social stimuli, such as faces (Rudebeck et al., 2006a
), implicating an important role for this frontal area, perhaps in conjunction with the amygdala (Bachevalier and Loveland, 2006
), in social perception.
 |
Summary and conclusion
|
|---|
Guided by the detailed accounts of connectivity among different subdivisions of the primate frontal cortex and by formal theories of decision making, a large number of recent lesion and physiological studies have begun to explore the contribution of the frontal cortex to the making and assessment of choices. Overall, the results from these studies suggest that the lateral, medial, and ventral aspects of the prefrontal cortex might be specialized for representing the relevant states of the animal's environment, updating the desirability of alternative actions, and predicting the values of rewards expected from different objects in the animal's environment, respectively. How these brain areas interact with one another and whether they are further differentiated to improve the animal's ability to behave optimally during complex social interaction remains an important topic for future research.
 |
Footnotes
|
|---|
Received April 6, 2007;
revised June 18, 2007;
accepted June 30, 2007.
This work was supported by the National Institutes of Health (D.L.), the Medical Research Council (M.F.S.R., M.E.W.), the Royal Society (M.F.S.R.), Human Frontier Science Program (M.S.), and a Grant-in-Aid for Scientific Research on Priority Areas from the Ministry of Education, Culture, Sports, Science, and Technology of Japan (M.W., M.S.).
Correspondence should be addressed to Dr. Daeyeol Lee, Department of Neurobiology, Yale University School of Medicine, 333 Cedar Street, Sterling Hall of Medicine C303, New Haven, CT 06510. Email: daeyeol.lee{at}yale.edu
Copyright © 2007 Society for Neuroscience 0270-6474/07/278170-04$15.00/0
 |
References
|
|---|
Amemori K, Sawaguchi T (2006) Contrasting effects of reward expectation on sensory and motor memories in primate prefrontal neurons. Cereb Cortex 16:1002–1015.[Abstract/Free Full Text]
Amodio DM, Frith CD (2006) Meeting of minds: the medial frontal cortex and social cognition. Nat Rev Neurosci 7:268–277.[CrossRef][ISI][Medline]
Bachevalier J, Loveland KA (2006) The orbitofrontal-amygdala circuit and self-regulation of social-emotional behavior in autism. Neurosci Biobehav Rev 30:97–117.[CrossRef][ISI][Medline]
Barraclough DJ, Conroy ML, Lee D (2004) Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7:404–410.[CrossRef][ISI][Medline]
Bechara A, Damasio H, Damasio AR (2000) Emotion, decision making and the orbitofrontal cortex. Cereb Cortex 10:295–307.[Abstract/Free Full Text]
Butter CM, Snyder DR (1972) Alterations in aversive and aggressive behaviors following orbital frontal lesions in rhesus monkeys. Acta Neurobiol Exp 32:525–565.[Medline]
Camerer CF (2003) Behavioral game theory. Princeton, NJ: Princeton UP.
Daw ND, Doya K (2006) The computational neurobiology of learning and reward. Curr Opin Neurobiol 16:199–204.[CrossRef][ISI][Medline]
Deaner RO, Khera AV, Platt ML (2005) Monkeys pay per view: adaptive valuation of social images by rhesus macaques. Curr Biol 15:543–548.[CrossRef][ISI][Medline]
Dum RP, Strick PL (1991) The origin of corticospinal projections from the premotor areas in the frontal lobe. J Neurosci 11:667–689.[Abstract]
Fellows LK (2004) The cognitive neuroscience of human decision making: a review and conceptual framework. Behav Cogn Neurosci Rev 3:159–172.[Abstract]
Fellows LK, Farah MJ (2007) The role of ventromedial prefrontal cortex in decision making: judgment under uncertainty or judgment per se? Cereb Cortex, in press.
Flombaum JI, Santos LR (2005) Rhesus monkeys attribute perceptions to others. Curr Biol 15:447–452.[CrossRef][ISI][Medline]
Freedman DJ, Riesenhuber M, Poggio T, Miller EK (2001) Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291:312–316.[Abstract/Free Full Text]
Funahashi S, Bruce CJ, Goldman-Rakic PS (1989) Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J Neurophysiol 61:331–349.[Abstract/Free Full Text]
Fuster JM, Alexander GE (1971) Neuron activity related to short-term memory. Science 173:652–654.[Abstract/Free Full Text]
Hampton AN, Bossaerts P, O'Doherty JP (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci 26:8360–8367.[Abstract/Free Full Text]
Hoshi E, Shima K, Tanji J (2000) Neuronal activity in the primate prefrontal cortex in the process of motion selection based on two behavioral rules. J Neurophysiol 83:2355–2373.[Abstract/Free Full Text]
Ito S, Stuphorn V, Brown JW, Schall JD (2003) Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302:120–122.[Abstract/Free Full Text]
Izquierdo A, Suda RK, Murray EA (2004) Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci 24:7540–7548.[Abstract/Free Full Text]
Izquierdo A, Suda RK, Murray EA (2005) Comparison of the effects of bilateral orbital prefrontal cortex lesions and amygdala lesions on emotional responses in rhesus monkeys. J Neurosci 25:8534–8542.[Abstract/Free Full Text]
Kennerley SW, Walton ME, Behrens TEJ, Buckley MJ, Rushworth MFS (2006) Optimal decision making and the anterior cingulate cortex. Nat Neurosci 9:940–947.[CrossRef][Medline]
Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176–185.[CrossRef][ISI][Medline]
Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M, Hikosaka O (2002) Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87:1488–1498.[Abstract/Free Full Text]
Kobayashi S, Nomoto K, Watanabe M, Hikosaka O, Schultz W, Sakagami M (2006) Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51:861–870.[CrossRef][ISI][Medline]
Lee D, Conroy ML, McGreevy BP, Barraclough DJ (2004) Reinforcement learning and decision making in monkeys during a competitive game. Cogn Brain Res 22:45–58.[CrossRef][Medline]
Leon MI, Shadlen MN (1999) Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24:415–425.[CrossRef][ISI][Medline]
Luppino G, Rozzi S, Calzavara R, Matelli M (2003) Prefrontal and agranular cingulate projections to the dorsal premotor areas F2 and F7 in the macaque monkey.
Matsumoto K, Suzuki W, Tanaka K (2003) Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301:229–232.[Abstract/Free Full Text]
Matsumoto M, Matsumoto K, Abe H, Tanaka K (2007) Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10:647–656.[CrossRef][ISI][Medline]
Miller EK, Cohen JD (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24:167–202.[CrossRef][ISI][Medline]
Nieder A, Freedman DJ, Miller EK (2002) Representation of the quantity of visual items in the primate prefrontal cortex. Science 297:1708–1711.[Abstract/Free Full Text]
Niki H, Watanabe M (1979) Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171:213–224.[CrossRef][ISI][Medline]
Öngür D, Price JL (2000) The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex 10:206–219.[Abstract/Free Full Text]
Padoa-Schioppa C, Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441:223–226.[CrossRef][Medline]
Petrides M, Pandya DN (1999) Dorsolateral prefrontal cortex: comparative cytoarchitectonic analysis in the human and the macaque brain and corticocortical connection patterns. Eur J Neurosci 11:1011–1036.[CrossRef][ISI][Medline]
Petrides M, Pandya DN (2002) Comparative cytoarchitectonic analysis of the human and the macaque ventrolateral prefrontal cortex and corticocortical connection patterns in the monkey. Eur J Neurosci 16:291–310.[CrossRef][ISI][Medline]
Procyk E, Tanaka YL, Joseph JP (2000) Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nat Neurosci 3:502–508.[CrossRef][ISI][Medline]
Rahman S, Sahakian BJ, Cardinal R, Rogers RD, Robbins TW (2001) Decision making and neuropsychiatry. Trends Cogn Sci 5:271–277.[CrossRef][ISI][Medline]
Roesch MR, Olson CR (2004) Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304:307–310.[Abstract/Free Full Text]
Rolls ET (2000) The orbitofrontal cortex and reward. Cereb Cortex 10:284–294.[Abstract/Free Full Text]
Rudebeck PH, Buckley MJ, Walton ME, Rushworth MF (2006a) A role for the macaque anterior cingulate gyrus in social valuation. Science 313:1310–1312.[Abstract/Free Full Text]
Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF (2006b) Separate neural pathways process different decision costs. Nat Neurosci 9:1161–1168.[CrossRef][ISI][Medline]
Rushworth MFS, Behrens TEJ, Rudebeck PH, Walton ME (2007) Contrasting roles for cingulate and orbitofrontal cortex in decision and social behavior. Trends Cogn Sci 11:168–176.[CrossRef][ISI][Medline]
Sakagami M, Niki H (1994) Encoding of behavioral significance of visual stimuli by primate prefrontal neurons: relation to relevant task conditions. Exp Brain Res 97:423–436.[ISI][Medline]
Seo H, Barraclough DJ, Lee D (2007) Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb Cortex, In press.
Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335–1338.[Abstract/Free Full Text]
Sugrue LP, Corrado GS, Newsome WT (2005) Choosing the greater of two goods: neural currencies for valuation and decision making. Nat Rev Neurosci 6:363–375.[CrossRef][ISI][Medline]
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge: MIT.
Takeda K, Funahashi S (2004) Population vector analysis of primate prefrontal activity during spatial working memory. Cereb Cortex 14:1328–1339.[Abstract/Free Full Text]
Tremblay L, Schultz W (2000) Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol 83:1864–1876.[Abstract/Free Full Text]
von Neumann J, Morgenstern O (1944) Theory of games and economic behavior. Princeton, NJ: Princeton UP.
Wallis JD (2007) Orbitofrontal cortex and its contribution to decision-making. Annu Rev Neurosci 30:31–56.[Medline]
Wallis JD, Miller EK (2003) Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18:2069–2081.[CrossRef][ISI][Medline]
Wallis JD, Anderson KC, Miller EK (2001) Single neurons in prefrontal cortex encode abstract rules. Nature 411:953–956.[CrossRef][Medline]
Wang Y, Shima K, Sawamura H, Tanji J (2001) Spatial distribution of cingulate cells projecting to the primary, supplementary, and pre-supplementary motor areas: a retrograde multiple labeling study in the macaque monkey. Neurosci Res 39:39–49.[CrossRef][ISI][Medline]
Watanabe M (1996) Reward expectancy in primate prefrontal neurons. Nature 382:629–632.[CrossRef][Medline]
Watanabe M, Sakagami M (2007) Integration of cognitive and motivational context information in the primate prefrontal cortex. Cereb Cortex, in press.
Wise SP, Murray EA (2000) Arbitrary associations between antecedents and actions. Trends Neurosci 23:271–276.[CrossRef][ISI][Medline]
This article has been cited by other articles:

|
 |

|
 |
 
H. Zheng and H.-R. Berthoud
Neural Systems Controlling the Drive to Eat: Mind Versus Metabolism
Physiology,
April 1, 2008;
23(2):
75 - 83.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
Y. B. Kim, N. Huh, H. Lee, E. H. Baeg, D. Lee, and M. W. Jung
Encoding of Action History in the Rat Ventral Striatum
J Neurophysiol,
December 1, 2007;
98(6):
3548 - 3556.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
B. W. Balleine
The Neural Basis of Choice and Decision Making
J. Neurosci.,
August 1, 2007;
27(31):
8159 - 8160.
[Full Text]
[PDF]
|
 |
|