QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

HOME  |   SEARCH  |   ARCHIVE  |   SUBSCRIBE  |   CONTACT  |   HELP

The Journal of Neuroscience, June 24, 2009, 29(25):7945-7947; doi:10.1523/JNEUROSCI.1876-09.2009

This Article Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Related articles in J. Neurosci. Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Boorman, E. D. Articles by Sallet, J. Search for Related Content PubMed PubMed Citation Articles by Boorman, E. D. Articles by Sallet, J.

 Previous Article  |  Next Article 

Journal Club

Editor's Note: These short, critical reviews of recent papers in the Journal, written exclusively by graduate students or postdoctoral fellows, are intended to summarize the important findings of the paper and provide additional insight and commentary. For more information on the format and purpose of the Journal Club, please see http://www.jneurosci.org/misc/ifa_features.shtml.

Mean–Variance or Prospect Theory? The Nature of Value Representations in the Human Brain

Erie D. Boorman^1,2 and Jerome Sallet^1,2

¹Department of Experimental Psychology, University of Oxford, Oxford OX3 9DU, United Kingdom, and ²Centre for Functional MRI of the Brain, University of Oxford, Oxford OX1 3UD, United Kingdom

Review of Hsu et al. (http://www.jneurosci.org/cgi/content/full/29/7/2231)

Whether investing in financial markets or deciding whom to ask on a date, our decisions are often taken in risky environments. Although traditional economic theories assume that we behave rationally in such environments, extensive evidence indicates that we are susceptible to systematic biases and inconsistencies when making risky decisions. For instance, people and other animals are not only sensitive to expected reward but also risk and can be either risk-averse or risk-prone depending on the context of a gamble. Two current theories have emerged in the field of "neuroeconomics" in an attempt to explain such phenomena. First, the mean-risk approach posits that the expected payoff (or mean reward) associated with a prospect is weighted by the prospect's risk (or known outcome variance) (D'Acremont and Bossaerts, 2008). For example, in one formulation called the mean–variance model, the expected value (V_MR) is equal to the mean payoff (µ) minus the outcome variance (²), which is weighted by the individual's susceptibility to risk (b): V_MR = µ – b², where b > 0 will produce risk aversion. A second theory, termed prospect theory, postulates the existence of separate nonlinear utility (i.e., reward magnitude) and reward probability (i.e., reward likelihood) functions (Rangel et al., 2008). According to prospect theory, the value of a prospect (V_PT) is determined by the product of nonlinear probability w(p_s) and utility u(x_s) functions across states, as follows:

where p_s is the probability of state s occurring, x_s is the return associated with state s, and n is the number of states. Because peoples' risky decisions can be well approximated by either theory, it has been difficult to arbitrate between these two views in the absence of neural evidence.

Recent functional magnetic resonance imaging (fMRI) and electrophysiological studies provide evidence in favor of both theories. In support of the mean-risk theory, it has been shown that the activity of dopaminergic cells in monkeys and the blood oxygen level-dependent (BOLD) response measured by fMRI in the dopaminoceptive ventral striatum, the anterior insula, and orbitofrontal cortex in humans scale in proportion to risk (Schultz et al., 2008; but see Niv et al., 2005). In contrast, the neural correlates of loss aversion and framing effects have been investigated with fMRI in humans (Rangel et al., 2008), pointing to the existence of a prospect theory-like nonlinear utility function. Nevertheless, a central question of prospect theory, whether the brain encodes a nonlinear probability function, and if so, where and when, remained largely unsettled. This was the issue tackled in a recent article by Hsu et al. (2009).

The authors designed an fMRI task in which they first presented human subjects with a single gamble for them to evaluate [Hsu et al. (2009), their Fig. 2]. For instance, participants might see that they have a 30/100 probability of receiving $20. After a fixation period, a second gamble that was close in the weighted expected value was presented alongside the first gamble, and subjects were asked to choose between the two. This experimental design separated in time the valuation and decision making processes, an important manipulation that enabled the authors to isolate the neural response to the valuation process uncontaminated by activity related to the decision itself. Importantly, no feedback was provided to the subjects, thereby preventing learning about reward probabilities during the experiment.

To infer each subject's probability-weighting function from behavior, the authors assumed a standard utility function for gains (there were no gambles involving losses in the experiment), u(x;p) = x^p, and fit various nonlinear probability-weighting functions proposed in the economic literature to subjects' actual choices. Because the one-parameter Prelec function fit nearly as well or better than the other functions tested and only contains one free parameter, it was used for the subsequent fMRI analyses [Hsu et al. (2009), their Fig. 1; Table 1]. This behavioral analysis demonstrated that 14 of 16 subjects exhibited nonlinear inverted S-shaped probability functions, and therefore overweighted small probabilities and underweighted mid-to-high probabilities. The remaining two subjects exhibited the opposite pattern.

The authors then set out to test whether the BOLD response at the time of the first gamble in any brain region varied parametrically with predictions from the model. In particular, they were interested in whether the expected value function derived from the prospect-theoretic model that is nonlinear in probabilities correlated with BOLD activity better than a standard expected utility function that is linear in probabilities (i.e., as follows:)

However, dissociating these two possibilities with fMRI poses a significant challenge because the two value functions are correlated. To achieve this, the authors used a cunning technique that decomposed the weighted form of the value function into two terms: the standard function that is linear in probabilities, and the nonlinear deviation between the prospect-theoretic function that is nonlinear in probabilities and the standard function that is linear in probabilities [Hsu et al. (2009), their Fig. 3A]. Notably, the sum of these two terms, multiplied by the utility function, equals the prospect theoretic nonlinear value function: {(p_s) + [w(p_s) – (p_s)]} u(x) = w(p_s) u(x). This model was then regressed against the BOLD signal in each subject. The authors reasoned that if the standard expected utility hypothesis is accurate, and probability is coded linearly in the brain, then there should only be regions exhibiting a significant effect of the first (linear) term. Alternatively, if the prospect theory hypothesis is accurate, and probability is coded nonlinearly in a manner predicted by the one-parameter Prelec function (i.e., with an inverted or regular S-shape probability function), then there should be regions that respond significantly and equally to both the linear and nonlinear components.

The authors identified such regions using two different analyses: an overlap method and a conjunction analysis. Because the conjunction analysis is statistically more conservative and easier to interpret, we have focused our discussion on the results from this analysis. The authors reported five peaks of activation exhibiting significant BOLD responses to both the linear and nonlinear terms that were not statistically different from each other at p < 0.5. Some aspects of the anatomical localization of the activations provided by the authors might be queried, but we feel that the first activation is likely to have its peak in the ventral anterior thalamus [(6, –3, 0); Hsu et al. (2009), their supplemental Table S5] and may extend into the pallidum [Hsu et al. (2009), their supplemental Fig. S3]. Notably, this region of the thalamus receives direct projections from the substantia nigra, and in turn projects to regions of the prefrontal cortex including the principal sulcus and orbital cortex in the macaque (Ilinsky et al., 1985). This activation may thus reflect afferent dopaminergic input. Two additional peaks near the superior frontal gyrus are difficult to interpret as they are located in the white matter [Hsu et al. (2009), their supplemental Table S5], but one peak (–15, 3, 48) may be associated with activation in the cingulate, a region that has previously been associated with value encoding (Rushworth and Behrens, 2008). Finally, activation peaks were found in the frontal operculum (–42, 0, 15) and the white matter adjacent to the insula and putamen [(–24, –6, 18); Hsu et al. (2009), their supplemental Table S5]. No region's activity reflected regular S-shape probability coding at the group statistical threshold used.

The loci of activations and the nature of nonlinear probability functions can be contrasted with those from two recent experiments (Berns et al., 2008; Tobler et al., 2008). Numerous factors may explain discrepancies between these studies: the way probabilities were presented to subjects (alphanumerically or learned through experience), the experience of feedback, the range of probabilities used, the context (positive or negative outcomes), and the specific analyses conducted. Nevertheless, these studies highlight regions (ventral anterior thalamus, cingulate cortex, dorsolateral prefrontal cortex) that receive some mesencephalic dopaminergic input and whose activity has been shown to be modulated by reward (Ilinsky et al., 1985; Head et al., 2004; Rangel et al., 2008).

Although not designed to test either mean-risk or prospect theories, previous cell recording studies in monkeys have shown coding of reward values and uncertainty by tuning curves (Sallet et al., 2007). Such coding is distinct from the nonlinear modulation of the BOLD signal and could suggest population rather than local processing. In addition, there is some evidence that dopamine neurons encode prediction errors, learning signals computed as the difference between obtained and predicted outcomes, adaptively (Schultz et al., 2008). It would therefore be interesting to test whether the nonlinear coding of probabilities reflects an absolute or adaptive coding.

These data constitute evidence that probability is encoded nonlinearly in the brain in a manner predicted by prospect theory during the valuation of a prospect for an impending decision. Together with fMRI evidence in favor of a nonlinear utility function (Rangel et al., 2008), it appears that neural evidence has now been uncovered to support the existence of two of the central features of prospect theory. Given the equally strong evidence for the existence of risk coding in the mammalian brain and thus, support for the mean-risk approach, these findings present an apparent paradox. Potential reconciliations include the possibility of discrete coding of risk and prospect theoretic value and the possibility that the value function itself is scaled by risk, as proposed by the mean–variance approach. Indeed, there is some evidence that dopamine responses to prediction errors scale according to the level of risk (Schultz et al., 2008), suggesting that reward value and outcome variance signals are combined at the level of individual dopamine neurons. Further experiments that manipulate value and risk independently should hopefully distinguish between these and additional possibilities.

Received April 20, 2009; accepted May 16, 2009.

Footnotes

This work was funded by the Wellcome Trust (E.D.B.). We thank Professor Matthew Rushworth for helpful comments on this manuscript.

Correspondence should be addressed to either Erie D. Boorman or Jerome Sallet, Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK, Email: erie.boorman{at}psy.ox.ac.uk or Email: jerome.sallet{at}psy.ox.ac.uk

Copyright © 2009 Society for Neuroscience 0270-6474/09/297945-03$15.00/0

References

Berns GS, Capra CM, Chappelow J, Moore S, Noussair C (2008) Nonlinear neurobiological probability weighting functions for aversive outcomes. Neuroimage 39:2047–2057.[CrossRef][Web of Science][Medline]

D'Acremont M, Bossaerts P (2008) Neurobiological studies of risk assessment: a comparison of expected utility and mean-variance approaches. Cogn Affect Behav Neurosci 8:363–374.[Abstract/Free Full Text]

Head D, Buckner RL, Shimony JS, Williams LE, Akbudak E, Conturo TE, McAvoy M, Morris JC, Snyder AZ (2004) Differential vulnerability of anterior white matter in nondemented aging with minimal acceleration in dementia of the Alzheimer type: evidence from diffusion tensor imaging. Cereb Cortex 14:410–423.[Abstract/Free Full Text]

Hsu M, Krajbich I, Zhao C, Camerer CF (2009) Neural response to reward anticipation under risk is nonlinear in probabilities. J Neurosci 29:2231–2237.[Abstract/Free Full Text]

Ilinsky IA, Jouandet ML, Goldman-Rakic PS (1985) Organization of the nigrothalamocortical system in the rhesus monkey. J Comp Neurol 236:315–330.[CrossRef][Web of Science][Medline]

Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9:545–556.[CrossRef][Web of Science][Medline]

Rushworth MF, Behrens TE (2008) Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 11:389–397.[CrossRef][Web of Science][Medline]

Sallet J, Quilodran R, Rothé M, Vezoli J, Joseph JP, Procyk E (2007) Expectations, gains, and losses in the anterior cingulate cortex. Cogn Affect Behav Neurosci 7:327–336.[Abstract/Free Full Text]

Schultz W, Preuschoff K, Camerer C, Hsu M, Fiorillo CD, Tobler PN, Bossaerts P (2008) Explicit neural signals reflecting reward uncertainty. Philos Trans R Soc Lond B Biol Sci 363:3801–3811.[Abstract/Free Full Text]

Tobler PN, Christopoulos GI, O'Doherty JP, Dolan RJ, Schultz W (2008) Neuronal distortions of reward probability without choice. J Neurosci 28:11703–11711.[Abstract/Free Full Text]

Related articles in J. Neurosci.:

Neural Response to Reward Anticipation under Risk Is Nonlinear in Probabilities

Ming Hsu, Ian Krajbich, Chen Zhao, and Colin F. Camerer
J. Neurosci. 2009 29: 2231-2237. [Abstract] [Full Text]  

This Article Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Related articles in J. Neurosci. Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Boorman, E. D. Articles by Sallet, J. Search for Related Content PubMed PubMed Citation Articles by Boorman, E. D. Articles by Sallet, J.

Home  |   Search  |   Archive  |   Subscribe  |   Contact  |   Help