Abstract
Value-based choices are influenced both by risk in potential outcomes and by whether outcomes reflect potential gains or losses. These variables are held to be related in a specific fashion, manifest in risk aversion for gains and risk seeking for losses. Instead, we hypothesized that there are independent impacts of risk and loss on choice such that, depending on context, subjects can show either risk aversion for gains and risk seeking for losses or the exact opposite. We demonstrate this independence in a gambling task, by selectively reversing a loss-induced effect (causing more gambling for gains than losses and the reverse) while leaving risk aversion unaffected. Consistent with these dissociable behavioral impacts of risk and loss, fMRI data revealed dissociable neural correlates of these variables, with parietal cortex tracking risk and orbitofrontal cortex and striatum tracking loss. Based on our neural data, we hypothesized that risk and loss influence action selection through approach–avoidance mechanisms, a hypothesis supported in an experiment in which we show valence and risk-dependent reaction time effects in line with this putative mechanism. We suggest that in the choice process risk and loss can independently engage approach–avoidance mechanisms. This can provide a novel explanation for how risk influences action selection and explains both classically described choice behavior as well as behavioral patterns not predicted by existing theory.
Introduction
Risk influences value-based choice in humans (Harrison and Rutström, 2008) and other animals (Kacelnik and Bateson, 1996). Whether outcomes entail gains or losses (i.e., their valence) also influences behavior (Kahneman and Tversky, 1979). In psychology and economics, the prevailing view of the relationship between these two variables is that outcome valence determines the effect of risk on choice, specifically leading to risk aversion for gains but risk seeking for losses. This relationship is a foundation stone in prospect theory (Kahneman and Tversky, 1979) and has been related to the psychophysical idea of diminishing marginal sensitivity (Weber and Johnson, 2008). It is also supported by classic experimental findings (Tversky and Kahneman, 1981; Camerer, 1989) and is used to explain phenomena such as why stock market traders hold losing stocks for too long (risk seeking) and sell winners too early (risk aversion) (Camerer, 1998). Here, we reexamine this dominant view of the relationship between risk and valence (i.e., risk aversion with gains and risk seeking with losses), and ask what are the likely biological mechanisms supporting the influence of these variables on choice.
Recent findings question the standard view of the relationship between risk and valence, for example in reports of risk aversion for gains but risk neutrality (not the predicted risk seeking) with losses (Laury and Holt, 2005). A plausible alternative hypothesis is that the valence and risk of economic stimuli exert independent influences on choice. Such independence not only allows for greater gambling with losses than gains as reported classically, but also similar gambling for each (Laury and Holt, 2005), or even greater gambling for gains than losses. Our first aim was to behaviorally test this hypothesis of independent risk and valence effects, which we sought to dissociate by manipulating task structure (Experiments 1 and 2) and exploiting interindividual differences (Experiment 3). We define risk as outcome variance as is standard in finance (Markowitz, 1952; Bossaerts, 2010).
A hypothesis of independent risk and valence effects was also motivated by evidence that competing neural valuation systems influence choice (Dayan, 2008). Processing of risk and valence by distinct neural systems would be more consistent with independent rather than linked behavioral effects. Candidate neural regions mediating such effects include the orbitofrontal cortex (OFC) and striatum implicated in loss aversion (Tom et al., 2007), as well as insula (Preuschoff et al., 2006) and parietal cortex (Platt and Glimcher, 1999; Huettel et al., 2005) implicated in risk. Thus, our second aim was to use fMRI to determine whether there was a neural dissociation in processing of stimulus risk and valence (Experiment 4).
A neurobiological perspective also enabled us to ask a third important question, namely how risk and valence influence action selection. One possibility is that these variables modulate values ascribed to actions by more goal-directed systems. Alternatively, they might act as appetitive or aversive stimulus features and influence a disposition to approach or avoid stimuli, in line with existing evidence that such approach–avoidance mechanisms underlie a variety of biases in humans and animals (Dayan, 2008; Guitart-Masip et al., 2010). We tested for these hypotheses in our neural data, and behaviorally in Experiments 5 and 6.
Materials and Methods
We report six independent experiments, in which we used a choice task that enables an independent manipulation of risk (measured as variance) and valence (gains or losses) in outcomes. We developed two variants of our task, comprising an “accept/reject” and a “selection” task. Experiment 1 assayed behavior in the accept/reject task and Experiment 2 in the selection task. In Experiment 3, participants undertook the accept/reject task on two separate days (1–3 d apart; mean, 2 d), receiving feedback and payment on the second day. Experiment 4 examined brain activity during the accept/reject task using fMRI. Finally, we tested reaction times (RTs) in the accept/reject (Experiments 5) and selection (Experiment 6) variants of the task. University College London Ethics Committee approved the study.
Participants
A total of 143 healthy participants in all took part after providing informed consent. The participant details per experiment were as follows: Experiment 1 (n = 16; mean age, 26 years; range, 19–70; 6 males), Experiment 2 (n = 24; mean, 23 years; range, 18–34; 3 males), Experiment 3 (n = 28; mean, 27 years; range, 19–62; 13 males), Experiment 4 (n = 22; mean, 22 years; range, 18–32; 6 males; 3 further participants excluded due to fMRI artifacts), Experiment 5 (n = 19; mean, 23 years; range, 19–31; 6 males; 1 further participant excluded as they only rejected), and Experiment 6 (n = 34; mean, 24 years; range, 19–36; 16 males; 1 further participant excluded who confused the buttons). Note that 8 of the 121 subjects in our behavioral experiments were aged over 32 years, and excluding these subjects does not alter the findings in any of our experiments.
Task
Accept/reject task (Experiments 1, 3, and 4).
In the accept/reject task (see Fig. 1), participants performed 200 trials presented in a random order, of which 100 were “gain trials” (all possible outcomes ≥0) and 100 were “loss trials” (all outcomes ≤0). In each trial, participants chose to accept or reject a lottery (four possible outcomes) compared with a sure option (£6 in gain trials; £−6 in loss trials). Each trial began with a fixation cross presented for 1–2 s (mean, 1.5 s), followed by viewing the options for 4020 ms; and finally a black square appeared to indicate participants had 1500 ms to input their choice by button press (the black square turned white when they chose). If participants did not respond, they received £0 on a gain trial and the maximum loss possible on a loss trial (£−12).
Our decision variables of interest were risk and valence. We manipulated risk by using a set of 100 lotteries (four possible outcomes, all ≥0; see Fig. 1b) in which we parametrically and orthogonally manipulated the degree of risk (variance; 10 levels) and expected value (EV) (10 levels). One-half of the set of lotteries had an EV above and one-half below the sure amounts, with careful matching of the lotteries above and below the sure amount. We presented each lottery in this set once to give 100 gain trials. To manipulate valence, we multiplied all outcome amounts by −1 to give 100 loss trials (i.e., all outcomes ≤0, and a sure option of £−6). This created a set of gain trials and a set of loss trials that were perfectly matched in their parametric modulations of risk and EV.
Participants began the day with an endowment of £12. After the experiment, one gain trial and one loss trial were picked at random and their outcomes were added to this endowment to determine a final payment. Participants could receive between £0 and £24. In Experiment 3, in which participants undertook the task on 2 separate days, they received feedback and payment after the second attendance. In Experiment 4, in which we used fMRI, all amounts were doubled.
Selection task (Experiment 2).
This variant aimed to change how individuals were influenced by valence (see Fig. 1). It was identical with the accept/reject task except that, whereas on every trial in the accept/reject task, individuals evaluated a lottery and accepted or rejected; in this variant, individuals evaluated two lotteries and selected between them. To manipulate risk, we again generated a set of 100 gain trials, in which we parametrically and orthogonally manipulated the difference in risk (10 levels of variance) and EV (10 levels) between the two lotteries (each with two possible outcomes, all ≥0). To manipulate valence, we multiplied all amounts by −1 to give 100 loss trials that were perfectly matched with the gain trials in their parametric manipulations of risk and EV.
Free response period tasks (Experiments 5 and 6).
Experiment 5 used the accept/reject task, and Experiment 6 used the selection task. These were exactly as described above, except that individuals could choose at any point during the 5520 ms for which the stimuli were presented (a black square was present throughout stimulus presentation and turned white when they chose).
Stimulus sets
Accept/reject task.
For our accept/reject task, we generated a set of 100 gain trials (ARMainList), in which we manipulated the difference in variance (ΔVar; 10 levels) and EV (ΔEV; 10 levels) of the lottery relative to the sure option of £6 (see Fig. 1b). We created this stimulus set in two stages. First, we generated a list of every possible trial within the following constraints: each lottery had four outcomes (i.e., four pie chart segments); outcomes were between £0 and £12, to militate against possible probability distortion effects at small probabilities (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992); the smallest allowable probability was 0.1; the smallest allowable probability increment was 0.05; and we controlled for lottery skewness. Second, from within this very large number of potential trials, we selected our set of 100 trials that were the closest match to our desired 10 levels of ΔVar and 10 levels of ΔEV.
We used ARMainList in Experiments 1 (behavioral), 4 (fMRI), and 5 (free response time). However, to check that our behavioral findings were not caused by characteristics of this specific lottery set, in Experiment 3 we also compared the ARMainList to two alternative sets. In Experiment 3, 11 of 28 subjects used ARMainList (maximum ΔEV, 1.25; maximum ΔVar, 23.9), and the remainder used one of two stimulus sets generated in the same way but with new lotteries and without skewness controlled [11 participants used ARAlternateList1 (maximum ΔEV, 1.35; maximum ΔVar, 23.8), and 6 participants used ARAlternateList2 (maximum ΔEV, 2.70; maximum ΔVar, 23.8)]. As the lottery set used did not affect the behavioral findings, we collapsed across lottery sets in Experiment 3.
Selection task.
For the selection task, we generated a set of 100 gain trials in the same way, although here manipulating the difference in EV (10 levels) and variance (10 levels) between two lotteries (each with two possible outcomes, ≥0). The difference in EV and variance between the options (maximum ΔEV, 1.9; maximum ΔVar, 18.3) was similar to that used in the accept/reject task.
Calculation of EV, variance, and skewness.
For a given lottery with N potential outcomes (m1, m2, … mN), with probabilities p = p1, p2, … pN, we define the EV, variance (Var), and standardized skewness (Skw) of the outcome distribution as follows:
Statistical analysis
All statistical tests used were two-tailed.
Reaction time analysis
We normalized each individual's RTs by taking the natural logarithm, mean-correcting, and dividing by the SD. However, our findings were the same regardless of using “raw” or normalized RTs. Due to a coding error, RTs are not reported for Experiments 1 and 3. Regression analysis on participants' RT data was conducted using the glmfit function in MATLAB.
Behavioral modeling
We modeled behavior in our accept/reject task, estimating model parameters on a single-subject level using maximum-likelihood analysis implemented in MATLAB. We compared models with different utility functions, using the Bayesian information criterion (BIC). Our model-based analysis of participants' choices sought to determine whether our decision variables of interest, risk and valence, influence choice. It also provided a trial-by-trial metric of risk for use in our fMRI analysis, and allowed us to ask whether our behavioral findings can be explained by probability distortion or choice randomness. In all our models, on each trial the subjective values (or utilities, U) of both options were computed using one of the utility functions below.
Impacts of risk and valence on choice.
We compared three models to ask whether behavior was influenced by risk and valence. First, in a very simple mean-only model (Mn_Only), individuals only cared about the mean of the options as follows:
Second, we asked whether choice was also influenced by risk, using a mean–variance model (Mn_Var). Specifically, risk is measured as variance. Here, ρ is a free parameter reflecting an individual's preference for variance, where a risk-neutral individual has ρ = 0; risk-averse, ρ < 0; and risk-seeking, ρ > 0, as follows:
Third, we asked whether both risk and valence influence choice, using a mean–variance–valence model (Mn_Var_Val). There is a ρgain parameter that reflects risk preference in gain trials and a ρloss parameter reflecting risk preference in loss trials. To preempt our results, this was the winning model in all six of the datasets as follows:
where ρ = ρgain for mean > 0; ρ = ρloss for mean < 0.
Expected utility model.
In addition to these models described above, we asked whether our data could be explained with a standard power utility model commonly used to model expected utility (Camerer, 2003). This model incorporates the impact of risk on choice, using a free parameter, κ, that reflects the concavity of the utility function and therefore the degree of risk aversion as follows:
Prospect theory model.
We further tested a utility function derived from prospect theory (Kahneman and Tversky, 1979), which in addition to the power utility function described above also incorporates the effects of valence and probability weighting. Here, the parameter λ reflects the degree of loss aversion, and the parameter π reflects probability distortion implemented with the Prelec probability weighting function (Prelec, 1998). We implemented two versions of this model. First, we used a reference point of zero, which has the virtue of simplicity with negative numbers coded as losses and positive numbers as gains, and this reference point is also equivalent to their expectation on arriving at the laboratory (it would result in their receiving the endowment of £12). Second, we used the sure amount in each trial as the reference point, such that all the trials become “mixed gambles.” However, we note that using the sure amount as the reference point turns the set of 100 gain trials and the set of 100 loss trials into two essentially identical sets of mixed gambles. This predicts choice should be the same with both sets, and therefore the model cannot explain our new finding of greater gambling in the gain trials compared with the loss trials in the accept/reject task or the reverse in the selection task.
Cumulative prospect theory.
Finally, we tested a cumulative prospect theory model (Tversky and Kahneman, 1992), which in its probability weighting uses rank ordering and cumulative probabilities. For a given lottery with N potential outcomes, we redefine outcomes relative to a reference point R, such that the outcomes are m−T, m−T+1, m−T+2…, R,… mN−2, mN−1, mN, with probabilities p = p−T, p−T+1, p−T+2…, pR, … pN−2, pN−1, pN. Overall utility, U = U− + UR + U+, is given as follows:
For m > R:
For m < R:
For m = R:
Noise in choice.
In all our models, on each trial the subjective value, or utilities (U), of both options (denoted here A and B) was computed using a utility function. These values were compared with generate a trial-by-trial probability of accepting the lottery, using a softmax function with a free parameter β (constrained between 0 and 20) that allows for noise in action selection as follows:
Finally, we asked whether valence acted by changing choice randomness. To the best fitting of the models above, we replaced the single free parameter in our softmax decision rule with separate parameters for gain trials (βgain) and loss trials (βloss).
Model fitting and comparison.
We fit data on an individual participant basis. We estimated best-fitting model parameters using maximum-likelihood analysis. Optimization was implemented with a nonlinear Nelder–Mead simplex search algorithm in MATLAB. We compared models using group Bayes factors, with the BIC penalizing model complexity (Schwarz, 1978).
Experiment 4: fMRI of the accept/reject task
fMRI data acquisition.
In a 3T Allegra scanner (Siemens), each participant underwent one functional run (515 volumes), acquired using a gradient-echo EPI sequence (46 transverse slices; TR, 2.76 s; TE, 30 ms; 3 × 3 mm in-plane resolution; 2 mm slice thickness; 1 mm gap between adjacent slices; z-shim, −0.4 mT/m; positive phase encoding direction; slice tilt, −30°) optimized for OFC and amygdala. We acquired a T1-weighted anatomical scan and local field maps.
fMRI data analysis.
Functional data were analyzed using standard procedures in SPM8 (Statistical Parametric Mapping; www.fil.ion.ucl.ac.uk/spm). fMRI time series were regressed onto a composite general linear model (GLM). The GLM contained boxcars for the length of time the lottery was displayed (5.5 s) to examine the decision-making process. Delta functions were also included for button presses, lottery onset to account for visual stimulus presentation, and for trials in which subjects failed to respond. We modeled our neuroimaging data using a 2 valence (gain, loss) × 2 choice (accept, reject) design. Additional parametric modulators were included, with the height of the boxcar modulated by the EV and variance of the lottery on each trial. The delta functions and boxcars were convolved with the canonical hemodynamic response function.
We report all activations at p < 0.05 that survive whole-brain correction using family-wise error at the cluster level (Friston et al., 2004), unless otherwise stated. Clusters were defined using a threshold of p < 0.005. For presentation, images are displayed at p < 0.001, uncorrected. Unless otherwise stated, small-volume correction (p < 0.05) was for a sphere of 10 mm radius around stated coordinates. For the contrast of loss > gain, we also used small-volume correction (p < 0.05) in anatomical regions of interest (amygdala and anterior insula) specified in the PickAtlas toolbox (Maldjian et al., 2003). Conjunction analyses were performed using the SPM8 conjunction null function (Friston et al., 2005).
In addition to using lottery variance (a key component of our winning behavioral model) as a parametric modulator in our imaging analysis, we analyzed our imaging data with three further GLMs. Instead of the EV parametric regressor, these GLMs used parametric regressors based on the subjective values (i.e., utilities) derived from our winning behavioral model. Specifically, one model used the subjective value of the lottery; a second used the chosen minus unchosen subjective value; and a third model used the chosen subjective value [this third model used a two choice (accept, reject) design for the onset regressors of interest to enable model estimation]. For each model, we first tested for whole-brain-corrected activity as above, and then in a more liberal analysis used small-volume correction (p < 0.05) for a sphere of 10 mm radius centered on any activity identified in regions previously linked to value computations (Kable and Glimcher, 2007)—specifically medial prefrontal, orbitofrontal, and posterior cingulate cortex defined using the PickAtlas toolbox as above, and ventral striatum defined using a 15 mm sphere centered around peak coordinates for positive stimuli from O'Doherty et al. (2004).
Results
Reversing the effect of valence independently of risk aversion
To behaviorally dissociate risk- and valence-induced influences on choice, we developed two variants of our task with different formats: the accept/reject task and the selection task. We first examined behavior in our accept/reject task (Experiment 1, n = 16; Fig. 1) (see Materials and Methods). In each gain trial, participants chose to accept a lottery (all outcomes ≥0) or reject the lottery and so receive £6 for certain. Across the set of 100 gain trials, we parametrically manipulated the degree of risk in the lottery (using 10 levels of variance) and orthogonally manipulated its EV (10 levels) (Fig. 1b). One-half of the lotteries had an EV above the sure amount and one-half below (mean EV across all 100 lotteries was equal to the sure option), which provided a simple metric of risk preference indexed as the proportion of riskier choices made (PropRisk; risk-neutral, 0.5; risk-averse, <0.5; risk-seeking, >0.5). To manipulate valence, we created 100 perfectly matched loss trials by multiplying all amounts in our gain trials by −1 (Fig. 1c).
Dissociating valence and risk related influences using task design. a–d refer to the accept/reject task. a, In each gain trial, individuals chose to accept a lottery (4 possible outcomes, all ≥0) or reject and so receive £6 for certain. b, We created a set of 100 gain trials that parametrically and orthogonally manipulated the degree of risk (defined as outcome variance; 10 levels) and EV (10 levels) of the lotteries. One-half of the lotteries had an EV above the sure amount and one-half below, metricating risk preference as the proportion of riskier choices (PropRisk; risk-averse, <0.5; risk-neutral, 0.5; risk-seeking, >0.5). c, Multiplying all gain trial amounts by −1 gave 100 loss trials with identical parametric manipulations. All 200 trials were presented in random order. d, Behavior in the accept/reject task (Experiment 1, n = 16). Individuals were risk averse overall (i.e., PropRiskall, <0.5). Valence also influenced choice, with more gambling for gains than losses (ImpValence = PropRiskgain − PropRiskloss). e–g refer to the selection task, in which again there were as follows: 100 gain trials with parametric and orthogonal manipulation of difference in risk and EV between the two options (e); and 100 loss trials created by multiplying the gain trial amounts by −1 (f). However, here in each trial individuals were presented with two lotteries to consider and select between. g, Behavior in the selection task (Experiment 2, n = 24): risk aversion overall was unaltered compared with the accept/reject task (i.e., PropRiskall, <0.5), but the direction of the valence effect was completely reversed. Error bars show SEM. *p < 0.05; **p = 0.005.
In our accept/reject task (Experiment 1, n = 16; Fig. 1), risk influenced choice overall, with individuals being averse to risk [PropRiskall, 0.40 ± 0.15 (SD); one-sample t test vs risk-neutral, t(15) = −2.9, p = 0.01; Fig. 1d]. This degree of risk aversion (i.e., on average, accepting gambles with EV of approximately £6.50 vs the sure amount of £6) is in keeping with the magnitude of risk aversion seen previously using tasks that measure risk preferences (Holt and Laury, 2002; Harrison and Rutström, 2008). We also extracted a simple metric for the impact of valence on choice from the difference in riskier choices in each domain (ImpValence = PropRiskgain − PropRiskloss). Individuals were also sensitive to valence (ImpValence, 0.11 ± 0.17; one-sample t test vs no effect of valence, t(15) = 2.6, p = 0.019). Strikingly, and against a prevailing expectation (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992), individuals gambled more for gain [PropRiskgain, 0.46 ± 0.18 (SD)] compared with loss outcomes [PropRiskloss, 0.35 ± 0.14 (SD); t(15) = 2.6, p = 0.019]. We precisely replicate these findings in Experiments 3 and 4 using the accept/reject task (Fig. 2). Previous work has reported a reduced difference in gambling between gains and losses (Laury and Holt, 2005), but such greater gambling for gains than losses has not, to our knowledge, been previously reported.
Replicating choice behavior. We strikingly replicate the behavioral effects of risk and valence in our further experiments using the accept/reject task (Experiment 3 when participants attended on two days, Experiment 4 using fMRI, and Experiment 5 with a free response time) and using the selection task (Experiment 6 with a free response time). The proportion of riskier choices overall (i.e., PropRiskall, shown by dotted lines) was similar in all experiments, with risk aversion shown by one-sample t tests against the null hypothesis of risk neutrality (i.e., PropRiskall, 0.5) in all datasets: Experiment 3, day 1 (t(27) = −2.2; p = 0.039); Experiment 3, day 2 (t(27) = −4.4; p < 0.0005); Experiment 4 (t(21) = −4.2; p = 0.0002), Experiment 5 (t(18) = −2.83; p = 0.01), and Experiment 6 (t(33) = −6.17; p = 5.9 × 10−7). Valence also influenced choice, but we again we reversed the direction of effect between the accept/reject task where individuals gambled more for gains than losses (Experiments 3, 4, and 5) and the selection task showing the opposite (Experiment 6). Error bars show SEM. *p < 0.05; **p = 0.005; ***p < 0.00005.
We next sought to modify our task to replicate classic findings of greater gambling for losses than gains (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992). One possible source of difference is the format in which decisions are presented, as illustrated by comparing our accept/reject task with the problems used in the classic paper establishing prospect theory (Kahneman and Tversky, 1979). In the former, each trial presented a different lottery to accept or reject, while in the latter each problem presented two options for individuals to select between. We therefore modified the task format to create a new selection task (Experiment 2, n = 24; Fig. 1) (see Materials and Methods). Again, there were 100 gain trials with parametric modulation of the difference in risk and EV between the two options in each trial, and 100 loss trials. As before, the proportion of riskier choices indexes risk preference (PropRiskall), and the difference in riskier choices between domains indexes the impact of valence (ImpValence). Crucially, however, in the selection task in each trial individuals were presented with two lotteries to consider and select between. Relative to our accept/reject task, we aimed to selectively reverse the effect from loss aversion, but leave the overall risk-induced effect unchanged.
As predicted, risk aversion overall was the same in the selection (PropRiskall, 0.42 ± 0.11) as in the accept/reject task (p > 0.4 for independent-sample t tests against PropRiskall in Experiments 1, 3, or 4). Furthermore, the magnitude of the valence effect was the same in the selection task (ImpValence, −0.16 ± 0.25) as in the accept/reject task (p ≥ 0.3 for independent-sample t tests against the ImpValence in Experiments 1, 3, or 4). However, the direction of this valence effect was now reversed, such that individuals selected the riskier option more for losses (PropRiskloss, 0.50 ± 0.17) than gains (PropRiskgain, 0.34 ± 0.16; t(23) = 3.1, p = 0.005). Thus, we dissociated the impacts of risk and loss on choice in a simple gambling task manifest in a reversal in direction of a valence-induced effect (not just attenuating it as shown previously), but leaving an overall risk-induced effect unchanged.
Behavioral modeling
Model-based analysis of our data confirmed that choice was best predicted by models incorporating the influences of both risk and valence (Fig. 3). In each of the six experiments in this study (using either the accept/reject or selection tasks; total n = 143), a mean–variance–valence model best predicted the data. This winning model enables us to extend ideas derived from financial economics that individuals respond to risk as measured by the variance in potential outcomes (Markowitz, 1952; Bossaerts, 2010), by incorporating valence. In the mean–variance–valence model, on each trial the subjective values, or utilities (U), of both options (A and B) were computed using the following utility function: U = Mean + ρ*Variance, where ρ = ρgain in gain trials and ρ = ρloss in loss trials. Here, ρgain is a free parameter reflecting risk preference in gain trials and ρloss reflects risk preference in loss trials. In each trial, the utilities were then compared to generate a probability of each action, with a softmax function allowing for noise in action selection (containing a free parameter β). This winning model in all six datasets explicitly incorporated outcome variance as a metric of risk, which we use in our neuroimaging analysis below.
Behavioral model comparison. In all six experiments examined separately, the mean–variance–valence model best predicted choice. Here, for illustration we show the combined results from the four experiments using the accept/reject task (Experiments 1, 4, 5 and day 1 in Experiment 3, giving n = 85; a); and the combined dataset from Experiments 2 and 6 using the selection task (n = 58; b). We plot the summed BIC for each model relative to that for the worst performing model (mean-only). The effects of risk and valence are seen clearly by comparing our three related summary statistic models: the mean-only model in which individuals care only about the mean value of the options (Mn_Only) is improved by adding the influence of risk in the mean–variance model (Mn_Var), which in turn is markedly improved by also accounting for valence in our mean–variance–valence model (Mn_Var_Val) that includes separate risk parameters for each valence. The expected utility (EUT) and prospetic models (here shown with a reference point of zero) also out-perform the mean-only model.
Model comparison also revealed that our findings were not explained by effects of choice randomness or probability weighting. With our winning model, replacing the single free parameter in our softmax decision rule (β) with separate parameters for gain trials (βgain) and loss trials (βloss) resulted in worse performance with all datasets. Risk-related parameters (ρ) from this winning model and the simpler metric (PropRisk) described above were very highly correlated for gain and loss trials in all six datasets (e.g., Experiment 4, gains, r = 0.94, p = 1.3 × 10−10; losses, r = 0.90, p = 7.9 × 10−9). Furthermore, it is notable that, in all six experiments, the mean–variance–valence model, explicitly incorporating variance, better predicted choice than expected utility or prospect theory models (Fig. 3), either when the prospetic models used a reference point of zero or the sure amount. Finally, we note that in absolute terms our winning mean–variance–valence model well predicted individuals' actual choices: it correctly predicted 76% [±8% (SD)] of participants' choices (probability of correct choice, >0.5) in the accept/reject task (n = 85), and correctly predicted 76% [±6% (SD)] of choices in the selection task (n = 58).
Independent and stable interindividual differences for risk and valence
We exploited interindividual differences to seek further evidence of behavioral independence between the influences of risk and valence. If these influences result from stable and independent processes, we can make two predictions: first, within individuals each influence should be consistent over time; and second, if they are independent, then knowing an individual's sensitivity to one influence would not predict sensitivity to the other. We tested these conjectures in Experiment 3, in which 28 participants performed the accept/reject task on two separate days (1–3 d apart). We found behavior on day 1 strongly predicted behavior on day 2 for both risk (PropRiskall, r = 0.77, p = 2.1 × 10−6) and valence (ImpValence, r = 0.84, p = 3.3 × 10−8; Fig. 4). However, crucially, these preferences were independent, with risk and valence effects showing no correlation on either day 1 (r = −0.021; p = 0.92; Fig. 3) or day 2 (r = 0.14; p = 0.47). This independence was also evident in our other five datasets (Experiments 1, 2, 4, 5, and 6), with R2 ≤ 0.07 in all datasets.
Individuals' preferences for risk and valence are consistent and independent. In Experiment 3, 28 participants performed the accept/reject task on 2 separate days. We show a striking consistency over days in individual preferences for both risk (a) (PropRiskall, r = 0.77, p = 2.1 × 10−6) and valence (b) (ImpValence, r = 0.84, p = 3.3 × 10−8). However, crucially, these preferences were independent, with risk and valence effects showing no correlation on either day 1 (c) (r = −0.021; p = 0.92) or day 2 (r = 0.14; p = 0.47), or in our other five datasets.
Dissociable neural activity relating to manipulations of risk and valence
We next used fMRI during our accept/reject task to ask whether activity in dissociable neural regions corresponded to our experimental manipulations of risk and valence, and to the independent preferences shown for each (Experiment 4, n = 22). We implemented a 2 valence (gain, loss) × 2 choice (accept, reject) analysis, with trial-by-trial metrics of risk (measured as the variance of the lottery) and EV. We report activity whole brain corrected (p < 0.05) at the cluster level, unless otherwise stated.
Our data revealed neural activity relating to our experimental manipulations of both risk and of valence, and that the activity relating to each was neuroanatomically dissociable. The degree of risk in the lottery positively correlated with activity in posterior parietal cortex, a region strongly associated with risk (Platt and Glimcher, 1999; Huettel et al., 2005; Mohr et al., 2010), and in middle temporal gyrus (Fig. 5, Table 1). Our manipulation of valence was expressed in greater activity for gains than losses in value-related (O'Doherty, 2004; Rangel et al., 2008) areas of orbitofrontal cortex and bilateral striatum (Fig. 5), as well as left dorsolateral prefrontal cortex and right posterior insula. Striatal activity was greatest in bilateral dorsal striatum (Table 1), a pattern previously seen when rewards are action related (O'Doherty et al., 2004; Tricomi et al., 2004), but also included ventral striatum bilaterally [e.g., Z-score ≥ 3.4 bilaterally at peak ventral striatum coordinates previously reported for positive stimuli in the study by O'Doherty et al. (2004)]. Furthermore, the neural regions corresponding to our manipulations of risk and valence were neuroanatomically dissociable. This was evident by using exclusive masking with a liberal threshold (p < 0.05, uncorrected) to show that the risk-related parietal activity still survived whole-brain correction having removed the valence-related voxels by exclusive masking; as was also the case for valence-related activity in OFC, striatum, and posterior insula having exclusively masked risk-related voxels.
Dissociable neural encoding of stimulus risk and valence. Experiment 4 (n = 22) used fMRI of the accept/reject task, which independently manipulates the degree of risk and the valence in outcomes. a, For valence, greater activity was seen for gains than losses in orbitofrontal cortex and bilateral striatum. b, Risk was measured as lottery variance, and this positively correlated with posterior parietal cortex activity. Exclusive masking shows this activity for valence and risk was anatomically dissociable (see main text).
fMRI results across subjects
We next tested for independent interindividual differences for risk and valence. Greater individual risk aversion (i.e., lower PropRiskall) predicted enhanced activity when accepting than when rejecting a risky option, in areas including the risk-related region of posterior parietal cortex and bilateral anterior insula/inferior frontal gyrus (IFG) (Fig. 6, Table 2). By contrast, the more an individual's choices were influenced by valence (ImpValence, defined above), the greater the enhancement of valence-related activity for gains (relative to losses) in right posterior insula. Again, exclusive masking (liberal threshold of p < 0.05, uncorrected) showed an anatomical dissociation of activity in these risk- and valence-related regions.
Approaching risk and loss: neural data. A possible mechanism by which valence and risk influence choice is by influencing the disposition to approach economic stimuli. Actions can be parsed into approach (accept) or avoidance (reject). a, In relation to valence, anterior insula/IFG demonstrates an interaction of choice (accept, reject) and valence (gain, loss). b shows this interaction was driven by increased activity when approaching (accepting) the lottery with losses (Lossaccept), which was the specific action to which individuals were most averse of the four possible actions in our task (Gainaccept, Gainreject, Lossaccept, Lossreject). Parameter estimates are taken from the peak for this interaction in right anterior insula/IFG. c, For risk, the more averse an individual was to risk (i.e., lower PropRiskall), the greater the activity when approaching (i.e., accepting) the risky option in areas including anterior insula/IFG. d, For illustration, we plot this correlation with risk preference (PropRiskall; risk-neutral, 0.5; risk-averse, <0.5; risk-seeking, >0.5) at the peak for this activity in right anterior insula/IFG. Error bars indicate SEM.
fMRI results between subjects, using second-level covariates related to risk and valence
Interestingly, we did not find any activity for losses (relative to gains) that survived whole-brain correction, and only by taking anterior insula (Mohr et al., 2010) and amygdala (De Martino et al., 2006; Guitart-Masip et al., 2010) as a priori regions of interest did left insula survive small-volume correction. We also note no activity surviving whole-brain correction correlated with our parametric manipulation of EV. Only by taking OFC as an a priori region of interest did we find a positive correlation with EV (21 47 −8; Z = 4.28; 44 voxels at p < 0.005, uncorrected; small volume corrected).
However, although we show above that our manipulations of risk and valence relate to dissociable neural regions, this does not preclude different encoding of risk for gain and loss outcomes. Testing this possibility addresses an important further question, as although our behavioral model captures the concept that valence changes the impact of risk on choice, how this occurs within the process of choice (involving option evaluation and action selection) cannot be determined from this behavioral model alone. In fact, the pattern of choices we observe can be explained by distinct hypotheses: first, in option evaluation valence could alter the encoding of risk (i.e., valence-dependent risk encoding); and second, action selection could be influenced by both risk and valence (e.g., both could be processed separately, involving valence-independent risk encoding) to give the observed pattern of choice.
We next asked whether our neural data support valence-dependent or -independent encoding of risk. By design, our parametric manipulation of risk in the gain trials is identical with that in the loss trials, and this allowed us to directly contrast these parametric regressors to compute their interaction. This directly tests for any valence-dependent difference in the strength of the relationship between risk and neural activity. In this analysis, there was no interaction in risk-sensitive parietal cortex between risk encoding for gain compared with loss trials [only 4 of the 316 voxels in the parietal cluster (Table 1) survive even at a liberal threshold of p < 0.01, uncorrected, for either the interaction of (riskgain > riskloss) or (riskloss > riskgain)]. Furthermore, a conjunction analysis between activity positively correlating with risk in the gain and loss trials demonstrated activity in the risk-sensitive regions identified by our main analysis (centered on coordinates 27 −70 31 in parietal and 57 −58 −2 in temporal cortices, small volume corrected). Thus, our data do not support valence-dependent risk encoding.
Neural data suggesting risk and loss can influence action selection through approach–avoidance mechanisms
Showing that stimulus risk and valence involve dissociable neuroanatomical regions does not explain how they influence action selection to influence choice in our accept/reject task. Recent work suggests choice results from multiple interacting decision systems (Dayan, 2008), and two of these are potentially relevant here. These are not mutually exclusive. One possibility is that risk or valence (or both) influence an individual's disposition to approach or avoid stimuli, by acting as appetitive or aversive stimulus features. Such approach–avoidance mechanisms appear to underlie a variety of biases in humans and animals (Dayan, 2008; Guitart-Masip et al., 2010). The key feature of neural activity related to such systems is its contingency upon specific pairings between stimulus properties and responsive actions (i.e., to approach appetitive and to avoid aversive stimulus properties). Second, risk and valence could modulate the values ascribed to actions by more sophisticated goal-directed systems, which assign values to actions by computing action–outcome associations and then evaluating the rewards associated with the different outcomes to choose between actions (Dayan, 2008). A central prediction here is a unified utility (or subjective value) signal that incorporates the stimulus features, in which activity is not contingent on the particular type of action that brings about a particular outcome.
We observed neural activity consistent with approach–avoidance mechanisms when individuals approached (accepted) the lottery, for both risk and valence (Fig. 6). With respect to risk, the more averse an individual was to risk (i.e., lower PropRiskyall), the greater the activity evoked when approaching (i.e., accepting) the risky option in areas including anterior insula/IFG (Fig. 6), a region known to support aversive representations (Calder et al., 2001; Seymour et al., 2007). In relation to valence, actions can be parsed into approach (accept) or avoidance (reject). Of the four possible actions in our task (Gainaccept, Gainreject, Lossaccept, Lossreject), individuals are least disposed to choose the lottery with losses. This specific action (Lossaccept), to which individuals were most averse, was the sole action associated with increased anterior insula/IFG activity (Fig. 6)—leading to an interaction between choice and valence [(Gainaccept + Lossreject) − (Gainreject + Lossaccept)] in anterior insula/IFG, an interaction also seen in presupplementary motor area (Table 1). Note that this interaction also suggests that the neural correlates of our experimental manipulation of valence are not only determined by the higher overall value of options in the gain than loss trials.
Interestingly, we did not find neural evidence surviving whole-brain correction for a unified value signal, a possibility we investigated using behavioral-model-derived utilities as parametric modulators (using chosen value, or chosen minus unchosen value, or the value of the lottery). However, a more liberal analysis in a priori value-related regions (OFC, medial PFC, posterior cingulate, and ventral striatum) revealed a positive correlation with chosen minus unchosen value in right ventral striatum (9 14 −5; Z = 3.4; 18 voxels; p < 0.03, small volume corrected). Note that the less robust nature of this evidence for unified utility may be explained by a number of factors (see Discussion), and we do not claim a system involving unified utility does not play an important role in this task as one of multiple contributing decision systems.
Testing behavioral predictions of approach–avoidance mechanisms
An approach–avoidance mechanism makes specific new behavioral predictions. It is known that individuals are slower to approach aversive stimuli and are faster to approach appetitive stimuli (Crockett et al., 2009; Guitart-Masip et al., 2011), measured using RTs. If risk and valence are indeed such stimulus features, this makes simple predictions. Regarding valence, individuals will be slower to approach (choose) options containing losses than gains. Regarding risk, this stimulus feature can be aversive, neutral, or appetitive depending on an individual's risk preference: thus, we predict that, when risk-averse, individuals will be slower to approach risk; when risk-neutral, they will show no RT difference; and when risk-seeking, they will be faster to approach risk.
In the experiments above, in each trial subjects had an imposed evaluation period of 4020 ms before acting within 1500 ms (Fig. 1). Here, to test our new predictions, subjects were free to respond at any point within the 5520 ms and undertook either the accept/reject task (Experiments 5, n = 19) or the selection task (Experiment 6, n = 34). In both new experiments, choice behavior precisely replicated that described above (Fig. 2). RTs conformed precisely to our predictions for valence (Fig. 7) and risk (Fig. 8). Valence influenced RTs, with individuals being slower to choose (approach) options with losses than with gains, both in the accept/reject task (gains, mean ± SD, RT, 2975 ± 574 ms; losses, 3189 ± 648; t(18) = 4.62, p = 2.1 × 10−4) and the selection task (gains, 2681 ± 472; losses, 3222 ± 525; t(33) = 13.04, p = 1.4 × 10−14). With risk, as its effect depends on individuals' subjective preference, we tested between subjects. In the accept/reject task, individuals' risk preference with gains (PropRiskgain) strongly predicted the RT effect when approaching (choosing) the riskier relative to the surer option (RTriskier − RTsurer) with gains (r = −0.89; p = 4.2 × 10−7); and risk preference with losses (PropRiskloss) strongly predicted the RT effect with losses (r = −0.75; p = 2.3 × 10−4). In the selection task, we see the same relationship between risk preference and RT effects with both gains (r = −0.82; p = 4.6 × 10−9) and losses (r = −0.70; p = 4.8 × 10−6). Furthermore, the pattern of RT effects was exactly as predicted: risk slowed approach when risk was aversive, induced no RT difference when risk was neutral, and speeded approach when risk was appetitive (Fig. 8).
Approaching loss influences reaction times as predicted by an approach–avoidance mechanism. As predicted by an approach/avoidance mechanism, individuals were slower to choose (approach) options with losses than with gains in both new experiments with a free response period: with the accept/reject task (Experiment 5, n = 19) (a); and with the selection task (Experiment 6, n = 34) (b). Strikingly, we also show the same influences in our previous experiments with an imposed evaluation period, suggesting these influences affect more action selection than option evaluation: with the accept/reject task (Experiment 4, n = 22) (c); and with the selection task (Experiment 2, n = 24) (d). For illustration, RT data is normalized for each subject. In each experiment, we parse trials into the four possible events: a gain trial and choose the surer option; a gain trial and choose riskier; a loss trial and choose surer, a loss trial and they choose riskier. Error bars show SEM. Paired t tests are shown. *p < 0.05; **p = 0.005; ***p < 0.0005.
Risk influences RTs and can be aversive, neutral, or appetitive. Risk influenced RTs as predicted by an approach–avoidance mechanism in both new experiments with a free response period: the accept/reject task (Experiment 5) (a); and the selection task (Experiment 6) (b). As the effect of risk depends on individuals' subjective preference, we looked between subjects. An individual's risk preference with gains strongly predicted their RT effect (RTriskier − RTsurer) with gains; and their risk preference with losses strongly predicted their RT effect with losses. In both tasks, we observe our predicted pattern, in which risk slowed approach when risk was aversive, risk induced no RT difference when risk was neutral, and risk speeded approach when risk was appetitive (see e for a diagram illustrating these predictions). We show the same RT effects in our previous experiments with an imposed evaluation period: with the accept/reject task (Experiment 4) (c); and with the selection task (Experiment 2) (d). Gains are in blue, and losses are in red. Regression lines are shown, which are not constrained in any way. For illustration, gray lines show risk neutrality in choice (i.e., PropRiskall, 0.5) and no RT difference (i.e., RTriskier − RTsurer = 0).
In light of these findings, we examined the reaction times in our previous experiments, in which in each trial there was an imposed evaluation period of 4 s before the 1.5 s choice period (Fig. 1). We conjectured that, if such RT effects were still seen in these experiments with an imposed evaluation period, that within the processes supporting choice this would be more suggestive of these influences affecting action selection rather than option evaluation. Indeed, our data do show the same RT effects when approaching risk and valence despite the imposed evaluation period. With respect to valence, individuals were slower to choose (approach) losses than gains in both the accept/reject task (Experiment 4: losses, mean ± SD, RT, 621 ± 105 ms; gains, 575 ± 87; t(21) = 3.16, p = 0.005) and the selection task (Experiment 2: losses, 611 ± 135; gains, 531 ± 103; t(23) = 6.58, p = 1 × 10−6). With respect to risk, they show the same RT effects consistent with risk being aversive, neutral, or appetitive (Fig. 8).
The proportion of missed trials was low [2 ± 3% (SD)], and there was no difference between tasks [accept/reject task, 2 ± 3% (SD); selection task, 2 ± 3% (SD); independent-samples t test, t(169) = 0.32, p = 0.75]. However, in keeping with longer RTs for loss than gain trials, there was a small increase in missed trials for losses [3 ± 4% (SD)] relative to gains [2 ± 3% (SD)], shown in a mixed ANOVA with missed trials as dependent variable (main effect of valence, F(1,169) = 30.9, p < 0.001; and no interaction between task and valence, F(1,169) = 3.2, p = 0.08). Note that removing the minority of trials involving lotteries containing possible zero outcomes (9% in the selection task; 16% in the accept/reject task in Experiments 1, 4, and 5) does not alter the findings in any of our datasets with respect to choice (more gambling for gains than losses in the accept/reject task and the reverse in the selection task; stable and independent interindividual differences for risk and valence) or reaction times (effects of risk and valence).
Finally, our RT data provided an opportunity to test an hypothesis related to the unified utility that may be associated with more goal-directed mechanisms. Specifically, it might be predicted that where the utilities of two options become closer that difficulty (and thus RT) would increase. We tested this by regressing RT against the absolute difference in utility (using behavioral-model-derived utilities) for each individual subject, and then bringing the regression coefficients from all individuals up to the group level where they are treated as a new response variable (analogous to group analysis in SPM) (Friston, 2004). In both tasks, with either a free response time or an imposed wait, as predicted the group mean regression coefficient was negatively signed (indicating longer RTs with smaller differences in utility), and one-sample t tests showed these regression coefficients were significantly different to zero (p < 0.001 for each of Experiments 2, 4, 5, and 6).
Discussion
Consistent with our hypothesis that risk and valence independently influence choice, we dissociated their effects both by manipulating task design and also by exploiting interindividual differences. These dissociations are not predicted by existing theory (Kahneman and Tversky, 1979; Tversky and Kahneman, 1992). Furthermore, our finding of greater gambling for gains than losses, not previously reported, is inconsistent with existing theory that predicts the opposite (Kahneman and Tversky, 1979). However, we can explain these new findings as well as classic findings (Kahneman and Tversky, 1979) within a biologically grounded, process-based account of choice that evolves from option evaluation to action selection (Corrado et al., 2009).
Regarding option evaluation, we show that risk and valence in economic stimuli engage neuroanatomically dissociable regions. Regarding action selection, our neural data are consistent with risk and valence in part exerting their influence through approach–avoidance mechanisms. For risk in particular, this extends “summary statistic” models of risk evaluation derived from financial economics (Markowitz, 1952; Bossaerts, 2010; Symmonds et al., 2011) and foraging theory (Stephens and Krebs, 1987), both by incorporating valence and linking them to action selection. Crucially given the correlational nature of fMRI data, this neural account makes new behavioral predictions that we then confirm experimentally in valence- and risk-dependent reaction time effects.
From a biological perspective, that risk and valence both influence choice is not unexpected. Risk sensitivity is phylogentically ancient (Real et al., 1982; Kacelnik and Bateson, 1996), and there are good computational reasons why animals should track risk in natural stochastic environments (D'Acremont and Bossaerts, 2008). Considerable biological evidence also exists for reward/punishment asymmetries (Dayan and Seymour, 2008) with loss aversion specifically reported in nonhuman primates (Chen et al., 2006). However, within this biological perspective, there is little that a priori favors the classically described relationship between these variables rather than the independent influences we show.
Our data also speak to how risk and valence may influence action selection. Mounting evidence suggests distinct valuation systems contribute to control of action, including both simpler systems that relate the value of particular states to innate behavioral repertoires like avoidance (Kim and Jung, 2006; Seymour et al., 2007) and more sophisticated goal-directed systems that use explicit models of the environment to select actions (Dayan, 2008). Our neural and RT data provide evidence that both loss and risk may in part influence choice via the former, by triggering avoidance or approach responses. In contrast, we do not find similarly robust neural evidence for the unified utility previously shown with simpler stimuli (Kable and Glimcher, 2007). One potential reason is that utility representations of more complex multiattribute stimuli may be distributed, and detectable only using multivariate but not standard mass univariate analyses (Kahnt et al., 2011). A further important question is how integration occurs between the influences of these different stimulus features and decision systems. Future work might usefully investigate where integration occurs within the process from action selection to motor implementation.
Regarding valence, our observation that stimuli containing loss induce avoidance can explain behavior across a variety of tasks. Previous work showed that framing a sure option as a loss biased individuals to avoid that sure option and choose a gamble instead (De Martino et al., 2006), a bias also elicited by incidentally presenting aversive conditioned stimuli with the sure option (Guitart-Masip et al., 2010). Avoidance of stimuli containing losses also explains a disposition not to choose “loss-gain mixed gambles,” which contain losses along with gains (Tom et al., 2007). With respect to elements of prospect theory (Kahneman and Tversky, 1979), while our data argues against the “reflection effect” (i.e., risk seeking with losses and risk aversion with gains), an approach–avoidance account is in fact consistent with “loss aversion” where losses have greater weight (“loom larger”) than gains.
Our observation that valence influences approach is also consistent with the context dependence we see for losses, where we reverse the direction of the loss-induced effect between our accept/reject and selection tasks. Context powerfully determines how animals react to aversive stimuli, such that rats in different contexts may respond to threat by fleeing, freezing, or even fighting (Blanchard and Blanchard, 1988; Dayan and Seymour, 2008). Although loss induces avoidance in both our tasks, in the selection task individuals had to select between two lotteries and so could not express avoidance by withdrawal, but instead could potentially avoid losses by selecting the riskier (higher variance) option. We note context effects in the same direction have been shown for mixed gambles, which when presented as in our accept/reject task were avoided more often than when presented as in our selection task (Ert and Erev, 2008). In the classic paper establishing prospect theory (Kahneman and Tversky, 1979), each problem presented two options for individuals to select between, leading to the same effect direction as our selection task.
That valence influences individuals from approaching (accepting) the lottery with losses in the accept/reject task is consistent with the enhanced anterior insula/IFG activity seen for this specific action. This is of interest as anterior insula is known to represent aversion (Calder et al., 2001; Seymour et al., 2007). Approach–avoidance mechanisms gave predictions that were confirmed in our RT experiments and explain previously reported longer RTs for losses than gains (Dickhaut et al., 2003). Our data also argue against the possibility that RT findings arise from greater computational demands induced by losses, because we observe the same RT effects even when subjects have an imposed evaluation period (Figs. 7, 8), and because the impact of losses differs between tasks in an action-specific way (Fig. 7). Furthermore, greater computational difficulty with losses would not explain our risk-related RT effects (Fig. 8).
Our valence-related neural data help reconcile previously discrepant neural findings. Greater activity for gains than losses in areas including value-related regions of striatum and orbitofrontal cortex (O'Doherty, 2004; Tom et al., 2007) may relate to the value difference between these trial types. However, for loss-related activity, while some studies report activity in regions associated with aversive processing, such as amygdala (De Martino et al., 2006) and anterior insula (Guitart-Masip et al., 2010), others do not (Tom et al., 2007). Crucially, we show that loss-related activity in anterior insula is driven by having to approach losses, and this can account for why such loss-related activity is reported in studies using contrasts that include choice as a factor (De Martino et al., 2006; Guitart-Masip et al., 2010), as in our valence-by-choice interaction. Our finding of loss-related activity in anterior insula rather than amygdala may reflect the involvement of the former in representing more complex aspects of aversive stimuli (Seymour et al., 2007).
With respect to risk, the overall proportion of riskier choices was similar in the accept/reject and selection tasks (Figs. 1, 2, PropRiskall), in which by design the magnitudes of the differences in risk between the two options in the trials was similar. Thus, regardless of the direction of the valence effect, most individuals were averse to risk overall. Our data suggest risk may be aversive or appetitive, which accords with ideas of an important affective component to this variable (Schonberg et al., 2011).
The degree of stimulus risk was encoded in parietal cortex, a finding that concurs with single-unit and previous fMRI data showing enhanced activity during risky decision making (Platt and Glimcher, 1999; Huettel et al., 2005; Mohr et al., 2010; Symmonds et al., 2011). Parietal cortex is known to express an interaction between number and space (Hubbard et al., 2005), suggesting this parietal risk representation may reflect the spread of an outcome distribution. Interestingly, we did not observe this correlation in insula, previously seen when risk is manipulated by altering win probability (Preuschoff et al., 2006). Insula has been associated with multiple aspects of risky decision making (Mohr et al., 2010), and here our anterior insula data relate to an approach–avoidance mechanism with risk (Fig. 6).
Finally, we demonstrate stable and independent interindividual differences for risk and valence, mirrored by dissociable neural correlates. Stability over time in the impact of loss has not been previously demonstrated and is interesting given possible genetic contributions to framing effects (Roiser et al., 2009). Stable risk preference concurs with work showing stability over months (Andersen et al., 2008). Functional segregation in insula for these preferences also accords well with its putative role in preferences (Singer et al., 2009) and marked functional segregation seen along its length (Caruana et al., 2011).
In conclusion, we dissociate the influences of risk and valence, consistent with a hypothesis that each independently influences choice. We explain these choices, as well as classical findings (Kahneman and Tversky, 1979), within a neurobiological account proceeding from option evaluation (with separable neural systems tracking risk and loss) to action selection (involving contributions from approach–avoidance mechanisms). For risk, we extend summary statistic models of risk evaluation derived from financial economics (Markowitz, 1952; Bossaerts, 2010) and foraging theory (Stephens and Krebs, 1987), by linking them to action selection through approach–avoidance mechanisms and by incorporating valence. More broadly, our account carries implications across the diverse disciplines to which existing theories have been influentially applied, including the economic (Camerer, 1998), cognitive (De Martino et al., 2006), and political sciences (Levy, 2003).
Footnotes
This work was supported by Wellcome Trust Programme Grant 078865/Z/05/Z (R.J.D.). The Wellcome Trust Centre for Neuroimaging is supported by core funding from Wellcome Trust Grant 091593/Z/10/Z. We thank Peter Dayan, Steve Fleming, and Benedetto DeMartino for helpful comments.
- Correspondence should be addressed to Dr. Nicholas D. Wright, Wellcome Trust Centre for Neuroimaging, 12 Queen Square, London WC1N 3BG, UK. n.wright{at}fil.ion.ucl.ac.uk
