Abstract
Some decisions are made after obtaining several pieces of information, whereas others are reached quickly. Such differences may depend on the quality of information acquired, as well as individual variability in how cautiously evidence is evaluated. The current study examined neural activity while subjects accumulated sequential pieces of evidence and then made a decision. Uncertainty was updated with each piece of evidence, with individual ratings of subjective uncertainty characterizing underconfidence when observing evidence. Increased uncertainty during evidence accumulation was associated with activity in dorsal anterior cingulate cortex, whereas greater uncertainty when executing a decision uniquely elicited lateral frontal and parietal activity. Greater underconfidence when observing evidence correlated with activity in ventromedial prefrontal cortex. These results suggest that neural mechanisms of uncertainty depend on the stage of decision-making (belief updating vs decision) and that greater subjective uncertainty when evaluating evidence is associated with activity in ventromedial brain regions, even in the absence of overt risk.
Introduction
People often make decisions after evaluating several pieces of information, such as when an investor tracks performance measures before purchasing a stock. The normative process of belief updating through evidence accumulation can be described using Bayesian calculations, in which the posterior likelihood of an event is computed by aggregating current evidence with previous information. Although Bayesian formalism describes the uncertainty that should be experienced by a rational decision-maker when choosing between options, it does not always reflect the subjective experience of uncertainty, which varies between individuals with access to the same information (Weber and Milliman, 1997). For example, some investors may be able to choose a stock after observing a few weeks of performance, whereas others may be uncertain even after following the stock for several months or years. This latter type of cautious decision-making has been observed in the laboratory, in which healthy subjects acquire more information than is needed to make a decision (Edwards and Slovic, 1965) and are underconfident when estimating event likelihoods (Edwards, 1968). Little is known about the neural mechanisms underlying individual differences in underconfidence, despite clear relevance for understanding psychiatric disorders characterized by pathological uncertainty, such as obsessive-compulsive disorder (Fear and Healy, 1997).
Previous neuroimaging studies of uncertainty have mostly focused on activity when making a decision or anticipating an outcome, with parametric increases in uncertainty correlated with greater activity in dorsal medial frontal/anterior cingulate cortex (MFC/ACC) (Volz et al., 2003; Grinband et al., 2006; Krain et al., 2006) and paralimbic regions, including lateral orbitofrontal cortex (OFC) and anterior insula (Critchley et al., 2001; Tobler et al., 2007; Preuschoff et al., 2008). In one study examining uncertainty during evidence accumulation, Huettel et al. (2005) found lateral frontoparietal and insular activity when subjects observed stimuli associated with greater uncertainty for an upcoming decision. However, it remains unclear how the brain updates uncertainty in response to each sequential piece of evidence and how such updating may be different from uncertainty experienced when actually executing a decision, when consequences for the outcome could be more salient.
In the current event-related functional magnetic resonance imaging (fMRI) study, participants rated subjective uncertainty while brain activity was measured separately on each of four sequential pieces of evidence, followed by a binary decision. The posterior probability computed from Bayes' rule using the binomial likelihood and known 50:50 prior (“objective uncertainty,” in contrast to ratings of subjective uncertainty) was dynamically updated as information was acquired. To the extent that individual differences in subjective uncertainty (i.e., the self-reported posterior) are attributable to variability in risk appraisal and negative affect/arousal when observing evidence, underconfidence should be associated with ventral prefrontal and insular activity (Bechara et al., 2000; Paulus and Stein, 2006). We further hypothesized that objective uncertainty during evidence accumulation would be tracked by regions involved in monitoring behaviorally relevant information, such as dorsal MFC/ACC (Carter et al., 1998), whereas executing a decision under uncertainty would additionally engage decision or outcome risk areas, such as OFC and anterior insula (Rolls, 2004; Preuschoff et al., 2008).
Materials and Methods
Subjects.
Twenty subjects free of psychiatric or neurological disorders were recruited to participate in the experiment. All subjects were right-handed, had normal or corrected-to-normal vision, and were not colorblind. Three subjects were excluded because they failed to understand the task adequately (e.g., they did not learn that all draws were from the same deck), leaving 17 subjects for analysis (mean age, 22.4 years; range, 19–32 years; 12 females). All subjects provided written informed consent as approved by the institutional review board of the University of Michigan Medical School.
Task.
Instructions and practice trials for the task were presented before the fMRI session. While in the fMRI scanner, subjects received 72 sequences in which subjective uncertainty was rated while observing four pieces of evidence used to make a decision. On 54 of these sequences, two decks each containing red and blue cards were displayed at the top of the screen. Deck A (left side of screen) contained 80% red cards and 20% blue cards, whereas deck B (right side of screen) contained 20% red cards and 80% blue cards. Subjects were informed of the percentages of red and blue cards contained in each deck, which were graphically represented by two rectangles containing a representative proportion of space colored in red and blue (Fig. 1). At the start of each sequence, one of the decks was selected by the experimenter, and subjects needed to identify which deck had been selected by viewing four sequential “draws” of colored cards from the chosen deck (“evidence accumulation”). On each draw, subjects rated how certain they were that the deck providing the draws was located on the left (deck A) or right (deck B) side of the screen using a 9-point scale with “uncertain” at the center and “certain” at either end. Each subject was informed that the choices located between uncertain and certain on the scale should be interpreted as evenly spaced intermediate categories between uncertainty and certainty. After viewing the four draws, subjects were given the option to choose which deck was supplying the cards or to decline to make a choice (“execution of decision”). After this, feedback was presented and a new sequence was started. Unlike during evidence accumulation, decision execution was associated with incentive consequences so that a correct choice resulted in a gain of 50 points, an incorrect choice resulted in a loss of 50 points, and declining to choose resulted in no change in points. Importantly, the amount of points obtained (based on accuracy of decision) determined a monetary bonus received at the end of the experiment. This aspect of the paradigm was aimed at increasing the motivational salience of uncertainty when executing the decision, where subjects could potentially win or lose money, relative to uncertainty when observing draws during evidence accumulation, where ratings would have no monetary consequences. This feature was designed to mimic real-world rewards and punishments, which normally are based on decisions stemming from the evaluation of an entire set of information that has been accumulated rather than on the appraisal of individual pieces of evidence. All responses were made using MRI-compatible button gloves attached to both hands.
Sequences of trials in the evidence accumulation task. a, Subjects viewed two decks containing equal but opposite ratios of red and blue cards. One of the decks was selected at the beginning of each sequence, and subjects determined which deck was selected by observing four draws of cards from the chosen deck. Subjective ratings of uncertainty regarding the identity of the deck supplying the cards were made on each draw. After observing all draws, a decision was made that would yield a win of 50 points if correct, a loss of 50 points if incorrect, or no change in points if declined. The amount of points won determined a monetary bonus at the end of the experiment. After this, feedback informed subjects which deck had been selected (data not shown). Evidence was varied to yield high uncertainty (type 2:2 sequences presenting 2 cards of each color), moderate to low uncertainty (type 3:1 sequences presenting 3 cards of one color and 1 card of the other color), or very low uncertainty (type 4:0 sequences presenting 4 cards all of the same color) regarding the identity of the deck providing the draws. To account for effects unrelated to uncertainty, control sequences in which one deck contained only blue cards and the other contained only red cards were also included (data not shown). b, Throughout each sequence, the Bayesian likelihoods changed dynamically as evidence accumulated, with each draw updating posterior probabilities associated with both decks. Solid black lines represent the posterior probabilities associated with the most likely deck (in this example, deck A) on each draw for the sequences depicted in a. Gray dashed lines illustrate probabilities associated with other possible sequences for type 2:2 and 3:1 sequences.
On each of these sequences, the likelihood of deck A or deck B being the deck supplying the observed draws changed dynamically as evidence accumulated, with each draw updating probabilities throughout a sequence. Subjects were informed that both decks had equal likelihood of being selected to supply the draws (prior of 50:50), with posteriors after each draw computed for both decks using Bayes' theorem:
where P(X) is the probability of one of the two decks supplying the cards, P(X′) is the probability that the other deck is supplying the cards, and P(Y) is the probability of observing a particular sequence of colored cards.
“Objective uncertainty” was defined as the calculated posterior probability for the most likely deck (based on the observed evidence) after each card was drawn (throughout this paper, we report probabilities as percentages). The closer the calculated posterior probability was to 100%, the less the objective uncertainty. This objective uncertainty measure can be conceptualized as the absolute difference between the calculated posterior and 50% (a smaller absolute difference represents larger objective uncertainty) and thus is related to other measures of uncertainty that calculate deviation from 50%, such as entropy. Importantly, the measure of objective uncertainty contrasts with a different measure we used, subjective uncertainty, which involved subjects' actual ratings (described below) rather than calculated posteriors. We have taken an “experience sampling” approach (Hertwig et al., 2004) by computing the posterior probability based on information directly available to the participant (the sequence of draws they actually observed) rather than computing the posterior probability of the true deck (i.e., the one actually supplying the cards). For instance, even in cases in which the true deck was deck A (containing 80% blue cards and 20% red cards), a red card on the first draw would be calculated with a posterior probability of 80% (for deck B) based on the evidence available to the subject at the time. This way, the uncertainty measure did not require that subjects have knowledge of the chosen deck and the calculated posterior could never fall below 50% for the more likely deck. By manipulating the order of presentation of red and blue cards in a sequence, the posterior probabilities that either deck was supplying the cards was varied on a draw-by-draw basis, reflecting the accumulation of evidence favoring one deck over the other, or not, depending on the sequence of draws.
The relative number of red versus blue cards presented in a sequence dictated whether the calculated posteriors (objective uncertainty) increased or decreased between the first and last draw of the sequence, with the posteriors on the last draw also determining the uncertainty associated with the decision. There were three types of sequences (18 presentations of each type): (1) “type 2:2” sequences presented two cards of each color (final posterior of 50% for each deck; associated with high objective uncertainty), (2) “type 3:1” sequences presented three cards of one color and one card of the other color (final posterior of 94.12% for the deck containing a majority of the dominant color observed; associated with moderate to low objective uncertainty), and (3) “type 4:0” sequences presented four cards all of the same color (final posterior of 99.61% for the deck containing a majority of the dominant color observed; associated with very low objective uncertainty). In addition to sequences in which each deck contained both red and blue cards, 18 control sequences in which one deck contained only blue cards and the other contained only red cards were included to account for nonspecific effects occurring over the sequence (such as fatigue or habituation) that were unrelated to objective uncertainty. Subjects were informed that the observed evidence would fairly represent what would occur by chance based on the probability distributions of the decks. Furthermore, subjects were told that the number of cards contained in each deck was “infinite” and, hence, that counting of cards would not aid performance.
Deck A supplied the cards on half of sequences across the entire experiment and within sequence types, whereas deck B supplied the cards on the other half. For sequences in which both red and blue cards were drawn (types 2:2 and 3:1), the order of cards was varied, with each order occurring in equal numbers when possible. Furthermore, the majority color among the four draws in a sequence corresponded to the majority color of the deck selected to supply the draws (i.e., three red cards and one blue card were drawn from a deck that had 80% red and 20% blue cards). The only instance in which this did not occur was on one type 3:1 sequence, in which three red cards and one blue card were drawn from a deck that contained 80% blue cards and 20% red cards. Consistent with our aim of having events in the task correspond to what would actually occur by chance, this sequence was used because the Bayesian probability of such an event is 5.88%, which is approximately equivalent to 1 of our total of 18 (5.56%) type 3:1 sequences.
Each sequence began by showing the two decks for 500 ms, followed by the four draws (2000 ms each) and then the decision (2000 ms). All stimuli remained on-screen until the 2000 ms had elapsed regardless of when individual responses (subjective uncertainty ratings and decision) were made. If responses were not made within this time frame, a message appeared on-screen asking subjects to speed up on future trials. After each draw and decision, an interstimulus interval blank screen was presented for 500–4500 ms to isolate the blood oxygenation level-dependent (BOLD) signal for each event. Feedback was jittered between 2000 and 4000 ms after the decision, followed immediately by the start of the next sequence. These timings were determined using a design optimization program written in Matlab (MathWorks) by R.C.W., which minimized mulitcolinearity between events for contrasts of interest.
fMRI acquisition and preprocessing.
MRI scanning occurred on a GE 3 T Signa scanner (LX [8.3] release). A T1-weighted image was acquired in the same prescription as the functional images to facilitate coregistration. Functional images were acquired with a T2*-weighted, reverse spiral acquisition sequence (gradient echo; repetition time, 2000 ms; echo time, 30 ms; flip angle, 90°; field of view, 20 cm; 40 slices; slice thickness/skip = 3.0/0 mm; matrix diameter of 71, equivalent to 64 × 64 voxels) sensitive to signal in ventral medial frontal regions (Yang et al., 2002). Subjects underwent six runs, each consisting of 171 volumes plus four initial, discarded volumes to allow for thermal equilibration of scanner signal. After acquisition of functional volumes, a high-resolution T1 spoiled gradient recalled echo (SPGR) scan was obtained for anatomic normalization.
Images were presented by a BrainLogics (Production Systems Technologies) digital MR projector, which provides high-resolution video (1024 × 768) by back projection.
Preprocessing was performed using the Statistical Parametric Mapping (SPM) 2 package (Wellcome Institute of Cognitive Neurology, London, UK), with the exception of realignment, which used MCFLIRT (Jenkinson et al., 2002), and slice-time correction, which was interpolated with an eight-point sinc kernel multiplied by a Hanning window. Realignment parameters were inspected as a proxy for subject movement to ensure that movement did not exceed either 3 mm translation or 1° rotation within each run. Parameters for anatomic normalization to the MNI152 brain, an average of 152 T1 images from the Montreal Neurological Institute (MNI), were derived from the high-resolution SPGR T1 image and applied to the time series of coregistered, functional volumes, which were resliced and smoothed with a 5 mm isotropic Gaussian smoothing kernel.
Data analysis.
Behavioral analysis examined mean subjective uncertainty ratings in a 2 (draw 1 and draw 4) × 3 (sequence types 2:2, 3:1, and 4:0) repeated-measures ANOVA and mean reaction times (RTs) in a 3 (draw 1, draw 4, and decision) × 3 (sequence types 2:2, 3:1, and 4:0) ANOVA. To examine whether RT to make the subjective uncertainty rating depended on the level of objective or subjective uncertainty experienced by subjects, separate univariate ANOVAs examined mean RT for calculated posteriors (50, 80, 94.12, 98.46, and 99.61%) and self-reported posteriors (50, 67.5, 75, 87.5, and 100%; see below for description of the transformation used to convert subjective ratings to probabilities). Finally, frequency of choosing the wrong deck (i.e., errors) and declining to choose a deck were also examined.
All analyses of fMRI BOLD signal were event related. Regressors specifying the onset of each draw separately, the decision, and feedback were convolved with the canonical hemodynamic response function using the general linear model as implemented in SPM2. Omission trials in which subjects did not make a response within the deadline were infrequent (8.29 ± 11.6 trials) and were included in the model to reduce error variance but were not analyzed further.
Four different models were used to analyze the data. To examine regions tracking objective uncertainty during evidence accumulation, one regressor specified for all draws, regardless of sequence type and time (i.e., draw number 1, 2, 3, or 4), was parametrically modulated by the posterior probability for the most likely deck (50, 80, 94.12, 98.46, and 99.61%) (model 1). First-level contrasts of this modulation identified regions in which brain activity became greater as the posterior for the most likely deck decreased (i.e., objective uncertainty increased); group-level analysis used one-sample t tests. Regions activated in group-level analyses were further examined to determine the pattern of activity over time (draw 1 vs draw 4) based on sequence type. To do this, a second model (model 2) specified separate regressors for each draw (1, 2, 3, and 4) for each sequence type, with no modulation. First-level contrasts were used to obtain parameter estimates for each model 2 regressor against baseline in those areas activated at the group level in model 1. These parameter estimates were averaged across all voxels in the activated region and separately submitted to 2 (draw 1 and draw 4) × 3 (sequence types 2:2, 3:1, and 4:0) repeated-measures ANOVAs. Values were extracted only from draws 1 and 4 because probabilities for these draws did not differ based on the ordering of the sequence, and there were not enough trials contained in the different combinations of orderings to reliably examine the pattern of activity for draws 2 and 3 based on sequence type.
To directly compare uncertainty-related activations when accumulating evidence versus executing a decision that contained incentive consequences (reward or punishment), a third model (model 3) specified separate regressors for the fourth (last) draw and when executing the decision, regardless of sequence type, each parametrically modulated by posterior probability for the most likely deck (similar to model 1, with the exception that only the fourth draw and the decision were parametrically modulated in model 3). Separate first-level contrasts of the modulations identified areas showing greater activation as the posterior for the most likely deck decreased on draw 4 alone and when executing the decision, as well as a direct comparison identifying regions correlated with posteriors when executing the decision but not on the fourth draw. Because no additional evidence was provided after draw 4—and hence, Bayesian probabilities were the same for draw 4 and the decision—differences identified in this comparison can be interpreted as resulting from an interaction between objective uncertainty and the psychological process involved (i.e., observing evidence on draw 4 vs making a decision).
To facilitate comparisons between regions tracking calculated objective uncertainty versus subjective uncertainty, subjective uncertainty ratings for each draw were converted into self-reported posteriors through a linear transformation. A rating of uncertain was converted to a 50% probability that either deck was supplying the draws, whereas a rating of certain for either deck was converted to a 100% probability, with intervals of 12.5% spanning intermediate ratings (62.5, 75, and 87.5%). Self-reported posteriors were then used in a fourth model (model 4) to parametrically modulate a regressor for all draws (just as in model 1, regardless of sequence type), with first-level contrasts of the modulation identifying regions in which brain activity became greater as self-reported posteriors decreased (i.e., subjective uncertainty increased). Models 1 and 4 were run separately to avoid the regressor multicolinearity that would be present in a unified model resulting from the high correlation between calculated and self-reported posteriors (r = 0.68).
The design also permitted the analysis of underconfidence during evidence accumulation by comparing calculated posteriors with self-reported posteriors and correlating these differences with brain activation while observing evidence. A difference score was obtained for each subject by subtracting self-reported posteriors from calculated posteriors for the most likely deck for each of the four draws, with larger scores representing greater underconfidence/subjective uncertainty. This type of underconfidence occurs when the point estimate of one's prediction is lower than the actual value and differs from a related bias of overly narrow confidence intervals around a prediction, i.e., second-order overconfidence (Lichtenstein et al., 1982). First-level contrasts obtained from model 2 (separate regressors for each draw, without parametric modulations) compared regions of activation for each draw regressor, averaged across sequence type, against baseline. Simple regression at the group level identified between-subjects correlations of neural activation with mean deviation score for each draw (regardless of sequence type) to identify brain regions showing increased activity for subjects exhibiting greater underconfidence during evidence accumulation.
For the models described above, all events in the task were specified as regressors (draws, decisions, and feedback) even if not the focus of the analysis. Furthermore, events in control sequences were also included in all models, but these events were not tested in the parametric analysis, i.e., they were not given Bayesian probabilities, and were not included in the contrasts. However, for comparison purposes, activity in those regions identified during noncontrol sequences is also presented for control sequences in the results and figures. Except when noted, all results are significant at p < 0.05, corrected for multiple comparisons across the whole brain at the cluster level (voxelwise threshold of z > 2.58).
Results
Behavioral
Mean subjective uncertainty ratings and RTs for these ratings are shown in Figure 2. Analysis of subjective uncertainty ratings based on draw number and sequence type revealed main effects and an interaction between factors (interaction, F(2,32) = 200.8, p < 0.001). Certainty increased between draw 1 and draw 4 on sequences in which all four cards were the same color (i.e., type 4:0) or in which three of the cards were the same color and one card was a different color (i.e., type 3:1) but decreased when two cards of each color were presented (i.e., type 2:2). Although updating of subjective uncertainty paralleled Bayesian probabilities, some subjects were consistently more underconfident than others (i.e., they rated subjective uncertainty higher), especially on the first draw (see scatter plots in Fig. 5).
Subjective uncertainty ratings and reaction times. Certainty decreased between the first and last draws on type 2:2 sequences but increased for type 3:1 and type 4:0 sequences. RTs decreased between the first and last draws for type 4:0 sequences but not for type 2:2 or type 3:1 sequences. In contrast, RTs decreased between the last draw, and the decision on type 2:2 and type 3:1 sequences and remained stable on type 4:0 sequences.
Analyses of RTs also identified main effects and an interaction between factors (interaction, F(4,64) = 18.5, p < 0.001) (Fig. 2). On type 4:0 sequences, RTs were faster when rating uncertainty on draw 4 compared with draw 1 but were similar for draw 4 and when making a decision. However, on both type 3:1 and type 2:2 sequences, RTs were similar when rating uncertainty for draws 1 and 4 and then decreased when making a decision.
Analysis of RT based on calculated posteriors indicated that subjects were significantly slower to make subjective uncertainty ratings when the Bayesian probability for the most likely deck was lower (i.e., objective uncertainty was higher) (F(4,64) = 21.3, p < 0.001; 50%, 941.7 ms; 80%, 975.8 ms; 94.12%, 857.2 ms; 98.46%, 769.6 ms; 99.61%, 665.1 ms). Similarly, there was a main effect of self-reported posterior probability on RT (F(4,106) = 9.1, p < 0.001; 50%, 977.9 ms; 62.5%, 1016.6 ms; 75%, 918.0 ms; 87.5%, 826.5 ms; 100%, 752.7 ms), indicating that ratings were slower when subjects reported more subjective uncertainty.
Subjects declined to choose a deck on 87.4% of type 2:2 sequences compared with 1 and 0% decline choices on type 3:1 and type 4:0 sequences, respectively. When subjects did choose a deck on type 4:0 and 3:1 sequences, accuracy was very high, with only one subject making one error on a type 3:1 sequence and no errors for any subjects on type 4:0 sequences.
Functional neuroimaging
Objective uncertainty when accumulating evidence
Results from model 1, in which all draws were modulated by their Bayesian probability regardless of draw number and sequence type, identified regions in dorsal anterior cingulate cortex (dACC), precuneus, and left postcentral gyrus that showed increasing activity as posterior probability decreased (i.e., objective uncertainty increased) (Table 1, Fig. 3). A post hoc repeated-measures ANOVA on dACC activity as evidence accumulated between the first and last draws (model 2) revealed an interaction between draw number and sequence type (F(2,32) = 7.8, p = 0.002) such that activity increased between the first and last draw for type 2:2 and 3:1 sequences but decreased for type 4:0 sequences (Fig. 3). Post hoc t tests (two-tailed) indicated that dACC activity did not differ between any of the three sequence types on draw 1 (p > 0.3 for all comparisons) but that type 2:2 activity was greater than type 3:1 (t(16) = −2.7, p = 0.017), and type 3:1 activity was greater than type 4:0 (t(16) = −2.8, p = 0.012) on draw 4. Because of the differential temporal pattern of activity in dACC based on sequence type, we hypothesized that this region was involved in detecting when information was inconsistent with expectations that had been established through previous observation of draws in the sequence. To directly test this hypothesis, we performed a secondary analysis in which draws 2, 3, and 4 were sorted based on whether they were consistent or inconsistent with previous draws in the sequence (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). Consistency or inconsistency was defined as a draw that, respectively, increased or decreased posterior probabilities. Results from this analysis supported our hypothesis, revealing robust activation in the same region of dACC for the comparison of inconsistent > consistent draws (supplemental Fig. 2, available at www.jneurosci.org as supplemental material).
Activity associated with increasing objective uncertainty when observing evidence and when executing a decision at p < 0.05 corrected for multiple comparisons across the whole brain
Neural activity related to increasing objective uncertainty during evidence accumulation. Increasing dACC and precuneus activity was associated with decreasing posterior probabilities (i.e., increasing objective uncertainty) when observing all draws, regardless of sequence type. In an analysis of the pattern of activity in these regions across draws for a given sequence type, dACC activity increased between the first and last draw on both type 2:2 and type 3:1 sequences, increasing the most for type 2:2 sequences, but decreased on type 4:0 and control sequences. Precuneus activity also increased over the course of evidence accumulation for type 2:2 and type 3:1 sequences, yet the amount of this increase did not differ based on sequence type. Values in parentheses represent posterior probabilities for the most likely deck. Color bar represents t scores.
Similar to dACC, an ANOVA on precuneus activation indicated no difference on draw 1 for any of the sequence types (p > 0.2 for all comparisons) and an increase in precuneus activity between draws 1 and 4 for types 2:2 and 3:1, but not type 4:0, sequences (interaction, F(2,32) = 9.0, p = 0.001). However, unlike dACC activity, the amount of increase between draws 1 and 4 was the same for type 2:2 and 3:1 sequences, as indicated by a lack of difference between these sequence types on draw 4 (p = 0.47). As can be seen from Figure 3, precuneus activity on draw 4 was greater for type 2:2 and 3:1 sequences compared with type 4:0 sequences (t(16) = −3.3, p = 0.004; t(16) = −3.5, p = 0.003, respectively).
Finally, given that reaction times were slower when calculated and self-reported posteriors were smaller (i.e., higher objective and subjective uncertainty), we sought to determine whether these relationships between neural activity and uncertainty were attributable to differences in RT. A supplemental model specifying the same regressors as model 1 and including RT as a covariate on each draw identified relationships between posterior probability and BOLD signal in the same regions as in model 1 (dACC: x = 6, y = 27, z = 30, z score = 3.43, k = 43; precuneus: x = −33, y = −87, z = 30, z score = 3.97, k = 315; postcentral gyrus: x = −57, y = −18, z = 51, z score = 5.0, k = 79) (supplemental Fig. 3, available at www.jneurosci.org as supplemental material). Thus, although response times for uncertainty ratings were different for varying levels of calculated and self-reported posterior probabilities, the brain activations correlated with uncertainty during evidence accumulation cannot be explained by this difference in RT.
Objective uncertainty when executing a decision
To compare uncertainty-related activations when observing evidence with those at the time of the decision, the fourth draw and decision were separately modulated by Bayesian probability (model 3). For the fourth draw only, uncertainty-related increases in activity were found in the same regions as for the entire sequence of draws (Table 1), although medial frontal activations spanned a larger region of cortex, encompassing anterior regions of MFC in addition to dACC. Increased activity in dorsal MFC/ACC was also related to greater uncertainty when executing the decision; in addition, decision-related uncertainty elicited activity in a distributed network of lateral frontal, parietal, and temporal cortex (Table 1, Fig. 4).
Neural activity related to increasing objective uncertainty on the last draw of evidence accumulation and during execution of the decision. Brain regions associated with increased neural activity as posterior probabilities decreased on draw 4 only (top) and during execution of the decision (middle). In the direct contrast (bottom), left lateral frontal, bilateral parietal, and medial frontal cortices showed increased activity with greater objective uncertainty during the decision but not on draw 4. Color bars represent t scores.
Although subjects selected the “decline to choose” option on the majority of type 2:2 sequences (87% of trials on average), it is possible that anticipation of receiving error feedback on trials in which subjects did not choose this option (i.e., when they guessed which deck was providing the draws), rather than greater uncertainty, could be influencing these findings. Thus, we conducted a secondary analysis that excluded all trials in which subjects guessed which deck was providing the draws on type 2:2 sequences. Results were essentially unchanged, identifying similar areas of activation, except for the right middle temporal region (supplemental Table 1, available at www.jneurosci.org as supplemental material), confirming that increasing activation in this widespread frontoparietal network when executing a decision under uncertainty was not attributable to greater expectation of receiving error feedback in response to guesses on type 2:2 sequences.
A direct comparison between regions modulated by posterior probability during evidence accumulation and execution of the decision revealed several areas that increased in activity as objective uncertainty became greater during the decision but not when observing the last piece of evidence (although probabilities were identical), including left hemisphere regions of OFC/inferior frontal cortex (IFC), superior and middle frontal gyri, and putamen, as well as bilateral dorsomedial frontal and inferior parietal cortices (Table 1, Fig. 4). Two regions in right precentral/postcentral gyrus and posterior MFC also emerged from the contrast; however, these activations were attributable to uncertainty-related decreases on draw 4 rather than uncertainty-related increases when executing a decision.
Subjective uncertainty and underconfidence when accumulating evidence
Neural regions that correlated with self-reported posterior probability for all draws during evidence accumulation (model 4) were similar to those correlated with calculated posteriors for all draws in model 1 (precuneus, left postcentral gyrus), albeit with reduced significance, indicated by lower z scores (Table 2). With correction across the whole brain, dACC activity was not related to subjective uncertainty; at an uncorrected threshold, a very small cluster of activation in dACC (−6, 21, 36; k = 5) was identified. This overlap in uncertainty-related activations, with smaller z scores in regions tracking self-reported compared with calculated posteriors, is consistent with our behavioral results indicating that subjective ratings were appropriately sensitive to changes in Bayesian probability overall but with notable deviations (i.e., underconfidence) exhibited by some subjects, potentially impacting neural activation at the group level.
Activity associated with increasing subjective uncertainty and underconfidence when observing evidence at p < 0.05 corrected for multiple comparisons across the whole brain
To examine brain regions involved in underconfidence, mean deviation scores (calculated minus self-reported posterior probability, with larger values corresponding to greater underconfidence) for each of the four draws were separately correlated with their respective parameter estimates at each voxel in the brain, regardless of sequence type (model 2). During the first draw, subjects with greater activity in ventromedial prefrontal cortex (VMPFC) exhibited greater underconfidence when making subjective ratings (Table 2, Fig. 5). Overlapping regions of VMPFC were also positively correlated with underconfidence on draw 2 (−9, 15, −21; k = 6), draw 3 (−12, 18, −24; k = 7), and draw 4 (−3, 9, −21; k = 13), although these activations were not significant when corrected for multiple comparisons. In addition, increased activity in right precentral gyrus and bilateral postcentral gyrus/paracentral lobule was associated with greater underconfidence on draw 4.
Greater VMPFC activity associated with underconfidence during evidence accumulation. A large region of VMPFC was related to underconfidence on draw 1 (orange), whereas smaller but overlapping areas were identified on draw 2 (blue), draw 3 (green), and draw 4 (red). Color bars represent t scores.
Discussion
The current study examined how the brain evaluates information when updating uncertainty about a future decision. Although dACC activity was correlated with objective uncertainty when observing evidence, the pattern of activity between the first and last draw suggests that this region was not tracking probabilities in a strict sense. Activity in dACC became greater between draws 1 and 4 as calculated posterior probabilities decreased (i.e., objective uncertainty increased) in type 2:2 sequences, yet it also increased (albeit to a lesser degree) when posterior probabilities increased (i.e., objective uncertainty decreased) in type 3:1 sequences. Indeed, even at an uncorrected threshold, we did not find any regions that behaved as if they were tracking objective probabilities (i.e., decreasing activity between draw 1 and draw 4 in type 4:0 and 3:1 sequences in parallel with increasing posteriors, and increasing activity in type 2:2 sequences in parallel with decreasing posteriors).
These findings may be interpreted in light of major theories of dACC functioning that have focused on evaluative and monitoring processes. Error detection theory suggests that dACC monitors mismatches between actual and intended responses (Scheffers and Coles, 2000), whereas conflict monitoring theory states that dACC detects conflict between coactivated responses (Botvinick et al., 2001). In contrast, Brown and Braver (2005) propose that dACC does not respond to errors or conflicts per se but instead is activated in situations in which error likelihood is greatest and, as such, has a role in risk prediction. Finally, reinforcement learning theory suggests that dACC is involved in learning after obtaining an outcome that is worse than expected, i.e., a reward prediction error (Holroyd and Coles, 2002), with later expansion of this theory to include outcomes that are merely different than expected (Oliveira et al., 2007), i.e., prediction errors not restricted to conditions of reward omission.
The pattern of results in dACC cannot be explained by existing error detection, conflict monitoring, or error likelihood theories. There were no overt errors during evidence accumulation, and both the response conflict associated with choosing a deck and the likelihood of making a future error in response to the decision would have been reduced in type 3:1 sequences as uncertainty decreased, yet our data show that the dACC signal increased between the first and last draws. Our findings are instead compatible with a role for dACC in responding to unexpected or unpredicted outcomes (Oliveira et al., 2007). During evidence accumulation, dACC may have increased between draws 1 and 4 on type 3:1 and 2:2 sequences because at least one draw was observed that was inconsistent with expectations established through previous observations of draws in a sequence (supplemental Figs. 1, 2, available at www.jneurosci.org as supplemental material), with the largest increase occurring in situations when expectations were repeatedly violated (i.e., type 2:2 sequences). In contrast, dACC activity did not increase as evidence accumulated on type 4:0 sequences because all draws consistently pointed to the same deck.
Frontal regions including OFC/IFC, superior and middle frontal gyri, and dorsomedial frontal cortex, as well as bilateral inferior parietal cortex, were uniquely associated with greater uncertainty when executing a decision at the end of the sequence compared with observing the last draw, even though objective uncertainty was the same. Although making ratings and executing the decision both involve choices regarding the identity of the deck supplying the cards, only the decision made at the end of the sequence is associated with incentive consequences. For this reason, unlike other studies that have identified OFC in uncertainty processing (Critchley et al., 2001; Tobler et al., 2007), the current paradigm was able to disentangle activations related to the cognitive experience of uncertainty from those associated with motivational processes such as reward risk that often vary along with uncertainty. The lack of significant modulation of OFC by uncertainty when subjects accumulated evidence in the absence of risk suggests that this region does not detect cognitive uncertainty per se, but instead responds to uncertainty only when there are incentive consequences. This is consistent with the well documented role of OFC in associating stimuli with the value of outcomes (Rushworth et al., 2007; Wallis, 2007), with lateral regions preferentially involved in processing punishment and disappointment (O'Doherty et al., 2001; Ursu and Carter, 2005). Indeed, uncertain sequences in our task should elicit a greater expectation of not receiving a reward, which may lead to disappointment when executing the decision. Alternatively, OFC/IFC activity may reflect a “safety signal” associated with opting out of making a decision, given previous work identifying a relationship between safe choices among risk-averse individuals and activity in a region of IFC located slightly superior to our own (Christopoulos et al., 2009).
Although we did find activity in OFC/IFC, we did not replicate previous findings linking decision uncertainty to nearby anterior insula (Volz et al., 2003; Grinband et al., 2006; Huettel, 2006; Preuschoff et al., 2008), which may be attributable to the fact that greater decision uncertainty in previous studies often also signals an increased likelihood for error. Errors robustly activate anterior insula, perhaps as a result of the aversive emotional response they elicit (Simmons et al., 2004), even in the absence of incentives or feedback (Taylor et al., 2007). Thus, the lack of insula activation in the present study may be attributable to our design, which dissociated increased uncertainty from error likelihood by allowing subjects to avoid an error by declining to make a decision on uncertain sequences.
Previous studies, including the analyses described above, have examined brain regions responding to uncertainty as defined by experimental stimuli. However, uncertainty is not only determined by external information but is also modulated by internal cognitive or emotional states, which vary between individuals. The VMPFC activity that correlated with underconfidence during evidence accumulation is not likely to be attributable to greater difficulty among underconfident subjects, because considerable evidence suggests that VMPFC activation decreases with cognitive effort (Simpson et al., 2001). Instead, we suggest that increased VMPFC activity may be related to greater arousal in autonomic and emotional systems when evaluating evidence. VMPFC has been conceptualized as part of a network important for integrating emotional-somatic states with factual information to guide decision-making (Damasio, 1996) attributable to strong interconnectivity with limbic/autonomic regions, including the amygdala, nucleus accumbens, hypothalamus, and midbrain (Kringelbach and Rolls, 2004; Price, 2007). VMPFC activity is associated with greater arousal, as measured by skin conductance (Critchley et al., 2000), and is part of the “default mode” network that deactivates during directed attention (Simpson et al., 2001). Damage to this region leads to an increase in risk-taking behavior (Clark et al., 2008), possibly because patients do not generate appropriate emotional-somatic anticipatory reactions before making a risky choice (Bechara et al., 2000).
Our results indicate that VMPFC activity is associated with relatively conservative ratings of certainty, even in the absence of overt risk when subjects are merely evaluating evidence. In theory, increased subjective uncertainty could be driven by a cognitive mechanism, such as an incorrect judgment of likelihood, or an emotional/motivational process involving the experience of increased arousal or appraisal of risk when evaluating evidence. Although correlational data cannot completely rule out the influence of aberrant cognition, our findings in VMPFC suggest that somatic/emotional mechanisms play an important role in determining subjective uncertainty, at least among healthy subjects. Of interest, individual variability in underconfidence was greater, and the relationship between VMPFC and underconfidence was stronger, on the first draw compared with later draws. This suggests that the neural signatures of conservatism have their greatest influence when observing limited information and that increasing evidence allows underconfident subjects to recover from initial uncertainty.
Overall, our data provide insight into how the brain updates beliefs during evidence accumulation and how emotional responses may interact with cognitive functioning to process uncertainty. Updating beliefs in a roughly Bayesian manner involves the activation of dorsal ACC regions to evaluate whether evidence is consistent or inconsistent with expectations, whereas the simultaneous activation of emotional-related processes by VMPFC may lead to a phenomenological experience of uncertainty that is greater (i.e., more underconfident) than what would be expected from the monitoring of dACC output alone.
Although results from the current study serve to elucidate the mechanisms of uncertainty during decision-making, limitations and questions remain for future research. While subjective ratings after each piece of information served a critical role in our design and analysis, it is possible that the mere process of rating uncertainty (in addition to experiencing uncertainty) lead to artificial reductions in emotionality (Taylor et al., 2003), which may have minimized the involvement of limbic/paralimbic brain regions in our task. Furthermore, methodological constraints required that we use a categorical scale for uncertainty ratings, yet a continuous visual-analog scale would have been preferable for detecting subtle differences between subjects. As such, our results may underestimate the full scope of brain regions involved in underconfidence. Despite these limitations, the current results represent an important step toward understanding how uncertainty is processed during different phases of decision-making and point to the need to investigate evidence accumulation in disorders characterized by pathological uncertainty.
Footnotes
This work was supported by National Institute of Mental Health National Research Service Award F32 MH082573 and University of Michigan fMRI Center pilot funding (E.R.S.), and National Institute of Mental Health Grant R01 MH071821 (S.F.T.).
- Correspondence should be addressed to Dr. Emily R. Stern, Department of Psychiatry, University of Michigan, 4250 Plymouth Road, 2506 Rachel Upjohn Building, Ann Arbor, MI 48109. emistern{at}med.umich.edu
