Many decisions are made under uncertainty; that is, with limited information about their potential consequences. Previous neuroimaging studies of decision making have implicated regions of the medial frontal lobe in processes related to the resolution of uncertainty. However, a different set of regions in dorsal prefrontal and posterior parietal cortices has been reported to be critical for selection of actions to unexpected or unpredicted stimuli within a sequence. In the current study, we induced uncertainty using a novel task that required subjects to base their decisions on a binary sequence of eight stimuli so that uncertainty changed dynamically over time (from 20 to 50%), depending on which stimuli were presented. Activation within prefrontal, parietal, and insular cortices increased with increasing uncertainty. In contrast, within medial frontal regions, as well as motor and visual cortices, activation did not increase with increasing uncertainty. We conclude that the brain response to uncertainty depends on the demands of the experimental task. When uncertainty depends on learned associations between stimuli and responses, as in previous studies, it modulates activation in the medial frontal lobes. However, when uncertainty develops over short time scales as information is accumulated toward a decision, dorsal prefrontal and posterior parietal contributions are critical for its resolution. The distinction between neural mechanisms subserving different forms of uncertainty resolution provides an important constraint for neuroeconomic models of decision making.
Decision making refers to the selection of actions based on the likelihood and potential value of possible outcomes. Decisions can be as simple as shifting the direction of gaze to receive an expected reward (Newsome et al., 1989; Platt and Glimcher, 1999) or as complex as choosing between two products based on one's current financial situation (Kahneman and Tversky, 1979; Tversky and Kahneman, 1981). Critical for all forms of decision making is accurate prediction: if an organism can confidently estimate the consequences of actions, optimal decisions can be made. Nevertheless, many decisions must be made with only limited information about their predicted consequences; that is, with psychological uncertainty or (if probabilities are estimable) economic known risks (Knight, 1921). Understanding how the brain resolves uncertainty during decision making is central to the emerging discipline of neuroeconomics (Glimcher, 2003).
Human neuroimaging studies have demonstrated that decisions under uncertainty, compared with certainty, evoke increased activation within the median wall of the posterior frontal lobe, an area referred to as frontomedian cortex (FMC) (Volz et al., 2003, 2004). The FMC may be critical for long-term development of stimulus-response contingencies and for hypothesis testing (Elliott and Dolan, 1998), which together reduce uncertainty over time. However, activation in a different set of regions, notably dorsolateral prefrontal cortex (dlPFC) and posterior parietal cortex (PPC), is evoked when subjects make decisions after an unexpected but behaviorally relevant stimulus (McCarthy et al., 1997; Kirino et al., 2000; Paulus et al., 2001). Furthermore, the amplitudes of the frontoparietal P300 electrophysiological potential (Squires et al., 1976) and of hemodynamic activation in dlPFC (Huettel et al., 2002) increase as decision stimuli become more unexpected. These results suggest that FMC and dlPFC/PPC might play complementary roles in uncertainty resolution, respectively supporting long-term and short-term modification of stimulus-response contingencies.
To test this suggestion, we adopted a novel decision task (Fig. 1A) that manipulated a different form of uncertainty than that used in previous paradigms. Decision uncertainty changed dynamically over a 6 s period as stimuli were presented that were consistent with one response or another. Across trials (Fig. 1B), some decisions were made with low uncertainty (i.e., 20%), and others were made with maximal uncertainty (i.e., 50%). While subjects performed this task, we measured blood oxygenation-level dependent (BOLD) activation using functional magnetic resonance imaging (fMRI).
We hypothesized that this task would evoke activation in different brain systems than that reported in previous studies of decision making under uncertainty. Because uncertainty in this task is (partially) resolved over the course of each trial, activation within regions such as dlPFC and PPC that are associated with activating and inhibiting stimulus-response contingencies based on moment-to-moment information (Miller and Cohen, 2001; Wallis et al., 2001; Huettel et al., 2002) should increase as uncertainty increases. However, under the hypothesis that FMC supports construction of long-term stimulus-response associations, as speculated by Volz et al. (2003), activation in FMC should not change with trial uncertainty. Such results, if obtained, would indicate that different sets of brain regions support different forms of uncertainty resolution.
Materials and Methods
Subjects. Data from 12 healthy volunteers (age range, 20-33 years; three females; nine males) were included in the analyses. All of the subjects had accurate behavioral performance and maximal head motion of less than one voxel, and all of the subjects gave written informed consent. This study was approved by the Institutional Review Board of Duke University Medical Center.
Experimental design. Each trial was composed of three phases: context, response, and feedback (Fig. 1 A). In the initial context phase, a series of eight shapes (each a circle or a triangle) were presented rapidly (duration, 400 ms; interval, 350 ms) at fixation. Each shape subtended ∼4° of visual angle. The number of shapes of each type on each trial was randomly drawn from the following probability distribution: 34% chance of eight shapes of one type, 30% of seven, 17% of six, 13% of five, and 6% of four of each type. The order of shapes within the series was randomized, and trials were equally likely to have more circles or more triangles.
After the offset of the last context shape by a variable (1850-4850 ms) interval, a single response cue (“?”) appeared at fixation for 1000 ms. Subjects indicated their decision by pressing with their right hand one of four buttons, arranged from left to right as follows: high confidence in a circle, low confidence in a circle, low confidence in a triangle, and high confidence in a triangle.
After a second variable interval (2000 to 3500 ms), a single feedback stimulus (either a circle or triangle) was presented for 1000 ms. The probability of each feedback stimulus depended on how many stimuli of that type had been presented: if eight, 80%; seven, 72.5%; six, 65%; five, 57.5%; and four, 50%. Thus, the outcome uncertainty associated with each trial type is given by the complement of these probabilities, and ranged from 20 to 50%. We used a minimal uncertainty of 20% rather than 0% so that all trial types could be followed by both positive and negative feedback.
Examples of the stimulus sequences used and the resulting uncertainty in the potential outcome are shown in Figure 1 B. Subjects were never explicitly told these feedback probabilities. However, they were instructed that as more stimuli of a given shape were presented, the shape was more likely to be correct. After the offset of the feedback stimulus, there was a 3500-6500 ms interval before the onset of the subsequent trial. During all interstimulus intervals, subjects were instructed to attend to a fixation cross at the center of the display.
Subjects participated in 7-10 experimental runs (mean, 9.2 runs), each ∼6 min in length, for an average total of 165 trials. At each break between runs, subjects received feedback as to their performance on the previous run. Subjects received $40 for participation in the fMRI session. The task was presented using the Compilable Imaging, Graphics, and Analysis Language display environment (Voyvodic, 1999). Before entering the scanner, each subject participated in a 30 min practice session in a behavioral testing laboratory.
fMRI data acquisition. Images were acquired using a 4.0 T General Electric (Waukesha, WI) scanner. We collected whole-brain BOLD images using a high-throughput T2*-weighted spiral-in pulse sequence with imaging parameters: repetition time (TR), 1500 ms; echo time, 35 ms; and flip angle, 60°. This field strength and pulse sequence provided excellent BOLD sensitivity for most cortical and subcortical regions, but significant signal loss was observed in the ventral frontal and temporal lobes because of local differences in magnetic susceptibility. We collected functional data from 34 axial slices with near-isotropic voxels of 3.75 × 3.75 × 3.8 mm. High-resolution three-dimensional full-brain spoiled gradient-recalled images were acquired to aid in normalization and coregistration.
In two subjects, an error in data acquisition resulted in a partial reduction in signal intensity within the inferior temporal cortex, inferior frontal cortex, and cerebellum. Because the regions of most interest to our hypotheses were unaffected, these subjects were included in all of the analyses.
Identification of task-related regions. Functional images were corrected for subject motion and time of acquisition within a TR and were normalized into a standard stereotaxic space [Montreal Neurological Institute (MNI), Montreal, Quebec, Canada] for intersubject comparison using SPM99 (Wellcome Department of Cognitive Neurology, London, UK). A smoothing filter 8 mm in width was applied after normalization.
From the overall time series for each subject, we extracted epochs that were time locked to the onset of the context stimuli on each trial. Trials on which the subject made a nonoptimal decision were excluded from additional analysis (4.9% of all trials), and the remaining trials were classified according to the amount of outcome uncertainty; that is, into five conditions: 20, 27.5, 35, 42.5, and 50%. Because our goals were to identify brain regions that were active to the task and evaluate whether those regions were influenced by uncertainty, we first used voxelwise analyses across all uncertainty conditions. For each condition in each subject, we calculated the z-score change from a peristimulus baseline (-4.5 to -1.5 s) at each time point in the epoch.
We then conducted a second random-effects analysis across all conditions and subjects to evaluate whether there was a significant change from baseline at each time point. Note that each condition was equally weighted in this analysis to preclude any biasing attributable to differential activation across conditions and/or different numbers of trials per condition. We calculated for each voxel the maximum t statistic from the eight time points from +3 to +13.5 s after the onset of the context stimuli; these bounds were chosen to encompass the predicted peak of the context-induced hemodynamic response, without making additional assumptions about its shape.
We next corrected this activation map for multiple comparisons using strict Bonferroni's correction (for ∼47,000 voxels in our normalized brain and eight time points of testing) at a type I corrected α value of 0.05. This resulted in a t threshold of 5.81 (df = 59) and an uncorrected α value of ∼1.3 × 10-7. Using a cluster-size threshold of six contiguous voxels, we found 40 foci of activation to be tested for uncertainty effects. We used such strict correction to ensure that we would have a high degree of confidence that the regions of interest (ROIs) used in the second stage of our analysis would be associated with task-related activation. The upper limit of the color maps on the activation figures is set at a type I corrected value of 0.000001 (t > 8.60; uncorrected, α = 2.6 × 10-12), and nearly all included regions had a maximal value above even that threshold. Note that this analysis does not introduce any bias into our subsequent analyses of uncertainty effects, because it treats all uncertainty conditions independently.
We do not report in this manuscript analyses of the feedback stimuli. Initial analyses revealed no significant regions of activation associated with feedback, which may reflect the fact that subjects' payment did not depend on their choices.
Assessment of significant effects of uncertainty within task-related regions. We next conducted a regression analysis to determine whether there were significant effects of uncertainty on the fMRI BOLD response in regions evincing task-related activation. After calculating an epoch-averaged waveform for each subject and ROI, we determined the time point at which there was maximal BOLD change across all conditions. We then calculated, for each of the five uncertainty conditions, the amplitude of the mean response within a five TR window centered on the BOLD peak. For each ROI, we constructed a simple regression analysis with independent variables of percentage uncertainty (five levels: 20, 27.5, 35, 42.5, and 50%) and dependent variables of peak BOLD signal change. We report in Tables 1 and 2 regions that had a significant main effect of uncertainty and a significant intercept, respectively, in the regression analysis.
To investigate whether the sequence of stimuli in the context phase had an effect on observed BOLD activation, we conducted three additional voxelwise analyses. First, we separated trials in which seven stimuli of one type and one of another type were presented (72.5% uncertainty condition) into two groups: those in which the odd stimulus occurred in the first four stimuli and those in which it occurred in the last four stimuli. Second, for epochs with six stimuli of one type (35% uncertainty), we compared trials in which both of the odd stimuli occurred in the first four stimuli with those in which both occurred in the last four. Third, for epochs with four stimuli, we compared trials in which there were many alternations between stimuli with those with fewer alternations. All of these analyses used t tests across subjects of the different waveforms at each time point (α = 0.001; cluster size, 6 voxels), because we had no a priori expectation about the shape and timing of these differences.
On each trial, the subjects indicated whether they predicted a circle or triangle to be presented in the feedback phase and whether they had high or low confidence in their decision. Mean response time was ∼870 ms, and this value did not significantly differ across uncertainty conditions (F(4,44) = 0.67; p > 0.10). As uncertainty decreased from 50 to 20%, subjects were increasingly likely to choose the more-prevalent stimulus and were increasingly likely to indicate that they had high confidence in their decision (Fig. 2). These results indicate that subjects learned the response contingencies and that our uncertainty manipulation had the desired effects on decision making.
We note that, although we recorded confidence judgments, those judgments were very highly correlated with our uncertainty manipulation, as shown in Figure 2. Therefore, our analyses below examine trials in which subjects predicted the more-probable stimulus, collapsing over expressions of high and low confidence.
Our initial analyses identified regions with significant increases in activation to the presentation of the context stimuli across all uncertainty conditions. Foci of significant task-related activation were observed in dlPFC, PPC, the anterior insula (aINS), FMC, and the thalamus, as well as in motor and visual regions. Latency to peak was earliest for regions within the occipital and inferior temporal lobes (6-7.5 s after trial onset), intermediate for parietal and anterior prefrontal regions (10.5 to 12 s), and latest for motor-related regions (15 to 16 s). This timing order reflects activation to the different task phases of viewing the context stimuli, forming a decision, and executing a response. Activation in posterior and medial prefrontal regions also peaked relatively late (15 s) in the epoch.
Within the set of regions evincing task-related activation, there were four areas with significant effects of uncertainty: posterior dlPFC, aINS, PPC, and the right thalamus (Table 1, Fig. 3A). Plotted in Figure 3B are the BOLD responses across conditions in the areas of frontal and insular cortices that exhibited uncertainty effects. Monotonic increases in BOLD activation were observed across conditions, from negligible activation in the 20% uncertainty condition to greatest activation in the 50% uncertainty condition. Figure 4 illustrates BOLD responses in the left and right intraparietal sulci of posterior parietal cortex. Again, BOLD activation was minimal in the 20% uncertainty condition and monotonically increased as uncertainty increased.
We additionally identified regions with a significant intercept in our regression analysis, which indicated a significant BOLD response independent of the uncertainty effect. Regions exhibiting such an uncertainty-independent main effect included areas implicated in motor and visual function (Table 2, Fig. 5). We note that, although FMC exhibited such an uncertainty-independent main effect, visual inspection of the time courses suggested that activation in the 20% uncertainty condition was less than the activation in all other conditions, which did not differ from each other.
There was no region for which both uncertainty and intercept factors were significant in the regression model, although the posterior left middle frontal gyrus (-44, -4, 36) (x, y, z) had a significant effect of uncertainty and a trend toward significance (p = 0.08) of the intercept, whereas the posterior left superior frontal gyrus (-25, -5, 52) had a significant effect of the intercept and a trend toward significance (p = 0.06) of uncertainty. The remaining regions identified as having significant task-related activation exhibited neither significant effects of uncertainty nor a significant intercept (Table 3).
Our experimental hypotheses were motivated by potential differences in the response to uncertainty in two sets of regions: (1) frontomedian cortices and (2) lateral prefrontal, parietal, and insular cortices. Therefore, we directly compared the effects of uncertainty on activation in these two sets of regions. We repeated the above regression analyses collapsing across FMC regions (coordinates are indicated in Tables 2 and 3) and across lateral regions listed in Table 1. We determined, for each subject, the main effect of uncertainty in each of the two sets, and then compared those sets of parameter estimates statistically. We found that the uncertainty effect was significantly greater in the set of lateral regions compared with FMC (t(11) = 2.90; p = 0.01), providing additional support for a dissociation between these systems.
Given these effects of uncertainty, we next evaluated whether the order of stimulus presentation influenced the pattern of BOLD activation. In the 27.5% uncertainty condition, subjects viewed seven of one shape and one of the other shape. We compared trials for which the incongruent event occurred in the first half of the context stimuli with trials in which the incongruent event occurred in the second half. Within the PPC, activation to late incongruent events (i.e., those inconsistent with an already planned response) was significantly greater than activation to early incongruent events at time points later in the epoch (Fig. 6). Centroids of this activation were located in the intraparietal sulcus bilaterally at MNI coordinates (-36, -57, 54) and (51, -52, 50). Analyses of two other aspects of stimulus sequence, the timing of odd events in the 35% uncertainty condition and the number of stimulus alternations in the 50% uncertainty condition, revealed no additional areas of significant activation associated with the sequence of the context stimuli.
The current study investigated the brain mechanisms underlying the resolution of uncertainty in decision making. The psychological concept of uncertainty is intimately related to the statistical concept of information (Shannon and Weaver, 1949; Garner, 1962), such that it depends on the number of potential outcomes and their estimated probabilities. In most previous studies of uncertainty, different stimuli were associated with different outcome probabilities (Critchley et al., 2001; Paulus et al., 2001; Volz et al., 2003). For example, Critchley et al. (2001) presented subjects with single nonface playing cards (e.g., ace through 10) and asked subjects to decide whether the ensuing card would be higher or lower. In this design, there is 0% outcome uncertainty for extreme cards (e.g., ace and 10) and up to 44% outcome uncertainty for medial cards (e.g., 5 and 6). Similarly, Volz et al. (2003) presented pairs of comic figures and asked subjects to judge which figure would “win” according to learned rules; depending on the pairing, the uncertainty in the winning outcome ranged from 0% to 40%. When subjects make decisions in such uncertain situations, activation is typically found in FMC, in either the medial frontal gyrus (Volz et al., 2003, 2004) or in the cingulate gyrus (Critchley et al., 2001).
Our experiment differed from these previous studies in that successive stimuli provided partial information toward an expected decision, with uncertainty reduced or increased depending on whether a stimulus confirmed or violated that expectation. Events that violate expectations have been demonstrated previously to evoke activation in dlPFC and PPC (McCarthy et al., 1997; Clark et al., 2000; Kirino et al., 2000; Huettel et al., 2002), and we found significant uncertainty-related activation within these regions. In contrast, activation in FMC was present during the task, but its amplitude did not change as a function of uncertainty. Together, these results suggest that uncertainty may influence distinct cognitive processes, depending on the decision task. When uncertainty influences the learning of relations between stimuli and responses, as reported by Volz et al. (2003), it evokes long-term strategic processes mediated by FMC. These processes were likely engaged on all trials in our design, because subjects refined contingencies between trial type (e.g., number of circles) and outcomes (e.g., probability of reward), but they were not influenced by the particular pattern of stimuli presented on each trial. In contrast, when decision making does not involve learning but requires selecting between plans for action based on stimulus information (as in the present study), uncertainty evokes short-term response-selection processes mediated by dlPFC and PPC (Schumacher and D'Esposito, 2002).
A striking and unexpected result was the measurement of only minimal BOLD activation when all eight stimuli supported the same prediction (Figs. 3, 4), although there remained a 20% chance that that prediction would be incorrect. For comparison, Critchley et al. (2001) found clear indications of increased activation within FMC and aINS (compared with no-uncertainty conditions) at an uncertainty level of only 11%. This difference may result from the same functional dissociation described above. If the active regions in the current experiment support the resolution of uncertainty toward a course of action, activation should be only evoked when successive stimuli point to different potential actions. However, such activation might be absent in our eight-stimulus condition because every single stimulus is consistent with the same response. In the latter case, the decision can be described as being made in the absence of behavioral uncertainty, although outcome (or reward) uncertainty is still present.
Along these lines, the strongest effects of uncertainty on decision making were observed in PPC, along the intraparietal sulcus. Early conceptions of parietal cortex described its function as supporting a control signal that relates stimuli to responses (Mountcastle et al., 1975). This perspective has been revised significantly by electrophysiological studies of decision making, which have demonstrated that neurons within the PPC of monkeys provide codes for decision variables such as probability and expected value (Shadlen and Newsome, 1996; Platt and Glimcher, 1999). Human neuroimaging studies have found PPC activation across a range of decision-making tasks (Paulus et al., 2001; Huettel et al., 2004). However, evidence for direct modulation of PPC activation by uncertainty has been equivocal. Volz et al. (2003, 2004) found that PPC activation increases as uncertainty increases, whereas Critchley et al. (2001) observed no significant uncertainty effects in PPC using a much simpler task (although PPC did show delay period activation that did not depend on uncertainty). We suggest that a primary function of PPC is to support the generation and modification of a set of context-appropriate responses (Bunge et al., 2002). As uncertainty increases in our task, subjects are faced with a more difficult generation process (i.e., there are more transitions and a longer time before response selection can take place). In support of this interpretation, we found that, when a single incongruent stimulus was presented, its timing influenced activation in PPC (Fig. 6).
We note that the regions discussed in the previous paragraphs, dlPFC and PPC, are commonly grouped as key nodes in a brain system for executive processing (McCarthy et al., 1997; Paulus et al., 2001; Bunge et al., 2002; Fincham et al., 2002). Given that our task requires multiple executive processes (e.g., maintaining shapes in working memory and selecting a context-appropriate response), an important question lies in the relationship between decision making under uncertainty and more general executive processes. A parsimonious interpretation based on our data and previous results is that, although resolution of uncertainty over time (as in the present study) invokes processes under the control of the executive system, the degree of uncertainty does not necessarily influence the degree to which control systems are engaged. As evidence for this perspective, activation in anterior dlPFC, which has been associated with a variety of behavioral control processes (Kirino et al., 2000; Rowe et al., 2000; Miller and Cohen, 2001; Cools et al., 2004), was present but did not significantly differ across uncertainty conditions.
We also observed clear uncertainty effects in insular cortex, which has historically been associated with affective stimuli, especially those from the chemical senses, and with sensory integration (Calvert, 2001; Adolphs, 2002; Wicker et al., 2003; Small et al., 2004). However, a number of recent studies have implicated insular cortex, especially anterior regions, in decision making under uncertainty (Critchley et al., 2001; Paulus et al., 2003; Volz et al., 2003). A potentially powerful framework for understanding insular activation has been advanced by Paulus et al. (2003), who suggest that the insula supports risk-based control processes, becoming increasingly active as subjects take actions with greater risk. This idea is tied to the somatic marker hypothesis (Damasio, 1996), such that increased activation in insular cortex (and other ventral regions) would reflect stronger negative somatic states (Bechara, 2001). A clear prediction of this framework is that different forms of uncertainty have different effects on insular and PPC activation. Uncertainty about future reward outcomes should evoke insular activation, whereas uncertainty about what behavior to choose should evoke PPC activation. Although the current study (and others in the literature) makes outcomes contingent on selection of the correct action, these forms of uncertainty could be distinguished by comparing stimuli with similarly known responses but different probabilities of reward.
In summary, we demonstrated using a novel task that the presentation of stimuli conveying greater or lesser amounts of uncertainty modulates activation in selected regions within a more general brain system for behavioral control. The results provided new evidence for parietal and prefrontal involvement in the resolution of uncertainty when subjects are faced with limited knowledge about the correct action to take.
However, several limitations of our study suggest avenues of future research. Although we had expected the delivery or omission of rewards to modulate activation in ventromedial frontal and striatal regions (Schultz et al., 1997; Fiorillo et al., 2003; Knutson et al., 2003), we found no significant effects of the feedback stimuli. As noted above, this absence may reflect the lack of explicit reward information during the experiment or the extended nature of the decision process. Also, because of the high magnetic-field strength and pulse sequence that were used, susceptibility arti-facts in orbitofrontal cortex (OFC) precluded inference about activation in that region. Given previous demonstrations that OFC damage increases selection frequency of risky alternatives (Bechara et al., 1994; Bechara et al., 2000) and that ventral prefrontal activation is observed in paradigms involving risk and reward (Elliott et al., 2000; O'Doherty et al., 2001), pulse sequences designed to ameliorate signal losses in that region will be critical for future work (Guo and Song, 2003). Furthermore, we were unable to examine the effects of decision confidence on activation attributable to its tight coupling to stimulus uncertainty. Internally attributed uncertainty, as could be evoked by soliciting a confidence judgment, has been shown to evoke dlPFC and PPC activation, perhaps because of working memory demands (Volz et al., 2004). The use of natural stimuli without a priori uncertainty values might allow dissociation of decision confidence and decision uncertainty.
This work was supported by National Institute on Drug Abuse Grant DA-16214 and National Institute of Neurological Disorders and Stroke Grants NS-41328 and NS-50329. G.M. was supported by a Senior Research Career Scientist Award from the Department of Veterans Affairs. We thank Evan Gordon, Beau Mack, Jonathan Smith, and Brent Warner for assistance in data collection and analysis.
Correspondence should be addressed to Scott A. Huettel, 163 Bell Research Building, Box 3918, Duke University Medical Center, Durham, NC 27710. E-mail:.
Copyright © 2005 Society for Neuroscience 0270-6474/05/253304-08$15.00/0