Abstract
External circumstances and internal bodily states often change and require organisms to flexibly adapt valuation processes to select the optimal action in a given context. Here, we investigate the neurobiology of context-dependent valuation in 22 human subjects using functional magnetic resonance imaging. Subjects made binary choices between visual stimuli with three attributes (shape, color, and pattern) that were associated with monetary values. Context changes required subjects to deviate from the default shape valuation and to integrate a second attribute to comply with the goal to maximize rewards. Critically, this binary choice task did not involve any conflict between opposing monetary, temporal, or social preferences. We tested the hypothesis that interactions between regions of dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC) implicated in self-control choices would also underlie the more general function of context-dependent valuation. Consistent with this idea, we found that the degree to which stimulus attributes were reflected in vmPFC activity varied as a function of context. In addition, activity in dlPFC increased when context changes required a reweighting of stimulus attribute values. Moreover, the strength of the functional connectivity between dlPFC and vmPFC was associated with the degree of context-specific attribute valuation in vmPFC at the time of choice. Our findings suggest that functional interactions between dlPFC and vmPFC are a key aspect of context-dependent valuation and that the role of this network during choices that require self-control to adjudicate between competing outcome preferences is a specific application of this more general neural mechanism.
Introduction
In many cases, the value of an object or specific features of an object can change depending on the situation. For example, the retractable roof of a convertible car is of little benefit during a cold Nebraska winter, but is a much greater asset in sunny Southern California. The car itself remains the same, but the value of its convertible top depends on the driving context. Here, we ask the question: How does the brain incorporate context and adjust value computations to fit a given circumstance?
Recent work has highlighted the role of lateral prefrontal cortex in modulating reward and value computation systems. Much of this evidence comes from situations where individuals endeavor to employ self-control to adjust the relative importance of immediate and long-term rewards in value computation and choice (McClure et al., 2004; Hare et al., 2009, 2014; Figner et al., 2010; Kober et al., 2010; Luo et al., 2012; Harris et al., 2013), or must trade-off monetary gains against social preferences or norms (Sanfey et al., 2003; Spitzer et al., 2007; Knoch et al., 2008; Baumgartner et al., 2011). In the example of dietary self-control, functional magnetic resonance imaging (fMRI) and electroencephalography (EEG) data suggest that dorsolateral prefrontal cortex (dlPFC) modulates activity in ventromedial prefrontal cortex (vmPFC) resulting in greater emphasis on the food's health relative to its taste in vmPFC value computations to conform to the dieting goal (Hare et al., 2009; Harris et al., 2013). Similarly, temporary inhibition of dlPFC through transcranial magnetic stimulation (TMS) resulted in altered vmPFC activity and choice patterns during tradeoffs between monetary gain and fairness or inequality (Baumgartner et al., 2011). One thing that all of these behaviors have in common is the presence of conflicting desires for or interpretations of the stimuli that must be balanced properly to achieve a goal (e.g., weight loss or fairness). It remains unknown whether dlPFC–vmPFC interactions also underlie context-dependent changes in valuation that do not involve self-regulation of conflicting desires.
We used a novel choice task to investigate the neural networks underlying flexible, context-dependent valuation with fMRI and compared these networks to those previously identified during choices with competing desires that required self-control. In this task, participants made choices over shapes with differing patterns and colors in contexts where the pattern or color would either influence the monetary outcome or not. Critically, the goal to maximize monetary reward in this task remained constant, and changes in context presented no conflict between outcome factors and no need for self-regulation of opposing desires. We found patterns of activity in and functional connectivity between vmPFC and dlPFC that are consistent with their respective roles in value computation and in signaling the need for modulation of computations. Our findings suggest that the modulatory influence of dlPFC on vmPFC value computations is a fundamental mechanism for flexible, context-sensitive valuation, and extend the scope of this mechanism beyond choices involving regulation or conflicting outcome preferences.
Materials and Methods
Subjects.
A total of 28 students from the University of Zurich or Swiss Federal Institute of Technology Zurich participated in the study. All participants were right-handed and were screened for nonsmoking, absence of color blindness, neurological diseases, and psychological disorders. Five subjects were excluded due to technical problems, and one subject did not pass the performance criterion of at least 90% correct choices (subject's error rate in the binary choice task was 18% compared with average error rate 6%). The remaining 22 subjects (16 male; mean, 22 ± 2.49 SD years; all right-handed) were included in the analysis. This study was approved by the Zürich Cantonal Ethics Commission and all subjects gave informed, written consent.
Task design and experimental setup.
The task involved binary choices between two reward options represented by visual stimuli with varying shapes, patterns, and colors. The task timing is described in Figure 1. All rewards were specified in experimental monetary units (MU) that were later converted into Swiss Francs (CHF 1: USD 1.06). The default reward level for each option was represented by one of four geometric shapes with each edge of the shape representing 1 MU: triangle (3 MU), square (4 MU), pentagon (5 MU), or hexagon (6 MU). On 60% of all trials, only the shape determined the reward levels (shape condition). On the remaining 40% of the trials, the pattern or color of the stimulus added or subtracted a value (−3, −1, +1, or +3 MU) to or from the shape value (bonus condition). Given that the average bonus amount from patterns and colors was zero, and that bonus values were randomly assigned, the average values of shape and bonus trials are expected to be equal. In fact, there was no significant difference in average stimulus value between shape and bonus trials for 21 of 22 subjects, although there was a small, but significant difference in one subject (shape value mean, 4.43 ± 1.15 SD; bonus value mean, 4.7 ± 2.41 SD, Mann–Whitney Wilcoxon test, U = 30,362, z = 1.78, p < 0.05). Note that our findings remained unchanged in two control analyses that either removed this single subject or added the mean value difference between shape and bonus trials as a covariate at the random effects level for all contrasts. Figure 1 shows an example of how the context determines the value of a stimulus and can lead to a change in what the most valuable option is as follows: e.g., a triangle (worth 3 MU by default) with a checkers pattern (−1 MU) and the color red (+3 MU) would have a value of 2 MU in a pattern bonus trial and 6 MU in a color bonus trial.
Timeline of the binary choice task. On each trial, the subject chooses between two visual stimuli that vary in shape, pattern, and color. Each attribute is associated with a monetary value that the subject has learned before scanning. At the beginning of each block of three to five trials, a cue that indicates which context applies to this block is presented for 1 s. In the shape context (60% of all trials) only the shape determines the value of a stimulus (top row), in the bonus context (40% of all trials), either the pattern (middle row) or the color (bottom row) add or subtract a value to or from the shape value. After a jittered fixation display the first choice pair is presented and the subject has up to 3.5 s to make a choice (mean RT = 1.17 s). Once a response is made, the chosen stimulus is highlighted with a white frame for 0.5 s. After choice feedback, a fixation cross is presented for a jittered interval of 4.5–8.5 s minus RT before the next trial begins.
In two practice-and-test rounds at a desktop computer, subjects learned the values associated with each stimulus attribute. In the initial practice round (60 choices), only the shape determined the reward levels. On each trial, subjects chose between two stimuli and were presented with a feedback on the value of the chosen stimulus. To test whether subjects had learned the shape–reward associations, they were then asked to indicate the value of a probe stimulus that was randomly drawn from the sample in a test round (40 trials). To proceed, subjects had to correctly specify the probe's value on at least 95% of the test trials; otherwise, another 20 test trials would begin. In the second practice round (60 choices), 40% of all choices included a bonus for the pattern or color on the shapes, as indicated by a cue preceding each block of choices. The exact color-bonus associations were counterbalanced across subjects. Subjects were then tested on this context-dependent integration of shape and bonus values in a second test round (40 trials) where subjects were asked to indicate the value of a probe stimulus. Again, subjects had to correctly specify the probe's value on at least 95% of the trials to proceed.
In the fMRI scanner, subjects played a total of 240 trials of the binary choice task. Choice pairs constituted of two randomly drawn shapes, and two colors and patterns each randomly drawn without replacement. This provides a total set of 64 different stimuli (4 × 4 × 4) and 2304 (64 × 36) possible sets of choice pairs. Subjects played a mean of 194.2 ± 4.7 SD trials with differently valued options (optimal choices possible), and 44.9 ± 4.5 SD trials with equally valued options (both choices equally good).
At the beginning of each block of 3–5 trials, if there was a change from the preceding block, then a cue indicated the context; i.e., whether only shape, or shape and pattern, or shape and color determined the stimuli's value during this block. Importantly, the shapes varied in color and pattern during all choices, but these attributes only mattered when specifically cued before a set of choice options was presented. To ensure that context conditions changed infrequently, the sequence of block lengths (3, 4, or 5 trials) was pseudorandomized to create 60 blocks with no more than two repetitions of any block length. The sequence of conditions was pseudorandomized to include 60% shape, 20% pattern, and 20% color trials without any continuous repetition of pattern or color blocks. Conditions changed on a mean 41.6 ± 0.8 SD trials, with a mean 17.9 ± 0.4 SD switches from shape to bonus condition. To minimize working memory requirements, the current context was shown on every decision screen (Fig. 1).
Subjects completed the scanning task in one continuous session of ∼31 min. To reduce fatigue effects in the subjects, we included three 13 s breaks in the fMRI paradigm. For 3 s, subjects saw the text “The task will continue in 10 s” followed by a 10 s countdown. Subjects were instructed to relax their eyes and fingers during these breaks while continuing to lie still at all times.
At the end of the fMRI task, three trials were randomly selected for payoff. The chosen stimuli's values in the selected trials were converted into CHF (with 1 MU = CHF 3) and added to a base pay rate of CHF 25 per hour. Subjects received a mean CHF 63 ± 4 SD.
MRI data acquisition and preprocessing.
All MRI data were acquired on a Philips Achieva 3.0 tesla whole-body scanner using an 8-channel sensitivity-encoded (SENSE) head coil. The functional session started off with a localizer scan followed by the binary choice paradigm implemented in MATLAB (MathWorks), using the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997; Kleiner et al., 2007). Visual stimuli were projected onto a screen that the subjects viewed through an angled mirror mounted to the head coil. Subjects made their responses with their right hands on a two-button response box. While subjects were playing the task, we acquired gradient echo T2*-weighted echo-planar images (EPIs; 755 volumes per session) with blood oxygen level-dependent (BOLD) contrast (41 slices per volume, field-of-view 200 × 126.5 × 200 mm, slice thickness 2.5 mm, 0.6 mm gap, in-plane resolution 2.5 × 2.5 mm, matrix 80 × 80, repetition time 2460 ms, echo time 30 ms, flip angle 77°) and a SENSE acceleration factor of 2. Volumes were acquired in axial orientation at a +15° tilt to the anterior commissure–posterior commissure line.
Before the functional run, field map images were acquired to correct for the inhomogeneity of the magnetic field. After the functional run, T1-weighted 3D fast-field echo images were acquired from each subject (181 slices, field-of-view 256 × 256 × 181 mm, slice thickness 1 mm, no gap, in-plane resolution 1 × 1 mm, matrix 256 × 256, repetition time 8.4 ms, echo time 3.89 ms, flip angle 8°).
Preprocessing of the functional images was implemented in the MATLAB-based software Statistical Parametric Mapping 8 (SPM8, version r5236; http://www.fil.ion.ucl.ac.uk/spm). First, we used the FieldMap toolbox to correct for distortions in the EPIs. Next, we used the New Segment toolbox to segment the T1 image into six tissue classifications (gray matter, white matter, CSF, bone, soft tissue, and air tissue), to apply this segmentation to the mean EPI image, to coregister all EPIs accordingly and to normalize all images to MNI standard space (Evans et al., 1993; http://www.bic.mni.mcgill.ca). Finally, we smoothed the image data with a 4 mm full-width at half-maximum Gaussian kernel.
fMRI data analysis.
The statistical analysis of the fMRI data were also performed in SPM8 (version r5236). We modeled the BOLD response with two general linear models (GLMs 1–2) that were estimated using SPM8's standard hemodynamic response function and a high-pass filter of 128 Hz, as well as correction for intrinsic autocorrelations. SPM8's internal masking threshold for the estimation of the β parameters was set to 0.4.
For the main analysis we defined GLM 1 with four regressors of interest as boxcar functions: (1) the display of shape cues (duration 1 s), (2) the display of bonus cues (duration 1 s), (3) choices in shape trials (duration = reaction times), and (4) choices in bonus trials (duration = reaction times). The two choice regressors were parametrically modulated by the respective chosen and nonchosen stimulus' values. Note that the parametric modulators for chosen and nonchosen values were only modestly correlated (maximum r across all subjects = 0.27) and that the parametric modulator for nonchosen values was orthogonalized with respect to the parametric modulator for chosen values. We added nuisance regressors for the 13 s breaks and six motion parameters. For every subject, we created contrast images for all regressors of interest and the parametric modulators.
We tested the contrast images in the following random effects models at the group level. First, to test which regions encode the relative value of the chosen stimulus we ran a one-sample t test on the contrast of chosen minus nonchosen value across all choice trials. Based on the resulting T-map (Fig. 2), we created two region-of-interest (ROI) masks of all voxels within anatomical masks of vmPFC and the posterior cingulate cortex (PCC), respectively, at a threshold of p < 0.001, uncorrected (peak MNI coordinates within ROI mask for vmPFC: 3, 51, −16; for PCC: −10, −52, 6). The anatomical masks were derived from the Automated Anatomical Labeling atlas as implemented in the WFU pickatlas v2.4 (Tzourio-Mazoyer et al., 2002; Maldjian et al., 2003). The vmPFC ROI mask was used for the functional connectivity analyses, and both ROI masks were used for the purpose of plotting mean β parameters (Fig. 3C).
Second, we tested the contrast of bonus minus shape choices to identify regions with increased activity during bonus choices. Based on the resulting T-map (Fig. 4), we created an ROI mask of the dlPFC/ inferior frontal gyrus (IFG) cluster at p < 0.001, uncorrected (peak MNI coordinates within ROI mask for dlPFC/IFG: −45, 6, 31; see Table 3), which was used to extract and plot β parameters (Fig. 5B).
For all random effects analyses, we applied whole-brain correction for multiple comparisons at the cluster level: We calculated the corrected cluster extent (kE) for each t test using Gaussian random-field theory as implemented in SPM8 with a cluster-defining individual voxel threshold of p < 0.005 to achieve a familywise error (FWE)-corrected statistical threshold of p < 0.05 (minimum kE > 50, range 50–65). Multiple-comparison corrections were applied at the whole-brain level in combination with an exclusion mask for CSFs that was derived from the Talairach Daemon database as implemented in the WFU pickatlas v2.4 (Lancaster et al., 1997, 2000; Maldjian et al., 2003). To better separate large clusters and produce more informative tables a cluster-defining threshold of p < 0.001, resulting in an FWE-corrected cluster extent of kE > 20 (range, 20–27), was used to generate Tables 1⇓–3.
Regions positively correlated with the difference between chosen and nonchosen stimulus values or vice versa
To investigate whether shape values are encoded differently in shape as opposed to bonus trials we set up a second GLM (GLM 2) that modulates bonus trials separately by the shape and the relevant bonus value. This model was set up for n = 18 subjects, because the relevant shape information was not logged during bonus trials for the first four subjects. In this GLM, we modeled the following regressors of interest as boxcar functions: (1) display of all cues (duration 1 s), (2) all choice trials modulated by chosen and nonchosen shape value (duration = reaction times), (3) choices in bonus trials modulated by chosen and nonchosen shape value and by chosen and nonchosen bonus value (duration = reaction times). Note that the parametric modulators for shape value in bonus trials represent an interaction term measuring changes in the slope of the correlation between the BOLD signal and shape values in bonus compared with all other trials (i.e., shape trials). Thus, the parametric regressors for shape value in all trials effectively explain the variance associated with shape values during shape trials, and the regressors for shape value in bonus trials should be interpreted as an implicit contrast of bonus and shape trials. The breaks and the motion parameters were again added as nuisance regressors.
For every subject we created contrast images for (1) chosen minus nonchosen shape value (relative shape value) in shape trials, (2) chosen minus nonchosen shape value in bonus versus shape trials, and (3) chosen minus nonchosen bonus value (relative bonus value) in bonus trials. We then tested these contrast images in one-sample t tests at the group level. We report results that survive cluster-level FWE-correction at a threshold of p < 0.05 for the whole-brain volume.
To illustrate the differential encoding of shape values in shape and bonus trials, mean β parameters for the parametric modulators for shape values were extracted from the vmPFC and PCC ROI masks using the MarsBaR toolbox (Brett et al., 2002). Figure 3C shows the mean βs for chosen and nonchosen shape values for shape and bonus trials.
Functional connectivity analysis: PPI.
We used psychophysiological interaction (PPI) analysis to assess the functional connectivity between vmPFC and dlPFC depending on three psychological factors-of-interest that were derived from GLM 1: (1) the display of the context cues, (2) choices in shape trials, and (3) choices in bonus trials. We extracted single-subject time courses from a volume-of-interest (VOI) within the vmPFC ROI mask described in Table 1 and Figure 2. For each subject, we identified the peak z value for the contrast of chosen minus nonchosen value across all trials and extracted the first BOLD signal eigenvariate from a 5 mm sphere around this individual peak. Next, we created PPI terms between this BOLD eigenvariate time course and the three psychological factors-of-interest. Last, the PPI terms, the VOI eigenvariate, and all regressors described in GLM 1 were entered into a new GLM (GLM-PPI). Single-subject contrasts were then computed for the PPI terms and tested in random effects analyses (one-sample t tests). We report results that survive cluster-level FWE-correction at a threshold of p < 0.05 for the whole-brain volume. To illustrate task-dependent changes in functional connectivity between vmPFC and dlPFC, mean β parameters were extracted from within the dlPFC/IFG ROI mask shown in Figure 4 using the MarsBaR toolbox and plotted in Figure 5B.
Last, we tested whether the changes in connectivity in bonus compared with shape trials were related to the reweighting of bonus attributes (i.e., colors or patterns) during stimulus value computations. For this, we used the β parameters for relative bonus values (GLM 2, contrast 3) in the vmPFC ROI mask as a covariate in the one-sample t test on the bonus PPI minus shape PPI contrast. At a cluster-level FWE-corrected threshold of p < 0.05 for the whole-brain volume, the resulting T-map in Figure 6 shows two regions in the dlPFC in which connectivity with the vmPFC increases in proportion to the relative bonus value encoding in vmPFC (see Table 6 for peak coordinates). These two functional clusters in the dlPFC, referred to as dlPFC/superior frontal gyrus (SFG) and dlPFC/middle frontal gyrus (MFG) were then used to extract mean β parameters for the bonus PPI minus shape PPI contrast using the MarsBaR toolbox. To illustrate the correlation between the context-dependent increase of vmPFC–dlPFC connectivity for bonus compared with shape trials and the relative bonus value encoding in vmPFC the β parameters in the two dlPFC clusters were then plotted against the β parameters for relative bonus values in the vmPFC ROI in Figure 6.
Results
Behavioral results
Subjects were highly accurate in all choice conditions. We assessed subjects' performance by computing the percentage of optimal choices (i.e., choosing the highest valued stimulus) in those trials in which an optimal choice was possible (i.e., with value left ≠ value right, mean 194.2 ± 4.7 SD trials). Overall, subjects made a mean of 95.5 ± 2.1% SD optimal choices. Although they made more optimal choices in shape compared with bonus trials, subjects were highly accurate in choosing the most valuable option in both cases (respective means = 96.7 ± 2.2% SD and 94.1 ± 2.7% SD; Mann–Whitney Wilcoxon test, U = 382.5, z = 3.29, p < 0.001). Subjects were also faster in making their choices in shape trials (mean 1.00 ± 0.34 s SD) than in bonus trials (mean 1.43 ± 0.60 s SD, Mann–Whitney Wilcoxon test, U = 43, z = 4.66, p < 0.001).
Neuroimaging results
Overall value encoding
The first goal of the fMRI analysis was to identify regions that encode the integrated value of the chosen stimulus across contexts. To do this we first estimated GLM 1 to test for regions showing BOLD responses that varied as a function of the relative values of the chosen and the nonchosen stimuli (i.e., chosen minus nonchosen). Figure 2 shows that activity in vmPFC, the ventral striatum and the PCC correlated positively with the difference between chosen and nonchosen stimulus value (p < 0.05, FWE-corrected at the cluster level; Fig. 2; Table 1). Activity in regions including the anterior insula, lateral prefrontal, anterior cingulate, and parietal cortices showed the opposite pattern, negatively correlating with the relative value of the chosen option (Table 1).
Overall value encoding. Activity in the vmPFC, ventral striatum, and PCC reflects the difference between chosen and nonchosen stimulus values at the time of choice. The figure shows statistical parametric maps of the random-effects analysis thresholded at a cluster-corrected FWE-threshold of p < 0.05, and projected on a template brain in MNI space. Colors indicate the magnitude of t values as shown by the color scale on the left. Circled regions were used as ROIs in further analyses.
Context-dependent value encoding
Next, we examined context-dependent changes in the encoding of shape and bonus attribute values. To do so, we estimated a second GLM (GLM 2) that measured changes in the effect for chosen and nonchosen shape value in bonus compared with shape trials, as well as the effect of chosen and nonchosen bonus values in bonus trials. Activity in both the vmPFC and PCC reflected relative bonus values during bonus trials in which those attributes contributed to the overall stimulus value, and thus, were important for optimal choices (p < 0.05, FWE-corrected at the cluster level; Table 2). Unlike bonus attributes, shape attributes always contributed to the overall stimulus value. However, the relative contribution of the shape to the overall stimulus value was greater in shape trials (shape alone determines value) compared with bonus trials (shape and pattern or color determine value). Consistent with the shape's contribution to the overall stimulus value, we found that, during shape trials, the activity in the vmPFC, ventral striatum, and PCC strongly correlated with the relative shape value (p < 0.05, FWE-corrected at the cluster level; Fig. 3A; Table 2). Furthermore, the correlation between BOLD activity and relative shape value significantly decreased during bonus compared with shape trials in both vmPFC and PCC (p < 0.05, FWE-corrected at the cluster level; Fig. 3B,C; Table 2). Thus, BOLD signals in vmPFC valuation networks increased or decreased their correlation with specific stimulus attributes in a manner consistent with context-dependent modulation of value computations.
Regions encoding the relative bonus value in bonus trials, the relative shape value in shape trials, and the relative shape value in bonus compared with shape trials
Default shape value encoding decreases in bonus contexts. A, B, Statistical parametric maps of the random effects analyses, color coded for the t values as indicated by the color bars, thresholded at a cluster-corrected FWE-threshold of p < 0.05, and projected on a template brain in MNI space. A, Activity in the vmPFC and PCC reflects the difference between the default shape value of the chosen and nonchosen option in shape trials. B, The default shape value encoding in these regions decreases in bonus compared with shape trials. C, The graphs show the mean (M) and SE of the β estimates for the shape values of the chosen, and nonchosen option in the indicated regions plotted for shape and bonus trials, separately.
Main effects of context
The second goal of the fMRI analysis was to identify areas that are sensitive to context changes that require a reweighting of the stimulus attributes to make an optimal choice. To do this, we tested the single-subject contrast images for bonus choices minus shape choices from GLM 1 in a one-sample t test. The resulting T-map shows that the bilateral dlPFC/IFG, inferior parietal lobule (IPL), anterior insula, dorsal anterior cingulate cortex (ACC), and ventrolateral PFC were more strongly activated for choices in the bonus than the shape condition (p < 0.05, FWE-corrected at the cluster level; Fig. 4; Table 3). Subjects made slightly fewer optimal choices and responded more slowly in bonus compared with shape trials suggesting that bonus trials may have been more difficult. To examine whether accuracy or reaction time (RT) could account for activation differences between bonus and shape trials, we reran the one-sample t test including accuracy and RT differences between bonus and shape trials as covariates. Including these covariates did not change the main effects of context reported above. In addition, computing a new GLM with an additional parametric regressor for RT on each trial also did not change the pattern of activation for bonus versus shape trials.
Context sensitivity. Activity in the dlPFC, IPL, anterior insula, dorsal ACC, and ventrolateral PFC is greater during choices in bonus compared with shape trials. The figure shows statistical parametric maps of the random-effects analysis, color-coded for the t values as indicated by the color bar, thresholded at a cluster-corrected FWE-threshold of p < 0.05, and projected on a template brain in MNI space. The circled region was used as ROI in the visualization of the functional connectivity effects in Figure 5.
Regions differentially active in bonus trials compared to shape trials
Changes in vmPFC functional connectivity with decision context
Our last goal for the fMRI analysis was to investigate changes in the functional coupling between vmPFC and dlPFC when context changes required reweighting of stimulus attributes to optimize choice outcomes. We did so by computing a PPI analysis (GLM-PPI) that included separate interactions between vmPFC BOLD signals and regressors representing both cue and decision time points. We first tested for changes in vmPFC connectivity during cues signaling a shift in decision context before the actual choice computations, and found that functional connectivity increased between vmPFC and the bilateral regions of dlPFC/IFG, IPL, and left anterior insula (p < 0.05, FWE-corrected at the cluster level; Fig. 5A,B; Table 4). Furthermore, a conjunction analysis showed considerable overlap between these dlPFC/IFG and parietal regions and the regions found to be more active for bonus compared with shape choices (Fig. 5C).
PPI effects seeded in the vmPFC when a context change is indicated. Functional coupling between the vmPFC (seed) and the dlPFC/IFG increases at display of the context cue, whereas coupling with the PCC decreases. A, Statistical parametric maps of the random-effects analysis, color coded for the t values as indicated by the color bar, thresholded at a cluster-corrected FWE-threshold of p < 0.05, and projected on a template brain in MNI space. B, Bar graph of the mean (M) and SE of the β estimates for the intrinsic coupling and cue PPI term for the vmPFC with the left dlPFC/IFG ROI circled in Figure 4. C, The PPI target regions (red), in particular the dlPFC/IFG, show substantial overlap (orange) with the regions that encode context sensitivity (yellow) as shown in Figure 4. Shown here are statistical parametric maps of the respective random-effects analysis and their overlap as indicated by the color legend, thresholds as specified in A.
Regions showing increased or decreased functional connectivity with the vmPFC during cue display
In addition, we sought to examine the vmPFC functional connectivity during bonus compared with shape choices, and especially whether this connectivity was related to the reweighting of bonus attributes (i.e., colors or patterns) during stimulus value computations. First, we computed a one-sample t test at the group level for the contrast bonus PPI minus shape PPI and found increased connectivity between vmPFC and the right parietal cortex, as well as decreased vmPFC connectivity with the PCC, the right superior temporal gyrus, and the ACC (p < 0.05, FWE-corrected at the cluster level; Table 5). Second, we computed a between-subjects correlation for the same bonus PPI minus shape PPI contrast and the β estimates for relative bonus value (GLM 2, contrast 3) in vmPFC (Fig. 2 for ROI). This analysis revealed that increased connectivity during bonus choices between vmPFC and left dlPFC/SFG, left dlPFC/MFG, PCC, and the left middle temporal gyrus was associated with stronger encoding of relative bonus values in vmPFC (Fig. 6; Table 6).
Regions showing increased or decreased functional connectivity with the vmPFC during choices in bonus compared with shape trials
Context-dependent vmPFC–dlPFC connectivity increases with the reweighting of stimulus attributes. The context-dependent connectivity (bonus PPI > shape PPI) between vmPFC and two regions in the dlPFC correlates positively with the relative bonus value encoding in the vmPFC at the time of choice. Top, Shown here are statistical parametric maps of the random-effects analysis (between-subjects correlation), color coded for the t values as indicated by the color bars, thresholded at a cluster-corrected FWE-threshold of p < 0.05, and projected on a template brain in MNI space. Bottom, Graphs show the single-subject means of the β estimates in the indicated regions for the bonus PPI minus shape PPI effect seeded in the vmPFC (x-axes) plotted against the single-subject means of the β estimates in the vmPFC for the difference between chosen and nonchosen bonus values (y-axes). The left and right scatter plots correspond to the left dlPFC/SFG and the left dlPFC/MFG clusters shown in the left and rightmost coronal sections.
Regions where the bonus PPI minus shape PPI contrast is associated with individual differences in vmPFC bonus value encoding
Discussion
This study demonstrates the contribution of dlPFC–vmPFC interactions to flexible, context-dependent value computations in the absence of opposing preferences that require self-control. We implemented a novel fMRI task in which context changes required subjects to adapt the integration of stimulus attributes to comply with the goal to maximize monetary rewards. Critically, our task did not involve any temporal tradeoffs or temptation conflicts. Consistent with their previously suggested roles in goal-directed choice (Rangel and Hare, 2010; Peters and Büchel, 2011; Bartra et al., 2013; Clithero and Rangel, 2014), the vmPFC encoded the relative values of the chosen options whereas the dlPFC responded to deviations from the default valuation context. Moreover, we showed that functional coupling between dlPFC and vmPFC increased when context changes required a reweighting of the stimulus attributes, and that the degree of dlPFC–vmPFC coupling was correlated with the signaling strength for context-specific attributes in vmPFC. These results suggest that dlPFC–vmPFC connectivity plays a pivotal role in context-dependent stimulus valuation that generalizes beyond decisions involving tradeoffs that require self-control.
For successful goal-directed decision making it is essential to evaluate the choice options in a context-dependent manner. We hypothesized that context specificity in stimulus evaluations may rely on a network of brain regions that we have previously proposed as a model of computational and neurobiological processes underlying self-control in the presence of competing preferences (Hare et al., 2009, 2011, 2014; Rangel and Hare, 2010; Harris et al., 2013). In this model, the brain first assesses stimulus attributes and then integrates the associated values into a net value. A key component of the model is that basic stimulus attributes (e.g., food taste) are always represented in the net value, whereas the values of more abstract or less dominant attributes are only integrated if they are relevant to the specific circumstances. To examine the generalizability of this framework for goal-directed choice beyond self-control situations, we designed a task that systematically varied decision contexts and the effects of stimulus attributes on the value of choice outcomes.
We found that the relative value of the chosen stimulus at the time of choice was encoded by vmPFC, ventral striatum, and PCC. Furthermore, the BOLD signal associated with the parametric encoding of a stimulus attribute was proportional to the contribution of that attribute to stimulus value. These findings are in line with previous research highlighting the role of vmPFC, ventral striatum, and albeit less consistently, PCC in the encoding of subjective value (Bartra et al., 2013; Clithero and Rangel, 2014). Together these previous findings suggest that disparate reward attributes and types can be integrated into value signals in the vmPFC. However, it should be noted that these observations have been made at the millimeter scale of fMRI resolution. Electrophysiological recordings in monkeys show that although vmPFC neurons do respond to different types of rewards, only a small proportion of neurons actually encodes an integrated value signal (Watson and Platt, 2012). Therefore, it will be important for future work to determine whether integrated value signals are primarily encoded at the single neuron or population levels. Our current results show that vmPFC and PCC encode and integrate values associated with stimulus attributes in a way that factors in the increased relevance of certain attributes in the light of context changes.
Compared with vmPFC, the role of PCC in value encoding has received less attention and is less well understood. The PCC is a prominent node in the default mode network and shows strong resting state coupling with the vmPFC (Fransson and Marrelec, 2008; Pearson et al., 2011). Anatomically, it is well positioned to integrate learned and remembered information while continuously detecting changes and monitoring reward outcomes (McCoy et al., 2003; Pearson et al., 2011). It has been suggested that PCC mediates the inhibition of parietal cortices, possibly to prevent distraction, and the activation of medial prefrontal cortex to increase attention to target stimuli (Small et al., 2003). Our data indicate that PCC and vmPFC are coactivated during value encoding. Interestingly, the functional coupling between vmPFC and PCC decreased in bonus choices, suggesting that these two regions may have independent functions in stimulus valuation and choice. Although the current study was not designed to dissociate vmPFC and PCC functions, future studies should endeavor to tease these functions apart.
In addition to changes in the value representations themselves, we examined brain regions that might signal the need to adjust the current or default manner of assigning values to each option. Note that in the paradigm used here, the function of such regions may include suppressing the response that is based only on the most salient attribute value (i.e., shape), but cannot be driven by preference conflicts (e.g., immediate gratification vs long-term benefits) because the goal of maximizing monetary payoffs today remains constant. Because we were particularly interested in dlPFC–vmPFC interactions when context changes initiate a reevaluation of the choice alternatives, we established a default attribute (shape) that always contributed to the net value and added bonus attributes (color, pattern) that became relevant only when specifically cued. We found that a cortical network of dlPFC, IPL, anterior insula, dorsal ACC, and ventrolateral PFC was more strongly activated for choices when the context required a deviation from the default valuation. Moreover, control analyses demonstrated that increased activity in these regions could not be explained by measures of task difficulty alone. In addition, several of these regions, but especially left dlPFC (IFG, MFG, and SFG), showed increased functional connectivity with vmPFC during context shifts.
The role of dlPFC in goal-directed choices is supported by growing evidence from fMRI and TMS studies (Knoch et al., 2006, 2008; Figner et al., 2010; Baumgartner et al., 2011). It has been shown that the dlPFC interacts with the vmPFC, and potentially modulates the valuation of food items in the vmPFC when self-control is used to comply with a dieting goal (Hare et al., 2009; Harris et al., 2013). Moreover, a combined TMS and fMRI study has shown that stimulation of the dlPFC leads to changes in activity in vmPFC during decisions in which fairness and monetary goals are in conflict (Baumgartner et al., 2011). Our study complements and extends this research by investigating context-dependent changes in valuation in the absence of any conflicting desires that might require self-regulation. We showed that dlPFC connectivity with vmPFC also increases in this more general case. Additionally, this connectivity increase is correlated with the context-initiated revaluation of the choice options in the vmPFC, such that context-specific reward attributes (e.g., color or pattern) are represented more strongly in vmPFC activity when dlPFC–vmPFC connectivity is higher. In line with previous work, our results suggest that dlPFC plays a critical role in detecting context changes and initiating the reweighting of stimulus attributes in vmPFC that is necessary for choosing the highest valued stimulus in a given choice situation.
In addition to goal-directed choice, a similar network of frontal, parietal, and cingulate regions has been associated with cue-driven, feature-based visual search (Egner et al., 2008). Particularly the posterior parietal cortex, the dorsal ACC, and the anterior insula have been shown to encode cues that indicate when a stimulus feature becomes relevant. This resembles our finding that these regions are more strongly activated when subjects need to attend to additional stimulus attributes. Such a mechanism of selective attention to stimulus features has also been demonstrated for choices under risk (FitzGerald et al., 2014) and learning in a changing environment (Wilson and Niv, 2011). The similarity of these networks across different aspects of cognition suggests that these regions play a fundamental role in optimizing behavior for specific environmental states and outcome goals.
Together, our data show that dlPFC–vmPFC connectivity during goal-directed choices is not specific to self-control contexts involving competing desires, but plays a broader role in flexible, contextually appropriate stimulus valuation. Although our findings in no way diminish the importance of dlPFC activity and coupling with vmPFC during choices that require self-control to resolve conflicts arising from opposing preferences (e.g., intertemporal, social norms), they do indicate that such dlPFC–vmPFC interactions cannot be equated with this type of self-control. Self-control choices represent an important set of decisions in which abstract and/or long-term benefits must be included in value computations together with more salient and/or immediate outcomes. Such decisions are of great importance because their outcomes directly affect a person's physical, social, and financial well being. Identifying neural mechanisms that are common not only to different self-control domains, but to context-dependent valuation in general may lead to novel targets and methods for interventions designed to support optimal goal-directed choices. Although it is often impractical and unethical to train individuals to resist temptation in the context of their specific choice pathology (e.g., gambling or drug addiction), practice in other context-dependent valuation paradigms might further develop this skill and facilitate its deployment during self-control challenges. The efficacy of such training programs remains to be determined, but in our opinion, they present an intriguing avenue for future research.
Footnotes
S.R. was supported by the German Academic Exchange Service (DAAD).
The authors declare no competing financial interests.
- Correspondence should be addressed to either of the following: Sarah Rudorf, Center for Economics and Neuroscience, University of Bonn, Nachtigallenweg 86, 53127 Bonn, Germany, rudorf{at}uni-bonn.de; or Todd Hare, Laboratory for Social and Neural Systems Research, Department of Economics, University of Zurich, Blümlisalpstrasse 10, 8006 Zürich, Switzerland, todd.hare{at}econ.uzh.ch
