Impulsivity and Self-Control during Intertemporal Decision Making Linked to the Neural Dynamics of Reward Value Representation

Apparatus.

E-Prime programs (Psychology Software Tools) controlled the behavioral task as well as the delivery of liquid rewards via a syringe pump (SP210iw; World Precision Instruments). Liquids from two 60 ml plastic syringes mounted on the pump were merged into one tube and then delivered to the participant's mouth through a plastic tube. The simultaneous use of two syringes allowed for a comfortable flow rate of 2.0 ml/s. The reward was delivered in 0.4 ml squirts but was experienced as a continuous flow. The amount of reward a participant received in a given trial was determined by the number of squirts.

Behavioral procedure.

The experiment consisted of two sessions (behavioral and fMRI) administered on separate days. In both of the sessions, participants were instructed not to drink any liquid for 4 h before the experiment. The behavioral session provided an estimate of each participant's delay-discounting rate. Previous work demonstrated that discounting behavior in this task is stable across sessions (Jimura et al., 2011). The results of the behavioral session were used for characterization of individual differences and optimization of the amount of immediate reward in the fMRI session (see fMRI session procedure, below). Before the experiment, participants were asked to choose one favorite drink that would serve as the reward from a list consisting of apple, orange, grape, grapefruit, and cranberry juices, lemonade, and water. No participants requested to change the reward drink in the fMRI session.

Behavioral task.

In both of the behavioral and fMRI sessions, participants performed the intertemporal decision-making task that we developed previously (Fig. 1A,B) (Jimura et al., 2009, 2011). At the beginning of each trial, two alternatives were presented at left and right locations on the screen: one involving a larger amount reward (20 or 40 squirts) available after a delay (10, 30, or 60 s) and a variable smaller amount available immediately. Participants were instructed to press one of two corresponding response buttons to indicate their preference. If the smaller, immediate amount was chosen, then a message appeared on the screen indicating that reward delivery could now begin. If the delayed reward was chosen, the participant had to wait to receive the reward. The choice period lasted from the onset of the choice presentation until participants' response, with the delay period starting immediately after the response. During the delay, a fixation cross was presented on the center of the screen. The background color of the screen was blue while the remaining time was 30–60 s, and red while remaining time 0–30 s. This was aimed to minimize the error of time estimation of reward outcome in the long delay condition.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Behavioral paradigm. A, Participants made a choice between larger amount of liquid available after seconds and smaller amount of liquid available immediately. B, In each trial, participants waited for the delay indicated after making a decision, and then drank the liquid before the next trial. The procedure was identical in the behavioral and fMRI sessions. C, Model-based value representation of future reward during the delay period. The subjective value model (orange) represents a gradual increase from the start of the delay toward reward outcome, whereas the anticipatory utility (blue) models an inverse pattern, a gradual decrease. The dotted and solid lines indicate economic and BOLD convolution models, respectively.

At the time of reward delivery, participants saw a visual message indicating the reward was ready. Importantly, participants were able to control the rate of liquid flow. Reward delivery continued as long as the button was held down; if the button was released, delivery paused and then resumed when the button was pressed again. During reward delivery, the amount remaining (in squirts) was displayed below a red horizontal bar whose length corresponded to the number of squirts still available. After the participant finished drinking, a fixation cross was presented.

Task trials had a constant onset asynchrony of 150 s independent of participants' choices. This procedure was implemented to prevent participants from selecting a strategy of repeatedly choosing the immediate option to maximize the reward attained per unit time (Fig. 1B). Because of this constant onset asynchrony, the optimal strategy to maximize reward rate is to always choose the delayed option. The duration of each trial thus varied, depending on participants' choice and the length of the delay period, from ∼24 to ∼84 s. Then, a variable intertrial interval (ITI) occurred, comprising both fixation and distractor task periods distributed across the ITI. The distractor tasks were included to minimize the effects of previous decisions, i.e., encouraging the treatment of each trial as a “one-shot” decision. Specifically, each task trial was followed by four pseudorandomized distractor trials, two trials of a Sternberg-type working memory task (Jimura et al., 2010, 2011) and two trials of a second delay-discounting task using hypothetical monetary rewards (Jimura et al., 2011). To inform participants of the type of task to be performed on the upcoming trial, a cue (i.e., “MONEY,” “JUICE,” or “MEMORY”) was presented for 2 s before each trial. Data from these distractor trials, concerning the relationship between working memory and delay discounting, will be described in a forthcoming report.

Behavioral session procedure.

Each participant performed the task sitting on a chair. To estimate individuals' delay-discounting rates, the current study used three delay conditions (10, 30, 60 s) for the larger amount (40 squirts), and two delay conditions (10, 30 s) for the smaller amount (20 squirts) (Jimura et al., 2009). The conditions were presented pseudorandomly, and the participants made three choices for each of the delay conditions. On the first trial of each delay condition, the choice was between a larger delayed amount and an immediate reward that was half of the delayed amount. For each delay condition, after the first trial, the amount of immediate reward was adjusted based on the participant's preceding choice. If the participant had chosen the smaller, immediate reward on the preceding trial, then the amount of the immediate reward was decreased by half (i.e., 10 and 5 squirts for the 40 and 20 squirt conditions, respectively); if the participant had chosen the larger, delayed reward on the preceding trial, then the amount of the immediate reward was increased by half (Jimura et al., 2009, 2011). The adjustment amount was five squirts in the third trial in the 40 squirt condition. The subjective value for the delayed reward was estimated to be equal to 1 ml (i.e., 2.5 squirts) more or less than the amount of immediate reward available on the last trial (third and second trials in 40 and 20 squirt conditions, respectively), depending on whether the delayed or immediate reward had been chosen on that trial. Although satiety and timing constraints resulted in only a few trials used at each delay and amount level for subjective value estimation, our previous work suggests that delay discounting in this paradigm is both robust and highly reliable at the individual level (test–retest reliability, r = 0.92) (Jimura et al., 2011).

Each experimental session began with two forced-choice trials and two practice trials, which familiarized participants with the choice procedure as well as with the rewards and delays. The syringes were refilled after every six trials, and the maximum amount of liquid that could be obtained in the session was 248 ml.

After the participants completed the task, they practiced drinking liquid though a plastic tube in a supine position in a mock scanner. They were instructed not to move their head while drinking, but were encouraged to move jaws and use muscles around the mouth to swallow the liquid. It is important that participants could fully control liquid flow into their mouth by holding down and releasing the button (see Behavioral task, above), which enabled the flow rate to be kept at a level that was comfortable without excess movement. After they completed five practice trials (three 16 ml and two 8 ml trials), they were asked whether they had any difficulties in drinking, and whether the practice drink was comfortable. No participants reported difficulties in drinking in the supine position or significant changes in comfort relative to a sitting position.

fMRI session procedure.

During fMRI scanning, participants engaged in intertemporal decision-making trials that were similar to those of the behavioral session. The primary difference was that the choice options for each trial were prespecified (rather than adjusted across the session), but set in an individualized manner based on discounting profile estimated from the behavioral session. Three conditions (60 s/40squirts; 30 s/40squirts; 30 s/20squirts) were used to measure brain activity during the delay period. The value of the IR was systematically manipulated such that, across trials, its value was one of the four different levels, 20, 40, 60, or 80%, of the subjective value of the delayed reward, estimated for each participant based on their choice profile in the behavioral session. This IR manipulation biased decisions toward delayed options, as the IR value was always smaller than the subjective value of delayed reward, providing more opportunity to measure brain activity during the delay period. Additionally, because the IR value was parametrically manipulated, it allowed examination of the effect of this variable on choice performance and brain activation.

When drinking liquid rewards, the participants were encouraged to use jaw movements and mouth muscles for swallowing, but not to move their head (a skill that had been practiced previously in a mock scanner setup; see Behavioral session procedure, above). Analyses of drinking related movements and their effects on image quality indicated that these were minimal (see Liquid delivery in fMRI session, below).

Each scanning run included six trials, with three or four scanning runs performed in the session, depending on the satiety of the participant. All trial variables (IR value, delay, amount of delayed rewards) were pseudorandomly intermixed within and across scans, with the constraint that the two middle IR levels were presented twice for each condition. Each experimental session began with two forced-choice trials and two practice trials, which familiarized participants with the choice procedure as well as with the rewards and delays. The maximum amount of liquid that could be obtained in the fMRI session was 344 ml.

It is important to note that the experimental design and procedure minimized any trial-by-trial learning and adjustment of choice biases across trials within the session. First, at the time of the scanning session, participants were highly familiar with the decision-making paradigm and associated reward consumption, having already experienced a previous behavioral session, mock-scanner training, and presession familiarization trials. Second, as described above, we previously found choice preferences to be highly stable across sessions (Jimura et al., 2011). Finally, the ITI (duration from the offset of liquid delivery to the onset of the next choice) was long (>1 min) and filled with distractor trials that disrupted memory for previous trial choices and encouraged participants to treat each trial as a one-shot decision.

Imaging procedures.

fMRI scanning was conducted on a whole-body Siemens 3T Trio System. A pillow and tape were used to minimize head movement in the head coil. Headphones dampened scanner noise and enabled communication with participants. Both anatomical and functional images were acquired from each participant. High-resolution anatomical images were acquired using an MP-RAGE T1-weighted sequence [repetition time (TR), 9.7 s; echo time (TE), 4.0 ms; flip angle (FA), 10°; slice thickness, 1 mm; in-plane resolution, 1 × 1 mm²]. Functional [blood oxygen level-dependent (BOLD)] images were acquired using a gradient echo-planar imaging sequence (TR, 2.0 s; TE, 27 ms; FA, 90°; slice thickness, 4 mm; in-plane resolution, 4 × 4 mm²; 34 slices) in parallel to the anterior–posterior commissure line, allowing complete brain coverage at a high signal-to-noise ratio. Each functional run involved 512 volume acquisitions.

Liquid delivery in fMRI session.

Liquid was delivered from the outside of the scanner room through a plastic tube, which enabled participants to consume the liquid during fMRI administration. The liquid consumption period partially covers the BOLD window associated with the (end of) delay period (Fig. 1C). Head motion was in fact greater during liquid consumption compared to button pressing during the money-discounting distractor task. However, the absolute magnitude of head motion during consumption was small, with maximum translations of (0.06, 0.09, 0.28 mm) ± (0.05, 0.05, 0.19 mm) (mean ± SD), and maximum rotations of (0.25, 0.08, 0.06) ± (0.17, 0.06, 0.03) degrees, along x-, y-, and z-axes, respectively.

Nonetheless, it is known that jaw movements can yield significant instability in echoplanar images, as based on nonhuman primate scanning (Keliris et al., 2007). However, inspections of our pilot scan showed such effects were minimal, due to the instructions and practice participants had with drinking in a supine position. In fact, instability may not have been present in the current study, as comparable EPI quality was present during consumption, compared to the fixation period. Finally, when examining BOLD activation during the consumption period, robust activations were observed, with prominent foci in somatotopic regions of mouth in the primary motor and sensory cortices, and in primary gustatory cortex. Given these results, we felt confident in assuming that movement-derived contamination during liquid consumption was minimal in the current study. Detailed results and activation maps of movement-related effects and consumption-related activity can be provided upon request.

Assessment of individual differences in delay-discounting effects.

From the behavioral session, the main effect of delay discounting was analyzed, first by conducting a repeated-measures one-way ANOVA with delay as a factor. A planned contrast on (log-) delay was then tested (Jimura et al., 2009, 2010). Participants were sorted into three groups—steep (STP), shallow (SHL), and intermediate (INT), based on visual inspection of their different discounting rates. The group effect was formally tested by entering group as an additional factor in the ANOVA. This test for group differences is more stringent than those based on model-fits of the data because it directly tests delay discounting (i.e., that the delay effect on subjective value will interact with group) without relying on any assumption regarding the direction or form of the effect.

Individual differences in delay discounting were quantified by calculating the area under the discounting curve (AuC) (Myerson et al., 2001; Sellitto et al., 2010; Jimura et al., 2011). The AuC represents the area under the observed subjective values at a given delay; more specifically, the AuC is calculated as the sum of the trapezoid areas under the indifference points normalized by the amount and delay (Myerson et al., 2001). Both subjective value and delay are normalized for purposes of calculating the AuC, which, as a result, ranges between 0.0 (maximally steep discounting) and 1.0 (no discounting). It has been argued that the AuC is the best measure of delay discounting to use for individual difference analyses, because it is theoretically neutral (i.e., assumption free) and also psychometrically reliable (Myerson et al., 2001).

An alternate approach to characterizing delay-discounting profile is to fit the discount function (either at the group or individual level) to a specific theoretical model. For example, the hyperbolic model of delay discounting, which is very popular and well validated (Rachlin et al., 1991; Green and Myerson, 2004; Kable and Glimcher, 2007), specifies that delay discounting can be characterized by the k parameter in the function A/(1+kD), where A is amount, and D is delay. The current study used AuC rather than k to characterize individual differences in delay discounting, because AuC is more psychometrically stable than k (Myerson et al., 2001) (see above). Nevertheless, the two measures tend to be highly similar in their characterization of individual differences in delay-discounting profiles. Indeed, this assumption was validated by a Spearman's rank-ordered correlation coefficient of 0.99 between individuals' discounting factor k and AuC for the current study. We did fit the group average discounting function with the hyperbolic model (using maximum likelihood estimation) so that a group-level k value could also be estimated; this value was subsequently used for model-based fMRI analyses of delay-period activity dynamics. However, the k value was not used in any individual difference analyses.

AuC was smaller in the smaller amount (20 squirts) condition than in the larger amount (40 squirts) condition (t₍₄₂₎ = 2.31; p < 0.05; based on two choices) (Jimura et al., 2009), replicating the amount effect observed in our previous report (Jimura et al., 2009). However, AuCs for the larger and smaller amount also correlated across participants (r = 0.61, t₍₄₁₎ = 4.91, p < 0.001), suggesting that individual differences in AuC were unrelated to the amount effect. Thus, for each participant, we averaged the AuCs for the larger and smaller amount conditions, and this averaged AuC was used in the imaging analysis to represent individual differences in delay discounting. Finally, it is important to note that AuC values and assignment to the discounting group were determined solely by performance in the behavioral session; therefore, individual differences analyses of the fMRI data (which were based on AuC) were unbiased.

Imaging analysis.

All functional images were first temporally aligned across the brain volume, corrected for movement using a rigid-body rotation and translation correction, and then registered to the participant's anatomical images to correct for movement between the anatomical and function scans. The data were then intensity normalized to a fixed value, resampled into 3 mm isotropic voxels, and spatially smoothed with a 9 mm full-width at half-maximum Gaussian kernel. Participants' anatomical images were transformed into standardized Talairach atlas space (Talairach and Tournoux, 1988) using a 12-dimensional affine transformation. The functional images were then registered to the reference brain using the alignment parameters derived for the anatomical scans.

A general linear model (GLM) approach was used to separately estimate parameter values for each event occurring in the task. Events were further convolved with a canonical hemodynamic response function. The focus of the analysis was on trials in which the delayed option was chosen. Consequently, both the distractor tasks and choice trials in which the immediate option was chosen were coded by events of no interest and not analyzed further.

On delayed option trials, a set of independent regressors coded for each of three trial periods: choice, delay, and reward delivery. The choice period was estimated by an event beginning with visual presentation of choice options and ending when the choice response was made. The duration of this period was thus equivalent to the choice reaction time. An additional parametric regressor during this period was used to code the IR value associated with the trial (four levels, normalized). Reward delivery was estimated with two events, a first transient event of no interest that coded the period during presentation of the instruction screen that indicated liquid was available, and a second longer event that coded (with a boxcar function) when the liquid was consumed (beginning with the first button press and ending when liquid delivery was completed).

The delay period before reward delivery was estimated by an event coded with two model-based regressors (Fig. 1C; additionally, on 60 s delay trials, a transient event of no interest was used to code the change in display color that occurred at the 30 s delay point). The first modeled a hypothesized dynamic increase in subjective value across the delay period, consistent with many neurocomputational and neuroeconomic models (Montague and Berns 2002; Berns et al., 2006; Kalenscher and Pennartz, 2008). We used a hyperbolic function to capture this pattern of delay-related increase, since this function has been used successfully in many theoretical accounts of delay discounting (Rachlin et al., 1991; Green and Myerson, 2004). To model delay-related SV dynamics with this function, we used SV = 1/(1 − kt), where t represents the time to reward delivery. The k parameter value was taken from the group mean discounting data in the behavioral session to provide a more stable fit of the model to delay-period activity. As illustrated in Figure 1C, the subjective value function gradually increases from the start of the delay toward the reward outcome. The second regressor coded AU, which reflects the positive utility derived from anticipation of a future reward (Loewenstein, 1987). Previous empirical and theoretical studies have postulated that AU is a complementary construct to SV, and thus maximal at the start of the waiting period and decreasing as the time to reward delivery grows near (Loewenstein, 1987; Berns et al., 2006). For simplicity, and to reflect the complementarity between AU and SV, we defined AU = 1 − SV. Importantly, although SV and AU are anticorrelated, when convolved with a hemodynamic response function, they produce dissociable BOLD regression functions (Fig. 1C). Specifically, the correlations between the SV and AU regressors were 0.18 and 0.51 for the 60 and 30 s conditions, respectively, allowing reasonable dissociation (cf. Otten et al., 2002). We also empirically double checked the dissociability in separate control GLM analyses in which only one of the two components was modeled. Highly similar results obtained when compared to the primary analyses, confirming that multicollinearity was not an issue.

It is worth noting that the form of the SV and AU representations incorporated in the analysis reflects a theoretical assumption, as some alternative models have postulated alternate functions to describe delay-discounting phenomena, such as exponential (Rachlin et al., 1991; Green and Myerson, 2004; Rangel et al., 2008) or subadditive (Read, 2001; Kable and Glimcher, 2010), rather than hyperbolic. Likewise, some accounts of AU postulate a linearly decreasing (Berns et al., 2006) or nonmonotonic delay function (i.e., peaking at some mid-delay point) (Loewenstein, 1987). Furthermore, some computational models do not even postulate that value representations are continuously maintained during delay periods (e.g., Brown et al., 1999; O'Reilly, 2006), and/or that value updates occur only in a cue-triggered fashion (O'Doherty et al., 2003; Redish, 2004; Seymour et al., 2004). Unfortunately, in the current data set we did not have sufficient statistical power to appropriately test and compare these alternative models in terms of their respective fits to the data. Thus, our use of the hyperbolic SV model, and the complementary AU model, is one of convenience, since these capture the broad qualitative characteristics of the functions we are interested in; we do not make any strong claims about the fit. Nevertheless, given the theoretical importance ascribed to distinctions between different discounting models, future research efforts could optimize the experimental design pioneered here to specifically focus on model comparison.

Parameter estimates for each subject were submitted to a group analysis using a voxelwise random-effects model. A whole-brain exploratory analysis was first performed to identify brain regions exhibiting subjective value and anticipatory utility effects during the delay period. Specifically, at each voxel, parameter estimates were identified that were significantly greater than zero at the group level (i.e., increased relative to baseline; p < 0.001), using cluster size thresholding to correct for multiple comparisons via the AlphaSim Monte Carlo procedure (http://afni.nimh.nih.gov/afni/). Significant voxel clusters were identified that surpassed a whole-brain corrected threshold of p < 0.05. Since this approach identified a large cluster that included both the vmPFC and VS (see Results) (Table 1), the VS effect was tested separately within an anatomically defined region of interest in VS created from TTatlas+tlrc Dataset (http://afni.nimh.nih.gov/afni/doc/misc/afni_ttatlas). Specifically, the VS was defined by voxels that fell within the region labeled as nucleus accumbens. Significance was then assessed by the threshold p < 0.05, corrected within the VS volume.

The analysis of IR value effects aimed to test whether the regions showing delay-period effects also showed choice-related effects in intertemporal decision making, as the IR manipulation provides a good estimate of decision value. Accordingly, to test for IR value effects, voxel clusters (p < 0.05 uncorrected) were identified within the regions of interest (ROIs) that showed delay effects. The vmPFC and VS ROIs were defined as regions showing the subjective value effect (p < 0.05, uncorrected), and anterior PFC (aPFC) ROIs were defined as regions showing the anticipatory utility effect (p < 0.05, uncorrected). Then, the size of the cluster was tested, correcting for multiple comparison within each ROI using a Monte Carlo procedure. Significant clusters were then reported using a volume-corrected threshold of p < 0.05. For individual difference effects, voxelwise Pearson correlation coefficients were computed between behavioral measurements of delay-discounting profile (i.e., AuC) and BOLD GLM parameter estimates for the choice period, IR value, delay period, and reward delivery effects. The significance of these voxel clusters was assessed using the same method as just described for IR value (for similar approaches, see Jimura and Braver, 2010; Jimura et al., 2010).

To illustrate the subjective value and anticipatory utility effects and their individual differences, the delay-related time courses were directly estimated (rather than fit to a specific model function; Figs. 3, 4). This was done via an independent first-level GLM analysis that removed the subjective value and anticipatory utility regressors from the model. After removing these regressors, the residual time courses were extracted from the delay period to visualize the effects that were present during this period. Before visualization, the time series was first subjected to a bandpass filter (high cut, 15 s; low cut, 150 s) to remove transient activity and low-frequency signal drifts.

Functional connectivity analysis: primary model.

A multilevel mixed effect general linear model (Raudenbush and Bryk, 2002) was used to examine functional connectivity relationships between vmPFC, VS, and aPFC. This type of multiple regression analysis does not directly test the causal pathway linking the regions, but instead assumes a particular connectivity pattern in which VS receives inputs from aPFC and vmPFC. We think this pattern is more plausible than other logically possible patterns. Specifically, as suggested by previous work, VS may represent the reward-related estimation error or discrepancy (Hare et al., 2008; Rolls et al., 2008; Peters and Büchel, 2010b; Daw et al., 2011) between the representation of SV signaled from vmPFC (Kable and Glimcher, 2007; Ballard and Knutson, 2009; Sellitto et al., 2010) and the current utility due to anticipation (i.e., the AU) signaled by aPFC (Koechlin and Hyafil, 2007; Schacter et al., 2007; Glimcher, 2009; Peters and Büchel, 2010b; Benoit et al., 2011).

The model included two levels, one for within-subject effects (i.e., IR value), and the other for between-subject effects (i.e., AuC). The within-subject level was modeled as follows: where β values indicate regression coefficients, X values indicate brain activity of each trial (see below), Y_IR denotes the amount of the IR on each trial, and ε is an error term. In this level, the VS activation was predicted by the aPFC and vmPFC activations (i.e., X_aPFC and X_vmPFC), the immediate reward (Y_IR), and their interactions with IR (i.e., Y_IR X_aPFC and Y_IR X_vmPFC).

Then, the higher-level between-subject effects were modeled as follows: where γ values indicate regression coefficients that were decomposed into interactions of the AuC and constant terms from the β values modeled in the lower-level model in Equation 1. In this level, the modeled parameters in Equation 4 were further predicted by the degree of delay discounting (AuC) of individuals. Because vmPFC did not show any individual differences effect of AuC (see Results), AuC-related influences were not included in Equations 4 and 5.

The ROIs were defined in a manner that was unbiased with respect to these effects: VS was defined anatomically in the left hemisphere based on the TTatlas+tlrc Dataset (see Imaging analysis, above); aPFC was defined in the left hemisphere, as a sphere with a 6 mm radius from the peak voxel of mean signal magnitude for the group-level AU effect; and vmPFC was defined similarly, but for the SV effect (for a similar approach, see Jimura et al., 2010).

Then in those ROIs, the fMRI time courses relative to fixation baseline were extracted from a 92 s epoch (62 s in short delay trials): 62 s (32 s) before the start of reward delivery to 30 s after the start of reward delivery. This time window fully covered the trial period from the onset of choices to when liquid consumption was complete. The activation data were high-pass filtered (cutoff, 150 s) to remove low-frequency signal drift. In the first analysis, all of the time points within the epoch were averaged together to create a single value for each trial. These values were extracted for each ROI, in each trial, and for each participant, and then included into the model, along with relevant coefficients coding the IR value for that trial and the AuC for that participant.

In a separate analysis, different models were constructed for each trial period separately (choice, delay, reward delivery). The choice period was defined as an epoch spanning from 62 s (or 32 s in short delay trials) to 52 s (or 22 s in short delay trials) before the start of reward delivery. The delay period was defined as 50 s (20 s) before the start of reward delivery to the start of reward delivery. The reward delivery period was defined as 2 s after the start of reward delivery to 30 s after the start of reward delivery. In the delay and reward delivery models, IR value effects were excluded as coefficients in the model (under the assumption that IR value effects did not influence these epochs). For each of the models, unknown parameters were all simultaneously estimated using the lme4 procedure in R (http://www.r-project.org).

Functional connectivity analysis control models.

To examine the specificity of model predictors in the multilevel mixed-effect analysis, the original model (i.e., VS activation influenced by aPFC and vmPFC inputs, and further modulated by IR and the AuC: Model I; see Multilevel mixed-effect GLM: primary model, above) was modified in the following ways and then tested.

In the first modification (Model II), vmPFC activation and interaction effects involving vmPFC were eliminated from the original model. In the second modification (Model III), aPFC activation and interaction terms were instead eliminated from the original model. These control models allowed to us examine whether the observed effects were due to the multicollinearity between the vmPFC and aPFC effects. It also allowed us to test whether inclusion of the aPFC and vmPFC improved model fitness in terms of Akaike's (1974) (AIC) and Bayesian information criteria (BIC) (Schwarz, 1978).

A second set of control models tested whether the results could be explained by averaging the epoch into a single value, or by the high degrees of freedom present in the model. To examine this, we tested models that involved replacement of aPFC- and vmPFC-related terms with plausible “substitute” brain regions that could serve as controls. Model IV replaced vmPFC with a lateral occipitotemoral region (LOT) that showed the SV effect during delay period, with a similar pattern to vmPFC (Table 1). Model V replaced aPFC instead, with a medial temporal lobe (MTL) region that showed the AU effect during delay period with a pattern similar to that of aPFC (Table 1). The regions of interest in LOT and MTL were defined by a homologous procedure used in the original model, as spheres with 6 mm radii from the peak voxel of mean signal magnitude in the subjective value (LOT) or anticipatory utility (MTL) effect across the participants.