Abstract
Large individual differences exist in the ability to delay gratification for the sake of satisfying longer-term goals. These individual differences are commonly assayed by studying intertemporal preferences, as revealed by choices between immediate and delayed rewards. In the brain, reward-based and goal-oriented decisions are believed to rely on the striatum and its interactions with other cortical and subcortical networks. However, it remains unknown which specific cortical-striatal tracts are involved in intertemporal decision making. We use connectivity analyses in both structural and functional MRI to further our understanding of the relationship between distinct corticostriatal networks and intertemporal preferences in humans. Our results revealed distinct striatal pathways that are differentially related to delay discounting. Structural and functional connectivity between striatum and lateral prefrontal cortex was associated with increased patience, whereas connectivity between subcortical areas and striatum was associated with increased impulsivity. These findings provide novel insights into how the anatomy and functioning of striatal circuits mediate individual differences in intertemporal choice.
Introduction
Decision making often requires the evaluation of outcomes that differ both in their magnitude and time of delivery (Ainslie, 1975; Loewenstein, 1996). Understanding individual differences in how people trade off reward magnitude and delay (i.e., delay discounting) is of high ecological and clinical relevance. Rates of delay discounting predict important outcomes in education, health, and savings and are abnormal in a range of psychiatric conditions, including addiction and attention-deficit hyperactivity disorder (Madden et al., 1997; Vuchinich and Simpson, 1998; Kirby and Petry, 1999; Mitchell, 1999; Marco et al., 2009; Shiels et al., 2009).
Research has suggested that multiple cognitive and neural processes underlie delay discounting (Kalenscher and Pennartz, 2008; Peters and Büchel, 2011; van den Bos and McClure, 2013). For example, more impulsive behavior can result from steep discounting of future rewards and/or oversensitivity to immediate rewards (McClure et al., 2004, 2007). Likewise, more patient behavior may result from control processes that bias attention away from immediate rewards and/or emphasize long-term goals (Hare et al., 2009; Figner et al., 2010; Hutcherson et al., 2012). It is likely that these processes operate in parallel while making intertemporal decisions (i.e., impulsivity could be the result of heightened reward sensitivity, reduced attention, or both). It is therefore hard to disentangle how specific processes contribute to delay discounting by studying behavior alone.
Studies have consistently shown that delay discounting involves cortical-basal ganglia circuits (for review, see Peters and Büchel, 2011). These circuits can be divided into two networks: a valuation network that is involved in estimating the incentive value of the different options and a control network that is involved in action selection, maintaining future goals, and inhibiting prepotent responses (Haber and Knutson, 2010; Kable and Glimcher, 2009; Figner et al., 2010; Miller et al., 2013; van den Bos and McClure, 2013). Important nodes in the valuation network include regions associated with the mesolimbic dopamine system, particularly the ventral striatum, amygdala, and ventromedial prefrontal cortex, whereas the control network includes the dorsal striatum, dorsal anterior cingulate cortex (dACC), dorsal and ventral lateral prefrontal cortex (dlPFC/vlPFC), and the posterior parietal cortex (pPC) (Peters and Buchel, 2011). Interactions between valuation and control networks likely occur in the striatum (Haber and Knutson, 2010). Some initial evidence for this can be found in recent studies showing that the structural integrity of the complete corticostriatal tract correlates with individual differences in discounting (Peper et al., 2013) and ADHD status (Liston et al., 2011). However, it is still unknown how separate cortical basal ganglia tracts specifically contribute to delay discounting. Furthermore, it is not understood how structural differences in these tracts are related to differences in brain function or whether trial-to-trial variations in functional connectivity between these brain regions account for variability in choices.
We address these questions using a combination of diffusion tensor imaging (DTI) and functional magnetic resonance imaging (fMRI) during a delay discounting task with real monetary rewards (Fig. 1). We related functional and anatomical measures of brain connectivity to delay discounting and a behavioral measure of time perspective (Zimbardo and Boyd, 1999).
Materials and Methods
Participants
We recruited 22 participants (12 female, mean age = 20.38 years, SD = 2.23) from a paid participant pool maintained by the Stanford University Psychology Department. Participants were paid $20 for participating in the MRI experiment, plus earnings from the discounting task (see below). The study was approved by the Stanford University Institutional Review Board and all participants gave written, informed consent before completing the task.
Participants completed the Raven Standard Progressive Matrices (Raven, 1941) for an estimate of their IQ and performed the stop-signal task (Verbruggen et al., 2008) to get a measure of response inhibition (stop signal reaction time, SSRT). We tested for possible sex differences in the analyses, but none was significant (or trending) for any of the brain-based analyses below, discount rates were not different, and no differences were found in IQ or SSRT. We also did not find correlations among IQ, SSRT, and discounting behavior (r = 0.24, p = 0.14 and r = 0.04, p = 0.72 respectively). It is likely that absence of a relation with IQ is the result of having a rather narrow IQ distribution that is at the high end of the curve (mean IQ = 116.8, SD = 8.4).
Behavioral measures
Delay discounting task.
Participants completed a total of 130 intertemporal choices inside and outside of the scanner (Fig. 1). The 60 trials outside of the scanner were determined by a staircase procedure. For this, the sooner smaller (SS) reward was fixed to $10 received today. The delay period (D) for the larger later (LL) reward was randomly chosen from a uniform distribution between 15 and 60 d in the future. The size of the LL reward was adjusted to converge toward the same subjective value as the SS outcome (V, in this task $10). We assumed that delay discounting is captured by a hyperbolic function: where A is the amount in dollars of the LL reward. The initial discount factor k was set to 0.02 and was increased or decreased when the participant chose the SS or LL option, respectively. For the first 20 rounds, the step size for changes is k was set to 0.01 and after that the step size decreased by 5% for each following step. After the participants completed 60 choices, we used the multivariate constrained minimization function (fmincon) of the optimization toolbox implemented in MATLAB for model fitting. To model trial-by-trial choices, we used a softmax function to compute the probability (PSS) of choosing the SS option on trial t as a function of the difference in VSS and VLL: where m is the decision slope and estimates response noise. Individual discount factors were determined as the value of k that maximized the likelihood of the observed choices.
The individual discount factors that resulted from this procedure were used to generate a subset of the choices in the delay discounting task that was presented to participants in the scanner (see below). The SS delays in the fMRI task included 0 (today) and 14 d and the LL delays included 14, 28, and 42 d. The different delays were equally divided over a total of 70 trials, resulting in 35 trials in which the SS was today and 35 trials in which the SS option was in the future (14 d). The SS rewards were randomly selected from a uniform distribution between $10 and $75. Following earlier studies (McClure et al., 2004, 2007; Figner et al., 2010), we determined LL reward size by adding a fixed percentage to the SS amount (.5%, 1%, 5%, 10%, 15%, 20%, 25%, 30%, 50%, or 75%) for 48 of the 70 trials. Next, for 22 of the 70 choices, we set the reward size of the LL exactly at the individually estimated indifference point using the estimated discount factors (cf. Rodriguez et al., 2014). These LL choices were randomly distributed over the task. At the end of the experiment, one trial was randomly chosen from the total of set of 130 choices and paid to the participant in the form of a (postdated) check. Note that there was a high correlation between estimated discount rates (k) when estimated separately for the prescanner and scanner tasks (r = 0.88, p < 0.001).
To get an individual difference measure of delay discounting behavior, we fit the hyperbolic discounting function (Equation 1) to each individual's aggregate behavior of the two discounting tasks using the fitting procedure described above. Next we log-transformed the discount factors given that the k estimates were not normally distributed.
Zimbardo Time Perspective Inventory.
There is accumulating evidence that differences in time perspectives are one of the important psychological constructs underlying individual differences in delay discounting (Bickel et al., 2006; Erbert and Prelec, 2007; Wittmann and Paulus, 2008; Zauberman et al., 2009; Radu et al., 2011). For example, opioid-dependent participants, who showed dramatically larger discount rates than a matched control group, also scored significantly lower on the future orientation (FO) scale and significantly higher on the present hedonism (PH) scale of the Zimbardo Time Perception Inventory (ZTPI) (Petry et al., 1998; Zimbardo and Boyd, 1999). Furthermore, individual differences in future thinking modulate the effect of temporal priming (Peters and Büchel, 2010) and predict developmental changes in discounting (Steinberg et al., 2009). Based on these findings, we assayed time perspective using the FO and PH subscales of the ZTPI.
The ZTPI was administered as an online questionnaire several days before the MRI experiment. Participants indicated on a 1–5 scale how applicable each of the 56 items was to him/herself. Items that comprised the FO scale included: “I believe that a person's day should be planned ahead each morning,” and “Thinking about the future is pleasant to me.” Example of items on the PH scale are as follows: “I feel that it's more important to enjoy what you are doing than to get the work done on time” and “I try to live my life as fully as possible, one day at a time.”
MR data acquisition and preprocessing
MRI data.
MR data were collected on a 3T GE Discovery MR750 scanner located at Stanford Center for Cognitive and Neurobiological Imaging. High-resolution T1-weighted images were first acquired (0.47 × 0.47 × 0.9 mm3, TR = 8.67 ms, TE = 3.47 ms, flip angle = 12°).
Diffusion-weighted imaging data.
Diffusion-weighted imaging (DWI) was performed at a resolution of 0.85 × 0.85 × 2 mm3, with 3 repeats of the b0 (no diffusion weighting image) and 2 repeats of each of 30 gradient directions at b1000 (TR = 9 s, TE = 89 ms). The FMRIB Diffusion Toolbox (FDT, http://www.fmrib.ox.ac.uk/fsl/fdt) was used to correct the DTI data for head movement and eddy currents, tensor model fitting, and generating fractional anisotropy (FA) maps. Data from the two acquisitions of each diffusion direction were averaged to improve the signal-to-noise ratio.
fMRI data.
Whole-brain, BOLD-weighted echoplanar (TR = 2000 ms, TE = 30 ms, flip angle = 77°, 44 total slices with 2 mm slice gap, 64 × 64 matrix) images were then acquired ∼30 degrees off the anterior commissure–posterior commissure plane to maximize signal in the ventral prefrontal cortex and ventral striatum. fMRI data were analyzed using SPM8 (http://www.fil.ion.ucl.ac.uk/spm/). The first five volumes were not analyzed to accommodate T1 equilibration. Given the known problems of motion for connectivity analyses (Power et al., 2012), we used ArtRepair software to correct for excessive movement (Mazaika, 2007). Images were realigned in ArtRepair to correct for movement, smoothed with a 4 mm full-width half-maximum Gaussian kernel, and motion adjusted. Deviant volumes resulting from sharp movement or spikes in the global signal were then interpolated using the two adjacent scans. No >8 (median = 0, mean = 0.9) of the volumes were interpolated within any subject. We then applied slice-timing correction to all images. Next, motion correction to the first functional scan was performed using a six-parameter rigid-body transformation. The motion-corrected images was coregistered to each individual's structural MRI using a 12-parameter affine transformation. Images were then resampled into 3 × 3 × 3 mm3 voxels and spatially normalized to the Montreal Neurological Institute (MNI) template by applying a 12-parameter affine transformation. Images were finally smoothed with a 4 mm isotropic Gaussian kernel and adjusted for global signal variation using a voxel-level linear model of the global signal.
Structural connectivity analyses
Tractography.
All diffusion image preprocessing and analyses were conducted using a combination of FSL tools and Nipy code (http://nipy.sourceforge.net/nipype/). Earlier studies have shown that the overall structural integrity of striatal fiber tracts is related to individual differences in impulsive behavior (Peper et al., 2013). Other studies have demonstrated that specific subsets of striatal tracts may be related to subcomponents of behavior (Behrens et al., 2003; Cohen et al., 2009; Draganski et al., 2008; Tziortzi et al., 2014). For example, the study by Cohen et al. (2008) used connectivity-based segmentation to show that novelty seeking was associated with connectivity strength of the subcorticostriatal tracts, but not with the corticostriatal tracts. Interestingly, their self-report measure of novelty-seeking relates (conceptually) to behavioral measures of impulsive behavior. The goal of our structural connectivity analyses was to identify specific corticostriatal and subcorticostriatal tracts. Next, we aimed to relate measures of structural connectivity strength with both functional connectivity and individual differences in discount rates.
FA is a measure commonly used in DTI studies to relate anatomical differences in white matter to behavior. Analyses of FA values are performed across the whole brain to identify regions of white matter related to behavior independent of particular axonal/fascicle tracts. In studies such as this one, in which we have particular interest in white matter pathways, seeded tractography is a superior approach because it allows analyses to target particular tracts in their entirety. Values derived from seeded tractography estimate the strength of pathways as a whole (Behrens et al., 2003; Cohen et al., 2009; Draganski et al., 2008; Tziortzi et al., 2014) and therefore provide a more direct test of hypotheses than is possible from voxel-level FA values.
Tractography analyses were performed in the subjects' native anatomical space and the results were output in MNI space by providing transformation parameters estimated via a 2-step procedure. First, the FA image was registered to each subject's high-resolution T1-weighted image with six degrees of freedom and a mutual information cost function. Next, the T1-weighted image was registered to the 1 × 1 × 1 mm3 MNI template using a nonlinear warping algorithm. The transformation parameters obtained from these two steps were concatenated to yield the mapping from the DWI to MNI space. The FDT toolbox was used to perform probabilistic tractography with a partial volume model (Behrens et al., 2003), allowing for up to two fiber directions in each voxel (Behrens et al., 2007). Dual-fiber models account for crossing fibers and thus yield more reliable results compared with single-fiber models. Five thousand sample tracts were generated from each voxel in the seed mask (striatum). Visual inspection ensured that tractography maps were successful and acceptable for further analysis. Tractography was performed separately for the left and right striatum and possible tracts were restricted to the hemisphere of origin using an exclusion mask of the contralateral hemisphere. The tractography results were used to identify anatomically distinct striatal segments in the segmentation step, next.
Segmentation.
To assess connectivity with extrastriatal regions, we used a set of 10 a priori masks defined as in Cohen et al., 2009 (Fig. 2A). These regions were based on single or a combination of the standard automated anatomical labeling (AAL) maps (AAL map number within parentheses): medial orbitofrontal cortex (mOFC: 28, 6, 26), vlPFC (10, 16), inferior frontal gyrus (pars triangularis, IFG: 14), dlPFC (4, 8), posterior cingulate cortex (68), ACC (32), dorsal ACC (dACC: 34), hippocampus (38), amygdala (42), and supplementary motor area (SMA: 20). These AAL numbers all correspond to the left hemisphere (subtract 1 for right hemisphere values). The striatum mask was obtained from the subcortical segmentation derived with freesurfer, to better account for anatomical hetereogeneity across subjects. Seed-based classification was done by first thresholding the images such that only voxels with at least 10 tracts terminating in one of the target regions were kept (Cohen et al., 2009; Forstmann et al., 2012). Finally, to tailor the cortical ROIs to the participants' individual anatomy, individually segmented gray matter (GM) and FA images were used to mask the ROIs. Following Tziortzi et al. (2013), the lower threshold for the GM mask was set at 0.25 and the FA mask upper threshold was set at 0.40. For the estimation of tract strength between the striatum and the target areas, MNI-space masks were normalized to each participant's native space using the inverse of the normalization parameters.
Following standard procedures, voxel values were converted into proportions; the value at each voxel was calculated as the number of tracts reaching the target mask for that voxel, divided by the number of tracts generated from the voxel (maximum 5000). This resulted in 10 value maps (for examples, see Fig. 2B), one for each target region, per participant. Finally, the striatum was segmented by assigning each voxel to the region with which it had the highest connection probability (Fig. 3, middle) using FSL utilities (Behrens et al., 2003; Johansen-Berg et al., 2004; Cohen et al., 2009; Tziortzi et al., 2014).
Note that the resulting segments are based on relative comparisons in connection strengths. Therefore, assigning a voxel to one target area does not exclude the possibility that tracts were found connecting to other target areas (for discussion, see Tziortzi et al., 2014). However, there are several reasons to believe the subdivision derived from one-to-one assignment is meaningful and helpful for better understanding of the functional subdivisions of the striatum and the related tracts. First, the ventromedial-to-dorsolateral connectivity profile resulting from the connectivity based segmentation procedure is consistent with the corticostriatal circuits previously identified in primates and other human diffusion MRI studies (Alexander et al., 1986; Behrens et al., 2003, Draganski et al., 2008 Cohen et al., 2009). Furthermore, a recent study using the same segmentation procedure showed that the homogeneity of dopamine release was significantly higher within the connectivity based segments compared with the classical structural subdivisions of the striatum (Tziortzi et al., 2014), supporting the hypothesis that the segmentation resulted in a meaningful functional subdivision of the striatum.
DICE coefficients.
To assess the intersubject spatial consistency of the striatal segments arithmetically, we calculated the DICE coefficient, which measures the volume overlap of striatal subdivisions across subjects. The DICE coefficient was estimated across subjects as the average overlap between a subject's striatal segment with striatal segments from all other subjects to determine whether the method and estimated connections were reproducible across subjects (individual subjects' scans were nonlinearly registered to the MNI template). The DICE coefficient was calculated as follows: where ∩ is the intersection of the ROI volumes and the norm (||) measures the number of voxels in the ROIs. The indices s and i refer to region number i in subject s. Note that the DICE value is calculate for each subject-region contrasted with the average from the group (i.e., ROIgroup,i) The DICE coefficient ranges from 0% for ROIs with no overlap up to 100% for identical ROIs. DICE values obtained for all striatal segments were high and satisfactory (between 0.36 and 0.65), except for regions connected to SMA and pPC (<0.05, see Table 1). Furthermore, for 4 and 5 participants (respectively), there were zero striatal voxels that had the highest probability of connection to the SMA and pPC. For this reason, these two target areas were excluded from further analyses.
Tract strength correlations with discounting.
To determine whether individual differences in discounting were related to the strength of white matter fiber tracts, we calculated the mean value of the tract probability within each individually determined striatal segment (8 per hemisphere, 16 in total). Although the voxels of each segment were determined based on relative comparisons of probability maps, the connection strength is not relative, but is the proportion of all generated tracts (5000 per voxel) that terminated in the corresponding target region. The resulting tract strength measure was correlated across subjects with log(k) using Spearman's rank-order correlation (tract strength values are non-normally distributed, so nonparametric correlations are most appropriate) with the following covariates: age, IQ, total intracranial volume, and size (number of voxels) of the individually defined striatal segment. To account for possible effects driven by outliers, we also performed robust regression analyses with the Huber weighting function (using the robustfit algorithm in MATLAB) after log transforming tract probability values, which yielded the same pattern of results. Effects were considered significant at an α of 3.125 × 10−3, based on Bonferroni correction for multiple comparisons (i.e., p = 0.05/16 striatal segments).
Functional connectivity analyses
To further explore how structural connectivity measures are related to neural mechanisms that underlie differences in discounting, we assessed functional connectivity of corticostriatal regions identified in DTI tracts using psychophysiological interaction (PPI) analysis. We hypothesized that there would be two possible ways in which structural connectivity might be related to individual differences in discounting. First, structural connectivity may be associated with an overall (mean) increase in functional coupling during the task, which biases all choices toward immediate or delayed rewards. Therefore, in our first set of analyses (PPI Model 1) we focused on functional connectivity between the striatum and those target areas that were identified in fMRI analyses to be involved in the discounting task. We generated a functional map to select the target areas that were involved in intertemporal decision making. Resultant functional ROIs were then matched to their structurally defined segments. For identified pairs of ROIs, we then tested for increased functional connectivity during the decision phase of the task relative to baseline and, if significant, whether this increased connectivity was also related to individual differences in tract strength and discount rates (k).
Second, we hypothesized that structural connectivity might be associated with more efficient processing of signals originating from those brain areas that are specifically involved in choosing LL over SS rewards (or vice versa). To test this hypothesis (PPI Model 2), we focused on functional connectivity between the striatum and those target areas that showed differential responses when participants made LL versus SS choices. Again, we generated a functional map to select the target areas that were involved in choosing LL and SS rewards and these functional ROIs were matched to their structurally defined segments. For these pairs of ROIs, we then tested for differential functional connectivity based on choice outcome and, if significant, whether this increased connectivity was also related to individual measures in tract strength and discount rate. These statistical analyses were performed using the PPI toolbox of SPM8.
PPI model 1
The fMRI time series data were modeled by a series of events convolved with a canonical hemodynamic response function (HRF). We set up a general linear model (GLM) with a single regressor for the decision phase, which was modeled as a fixed event of 3 TR duration (6 s). We used individual model-based parameters estimates of discount rate (k) to generate trial-by-trial measures of the total subjective value of the chosen option (Vchoice) and the difference in subjective value between the SS and LL choice (Vdiff; i.e., inverse of decision conflict or difficulty). These two measures were entered as covariates of no interest. The model also included session constants and motion parameters as regressors of no interest. All regressors were convolved with the canonical HRF and regressed against the BOLD signal. This GLM was used to generate first-level, single-subject contrast maps that identified voxels that were generally involved in making a decision (choice − baseline) while controlling for the subjective value of the chosen option and the amount decision conflict. Finally, we calculated second-level group contrasts using one-sample t tests on the single-subject contrasts (Fig. 4).
Next, we determined whether there was significant task-related activation (choice − baseline) in the cortical and subcortical masks that were used for the structural segmentation. This analysis revealed significant responses in the bilateral dlPFC, bilateral IFG, bilateral amygdala, bilateral ACC, and bilateral dACC (FWE corrected, p < 0.05). We extracted the mean BOLD time series from the voxels within a 6-mm-radius sphere surrounding the activation peak of the [choice − baseline] contrast within each cortical target masks (Table 1). These spheres were combined with individual GM masks to ensure that analyses did not include signals from nonbrain or white matter voxels. Given the high correlation between time series in the bilateral amygdala, dACC, and ACC regions (r = 0.89, r = 0.87, and r = 0.91, respectively), we extracted the mean BOLD signal from the sphere surrounding the peak voxel within combined (bilateral) functional masks for these regions. Naturally, we also combined the corresponding bilateral striatal segments associated with the amygdala and ACC regions for comparative analyses. Variance associated with the six motion regressors was removed from the extracted time series. The time courses were then deconvolved based on the model for the canonical hemodynamic response to construct a time series of neural activity following the procedures outlined in Gitelman et al. (2003).
Subsequently, for the PPI analysis, we estimated a GLM for every subject that included the following three regressors in addition to the motion parameters and the two covariates of no interest (Vchoice and Vdiff): (1) an interaction between mean BOLD response in the sphere centered on each target area and the mean centered decision phase regressor convolved with the canonical HRF; (2) a regressor specifying decision phases as an indicator function convolved with the canonical HRF; and (3) the original BOLD eigenvariate from the target area (i.e., the first principal component of time-series from the voxels within the 6 mm sphere). Single-subject contrasts were calculated after estimation of the GLM.
Finally, we used the Marsbar toolbox to extract mean PPI coefficients from first-level, single-subject contrasts using the striatal segments that corresponded with the target area used for the PPI analyses. Overall, this analysis procedure produced highly individualized measures of functional connectivity within corticostriatal and limbic-striatal tracts. These PPI coefficients were correlated with measures of structural connectivity and discount factors.
PPI model 2
The GLMs for PPI Model 2 was the same as PPI Model 1, except that we added a choice regressor that was 1 for choices in which subjects indicated a preference for the LL reward and 0 when the SS reward was chosen. First, we determined the areas that were involved in choosing LL over SS (and vice versa). These analyses revealed that there was significant activation in the right dlPFC, cerebellum, and visual cortex for the LL > SS contrast (Table 1). We did not find any significant responses in the brain for the SS > LL contrast (both FWE corrected, p < 0.05). Finally, we extracted the mean BOLD time series from the voxels within a 6-mm-radius sphere surrounding the activation peak of the right dlPFC for the PPI analyses. Following the procedure of PPI Model 1, the PPI coefficients extracted from the striatal segment were correlated with measures of structural connectivity and discount rates.
The significance of all whole-brain analyses are reported corrected for multiple comparisons using the familywise error (p < 0.05) as implemented in SPM8. The following right dlPFC, amygdala, and left IFG PPIs were constrained to those tracts that showed a significant structure–behavior relationship. Therefore, results were considered significant at an α of 0.016, based on Bonferroni correction for multiple comparisons (i.e., p = 0.05/3 tracts).
Replication data
To strengthen the conclusions about the relations between structural connectivity and individual differences in temporal discounting, we added a replication sample of an additional 23 participants (15 female, mean age = 24.6 years, SD = 7.6) from a paid participant pool maintained by the Stanford University Psychology Department. Participants were paid $20 for participating in the MRI experiment, plus earnings from the discounting task. The study was approved by the Stanford University Institutional Review Board and all participants gave written, informed consent before completing the task. The participants performed the same prescanning discounting task as described above.
MR data were collected on the same 3T GE Discovery MR750 scanner located at Stanford Center for Cognitive and Neurobiological Imaging. High-resolution T1-images weighted images were first acquired (0.47 × 0.47 × 0.9 mm3, TR = 8.67 ms, TE = 3.47 ms, flip angle = 12°). DWI was performed at a resolution of 0.85 × 0.85 × 2 mm3, with 3 repeats of the b0 (no diffusion weighting image) and 1 repeat of 60 gradient directions at b1000 (TR = 9 s, TE = 89 ms).
Results
Behavioral results
We assayed intertemporal preferences in two ways. First, we had subjects complete a number of monetary intertemporal choices. For this task, we asked people to indicate their preferences between pairs of smaller, sooner and larger, later payments. One of the selections was paid after the experiment. We assumed a hyperbolic discount function and determined each subject's best-fitting discount rate (k) for use as a summary measure of delay discounting. We analyze log-transformed discount rates in all analyses below because log(k) is closer to a normal distribution than is the distribution of k values. Second, we assessed subjects' attitudes toward immediate and delayed outcomes using the ZTPI, a questionnaire that measures independent factors of PH and FO.
Discount rates varied by an order of magnitude across study participants (kmin = 0.009 and kmax = 0.11; Fig. 1B). These discount rates were also related to self-reported measures of PH and FO. Specifically, the higher the participants' score on PH, the higher their discount rate (r = 0.37, p < 0.02; Fig. 1C); conversely, the higher the participants' score on FO, the more patient were their choices (r = −0.39, p < 0.01; Fig. 1C). PH and FO were not correlated (r = 0.26, p = 0.54), consistent with Zimbardo and Boyd's (1999) assertion that they capture independent factors in reward valuation. Moreover, the difference score between FO and PH was a stronger predictor of discounting (r = −0.81, p < 0.001) than were FO and PH individually (z = 2.18, p < 0.016 and z = 2.44. p < 0.009, respectively). Together, these results suggest that discounting was associated with two independent and complementary personality traits.
Imaging results
Our primary interest was in identifying roles for separate corticostriatal and subcorticostriatal loops in intertemporal choice. We did this by way of a multipart analysis that first identified ROIs using correlations between discounting behavior and anatomy and then tested for consistent relationships between behavior and functional brain activity in these ROIs acquired during a delay discounting task.
We began by determining the anatomical organization of the human striatum on the basis of the relative strength of white-matter fiber connections from cortical and subcortical seed regions by applying a classification procedure to label each voxel in the striatum according to the target structure to which that the voxel was most strongly connected (Behrens et al., 2003; Cohen et al., 2009; Draganski et al., 2008; Tziortzi et al., 2014). This procedure resulted in individual segmentation maps with a ventromedial-to-dorsolateral gradient that was organized in bands of similar connectivity to subcortical and cortical regions (Fig. 2). Next, these individual segmentation maps were used to generate tract strength scores by calculating the mean value of connection probability for each identified striatal segment in each subject. We then tested for a relationship between tract strength and individual discount rates, log(k). Finally, based on these behavioral correlations, we selected individualized anatomical ROIs for functional connectivity (PPI) analyses to further investigate the relationship between structural/functional connectivity and discounting behavior.
Structural connectivity
Using DTI and probabilistic tractography, we defined a measure of connection strength between the right and left striatum and a set of cortical and subcortical targets (Fig. 2A). Tract strength was correlated across subjects with log(k) using Spearman's rank-order correlation (tract strength values are non-normally distributed) while accounting for the following covariates: age, IQ, and size of the individually defined striatal segment. Individual differences in discounting behavior were negatively associated with tract strengths linking to the right dlPFC (r = −0.66, p < 0.001) and left IFG (r = −0.58 p < 0.001) and were positively associated with tract strength to the bilateral amygdala (r = 0.52, p < 0.001; Fig. 3). Trending correlations were found for the left dlPFC (r = −0.40, p = 0.008), right vlPFC (r = −0.46, p = 0.006), left mOFC (r = 0.37, p = 0.01), and right hippocampus (r = 0.40, p = 0.01), however these areas did not survive Bonferroni correction for multiple comparisons. It is notable that increased striatal connection strength with dorsal-lateral areas (including dlPFC and IFG) was associated with more patient behavior, whereas connection strength with ventral-medial and limbic areas (including the bilateral amygdala) was associated with more impulsive behavior. Interestingly, the strengths of these tracts were not correlated. There was a trending correlation between right dlPFC and left IFG connectivity strength (r = 0.36, p = 0.12), but neither was correlated with the amygdala tract strength (r = 0.03, p = 0.12 and r = −0.25, p = 0.25, respectively).
Next, we tested whether the connectivity strength of these tracts was associated with individual differences in FO and PH. To determine whether the correlations with FO and PH were significantly different, we performed pairwise comparisons of the Fisher's Z-transformed correlation coefficients. These analyses revealed that there was a significant dissociation between dorsolateral and ventromedial striatal tracts and individual differences in FO and PH (Fig. 5). Connection strength with the dorsolateral areas was associated with significant higher scores on FO compared with PH, whereas connection strength with the ventromedial areas was associated with higher scores on PH compared with FO (FO > PH: dlPFC, z = 2.52, p < 0.005; vlPFC, z = 2.49, p < 0.006; IFG, z = 2.48, p < 0.006. PH > FO: amygdala, z = 3.12, p < 0.001; hippocampus, z = 2.16, p < 0.017; mOFC, z = 2.23, p < 0.01).
Functional connectivity
PPI model 1
We hypothesized that anatomical measures of structural connectivity along different striatal tracts would be evident in functional brain activity. In particular, we wished to test for differences in functional connectivity (or task-related correlations) across corticostriatal and subcorticostriatal pathways. To identify such relationships, we performed functional connectivity analyses on those regions that were involved in intertemporal decision making. Functional images were acquired as participants completed the delay-discounting task discussed above. First, we tested for areas that were more engaged during the decision period versus the intertrial period (controlling for total expected value, difference in expected values between the smaller sooner and larger later outcomes, and choice). These analyses revealed that there was a large network of cortical and subcortical areas engaged during the decision phase (Fig. 4, Table 2). However, our subsequent analyses revealed that only for the amygdala and the right dlPFC was there a significant change in connectivity during the choice period; that is, we found significant positive striatum–amygdala coupling and significant negative striatum–right dlPFC coupling during the choice phases of the task relative to baseline (Fig. 6A).
Importantly, for both the amygdala and the right dlPFC, we found that individual differences in functional connectivity during choice versus baseline correlated with structural connectivity (r = 0.75, p < 0.001 and r = −0.56, p < 0.014, respectively) and discount rates (r = 0.65, p < 0.008 and r = 0.57, p < 0.01, respectively; Fig. 6B). Therefore, increased positive striatum-amygdala functional connectivity during decision making is related to stronger striatum–amygdala structural connectivity and increased rates of discounting (more impulsive behavior). Conversely, increased negative striatum–dlPFC functional connectivity during choice is related to stronger striatum–dlPFC structural connectivity and decreased rates of discounting (less impulsive behavior).
PPI model 2
Next, we hypothesized that structural connectivity would be associated with more efficient processing of signals originating from those brain areas that were specifically involved in choosing LL over SS (or vice versa). To test this hypothesis, we focused on functional connectivity between the striatum and those target areas that were differentially activated when participants made LL versus SS choices. These analyses revealed that, of our target areas, only a single region was differentially activated based on choice outcome (Table 1). Specifically, we found that the right dlPFC (Fig. 7) was more engaged when participants indicated a preference for the delayed rewards (LL) versus a preference for the sooner rewards (SS). No brain regions showed a significant effect in the contrast of SS versus LL choices. More importantly, our PPI analyses also revealed that the right dlPFC evinced increased negative coupling with the striatum for LL versus SS choices. Therefore, choosing LL rewards was associated with increased negative coupling between the right dlPFC and the striatum. Interestingly, we also found that individual differences in functional connectivity for LL versus SS trials correlated with individual differences in structural connectivity (r = −0.5 p < 0.01) and discount rates (r = 0.64, p < 0.003; Fig. 7). Therefore, those individuals that were overall more patient showed a greater difference in striatum–dlPFC functional connectivity for LL compared with SS choices and had greater striatum–dlPFC structural connectivity.
Finally, exploratory analyses revealed that the striatum–dlPFC connectivity measures assessed based on decision making versus baseline (Model 1) and based on choice outcome (Model 2) were significantly correlated (r = 0.43, p < 0.05). This raises the question of whether the earlier reported relationship between connectivity during the choice compared with baseline and discounting may have been driven by the increased connectivity during LL choices. To control for this possibility, we performed a stepwise linear regression in which we entered connectivity measures for [LL vs SS] and [choice vs baseline] as independent variables and discount rate (k) as the dependent value. In this linear model, the connectivity of [LL vs SS] remains a significant predictor of individual difference in discounting (β = 0.52, p < 0.01), but the [choice vs baseline] connectivity measure is no longer significant (β = 0.21, p = 0.3). This suggests that it is specifically the increased striatum–dlPFC connectivity for LL choices that is associated with individual differences in discounting.
In sum, analyses of the fMRI data showed that several of the cortical and subcortical target areas identified in the DTI segmentation analyses were functionally involved in the intertemporal choice task. These regions included the right dlPFC, left IFG, and the amygdala. However, only the right dlPFC and the amygdala also showed increased functional coupling related to intertemporal decision making. Interestingly, these two areas showed qualitatively distinct relationships between functional coupling and discounting behavior, suggesting two different mechanisms by which they influence behavior. First, the strength of functional coupling between amygdala and the striatum during the choice period was related to greater discounting of future rewards. Conversely, participants who discounted the future less showed increased negative coupling between the dlPFC and the striatum based on choice outcome.
The reported associations between functional connectivity and discounting were selective with respect to other cognitive variables. Specifically, we collected additional cognitive measures for all subjects, including visuospatial IQ and SSRT (Verbruggen et al., 2008). However, individual difference in IQ and SSRT did not correlate with functional connectivity measures in the selected tracts (all p > 0.23, uncorrected). Moreover, all of the results relating delay discounting to anatomical and functional connectivity were significant even though IQ and SSRT were controlled for in all analyses.
Conditional process analyses
Based on our results showing a relationship between measures of structural and functional connectivity and discounting behavior, we decided to explore these associations further with a mediation analysis. Specifically, we tested whether the effect of structural connectivity on log(k) was mediated by functional connectivity. To test the mediation hypothesis, we used the Hayes' PROCESS algorithm (Hayes, 2012). We calculated the 95% bias corrected bootstrap confidence intervals (CIs) of the indirect effect on the basis of 5000 bootstrap samples. When the CI ranges does not include zero, this is considered support for a significant mediation effect. The 95% confidence interval around the indirect effect ranged from 0.17 to 0.52 for the dlPFC tract and from 0.22 to 0.38 for the amygdala tract, indicating that the relationship between structural integrity and discount rates was mediated by functional connectivity in both tracts (Fig. 8). In sum (although no statistical methods proves causality), the mediation analyses supports the hypotheses that: (1) increased dlPFC-striatum tract integrity leads to increased negative functional coupling between dlPFC and the striatum during LL versus SS choices, which is in turn leads to more far-sighted preferences, and (2) increased amygdala–striatum tract integrity leads to increased positive functional coupling between amygdala and the striatum during choice, and in turn more impulsive preferences.
Specificity of behavior connectivity results in dlPFC
Our functional imaging results indicated that only a subsection of anatomically defined dlPFC is involved in the deciding between immediate and delayed rewards. Inspired by these results and the relatively large size of the anatomically defined region, we decided the further explore several tracts that connect to different subregions of the dlPFC. We divided the dlPFC in three regions using the Brodmann areas (BAs) with which it intersects. This resulted in one region that intersected with BA8 and BA9, one that intersected with BA46 and BA47, and one that intersected with BA10 (Fig. 9).
Using these three dlPFC regions, we applied the probabilistic tracking algorithm with these new targets. Segmentation of the striatum produced the segments shown in Figure 9. These striatal segments were used to further explore the relationship between tract strength and discount rates and functional connectivity.
We found a significant correlation between tract strength and discount rates for the BA8–BA9 region (r = −0.64, p < 0.001) and the BA46–BA47 region (r = −0.51, p < 0.007), but not for the BA10 region (r = −0.23, p = 0.28). Next, we investigated the relationship between functional and structural connectivity within these tracts. These analyses revealed that choice-related differences in functional connectivity were significant correlated with structural connectivity for the BA8–BA9 tract but not for the BA46–BA47 tract (i.e., PPI Model 2; r = −0.59 p < 0.008 and r = −0.44 p < 0.05, respectively). These results are consistent with the fact that the peak of the functional activity in the LL > SS contrast was located in BA9 and suggest that an even more specific striatal tract is associated with interindividual differences in temporal discounting.
Replication analyses
Finally, to test the robustness of our structural connectivity results, we performed the same set of analyses on an independent dataset that was collected for different purposes. Discount factors from this group, estimated using the same hyperbolic model, were comparable to those found in the initial dataset, ranging from kmin = 0.009 to kmax = 0.06.
The seed-based classification produced a very similar pattern of striatal segmentation as above. Visual inspection showed the same dorsal-lateral to ventral-medial gradient, and the DICE values obtained for all striatal segments were satisfactory (between 0.42 and 0.62; Table 1). The SMA and pPC regions showed similar low DICE values (<0.04) and also did not exist for 6 participants. These regions were therefore again excluded from further analyses.
More importantly, using the methods described above, we tested for a relationship between tract strength and delay discounting, log(k), using Spearman's rank-order correlation. This analysis produced the same general pattern of results as above: striatal connection strength with dorsal-lateral areas was associated with more patient behavior, whereas connection strength with ventral-medial and limbic areas was associated with more impulsive behavior. More specifically, individual discount rates were negatively associated with tract strengths linking to the right dlPFC (r = −0.58, p < 0.001), left dlPFC (r = −0.55 p < 0.003), and positively associated with tract strength to the bilateral amygdala (r = 0.52, p < 0.001; Tables 2, 3). A trending correlation was found for the left IFG (r = −0.41, p = 0.069). In a final step, we also replicated our specification analyses (see above) showing that both the strength of BA8–BA9 and BA46–BA47 tracts (r = −0.62, p < 0.001, and r = −0.54, p < 0.006, respectively) correlated with individual differences in discount rates, but this was not the case for the BA10 tract. In sum, although the replication dataset did not have functional data, it fully supports our findings from our primary analyses.
Discussion
Prominent models of intertemporal choice, such as the hyperbolic model, are able to describe individual time preferences with high accuracy, allowing for the detection of subtle differences in delay discounting across individuals. However, given the high degree of variability in discounting factors observed within subjects across experiment contexts, we (and others) have argued that intertemporal preferences derive from multiple cognitive and neural processes that are not captured by existing models (Jimura et al., 2011; Peters and Büchel, 2011; van den Bos and McClure, 2013).
Previous research on intertemporal decision making has led to the well established hypothesis that at least two qualitatively different processes contribute to valuing delay rewards (Ainslie, 1975; Shefrin and Thaler, 1981; Loewenstein, 1996; Peters and Büchel, 2011). One process is relatively myopic and has been associated with the ventral striatum and associated structures, such as the ventromedial prefrontal cortex (McClure et al., 2004; Hare et al., 2009). The second process is hypothesized to subserve control functions that promote far-sighted choices in the face of immediate temptation (Figner et al., 2010). The ZTPI classifies attitudes about the future along similar lines, and we included this questionnaire in our study based on this correspondence. Together, our results largely support the dichotomy suggested by the primary motivational/regulatory control hypotheses. Anatomical tracts associated with the former (motivational) process (amygdala to ventral striatum) are associated with increased myopia and greater self-reported PH. Tracts associated with the latter (control) process (lateral prefrontal cortex to dorsal striatum) are associated with reduced impulsivity and larger measures of FO. The novel contributions stemming from these findings are threefold. First, we found that these associations are independent in the sense that measures associated with motivational processes do not predict measures associated with control processes within individuals. Second, our results suggest that the motivational and control processes influence individual differences in discounting behavior through different functional mechanisms. Finally, given known connections between dorsal and ventral striatum (Haber and Knutson, 2010), we can hypothesize that the striatum may be one site where motivational and control processes interact. The nature of this interaction is an important avenue for future research because it may be the basis for some forms of self-control.
Our primary analyses are based on tract likelihoods and PPI measures of functional connectivity. These measures are still somewhat debated and the correct interpretation of results with respect to neuroanatomy is still being determined. For example, numerous structural properties likely contribute to tract likelihood, including axon caliber, axonal density, membrane permeability, and myelination (Jones et al., 2013). Regardless, given that myelination, axon caliber, and axon density all positively influence the conduction of action potentials within neuron networks (Paus, 2010), it is commonly assumed that the structural connectivity measures provide at least partial information about the functional effectiveness of connections. The advantage of our current design is that we were able to provide further support for this hypothesis by showing that the effect of individual differences in structural connectivity are mediated by changes in functional connectivity. Finally, both probabilistic tractography and PPI are in principle unable to resolve the direction of connectivity. However, based on animal-tracing studies, it is highly likely that the tracks between the striatum and our selected seed regions are afferent (i.e., providing input to the striatum; Haber and Knutson, 2010).
Our results are consistent with the general results of Peper et al. (2013) showing that individual difference in corticostriatal tract integrity is related to discounting. However, we have identified more specific tracts and have shown that different corticostriatal tracts have opposing effects on discounting. Our results are also consistent with several studies showing that activity in the lateral PFC is associated with an increase in the likelihood of selecting larger delayed outcomes over smaller sooner outcomes (Peters and Büchel, 2011). Although there is still a debate about the exact function played by the dlPFC in intertemporal choice, it is generally agreed that the dlPFC is centrally involved in producing goal-directed behavior (Miller and Cohen, 2001), which generally favors the larger, later rewards in intertemporal choices, and that the dlPFC affects choice by regulating brain valuation processes (Hare et al., 2009; Kable, 2010). Our results extend these claims in two important ways. First, we found that individual differences in structural connectivity specifically in the dlPFC–dorsal striatum pathway predicted individual differences in discounting (more specifically, the pathway connected to BA8–BA9). Second, we showed that this effect is mediated by increased negative functional coupling, particularly during choices for delayed rewards. Together, these results support the hypothesis that an inhibitory interaction between dlPFC and striatum is important for intertemporal decision making.
Similar to the right dlPFC, the structural connectivity measure for the left IFG predicted less impulsive behavior; however, we found no relation between IFG tract strength and functional connectivity. This dissociation might reflect the fact that IFG–striatal connectivity is associated with patient behavior through its involvement in impulse control (Aron et al., 2007). This would be consistent with our findings that IFG–striatal connectivity is correlated with trait FO, but is not specifically engaged in the delay-discounting task.
More interestingly, we also found that increased connectivity strength between the ventral striatum and the amygdala predicted more impulsive behavior. Although the function of the amygdala has traditionally been related to fear-related processes, more recent theories ascribe the amygdala a more general role in the evaluation of choice options (Morrison and Salzman, 2010). More specifically, the amygdala has been considered part of an “impulsive system,” which triggers emotional responses to immediate outcomes while taking into account current (physiological) state (Bechara, 2005; Morrison and Salzman, 2010; Gupta et al., 2011). Previous imaging studies have shown that amygdala activation was associated with individual differences in discounting, such that increased amygdala activity predicted more impulsive behavior (Hoffman et al., 2008; Pine et al., 2010). Similarly, Cohen et al. (2008) found that the connection between striatum and amygdala, but not the cortical tracts, was associated with self-reported novelty seeking, a trait related to impulsiveness. However, it was unclear from these studies how the computations in the amygdala-biased discounting behavior. Our results suggest that increased amygdala input to the ventral striatum may enhance the incentive values of the immediate rewards (Everitt et al., 1999), resulting in steeper discounting. If this is true, then individual differences in amygdala–ventral striatum connectivity may furthermore moderate the effects of internal states on discount factors. For example, low blood glucose levels may have a greater impact on discounting for those individuals that show strong amygdala–ventral striatum connectivity (Wang and Dvorak, 2010). Consistent with this hypothesis, Pine et al. (2010) found that the effect of L-DOPA on discounting was mediated by the level of differential amygdala activity.
Our results are also relevant to recent findings related to clinical populations (i.e., addiction and ADHD). For example, a strong coupling between the amygdala and ventral striatum may underlie the steeper delay discounting in response to internal states or conditioned stimuli that trigger drug consumption in substance-dependent populations (cf. cigarette and cocaine addiction in Madden and Bickel, 2009). For example, drug craving in dependent individuals, elicited via the presentation of cues associated with drug use, is associated with increased activity in the amygdala and ventral striatum (Kilts et al., 2001). Therefore, individuals who have stronger amygdala–ventral striatal connectivity may be more likely to experience craving in the presence of drug cues that in turn result in more impulsive responding and subsequent drug use. Furthermore, ADHD is also recognized as a disorder associated with steeper rates of delay discounting and impulsivity (Schweitzer and Sulzer-Azaroff, 1995; Marco et al., 2009). The dual-pathway or multiple-pathway model in ADHD (Sonuga-Barke, 2003; Castellanos et al., 2006) emphasizes that there could be disturbances in either a cognitive control or reward responsivity system (Castellanos et al., 2006; Scheres et al., 2006) and their connections (Tomasi and Volkow, 2014). Recent evidence from a study examining the relationship between behavioral measures of delay discounting and resting-state functional connectivity MRI measures suggests that greater impulsivity in ADHD is associated with stronger striatal–anterior PFC connectivity compared with control subjects (Costa Dias et al., 2013). Future research should consider how differences in structural connectivity of separate striatal tracts are associated with particular vulnerabilities for impulsive behavior or addiction.
Our findings provide an interesting window into possible mechanisms underlying increased impulsive behavior associated with developmental populations and psychopathology. First, it has become commonly accepted that there are different developmental trajectories associated with the motivational limbic/striatal and control lateral PFC systems (Somerville and Casey, 2010; Luna et al., 2013). The reward system is thought to develop in early adolescence, whereas the cognitive-control system is thought to develop later and in a more linear fashion. The developmental imbalance between these regions is thought to result in greater impulsivity, particularly in early adolescence. Finally, ADHD is also recognized as a disorder associated with steeper rates of delay discounting and impulsivity (Marco et al., 2009). Similar to the developmental models, the dual-pathway or multiple-pathway model in ADHD (Castellanos et al., 2006) emphasizes that there could be disturbances in either a cognitive control or reward responsivity system. We suspect that continued developments in understanding how different neural systems contribute to intertemporal preferences will enable even more refined insights into the developmental and psychopathological differences in impulsive behavior.
In conclusion, our study expands our understanding of the neural mechanisms of intertemporal choice. Together, the findings provide support for the idea that intertemporal preferences are generated by complementary but distinct neural mechanisms that are associated with specific corticostriatal and subcorticostriatal networks. By combining measures of functional connectivity, white matter, and behavior, we were able to integrate different findings from earlier functional and structural studies and provide important insights into how functional connectivity mediates the relationship between structure and individual differences in discounting behavior within a healthy population.
Footnotes
This work was supported by The Netherlands Organisation for Scientific Research (NOW Rubicon Grant #446-11-012) and the National Institute of Mental Health, National Institutes of Health (Grant R01 091068 to J.B.S. and S.M.M.).
The authors declare no competing financial interests.
- Correspondence should be addressed to either of the following: Wouter van den Bos, Center for Adaptive Rationality, Max-Planck-Institute for Human Development, Lentzeallee 94, 14195 Berlin, Germany, vandenbos{at}mpib-berlin.mpg.de; or Samuel M. McClure, Department of Psychology, Stanford University, 450 Serra Mall, Stanford, CA 94305. smcclure{at}stanford.edu