Abstract
Frontostriatal circuits have been implicated in reward learning, and emerging findings suggest that frontal white matter structural integrity and probabilistic reward learning are reduced in older age. This cross-sectional study examined whether age differences in frontostriatal white matter integrity could account for age differences in reward learning in a community life span sample of human adults. By combining diffusion tensor imaging with a probabilistic reward learning task, we found that older age was associated with decreased reward learning and decreased white matter integrity in specific pathways running from the thalamus to the medial prefrontal cortex and from the medial prefrontal cortex to the ventral striatum. Further, white matter integrity in these thalamocorticostriatal paths could statistically account for age differences in learning. These findings suggest that the integrity of frontostriatal white matter pathways critically supports reward learning. The findings also raise the possibility that interventions that bolster frontostriatal integrity might improve reward learning and decision making.
Introduction
Functional neuroimaging studies have consistently implicated frontostriatal regions innervated by ventral tegmental dopamine projections in reward processing (O'Doherty, 2004; Knutson and Cooper, 2005). Across the adult life span, these studies have revealed both similarities and differences in frontostriatal recruitment in older adults. Although investigators have documented intact striatal responses to reward cues when tasks do not involve learning (Samanez-Larkin et al., 2007; Cox et al., 2008; Samanez-Larkin et al., 2011), age differences in striatal function have been reported in the context of probabilistic learning (Aizenstein et al., 2006; Mell et al., 2009; Samanez-Larkin et al., 2010). This pattern of age-related similarities in reward tasks that do not require learning but differences in those that require learning is consistent with a larger body of behavioral findings (Mata et al., 2011). Since striatal recruitment is similar in response to reward cues but decreased or more variable during reward learning as a function of age, it seems unlikely that age differences in reward learning are due solely to persisting striatal dysfunction. Instead, age differences in reward learning may depend upon a broader network of regions that project to the striatum, including the prefrontal cortex.
Healthy aging is associated with changes in brain structure as well as function (Rubin, 1999; Buckner, 2004), and both cross-sectional and longitudinal evidence suggests that aging may compromise white matter microstructural integrity (Sullivan and Pfefferbaum, 2006; Madden et al., 2009; Sullivan et al., 2010). Age differences are more evident in frontal cortical than in posterior cortical pathways (Sullivan and Pfefferbaum, 2006; Madden et al., 2009).
Diffusion tensor imaging (DTI) allows investigators to assess microstructural qualities of specific white matter tracts in humans (Basser et al., 1994). Unlike strict volumetric measures, DTI metrics like fractional anisotropy (FA) are thought to reflect a combination of fiber density, axonal diameter, and myelination in axonal projections, which we collectively refer to here as “integrity.” Findings using these methods reveal ascending spirals of connectivity emanating not only between midbrain nuclei and the striatum but also between the striatum, thalamus, and the prefrontal cortex (Draganski et al., 2008; Cohen et al., 2009) consistent with tract tracing studies in primates (Haber, 2003). Ventromedial aspects of these circuits have been associated with motivation, whereas dorsolateral circuits have been more associated with translation to motor activity (Haber and Knutson, 2010).
Although prior studies have demonstrated associations between age, implicit sequence learning, and white matter tract integrity in the dorsal striatum and connected dorsolateral prefrontal cortex (Bennett et al., 2011), here we instead focused on ventromedial tracts typically associated with reward learning. Thus, we combined DTI with a probabilistic reward learning task in a community adult life span sample to determine whether the white matter integrity of frontostriatal pathways might mediate the influence of age on reward learning.
Materials and Methods
A survey research firm recruited healthy human adults who were representative of San Francisco peninsula residents with respect to sex, income, education, and ethnicity. Subjects were sampled uniformly across adulthood and screened for dementia (Mini-Mental >26). Data from 25 adults (ages = 21–85 years, mean = 55, SD = 17; 16 females) who completed a DTI scan are included in analyses of age effects. All performance-based analyses were conducted on a subset of 22 adults (ages = 29–85 years, mean = 58, SD = 15; 14 females) who completed a reward learning task (the excluded subjects completed only half the task). Written informed consent was obtained from all subjects; the study was approved by the Institutional Review Board of Stanford University. Subjects received fixed payment of $20 per hour and cash equivalent to their total task earnings. These data come from a subset of subjects included in a study of age differences in financial decision making (Samanez-Larkin et al., 2010).
A behavioral measure of probabilistic reward learning, the Monetary Incentive Learning task, was adapted from conventional reinforcement learning tasks (Kim et al., 2006; Pessiglione et al., 2006) and has been used in our recent work (Samanez-Larkin et al., 2008; Knutson et al., 2011). The goal of this task is to maximize earnings by learning from feedback to select the better cue in each of three conditions (gain acquisition, loss avoidance, neutral). One pair of fractal cues was assigned to each condition. On each trial, subjects viewed a single pair of cues (from one of the three conditions), chose one from the pair, and viewed the outcome of their choice. On average, one of the cues yielded a better outcome, while the other yielded a worse outcome. In gain cue pairs, the better cue had a higher probability of returning gains (66% +$1.00; 33% +$0.00) than the worse cue (33% +1.00; 66% +$0.00). Similarly, in loss cue pairs, the better cue had a higher probability of returning non-losses (66% −$0.00; 33% −$1.00) than the worse cue (33% −$0.00; 66% −$1.00). In neutral cue pairs, choice of either cue had no impact on outcomes (100% $0.00). The computer assigned each cue to either the better or worse outcome distribution at the beginning of the task. Different cues were used for practice and experimental sessions. Subjects were informed about cue probabilities before the practice session and told to try to maximize their earnings throughout the experiment. Each experimental run included 24 trials per condition in a pseudorandomly mixed order, for a total of 72 trials.
Previous research indicates that performance in the gain and loss learning conditions is uncorrelated during initial learning (Knutson et al., 2011). The first half of the trials (first 12) in this sample revealed a similar lack of correlation, r = 0.18, p = 0.39. As learning reached asymptote, however, gain and loss learning performance grew more correlated; over all 24 trials, gain and loss learning were significantly correlated, r = 0.42, p < 0.05. We did not observe an age by condition interaction in learning, p > 0.05, suggesting that any age differences observed were primarily due to common learning from probabilistic feedback. In all analyses, we adopted a general measure of learning that combined performance across gain and loss conditions. This measure of learning performance was the percentage of choices that matched the “better” cue (i.e., had the higher probability of a gain or non-loss) across the task.
DTI data were acquired on a 1.5-Tesla GE Signa LX scanner using a standard quadrature head coil. The protocol used eight repetitions of a 90 s whole-brain scan. Scans were averaged across the 12 min session to improve signal quality. The pulse sequence was a diffusion-weighted single-shot spin-echo, echo planar imaging sequence (time to echo, 63 ms; time to repetition, 6 s; field of view, 260 mm; 128 × 128 matrix; ±110 kHz bandwidth; partial k-space acquisition; 40 diffusion directions). We acquired 60 axial, 2-mm-thick slices (no skip) for two b-values (0 and 900 s/mm2). A high-resolution T1-weighted anatomical image was also collected using a sagittal 3D spoiled gradient sequence (1 × 1 × 1 mm voxels). Anatomical landmarks were manually defined in the anterior/posterior commissure and midsagittal plane to guide a rigid-body transform to convert the T1-weighted images to anterior/posterior commissure (AC-PC) aligned space.
Eddy current distortions and subject motion in the diffusion-weighted images were removed using a 14-parameter constrained, nonlinear coregistration based on the expected pattern of eddy-current distortions given the phase-encode direction of the acquired data (Rohde et al., 2004). Each diffusion-weighted image was registered to the mean of the (motion-corrected) non-diffusion-weighted (b = 0) images using a two-stage coarse-to-fine approach that maximized the normalized mutual information. The mean of the non-diffusion-weighted images was aligned to the T1 image using a rigid-body mutual information algorithm. All raw images from the diffusion sequence were resampled to 2 mm isotropic voxels by combining the motion correction, eddy-current correction, and anatomical alignment transforms into one omnibus transform and resampling the data using a seventh-order b-spline algorithm based on code from SPM5 [K. Friston and J. Ashburner (2004) Statistical Parametric Mapping, available at: http://www.fil.ion.ucl.ac.uk/spm/software/spm5/]. An eddy-current intensity correction (Rohde et al., 2004) was applied to the diffusion-weighted images after resampling. The rotation component of the omnibus coordinate transform was applied to the diffusion-weighted gradient directions to preserve their orientation with respect to the resampled diffusion images. The tensors were then fit using a linear least-squares algorithm. Visual inspection confirmed that DTI and T1 images were aligned to within a few millimeters.
To probabilistically track primary pathways of interest, we defined four gray matter foci on individual subjects' structural images. Spheres (8 mm diameter) were placed in the ventral tegmental area (VTA), nucleus accumbens (NAcc), dorsomedial thalamus (DMThal), and the medial prefrontal cortex (MPFC). DTI fiber tractography and data analysis were performed with freely available software (mrDiffusion, ConTrack, and QUENCH; http://vistalab.stanford.edu/software). We identified four lateralized fiber bundles that extended between VTA-NAcc, NAcc-DMThal, DMThal-MPFC, and MPFC-NAcc. Although the NAcc-DMThal tracts pass through the pallidum (Alexander et al., 1990; Haber, 2003), we did not define a separate seed in the pallidum. We limited our seeds to regions of the network where pathways significantly changed direction. A probabilistic tractography algorithm (Sherbondy et al., 2008) was used to generate a set of 50,000 candidate pathways connecting the pairs; seeds were placed in both regions to ensure symmetric tracking. Candidate pathways were scored using the ConTrack scoring algorithm described by Sherbondy et al. (2008). Only the top scoring 1% of pathways were retained; these pathways were then considered the most likely pathways connecting the pair of regions. The volume of tracts was not correlated with age (all |r| values < 0.13, p > 0.48). The white matter integrity of their connection was summarized by calculating the mean FA across all the points connecting lateralized pairs and then averaged bilaterally, to reduce the number of statistical comparisons given a lack of specific hypotheses about laterality in the existing literature. Robust regression analyses examined relationships between these measures of FA and both age and task performance. All reported coefficients are standardized βs.
To identify control pathways, whole-brain tractography was implemented with a deterministic, streamlines tensor-tracing algorithm (Conturo et al., 1999; Mori et al., 1999; Basser et al., 2000). Fibers were traced in native subject space by seeding voxels with FA > 0.3 and implementing standard tractography parameters (step size, 1 mm; FA threshold, 0.15; length threshold > 20 mm; angle threshold, 30°; seed voxel offsets, [−0.25, 0.25]; offset jitter, 0). Resulting fiber tracts were automatically classified into 20 fiber structures (Wakana et al., 2007; Hua et al., 2008; Zhang et al., 2008). Specifically, we manually defined reference regions of interest (rROIs) describing two waypoints for each of the tracts described by Wakana et al. (2007). The rROIs were drawn in MNI space on the ICBM-DTI-81 atlas and warped from MNI space into each individual's diffusion space. Fibers were retained if they passed through any pair of rROIs. Some fibers passed through the rROIs for more than one major tract, so we applied an additional inclusion criterion by warping each fiber to MNI space and measuring the approximate overlap between the fiber points and each of the major JHU atlas tracts (Wakana et al., 2007). Fibers were classified as representing the tracts with which they had the highest degree of overlap.
The set of 20 resulting fiber structures (forceps major and minor plus 9 bilateral tracts) were extracted for each individual and then reduced to 11 by averaging across bilateral tracts (corticospinal tract, cingulum-hippocampus, cingulum-cingulate gyrus, inferior fronto-occipital fasciculus, inferior longitudinal fasciculus, superior longitudinal fasciculus, superior longitudinal fasciculus temporal, uncinate fasciculus, and anterior thalamic radiation). Of these pathways, we selected four tracts that showed reduced FA with age (all p values < 0.05), but were anatomically distinct from the cortical and subcortical tracts of primary interest. FA along these control paths was included in analyses to covary general age differences in white matter integrity.
We conducted power analyses based on average effect sizes from a recent study of age and performance differences in white matter integrity between the dorsolateral prefrontal cortex and caudate (Bennett et al., 2011). Although our study was focused on ventromedial frontostriatal tracts, this was the only published study that examined both age and learning performance relationships with frontostriatal white matter. For white matter and age effects, at least 19 subjects are required to achieve power of 0.8 for an estimated f2 of 0.46. For white matter and learning performance effects, at least 22 subjects are required to achieve power or 0.8 for an estimated f2 of 0.40. Thus, our sample size was sufficient to detect similar effects.
Results
Age was related to reduced white matter integrity in some, but not all, pathways of interest (Fig. 1, N = 25). Specifically, age-associated reductions in fractional anisotropy were significant in the DMThal-MPFC pathway, β = −0.35, p < 0.05; and MPFC-NAcc pathway, β = −0.43, p < 0.05, but not in the NAcc-DMThal pathway, β = −0.07, p = 0.70. Conversely, fractional anisotropy was positively associated with age in the VTA-NAcc pathway, β = 0.46, p < 0.05.
A–D, White matter tracts in a representative subject between the VTA-NAcc (A), NAcc-DMThal (B), DMThal-MPFC (C), and MPFC-NAcc (D). The thalamocortical (C) and corticostriatal (D) tracts were associated with both age (N = 25) and learning (N = 22). Scatter plots are simple pairwise correlations. E, A combined measure of thalamocorticostriatal (TCS) white matter integrity mediates age differences in probabilistic reward learning (N = 22). Path coefficients are standardized βs. *p < 0.05, two-tailed.
Individual differences in white matter integrity were also associated with reward learning performance (Fig. 1; N = 22). After controlling for age, better reward learning performance was associated with higher fractional anisotropy in the DMThal-MPFC pathway, β = 0.45, p < 0.05, and MPFC-NAcc pathway, β = 0.35, p < 0.05, but not in the NAcc-DMThal pathway, β = 0.30, p = 0.13, or the VTA-NAcc pathway, β = 0.15, p = 0.53.
Of the four primary pathways of interest, fractional anisotropy in two (DMThal-MPFC, MPFC-NAcc) showed similar relationships with age and with task performance controlling for age, and fractional anisotropy in these two tracts was correlated, β = 0.41, p < 0.05. Thus, these tracts were averaged into a unified ventromedial thalamocorticostriatal index for subsequent analyses.
Hierarchical regression analyses tested the hypothesis that white matter integrity in this specific thalamocorticostriatal pathway mediated age differences in probabilistic reward learning (N = 22). In step 1, age was associated with poorer learning, β = −0.46, p < 0.05. In step 2, age was associated with decreased thalamocorticostriatal white matter integrity, β = −0.40, p < 0.05. In step 3, increased thalamocorticostriatal white matter integrity was associated with better learning, β = 0.49, p < 0.05, controlling for age. In the final step, the age effect was reduced to nonsignificance after including thalamocorticostriatal white matter integrity in the model, β = −0.26, p = 0.12, consistent with full mediation (Fig. 1E).
Three individuals with mean learning scores <50% showed a stronger preference for the less optimal cue than the optimal cue in at least one condition. If these subjects are excluded, correlations of learning with white matter integrity in the DMThal-MPFC, β = 0.38, p = 0.09, and the MPFC-NAcc, β = 0.43, p < 0.05, remain relatively unchanged, and all paths in the mediation model remain significant, even with this reduced sample size (all p values < 0.04).
The associations between tract integrity and reward learning were selective, both with respect to other cognitive variables and with respect to the pathways implicated. Ventromedial thalamocorticostriatal integrity was not correlated with performance on other cognitive measures including numeracy, the trail-making test, digit span, or letter-number sequencing (all p values > 0.14). Including any of these variables in the model did not eliminate the significant relationship of white matter integrity with learning performance controlling for age. We also examined the anatomical specificity of these effects using other major tracts shown to decline with age in longitudinal studies (Teipel et al., 2010). The present relationship remained significant (p < 0.05) when controlling for the integrity of these tracts (inferior fronto-occipital fasciculus, inferior longitudinal fasciculus, uncinate fasciculus, and cingulum bundle). Thus, the observed relationships were selective to thalamocorticostriatal pathways and not simply due to general differences in cognitive ability or global white matter deterioration.
Discussion
As predicted, white matter integrity was associated with probabilistic reward learning. Specifically, increased white matter integrity in thalamocortical and corticostriatal paths was associated with better reward learning performance. Moreover, diminished integrity in these paths mediated the association between age and poor performance. Finally, consistent with specificity, individual differences in other cognitive variables and the integrity of other paths could not account for these associations. Although current analyses focused on the role of structural integrity in mediating age differences, relationships between white matter integrity and reward learning hold when controlling for age. Thus, the findings reveal a more general relationship between individual differences in thalamocorticostriatal integrity and probabilistic reward learning.
These findings build upon and extend existing documented associations between age differences in white matter and a wide range of cognitive variables including processing speed (Burgmans et al., 2011; Salami et al., 2012), task-switching (Gold et al., 2010), top-down attention (Madden et al., 2007), and implicit sequence learning (Bennett et al., 2011). The study additionally demonstrates a specific mediated effect of thalamocorticostriatal white matter tract integrity on age differences in probabilistic reward learning. Since we did not observe age differences in tract integrity along the subcortical thalamostriatal pathways, behavioral and age effects were limited to tracts connected to the prefrontal cortex.
White matter integrity is thought to facilitate information transfer through neurons. Since thalamocorticostriatal pathways play important roles in reward learning and decision making (Haber and Knutson, 2010; Seger et al., 2010), the structural integrity of these pathways may facilitate functional throughput. Although DTI cannot presently provide information about the direction or chemical specificity of neural projections, convergent inferences may be possible by incorporating existing knowledge about the anatomy and neurochemistry of these pathways (Alexander et al., 1986, 1990; Haber, 2003). Thalamocortical and corticostriatal projections through the medial prefrontal cortex typically use the neurotransmitter glutamate. Projections from the prefrontal cortex to the striatum are thought to provide reentrant input that can flexibly guide motivated behavior toward specific goal objects (Haber and Knutson, 2010). Thus, it is possible that reduced structural input through the medial prefrontal cortex to the ventral striatum might impede the formation of new reward associations even while leaving general motivation processes relatively intact.
Unexpectedly, age was positively associated with subcortical fractional anisotropy in the VTA-NAcc pathway, which might indicate enhanced integrity. However, there may be confounding structural factors that contribute to this effect such as contamination from greater age-related fractional anisotropy in nearby subcortical gray matter structures (e.g., the putamen) attributed to increased iron accumulation or interstitial fluid (Pfefferbaum et al., 2010). Future studies might better characterize potential alternative contributions to indices of this pathway's integrity.
These findings elucidate a novel link from brain structure to behavioral function but have a few limitations. The issue of crossing fibers (and, more generally, fiber directional coherence) poses challenges for the interpretation of diffusion data (Douaud et al., 2011). Although the present probabilistic tracking methods are generally resistant to the effects of crossing fibers (Sherbondy et al., 2008), observed differences in fractional anisotropy may still partially depend upon fiber direction coherence. Additionally, while similar to other samples in the literature (and justified by a power analysis), the sample size is not large. Finally, since the sample is cross-sectional, future research should determine whether changes in white matter precede changes in learning or vice versa.
The evidence that a neurobiological mechanism can partially account for age differences in learning does not imply that age-related reductions in learning are inevitable. Interventions may have the capacity to alter connective structure. Emerging evidence from training studies suggests that white matter plasticity can extend into old age (Lövdén et al., 2010; Engvig et al., 2011), raising the possibility that future targeted interventions may be able to enhance structural integrity and associated learning in individuals of all ages.
Footnotes
This research was supported by the Financial Industry Regulatory Authority Investor Education Foundation and the National Institute on Aging (grant R21-AG030778 to B.K., and grants F31-AG032804 and F32-AG039131 to G.R.S.-L.). Thanks to Dennis Chan, Stephanie Greer, Robert Ruelas, Andrew Trujillo, and Daniel J. Yoo for assistance with data collection, Roland Bammer for use of the diffusion imaging sequence, and Brian Wandell for software and support (R01-EY015000).
- Correspondence should be addressed to Gregory R. Samanez-Larkin, Psychological Sciences, Vanderbilt University, 111 21st Avenue South, Nashville, TN 37240-7817. g.samanezlarkin{at}vanderbilt.edu