Abstract
Why are some individuals more susceptible to the formation of inflexible habits than others? In the present study, we used diffusion tensor imaging to demonstrate that brain connectivity predicts individual differences in relative goal-directed and habitual behavioral control in humans. Specifically, vulnerability to habitual “slips of action” toward no-longer-rewarding outcomes was predicted by estimated white matter tract strength in the premotor cortex seeded from the posterior putamen (as well as by gray matter density in the posterior putamen as determined with voxel-based morphometry). In contrast, flexible goal-directed action was predicted by estimated tract strength in the ventromedial prefrontal cortex seeded from the caudate. These findings suggest that integrity of dissociable corticostriatal pathways underlies individual differences in action control in the healthy population, which may ultimately mediate vulnerability to impulse control disorders.
Introduction
According to dual-system accounts, action control is determined by the balance between a goal-directed and a habitual system (Dickinson, 1985; de Wit and Dickinson, 2009). The goal-directed system encodes the relationship between actions and their consequences and therefore allows healthy individuals to flexibly direct their actions toward currently desirable outcomes (O). Conversely, stimulus–response (S–R) associations drive action selection in the habit system. In a stable context, habitual control has the advantage of efficiency, but the drawback is that it can lead to “slips of action” toward outcomes that are currently not desirable, such as erroneously taking the elevator to a floor where one used to work.
Disruption of the balance between goal-directed and habitual action control has been proposed to underlie dysfunctional behavior in, for example, obsessive-compulsive disorder (Gillan et al., 2011), drug seeking (Everitt et al., 2001), and obesity (Volkow and Wise, 2005). However, even among healthy, young people, there may be substantial individual differences in the propensity to over-rely on habits at the expense of flexible, goal-directed action control, which may ultimately constitute a vulnerability factor for impulsive–compulsive behavior. In the present study, we investigated the neural basis of such individual differences.
Recent animal lesioning studies, as well as human functional neuroimaging research, suggest that distinct striatal subdivisions may support goal-directed versus habitual behavioral control (Balleine and O'Doherty, 2010). Specifically, fMRI research has provided evidence that, whereas the caudate is engaged during goal-directed action (Tanaka et al., 2008), the posterior putamen is involved in habit learning (Tricomi et al., 2009). Furthermore, white matter tractography studies (Lehéricy et al., 2004; Draganski et al., 2008) have shown that the caudate is strongly interconnected with the ventromedial prefrontal cortex (vmPFC), a cortical region that has been implicated previously in goal-directed action (Hampton and O'Doherty, 2007; Valentin et al., 2007; Tanaka et al., 2008; de Wit et al., 2009b; Gläscher et al., 2009; Tricomi et al., 2009). In contrast, the posterior putamen projects heavily to (pre)motor cortex (PMC), suggesting that this corticostriatal pathway may be well suited to support habitual responding.
The aim of the present study was to investigate directly the importance of anatomical connectivity within these corticostriatal pathways for the balance between goal-directed and habitual action control. To this end, we measured individual differences in action control on an instrumental learning task (de Wit et al., 2007, 2009b, 2011, 2012; Gillan et al., 2011), in which healthy, young volunteers were trained to perform instrumental responses to obtain rewarding outcomes. In the critical test, some outcomes were devalued, allowing us to assess participants' ability to selectively respond toward the still-rewarding outcomes. Vulnerability to “slips of action” toward no-longer-rewarding outcomes was taken as an index of relative habitual and goal-directed control and was related to individual measures of diffusion tensor imaging (DTI) tractography of striatal white matter connectivity, as well voxel-based morphometry (VBM) measures of gray matter integrity.
Materials and Methods
Participants
We recruited 24 healthy volunteers from the campus of the University of Amsterdam, with some participating for course credits while others were paid approximately €58. One volunteer was excluded as a result of previous experience with the instrumental learning task, leaving 23 participants (10 males) between the ages of 18 and 26 years (mean ± SD, 21.2 ± 2.1). Participants underwent telephone screening according to standard neurological and MRI exclusion criteria and gave their written informed consent before participation. The study was approved by the local ethics committee and was conducted in accord with relevant laws and institutional guidelines.
Study procedure
Participants underwent MRI scanning at the Amsterdam Medical Centre. Behavioral testing took place on two separate days at the University of Amsterdam. We report performance on the instrumental learning paradigm, as well as on the O-Span test of working memory (Turner and Engle, 1989) and NLV test of verbal intelligence quotient (IQ) (Schmand et al., 1992) (Dutch equivalent of the National Adult Reading Test). In addition, participants received a number of other unrelated tasks and questionnaires that will be reported elsewhere.
Instrumental learning paradigm
Participants were trained and tested on an instrumental learning paradigm (programmed in Visual Basic 6.0), consisting of four stages: a learning phase, an outcome-devaluation test, slips-of-action (and baseline) tests, and paper-and-pencil questionnaires of contingency knowledge. The results of the slips-of-action test were related to our MRI metrics. The basic design is briefly related below, but we refer the reader to previous publications for more detailed descriptions of stages 1 and 2 of this task (de Wit et al., 2007) and of the slips-of-action test (Gillan et al., 2011; de Wit et al., 2012).
Stage 1: instrumental learning.
On each trial, a fruit on the outside of a box on the screen functioned as a discriminative stimulus signaling which response (right or left key press) would be rewarded with a fruit inside the box and credits (Fig. 1a). For example, participants could learn by trial and error that grapes signaled that pressing the right key would be rewarded with an orange inside the box and credits, whereas pressing the left key would lead to an empty box and no credits. Participants were instructed that they should try to gain as many fruits and credits as possible by responding as fast and accurately as possible and that they would receive 1 euro-cent for each credit.
Illustration of instrumental task. a, contains an illustration of the instrumental learning stage. In this example, grapes (on the outside of the box) signal that pressing the right key will be rewarded with an orange (inside the box) and credits, whereas pressing the left key will not be rewarded. b contains a schematic depiction of the congruent, standard, and incongruent contingencies. c illustrates the slips-of-action test. In this example, the initial instruction screen shows the pineapple and pear outcomes with a cross superimposed, indicating that these will now lead to the subtraction of credits. The other four fruit outcomes are still valuable. After the instruction screen, participants are presented with a rapid succession of fruit stimuli (on the outside of the boxes) and are asked to press the correct keys when a stimulus signals the availability of a still-valuable outcome inside the box (“Go”) but to refrain from responding when the outcome inside the box has been devalued (“No-Go”). In this particular example, participants should press when the grapes stimulus is presented but refrain from responding on trials with the bananas stimulus. d shows the dual systems that are hypothesized to underlie performance on the slips-of-action test, with the S–O–R goal-directed system allowing for successful test performance based on outcome anticipation and evaluation, but the S–R habitual system activating responses directly and thereby giving rise to slips of action toward no-longer-valuable outcomes.
There were three discrimination types (Fig. 1b). For the congruent discrimination, the fruit stimulus was the same as the outcome. Therefore, the congruent stimuli always acted as a reminder of the available outcomes on that trial. In contrast, in the incongruent discrimination, each fruit functioned as a stimulus and as an outcome for opposing responses. For this reason, encoding of the outcomes should cause response conflict with the appropriate S–R associations. Indeed, previous studies have demonstrated that participants adopt an S–R habitual strategy to solve this discrimination (reflected in poor performance in the test stages). Finally, we included a standard discrimination, in which four different fruit pictures only ever functioned as either stimulus or outcome and which therefore lacks the stimulus–outcome confound of the congruent and incongruent discriminations. Performance on standard trials can be supported by both the (S–R) habit system and the (S–O–R) goal-directed system.
During each of six blocks, each stimulus was presented twice, in random order. Before commencement of this stage of the task (as well as the test stages), participants received a demonstration with food pictures that were not used in the actual experiment.
Stage 2: outcome-devaluation test of action-outcome knowledge.
The instructed outcome-devaluation test was conducted to check that participants had learned about the R–O contingencies. On each trial, two open boxes with fruit inside were presented. A cross was superimposed on one of these fruits, indicating that it was no longer worth credits. Participants were instructed to press the key that previously earned the still-valuable fruit. They were no longer provided with feedback. During this test, all outcomes were devalued twice.
Stage 3: slips-of-action test.
Each of six 24-trial blocks began with a 5 s screen displaying all six fruit outcomes, with two of these (pertaining to different discrimination types and opposite key presses) shown with a cross superimposed indicating that these were devalued (Fig. 1c). Subsequently, participants were shown a succession of 2 s screens with closed boxes with the fruit stimuli on the front (separated by a 1 s intertrial interval). Participants were instructed to press the appropriate key before the stimulus disappeared if it signaled the availability of a still-valuable outcome but to withhold the response if the stimulus was linked to a now-devalued outcome. Although participants did not receive direct feedback, they were told that they were still gaining credits for correct responses for still-valuable outcomes but losing one credit for each response toward a now-devalued outcome. As illustrated in Figure 1d, participants could only perform well on this test if they were able to base responding on the signaled outcomes and their current values (via S–O–R associations in the goal-directed system, e.g., grapes–orange–right). Conversely, they should find it challenging to withhold responses toward devalued outcomes if they formed strong direct S–R associations (in the habit system, e.g., grapes–right), which should trigger responses regardless of the current value of the available outcome. To assess performance, we calculated difference scores by subtracting the percentage of responses to stimuli predicting now-devalued outcomes from that to stimuli predicting still-valuable outcomes.
We also administered a baseline test that was identical to the slips-of-action test, except that participants were shown the six stimulus fruits with two of these devalued, and were instructed to base responding on stimulus (as opposed to outcome) value. This test does not require outcome retrieval and as such there is no competition between the goal-directed and habitual system. Difference scores were calculated by subtracting responding to now-devalued stimuli from responding to still-valuable stimuli. The order of tests was counterbalanced across participants.
Stage 4: questionnaires of contingency knowledge.
Explicit knowledge of the contingencies was assessed through paper-and-pencil questionnaires. On the S–R questionnaire, participants had to indicate which response was correct for each of the six stimuli. On the S–O questionnaire, they had to indicate which fruit outcomes were linked to each of the six stimuli. Finally, the R–O questionnaire required participants to indicate which response earned each of the six fruit outcomes. Participants were always shown one question at a time. The order of questionnaires was counterbalanced across participants.
Behavioral analyses
All data were analyzed using SPSS version 15. We conducted ANOVAs, which included the factor discrimination (congruent/standard/incongruent). Bonferroni's corrections were adopted for pairwise comparisons. All p values are based on Greenhouse–Geisser sphericity corrections, and original degrees of freedom are reported. All significant (p < 0.05) higher-order interactions with discrimination are reported. Instrumental performance was also correlated with working memory and verbal IQ (using two-tailed Spearman's analyses).
Neuroimaging procedures
DTI acquisition.
Diffusion-weighted data (DTIs) and a high-resolution three-dimensional (3D) T1-weighted image were acquired with an eight-channel array head coil, on a Philips 3 T scanner. Diffusion-weighted data were acquired using spin-echo echo planar imaging (TR, 9 s; TE, 65 ms; 60 2-mm-thick axial slices; matrix size, 112 × 112; in-plane resolution, 2 × 2 mm2). For diffusion directions, 50 equidistant points along a unit sphere were created. To improve the signal-to-noise ratio, the average of four consecutive scans was obtained. FSL (for FMRIB Software Library) tools (www.fmrib.ox.ac.uk) were used in all analyses (Smith et al., 2004; Woolrich et al., 2009). FSL Betcrawler (Smith, 2002) was applied to automate extraction of the brains from the T1 and DTI images, and these were manually checked and re-extracted if not successful.
VBM preprocessing.
Using FSL VBM tools (Smith et al., 2004) and an optimized VBM protocol (Good et al., 2001), a study-specific gray matter template was created, based on the individual T1 scans. After tissue-type segmentation, gray matter was registered to the standard MNI152 brain and then averaged across participants. Individual participant gray matter images were then nonlinearly re-registered to the group template and smoothed using a Gaussian kernel of 4 mm. Finally, individual participant 3D files were merged into a single 4D file.
DTI preprocessing and probabilistic tracking.
Four scans were averaged for each participant. Preprocessing of DTI data followed the standard FSL protocols of eddy correction [using FMRIB (for Functional MRI of the Brain) Diffusion Toolbox to align all images to a reference b0 image and then linearly transform them (Behrens et al., 2003)], brain extraction, and the fitting of diffusion tensors. Following this process, the diffusion parameters were sampled for each voxel using FSL bedpostX (Behrens et al., 2007).
Probabilistic tractography was run using FSL probtrackX from within each participant's diffusion space. Seed masks were created in normalized MNI space (using probabilistic masks from the Harvard–Oxford subcortical atlas) for the caudate and putamen. We created separate anterior and posterior putamen seed regions (delineation at y = 2; with anterior > 2 and posterior ≤ 2) because only the latter has been implicated in habit learning (Tricomi et al., 2009) and because previous DTI studies have demonstrated differential connectivity patterns for these subdivisions (Lehéricy et al., 2004; Draganski et al., 2008). The volumes of these seed regions were comparable: 1087 voxels (8696 mm3) for the caudate, 900 voxels (7200 mm3) for the anterior putamen, and 988 voxels (7904 mm3) for the posterior putamen. The following FSL probtrackX parameters were chosen: 5000 samples; curvature threshold, 0.2; no waypoint, exclusion, or termination masks; no advanced options.
Probabilistic tracking builds a connectivity distribution between a predefined seed structure and every other voxel in the brain, based on the most probable diffusion direction. Five thousand samples were sent from each voxel in the seed region, providing a quantitative measure of the probability of connectivity with other voxels in the brain. Tracking was performed in diffusion space, after the seed masks had been linearly transformed from MNI space and visually inspected. The tractography results were then transformed back to MNI space to produce a whole-brain image for each seed region, showing for each voxel the number of received samples. The relative strength of the white matter tract between the seed region and each voxel can be indirectly inferred from these received sample values according to a well-established method (Johansen-Berg and Rushworth, 2009), although it is not possible to pinpoint from which of the specific voxels in the seed mask the samples were sent. Individual participant 3D files were merged into a single 4D file for each of the three seed regions. Figure 2 illustrates average corticostriatal connectivity, across all participants in the present study, separately for the three striatal seed regions. In line with previous DTI studies (Lehéricy et al., 2004; Draganski et al., 2008), we found evidence for differential connectivity patterns.
Striatal tractography. Raw tractography data shown separately for the three striatal seed regions, at x = −18, y = −2, z = −12 (thresholded at 0.02% of total samples sent from the seed mask). Results are shown in neurological convention (right is right).
VBM and DTI statistical analyses.
Nonparametric voxelwise statistical testing was performed using FSL Randomize, a tool that combines general linear modeling and permutation testing (25,000 permutations). The relationship between tract strength and behavioral performance was assessed by correlating four unique behavioral scores (see below) with the tract values at each voxel, independently for each of the seed regions. Gray matter density was also correlated with performance. Anatomical locations of significant voxels were determined by reference to the Jülich Histological Atlas (included as part of FSLview) for the cortical regions (Eickhoff et al., 2005). We will report significant findings from the whole-brain VBM analysis only for the striatal and cortical components of the networks identified in the DTI analysis (as listed in Table 1; and additionally the network implicated in response conflict resolution).
Individual difference scores (responding toward valuable outcomes minus devalued outcomes) on standard trials of the slips-of-action test were used as an index of relative goal-directed and habitual control, with poor performance indicating reliance on an habitual strategy. As a secondary analysis, difference scores on incongruent trials of the slips-of-action test were related to tract strength to investigate the networks involved in resolution of outcome-induced response conflict. Finally, as control analyses, we also related performance on both standard and incongruent trials of the baseline test to tract strength.
After voxelwise correlations against the behavioral regressors, FSL Randomize assessed the significance of the model fit by comparing each statistic to a null distribution, which was generated by randomly shuffling the original dataset 25,000 times. Threshold-free cluster enhancement (TFCE) was used to boost signal in areas that exhibit spatial clustering (Smith and Nichols, 2009). TFCE uses an image algorithm that enhances the signal intensity within cluster regions, while retaining voxelwise spatial information. Resulting statistical maps were thresholded at p < 0.005. A significant relationship between white matter tractography values and behavioral regressors at a particular voxel implies variable white matter architecture between (some part of) the seed region and the voxel in question. Without lowering the statistical threshold, it is not possible to precisely visualize the entire length of the specific white matter tract in question. However, a number of studies have verified that the tractography follows known white matter pathways (for review, see Johansen-Berg and Rushworth, 2009), and our raw tractography results (as illustrated in Fig. 2) are in line with those previously demonstrations (Lehéricy et al., 2004; Draganski et al., 2008). To protect against false positives, data were only retained for those voxels in which at least half of the participants (n = 11) had tracts to that voxel from the seed mask (for a similar approach, see Cohen, 2011a). In addition, only clusters of at least 20 contiguous voxels (160 mm3) are reported. Descriptive plots showing the mean tractography values of significant voxels correlated against the behavioral scores were generated to exclude the possibility that outliers were driving the results (see Results).
We do not report results in the cerebellum because, for some subjects, this region was only partly in the field of view. Furthermore, for two participants, the PMC was not completely within the field of view. Exclusion of those participants did not affect the pattern of results reported here.
Results
Behavioral results
Statistical analyses are presented separately for the four stages of the instrumental task.
Stage 1: instrumental learning
Mean percentages of correct responses (across the 23 participants) during six blocks of instrumental training are displayed in Figure 3a, with 50% indicating chance-level performance. A significant effect of block [F(5,110) = 30.44, mean squared error (MSE) = 339.6, p < 0.0005] reflects that participants gradually learned to perform the correct response for each stimulus to gain rewarding outcomes.
Behavioral performance on the instrumental task. a shows mean levels of performance during instrumental training, separately for the congruent (open circles), standard (open squares), and incongruent (filled circles) trials, across six consecutive blocks. b shows the mean difference scores (responding toward valuable minus devalued outcomes/stimuli) for the slips-of-action test (filled bars) and the baseline test (open bars). For both a and b, error bars reflect SEMs, and these graphs have a non-zero origin at 50% (which denotes chance level for the training stage, whereas chance-level performance on the slips-of-action test would be at 0).
In the present study, we replicated previous demonstrations of superior performance on standard and congruent trials relative to incongruent (F(2,44) = 8.30, MSE = 711.6, p = 0.001). We have previously proposed that this “congruence effect” is attributable to outcome-induced response conflict in the incongruent discrimination, which forces participants to rely on S–R habit learning (de Wit et al., 2007, 2009b). On the final block of training, performance was above chance level for all three discriminations (p values <0.0005).
Stage 2: outcome-devaluation test of action-outcome knowledge
A main effect of discrimination (F(2,440) = 23.47, MSE = 604.8, p < 0.0005) mirrored that observed in training. Performance on both congruent and standard trials was superior to that on incongruent (p values <0.0005), with mean accuracy percentages of 91, 88, and 47% (SEMs of 4.9, 4.1, and 7.6), respectively. Furthermore, in contrast to congruent and standard test performance, incongruent performance did not differ from chance level (p values <0.0005 vs p = 0.7). Therefore, participants successfully used action-outcome knowledge to direct their responses toward still-valuable outcomes on congruent and standard test trials but not on incongruent trials.
Stage 3: slips-of-action test
Figure 3b displays the mean difference in responding for valuable minus devalued fruit outcomes in the slips-of-action test (filled bars). This difference score is a measure of relative goal-directed and habitual control, with a score of 100 indicating perfect goal-directed performance (responses were made to all stimuli linked to still-valuable outcomes and withheld appropriately to all stimuli linked to devalued outcomes). Conversely, a score of 0 would indicate habitual (S–R) responding that is insensitive to current outcome value. Difference scores are also shown for the baseline test (open bars), which was included to control for general task characteristics. These individual difference scores were used as behavioral regressors in the neuroimaging analyses.
Independent samples t tests confirmed that performance was above chance level for all discriminations across the slips-of-action and baseline tests (p values <0.0005). The overall ANOVA only yielded a significant interaction between test version and discrimination (F(2,44) = 5.78, MSE = 368.9, p < 0.01). Whereas performance on the slips-of-action test and baseline test was statistically indistinguishable on congruent and standard trials (t values = −1.00 and −0.72, p values = 0.3 and 0.5, respectively), incongruent performance was significantly worse on the slips-of-action test (t = −2.65, p = 0.02).
To conclude, participants showed good performance on congruent and standard trials of the slips-of-action test, which was at a similar level overall as on the baseline test, in which responding had to be based on (antecedent) stimulus value as opposed to (consequent) outcome value. However, although participants showed good performance on the incongruent baseline test, we found evidence for impaired performance when outcome retrieval was required on incongruent trials of the slips-of-action test.
Stage 4: questionnaires of contingency knowledge
Participants successfully acquired explicit knowledge of the relationships between stimuli, responses, and outcomes during the learning stage. In the present study, there was no significant main effect of S–R/R–O/S–O association type (F(2,44) = 2.73, MSE = 0.193, p = 0.09) nor of discrimination (F(2,44) = 1.50, MSE = 0.544, p = 0.2). Mean scores (ranging between 0 and 2) on the S–R/R–O/S–O questions were as follows: congruent = 1.8/1.7/1.7; standard = 1.9/1.6/1.7; incongruent = 1.7/1.4/1.6. Therefore, it appears that participants were able to extract the contingency rules for all three discrimination types.
Controlling for working memory and IQ
The mean ± SEM score on the O-Span test of working memory was 0.60 ± 0.04). Spearman's correlational analyses were conducted to investigate whether outcome-based responding on the standard and incongruent trials of the slips-of-action test was related to individual differences in working memory. However, we failed to find evidence for such a relationship (ρ values = 0.26 and 0.21, p values = 0.2 and 0.3). Verbal IQ was also unrelated to task performance (mean ± SEM score was 84 ± 0.95; ρ values = 0.09 and 0.06, p values = 0.7 and 0.8).
Summary of main behavioral findings
Participants successfully learned to perform instrumental responses to earn rewarding outcomes, and we replicated previous studies that showed faster learning of congruent and standard discriminations relative to incongruent (de Wit et al., 2007, 2009b, 2012). This congruence effect may be attributable to response conflict induced by the incongruent outcomes. Indeed, a subsequent outcome-devaluation test demonstrated (outcome-based) goal-directed action only on congruent and standard trials.
The slips-of-action test was conducted to directly assess relative goal-directed and habitual control. Importantly, performance on the critical, standard trials of the slips-of-action test was statistically indistinguishable from that on the baseline test. However, in line with the idea that different mechanisms underlie performance on these two tests, a correlational Spearman analysis showed that level of performance on standard trials of the slips-of-action test and baseline test was not related (ρ = 0.11, p = 0.6). The baseline test therefore offers an adequate control for our investigation of the neural basis of relative goal-directed and habitual action control.
Neuroimaging results
Corticostriatal connectivity correlates with individual differences in goal-directed versus habitual action control
To relate individual measures of anatomical corticostriatal connectivity to action control, we correlated caudate- and putamen-seeded tractography with performance on standard trials of the slips-of-action test task (for all significant results, including those outside the regions that have been implicated previously in goal-directed and habitual behavior, see Table 1). In line with our expectations, individual differences in estimated tract strength in voxels in the vmPFC seeded from the caudate were predictive of goal-directed performance, such that participants who showed stronger connectivity were better able to base their responding on prediction and evaluation of available outcomes (for significant voxels and scatter plots, see Fig. 4, top). Conversely, we found a negative relationship between behavioral performance and estimated tract strength in voxels within the PMC seeded from the posterior putamen (Fig. 4, bottom). Therefore, we provide evidence that connectivity within dissociable corticostriatal circuits underlies the balance between goal-directed and habitual action control in humans.
Significant neural correlates of successful, goal-directed performance on the slips-of-action test
Significant relationships between performance on the slips-of-action test and estimated white matter tract strength in corticostriatal pathways. a depicts our main tractography findings: whereas estimated caudate–vmPFC tract strength (shown here at x = −12, y = 48, z = −2) was a positive predictor of goal-directed performance (as reflected in difference scores on the standard trials of the slips-of-action test), connectivity between posterior putamen and PMC (shown at x = 34, y = −6, z = 56) was a negative predictor. b shows scatter plots with the corresponding connectivity–behavior correlations. The negative correlation between performance and posterior putamen–PMC tract strength (illustrated in the bottom scatter plot) remained significant after removing the outlier (R2 = 0.34).
Finally, analyses with the anterior putamen seed region revealed a significant positive relationship between behavioral performance and connectivity between the anterior putamen and the PMC and the opercular cortex (extending into temporal and insular cortices). These results suggest, therefore, that circuitries involving posterior and anterior parts of the putamen are functionally dissociable, with the former predominantly supporting habitual control and the latter goal-directed control.
Relating standard performance on the baseline test to striatal connectivity
The baseline test provides an important functional control condition, because it is identical to the slips-of-action test except that it requires neither prediction nor evaluation of consequent outcomes. Importantly, the only finding within the networks identified in the slips-of-action test analyses was a positive relationship between baseline test performance and caudate–PMC connectivity [peak (x, y, z) coordinates: (−12, −6, 60); (−48, 4, 50); combined cluster size = 79 voxels]. Striatal connectivity with the vmPFC was not a positive predictor of performance on standard trials of the baseline test, nor did we find evidence for a negative relationship between performance on the baseline test and posterior putamen–PMC connectivity. Therefore, although mean levels of behavioral performance were similar on the slips-of-action and the baseline tests, only the former is dependent on relative tract strength in the caudate–vmPFC and posterior putamen–PMC pathways.
To provide additional evidence for this claim, we proceeded to extract the mean values of the significant vmPFC voxels in the analysis of standard trials of the slips-of-action test, and connectivity-behavior correlations were calculated separately for the slips-of-action versus the baseline test. We then used Fisher's transform to establish that these correlations did indeed differ significantly from each other (Z = 2.93, p < 0.005). The same procedure was applied to the significant PMC voxels from the posterior putamen analysis, and again connectivity between these regions appeared to be uniquely related to the slips-of-action test (Z = 2.47, p < 0.01). Finally, the same was true for the unexpected correlation between behavioral performance and connectivity between the anterior putamen and opercular cortex extending into temporal and insular cortices (Z = 3.52, p < 0.005) but not for connectivity between this seed region and the PMC (Z = 0.42, p = 0.67), which means that the latter pathway may contribute similarly to performance on the two tests.
In conclusion, these results exclude the possibility that our primary findings concerning the slips-of-action test are attributable to general task characteristics and therefore confirm the specific relevance of the caudate–vmPFC and posterior putamen–premotor connectivity to goal-directed and habitual learning, respectively.
DTI predictors of resolution of outcome-induced response conflict on incongruent trials
We have previously suggested that goal-directed performance of the incongruent discrimination is possible only if a conflict resolution process is engaged to overcome outcome-induced response conflict, potentially by encoding items differentially depending on their functional role of stimulus and outcome (Dickinson and de Wit, 2003; de Wit et al., 2006, 2009a). In line with previous studies linking response conflict resolution to anterior cingulate cortex (ACC) functioning (Botvinick et al., 2004), the present study yielded a significant positive correlation between successful performance on incongruent trials of the slips-of-action test and caudate–ACC (extending into the paracingulate gyrus) tract strength [peak (x, y, z) coordinates: (−10, 44, 2); (14, 42, 20); (4, 32, 16); (−4, 38, 20); (10, 28, 40); with a combined number of 244 voxels; Fig. 5].
Scatter plot with brain connectivity–behavior correlation for resolution of outcome-induced response conflict. The mean estimated tract strength (from all significant clusters) between the caudate and all significant clusters within the ACC (depicted on the right, shown at the peak voxel at x = −4, y = 38, z = 20) correlated positively with successful conflict resolution (as reflected in the difference scores on incongruent trials of the slips-of-action test). This graph has a non-zero origin.
Caudate–ACC connectivity was not significantly related to successful performance on the incongruent trials of the baseline test. However, we should treat this null result with caution, because the overall higher level of behavioral performance on incongruent trials of the baseline test means that the lack of correlation could be attributable to a ceiling effect. To further investigate the specific relevance of ACC connectivity for performance on the slips-of-action test, we extracted the mean values of the significant ACC voxels in the analysis of the incongruent trials of the slips-of-action test and calculated separate correlations for the slips-of-action versus the baseline test. We then used Fisher's transform to establish that these correlations differed significantly from each other (Z = 2.69, p < 0.01). In summary, these results suggest that the ACC is involved specifically in the resolution of outcome-induced response conflict.
Relating gray matter density to individual differences in action control
The whole-brain VBM analysis revealed a significant negative relationship between performance on standard trials of the slips-of-action test and gray matter density of the posterior putamen extending into pallidum [peak coordinate (x, y, z): (22, 0, 2); cluster size = 55 voxels; Fig. 6). An additional analysis of standard trials of the baseline test did not provide evidence for a negative correlation with gray matter density of the putamen.
Scatter plot with gray matter–behavior correlation for goal-directed action. Gray matter density in the posterior putamen (depicted on the right, shown at the peak voxel at x = 22, y = 0, z = 2) was a negative predictor of goal-directed performance (as reflected in the difference scores on standard trials of the slips-of-action test). In the scatter plot, circles drawn around the dots indicate additional data points at those values. This graph has a non-zero origin.
Gray matter density in other parts of the networks identified in the DTI analyses was not significantly related to performance on the slips-of-action test. To provide additional evidence against the possibility that the significant target regions identified in the DTI analyses were confounded by individual differences in cortical shape, we conducted additional VBM analyses in which we extracted the gray matter density values from the exact same cortical voxels as those found to be significant in the tractography analyses and investigated whether these values correlated with our performance measures (for similar procedure, see Cohen, 2011b). First of all, there were no significant VBM–behavior correlations for the pathways hypothesized to underlie goal-directed action and habit learning (for performance on standard trials of the slips-of-action test: ρ = −0.05, p = 0.44 for vmPFC; ρ = −0.10, p = 0.98 for PMC). Furthermore, we also found no significant relationship between our gray matter density in the ACC region that was implicated in conflict resolution by the white-matter analysis (for performance on incongruent trials of the slips-of-action test: ρ = 0.24, p = 0.27). These analyses suggest that our principal tractography results are not driven by variability in geometric brain shape.
Finally, the white matter connectivity analyses also yielded the unpredicted result that connectivity between the anterior putamen and cortical regions (PMC and opercular cortex extending into temporal and insular cortices) was predictive of goal-directed action. The VBM control analyses once again failed to find a significant correlation for the opercular cortex (for performance on standard trials of the slips-of-action test: ρ = 0.03, p = 0.90), but there was in fact a significant correlation between gray matter density values in the PMC region and goal-directed performance (ρ = −0.458, p = 0.028). Therefore, we should treat the anterior putamen–PMC connectivity finding with caution, because we cannot exclude the possibility that gray matter variability contributed to this finding (and also in light of the null result yielded by the control analyses reported in the context of the baseline test).
Discussion
We provide evidence that dissociable corticostriatal networks underlie individual differences in vulnerability to slips of action that are thought to result from competition between goal-directed and habitual action control. Whereas estimated white matter tract strength in the vmPFC seeded from the caudate was predictive of the ability to selectively respond to still-rewarding outcomes, tract strength in the PMC seeded from the posterior putamen predicted vulnerability to slips of action toward no-longer-rewarding outcomes. We also provide evidence for regional specificity within the putamen, with connectivity between the anterior part and opercular cortex (extending into temporal cortex and insular cortex) correlating positively (as opposed to negatively) with performance. Furthermore, vulnerability to slips of action was greater in individuals with high gray matter density in the posterior, but not the anterior, putamen.
The structural integrity of dissociable corticostriatal white matter tracts has not been related previously to the balance between goal-directed and habitual behavior, but our findings converge with functional neuroimaging research that has consistently implicated the same region of the vmPFC in goal-directed action (Valentin et al., 2007; de Wit et al., 2009b) and on the basis of which the caudate and posterior putamen have been proposed to play a role in goal-directed and habitual control, respectively (Tanaka et al., 2008; Tricomi et al., 2009; Balleine and O'Doherty, 2010). Furthermore, previous fMRI studies of procedural sequence learning have shown transfer of activation from anterior toward posterior striatum with progressive training (Jueptner et al., 1997; Lehéricy et al., 2005). Although these studies did not directly assess behavioral sensitivity to outcome devaluation, extensive training would be expected to shift the balance from (outcome-based) goal-directed toward (stimulus-based) habitual control, a transition that may have been mediated by a shift in recruitment of the anterior subdivision of the putamen toward the posterior (“sensorimotor”) putamen. Moreover, a recent fMRI that did establish insensitivity to outcome devaluation as a consequence of extensive training (in humans) has also implicated specifically the posterior putamen (extending into pallidum) in S–R habit learning (Tricomi et al., 2009). Therefore, our findings extend previous studies by demonstrating that these brain regions are critically embedded within dissociable corticostriatal pathways.
Our evidence for a dual-system neuroarchitecture is also in line with recent DTI studies that have provided evidence for distinct cortical projections (Lehéricy et al., 2004; Leh et al., 2007; Draganski et al., 2008) of the caudate, anterior and posterior putamen. These patterns of anatomical connectivity converge with resting-state research into functional connectivity (Di Martino et al., 2008; Barnes et al., 2010) and with evidence from animal tracing studies (Haber, 2003; Haber et al., 2006). Furthermore, recent animal research has shown that DTI and histological fiber tracing methods yield comparable results (Dyrby et al., 2007). The present study also adds to increasing evidence that the relatively novel method of performing voxelwise correlations between individual differences in behavior and structural white matter tractography can be used to inform theoretical models of learning/memory and action control (Cohen et al., 2008; Harsay et al., 2011; Schott et al., 2011).
Importantly, our results were not attributable to individual differences in IQ or working memory span. Furthermore, the baseline test was used to exclude the possibility that the critical brain–behavior correlations reflect general task characteristics of the slips-of-action test, thereby providing additional evidence for the specific relevance of these networks for dual-system action control.
Resolution of outcome-induced response conflict
The ACC has long been implicated in overcoming response conflict that arises from competing S–R associations (Botvinick et al., 2004). However, we have previously provided evidence that the homolog brain region in rats (the dorsomedial PFC) is also important for outcome-induced response conflict, which is engendered when learning about incongruent outcomes interferes with stimulus control over instrumental behavior. Temporary inactivation of the dorsomedial PFC (through local infusion of a GABA agonist) selectively disrupted performance on incongruent trials of an instrumental learning task (de Wit et al., 2006, 2009a), without disrupting the expression of goal-directed action per se.
The behavioral results of the present study support the idea that outcome encoding causes response conflict on incongruent trials; participants performed relatively poorly on incongruent training trials, failed to respond on the basis of response–outcome associations in the outcome-devaluation test, and tended to be relatively impaired on incongruent trials of the slips-of-action (but not the baseline) test. To investigate the neural basis of resolution of outcome-induced conflict, we related striatal tractography to inter-individual variability in performance. In line with the aforementioned animal studies, we found that successful performance was positively correlated with connectivity between the caudate and the ACC. Therefore, our findings support the view that the ACC is involved in the resolution of response conflict that is caused by encoding (incongruent) outcomes in humans as well as animals. The precise mechanism by which the incongruent discrimination can be solved in a goal-directed manner remains to be established but may be related to successful deduction of the incongruent rule as supported in the present study by participants' excellent explicit knowledge of the incongruent contingencies.
Relevance for clinical psychology
An imbalance between goal-directed and habitual behavioral control has been hypothesized to underlie impulsivity (Hogarth et al., 2012) and dysfunctional behavior in a wide range of clinical conditions (Robbins et al., 2012), e.g., obsessive-compulsive disorder (Stein et al., 2009), drug abuse (Everitt et al., 2001), and obesity (Volkow and Wise, 2005). For most conditions, however, direct evidence in humans is still lacking. The instrumental paradigm used in the present study provides a useful tool to investigate these hypotheses within the framework of carefully operationalized constructs derived from animal learning theory (Dickinson, 1985; Balleine and O'Doherty, 2010). Indeed, we have recently used this task to demonstrate greater vulnerability to slips of action in obsessive-compulsive disorder, in line with the idea that these patients tend to over-rely on habitual control (Gillan et al., 2011). Furthermore, we were able to provide evidence that dopamine is involved in establishing the balance between goal-directed and habitual action control in humans (de Wit et al., 2011, 2012). On the basis of our present findings, we propose that individual differences in corticostriatal connectivity may constitute a vulnerability factor for clinical conditions that involve over-reliance on habits.
Main conclusions
Estimated strength of dissociable corticostriatal fiber tracts is predictive of vulnerability to slips of action toward no-longer-rewarding outcomes. Our results suggest that a network involving the caudate and the vmPFC subserves flexible goal-directed action that is mediated by goal anticipation and evaluation. Conversely, anatomical connectivity between the posterior putamen and PMC supports S–R habitual control. Therefore, the present study provides the first evidence that white matter integrity underlies individual differences in the balance between goal-directed and habitual action control.
Footnotes
This study was supported by the Netherlands Organization for Scientific Research. We thank James Cavanagh for his contributions to the neuroimaging analyses, Jessika Buitenweg and Martine Groefsema for their contributions to running this study, and Tarik Barri for programming support.
The authors declare no competing financial interests.
- Correspondence should be addressed to Sanne de Wit, Weesperplein 4, 1018 XA Amsterdam, The Netherlands. s.dewit{at}uva.nl
