Abstract
The vigor with which a participant performs actions that produce valuable outcomes is subject to a complex set of motivational influences. Many of these are believed to involve the amygdala and the nucleus accumbens, which act as an interface between limbic and motor systems. One prominent class of influences is called pavlovian–instrumental transfer (PIT), in which the motivational characteristics of a predictor influence the vigor of an action with respect to which it is formally completely independent. We provide a demonstration of behavioral PIT in humans, with an audiovisual predictor of the noncontingent delivery of money inducing participants to perform more avidly an action involving squeezing a handgrip to earn money. Furthermore, using functional magnetic resonance imaging, we show that this enhanced motivation was associated with a trial-by-trial correlation with the blood oxygenation level-dependent (BOLD) signal in the nucleus accumbens and a subject-by-subject correlation with the BOLD signal in the amygdala. Our data dovetails well with the animal literature and sheds light on the neural control of vigor.
Introduction
Stimuli that are pavlovian predictors of outcomes exert an influence over the vigor of participants' instrumentally trained responses both for those outcomes and for others. This pavlovian–instrumental transfer (PIT) effect, which occurs without any formal association between pavlovian and instrumental contingencies, has been extensively studied in animals (Estes, 1943; Rescorla and Solomon, 1967; Lovibond, 1981, 1983; Colwill and Rescorla, 1988; Rescorla, 1994; Holland et al., 2002; Holland and Gallagher, 2003) (for review, see Dickinson and Balleine, 1994, 2002) and is known to affect mainly habitual rather than goal-directed actions (Holland, 2004; Yin and Knowlton, 2006). PIT is one of the key paradigms used to explore motivational aspects of the interaction between instrumental and pavlovian conditioning (Dickinson and Balleine, 1994, 2002; Holland, 2004; Niv, 2007), and depends crucially on the integrity of the amygdala and the nucleus accumbens (NAcc). These two structures have long been implicated in the motivational interface between limbic and motor systems (for review, see Mogenson et al., 1980; Mogenson and Yang, 1991; Cardinal et al., 2002; Balleine, 2005; Balleine and Killcross, 2006).
A broad range of common behaviors in humans may involve influences associated with PIT. Priming of goal scripts and higher mental processes (Bargh and Chartrand, 1999) has been previously linked to the transfer of the preexperimentally learned pavlovian value of the prime (Solarz, 1960; Staats and Warren, 1974). Continued drug seeking in addiction, especially relapse in the presence of drug cues, has also been considered to reflect PIT effects (Ludwig et al., 1974) (for review, see O'Brien et al., 1998). Because the paradigms used in these literatures used stimuli that have acquired their pavlovian value preexperimentally, pavlovian learning was not under experimental control. Although pavlovian learning was controlled in studies of human conditioned suppression of instrumental responding in the presence of an aversive conditioned stimulus (CS) (Di Giusto et al., 1974; Punch et al., 1976), the transfer test was not performed under extinction, as is typical in the animal literature, thereby allowing new learning of both instrumental and pavlovian values to influence the results. Because of its considerable theoretical significance, our study attempted to extend these previous studies by providing a formal demonstration of appetitive human PIT, using a standard paradigm from animal PIT.
Here we adapted a standard animal PIT paradigm for use with humans in the context of functional magnetic resonance imaging (fMRI). Participants were first taught two separate relationships: (1) a purely predictive (i.e., pavlovian) association between a combined auditory and visual stimulus (CS+) and monetary reward; and (2) a contingent instrumental association between squeezing a handgrip and the delivery of money. We then monitored the frequency with which subjects squeezed the handgrip in extinction (i.e., without any money being delivered) in the presence of either CS+ or other (control) stimuli. We predicted that the frequency of handgrips would increase in the presence of the CS+ relative to the other stimuli and sought to study the association between this effect and the blood oxygenation level-dependent (BOLD) signal in regions associated with PIT in animals, specifically the amygdala and NAcc.
Materials and Methods
In the initial instrumental learning phase, participants learned to perform a free-operant instrumental task that involved squeezing a handgrip to obtain monetary reward. Previous work has shown that interval schedules are more conducive to the development of habitual instrumental response relative to ratio schedules, and that habitual response is more vulnerable to the effects of pavlovian cues (Holland, 2004; Yin and Knowlton, 2006). Interval schedules were somewhat difficult to administer, because animal studies typically use tonic CSs, but frequency drift in the fMRI scanner environment and other issues to do with the analysis of imaging results from block designs required that block length be kept fixed and relatively short (Henson, 2006). Consequently, we used a schedule that rewarded participants in a probabilistic rather than an interval-dependent manner. Two 1 s time windows per block were selected randomly, and participants could obtain a reward if they had squeezed the power grip with force that reached criterion within that time window.
As in animal studies, the CSs for the following pavlovian task were tonic stimuli (12 s long). Following pilot data suggesting that some participants failed to exhibit conditioning to a simple auditory CS, we used a compound CS, namely a combination of a background fractal image and a sound. In this task, participants “removed” a patch, which “hid” a monetary reward, by pressing a key whenever the patch was displayed on the computer monitor. The patch hid a reward during the CS+ but no reward during the CS−. During the baseline condition, no patches were presented. Although participants knew that reward was not contingent on their response during this pavlovian stage, our task might be considered to have an instrumental component, in that participants had to make a key press to register the stimulus. The prime motivation for this task structure was to make it as analogous as possible to animal PIT studies, which measure pavlovian conditioning to the CS+ relative to control conditions through the response frequency of trained nose poke or magazine entry actions, performed to reveal the presence of a reward and consume it (Dickinson et al., 2000; Holland, 2004; Corbit and Balleine, 2005). Our setting allowed us to use the parallel measure of key press latency for the same purpose: although the key-press action itself was instructed, the expected differential key-press latency for the different CSs could only be based on their learned value. As in animal studies, the motor action required for the pavlovian and the instrumental tasks were completely orthogonal, the former requiring a key press with the nondominant hand, and the latter a power-grip squeeze with the dominant hand. It may be that there is some degree of interference between the key press and the gripping, but we would have expected that only to weaken our results.
Participants then returned to the instrumental task and engaged in it until the extinction phase, during which no reward was given, but the CSs were presented in a pseudorandom order (Fig. 1). At the end of the experiment, we debriefed participants as to whether they gripped more, or less, as a function of CS identity. We report results from two studies, one which involved a longer extinction period, for which we only have behavioral data, and one with briefer extinction, for which we have both behavioral and functional neuroimaging data. Apart from the length of the extinction period, the procedure in both studies was identical.
The PIT paradigm used in experiments 1 and 2. Stage 1, In the pavlovian conditioning stage, participants are exposed to repeated pairings of the CS+ (a visual background and a sound) and a US (monetary reward of 20 pence), as well as presentations of a CS− that is not associated with reward. Here participants pressed a key to remove a patch that hid either a coin (CS+) or a coin with a superimposed red X (CS−). During the baseline CS, no patches were present; thus, there was no opportunity for reward. Each CS block lasted 12 s. Stage 2, During instrumental learning, participants were trained to squeeze a handgrip to obtain the same reward. Each block lasted 12 s. Stage 3, The critical PIT test took place under extinction and included presentation of the three CSs in a random order (here only the CS+ block is depicted). The participant was allowed to continue responding instrumentally.
Participants
Sixteen adult participants (mean age, 31.31 years; SD, 11.71; six females) took part in the behavioral experiment 1, and 17 right-handed adult participants (mean age, 26.30 years; SD, 5.47; 10 females) took part in the combined behavioral and fMRI experiment 2. One additional participant was excluded from further analysis because of excessive motion (>10 mm). All participants were screened for neurological/psychiatric history.
Materials
Pavlovian stimuli.
Three visual images and three sounds were combined randomly to create three compound pavlovian stimuli. These were randomly assigned as CS+, CS−, or baseline conditions. The visual images comprised 100 × 130 mm rectangles with a fractal pattern of a green, light-blue, or purple hue. The sounds were 12 s samples of a cello playing a G3 note, a French horn playing an E4 note, or a saxophone playing a G5 note. The unconditioned stimulus (US) was a 50 × 50 mm image of a 20-penny coin, presented in the middle of the screen superimposed on either the pavlovian or the instrumental visual image.
Instrumental apparatus and stimuli.
For the instrumental task, we used a hand grip apparatus molded from plastic cylinders that compressed an air tube [as described previously by Pessiglione et al. (2007)]. The tube was connected to a transducer (Honeywell, Morristown, NJ) that converted air pressure into a voltage output. Thus, variation in compression of the two cylinders by isometric handgrip resulted in a differential voltage signal that was linearly proportional to the force exerted. The signal was fed into the PC controlling stimulus presentation via a signal conditioner (CED 1902; Cambridge Electronic Design, Cambridge, UK). The dynamic value of the recorded signal was used to provide participants with a real-time visual feedback that veridically reflected the force being exerted on the hand grip, displayed as a “mercury” level moving up and down within a thermometer. The thermometer was presented over a background of a visual image similar to the pavlovian stimuli, with an orange hue, and was always accompanied by the sound of a waterfall.
Procedure
Participants were exposed to three distinct task conditions: instrumental learning, pavlovian conditioning, and PIT. Participants' maximum grip force was measured before the experiment began, and they then practiced squeezing the grip to move the mercury in the computerized thermometer up and down. Participants in the behavioral experiment performed all tasks in a quiet test room. All participants in the scanning experiment performed the pavlovian conditioning stage and PIT stage (24 instrumental learning blocks followed by the transfer test; see below) in the scanner environment, where there is background scanner noise. For practical considerations, the first nine participants performed the initial instrumental learning outside the scanner, whereas the others performed it inside the scanner; the site of instrumental learning did not interact with any experimental factor, and this factor was not considered further.
Initial instrumental learning.
Participants learned to squeeze the grip to gain monetary reward. The task consisted of 24 12 s “task on” blocks, during which the thermometer was presented, followed by a 4–12 s “off” period (average duration 8 s), during which participants were asked to fixate their gaze on a cross-hair that appeared in the middle of the screen and relax their hand (so that the baseline could be recalibrated). Participants were instructed that during the 12 s presentation of the thermometer, they were free to choose when to squeeze the grip. Participants were further instructed that each grip they make should bring the mercury to the maximum and down again, and were asked not to be concerned with the speed of the grip. They were also told that there were up to three special time windows per block and that they would gain one 20-penny coin if they happened to squeeze the grip during one of these time windows. Finally, they were assured the monetary reward was real and encouraged to use their intuition as a guide to decide when to grip. The criterion force required to reach the top of the thermometer was either 50 or 70% of participants' maximum force, and changed randomly every second. Two 1 s periods in each block were randomly selected as “rewarded time windows.” If participants reached criterion within that time window, the coin US was displayed for 1 s.
Pavlovian conditioning.
The task consisted of 36 12 s “task on” blocks, during which either the CS+ or the CS− were presented, followed by 4 s “off” period, during which the baseline stimulus was presented. A cue, followed by the US, was presented once every 4 s during each “on” block. Its onset time was sampled from a normal distribution that spanned the first 2 s of that 4 s period. Participants were asked to make speeded key presses (on a computer keyboard for experiment 1, or an MRI-compatible response box for experiment 2) with their nondominant index finger when they saw a gray patch on the screen. Pressing the key removed the patch and triggered the display of the US. Participants were explicitly informed that, unlike for the grip task, what they would gain was not dependent on performance (key press or speed). To further emphasize this point, participants were notified that the patch would be removed after 1 s even if they did not make a key press and that their speeded response was required “to allow the experimenter to gauge their attention to the task.”
At the end of the task, conditioning was assessed using choice and rating measures. A forced-choice assessment consisted of presenting pairs of stimuli, one after the other. The pairs consisted of all six combinations and orders of the three pavlovian stimuli (CS+, CS−, baseline) with the order of pair presentation being random. Participants selected which stimulus they preferred. Choice scores per stimulus were computed as the number of times each stimulus was preferred. The rating assessment consisted of presenting participants with the three stimuli in a random order requiring ratings for pleasantness using a 0–4 scale (0: very unpleasant; 1: unpleasant; 2: neutral; 3: pleasant; 4: very pleasant).
PIT.
Instructions for this phase were identical to those for instrumental learning. Participants first received 24 instrumental learning blocks: 12 blocks identical to the instrumental learning blocks (i.e., two rewarded time windows), followed by 12 partial extinction blocks (one rewarded time window). The partial extinction blocks, just before the transfer test, were used on the basis of evidence that PIT is larger after some extinction has already taken place (Dickinson et al., 2000). The transfer test followed the partial-extinction blocks, with 18 (experiment 1) or 36 (experiment 2) full-extinction blocks (zero rewarded time windows). For the transfer test, the pavlovian visual stimuli replaced the instrumental visual image, and the auditory stimuli were superimposed on the instrumental waterfall sound so that both could be heard together. One of the CSs was presented for the entire duration of each one of the extinction blocks. Block order was pseudorandomized by cycles of three, so that each one of the three stimuli was presented in each cycle of three consecutive extinction blocks.
Image acquisition
A 1.5T Siemens SONATA system (Siemens, Erlangen, Germany) was used to acquire both T1-weighted anatomical images and T2*-weighted MRI transverse echoplanar images (EPIs) (64 × 64 mm, 3 × 3 mm pixels, echo time = 90 ms) with BOLD contrast. The EPI sequence was optimized for maximizing signal in inferior brain regions (Weiskopf et al., 2006). Each EPI comprised 48 2-mm-thick contiguous axial slices taken every 3 mm, positioned to cover the whole cerebrum, with an effective repetition time of 4.32 s per volume. In total, 213 volumes were acquired in a single scanning session. The first five volumes were discarded to allow for T1 equilibration effects.
Imaging analysis
The data were analyzed using Statistical Parametric Mapping (SPM5; Wellcome Trust Centre for Neuroimaging, London, UK, http://www.fil.ion.ucl.ac.uk/spm). All volumes were realigned to the first volume to correct for interscan movement. Squeezing the power grip could cause task-related head movement. To remove unwanted movement-related variance without removing variance attributable to the motor task, images were realigned and unwarped in SPM5 (Andersson et al., 2001). Given the observed variance after realignment and the realignment parameters, estimates of how deformations changed with participant movement were made, which were subsequently used to minimize movement-related variance. The motion-correction parameters were also used to determine whether head motion differed significantly between the conditions; there were no significant differences in translation or rotation motion between conditions, p > 0.10 in every case. The mean of the motion-corrected images was then coregistered to the individual's structural MRI using a 12-parameter affine transformation. To correct for their different acquisition times, the signal measured in each slice was shifted relative to the acquisition of the middle slice using sinc interpolation in time. The resulting volumes were then normalized to a standard EPI template based on the Montreal Neurological Institute (MNI) reference brain in Talairach space (Talairach and Tournaux, 1998) and resampled to 3 × 3 × 3 mm3 voxels. All normalized images were then smoothed with an isotropic 8 mm full-width half-maximum Gaussian kernel to account for intersubject differences and allow valid statistical inference according to Gaussian random field theory (Friston et al., 1995a,b). The time series in each voxel were high-pass filtered at 1/128 Hz to remove low-frequency confounds and scaled to a grand mean of 100 over voxels and scans within each session.
A random-effects, event-related, statistical analysis was performed with SPM5. First, we specified a separate general linear model for each participant. According to the behavioral data, PIT was significant during the first four full-extinction cycles. To take this into account, the critical regressors focused on fMRI data acquired during these four PIT cycles (Fig. 2). Three regressors represented the three block conditions: CS−, CS+, and baseline. Three additional regressors represented the tonic parametric modulation of activity in each block by grip frequency, entered as a standardized, mean-corrected number of grips per block. The seventh regressor of no interest modeled phasic grip activity as single events with a δ-function vector. Pre-PIT task blocks were modeled in the same way, with one regressor modeling blocks and a second modeling grip frequency per block; the last regressor of no interest modeled all post-PIT blocks (the final two blocks). Regressors were convolved with a generic hemodynamic response function and entered into a nine-column design matrix. The mean of each scan run was removed on a voxelwise basis.
The model used to analyze the fMRI PIT data. Regressors 1, 3, and 5 represented the baseline, CS+, and CS− blocks, respectively. Regressors 2, 4, and 6 represented the tonic parametric modulation of activity in each block by grip frequency. Regressor 7 accounted for phasic grip activity as single events with a δ-function vector. Regressors 8 and 9, not pictured here, modeled pre-PIT task blocks and post-PIT blocks. The difference in grip frequency modulation as a function of CS+ and CS− was examined by contrasting regressors 4 and 6. The global PIT contrast examined the correlation of global PIT with the contrast of regressors 3 and 5.
PIT effects were identified on the basis of two critical contrasts. First, to identify neural activity associated with enhanced grip frequency as a function of CS context within subjects, we contrasted the parameter estimates of parametric grip frequency modulators during CS+ and CS− blocks. The contrast images were then entered into a second-level analysis, using a one-sample t test (we excluded one participant from this contrast because she gripped exactly the same number of times during each block in which the CS− was presented). This contrast measures the CS-dependent coupling between grip frequency and BOLD, but explicitly discounts signals that are constant during the presentation of each CS that might influence the excess gripping. To take such signals into account, global PIT was computed per participant as a proportional increase in average number of grips in the CS+ blocks relative to the CS− blocks. To identify neural activity associated, between subjects, with enhanced grip frequency as a function of CS context, we contrasted the CS+ and CS− blocks, and entered this contrast image to a one-sample t test, with global PIT as a covariate of interest. Because participants who show more PIT may also have increased head motion in the CS+ block relative to the CS− block, we also included six additional covariates of no interest, representing difference between the six translation and rotation motion-correction parameters in the CS+ and CS− blocks. This second-level model was also used to examine mean differences in activation for CS+ relative to CS− (CS context effect). This CS context contrast thus covaried out between-subject differences in global PIT (as well as potential motion differences). By including frequency modulation in the first-level model, the CS context contrast also covaried out within-subject PIT. Notably, because the first-level model included both overall CS context effects, as well as activity related to phasic handgrip squeezing, both the within-subject and between-subject PIT contrasts are uncontaminated by these contextual and motoric factors.
Single subject T1-weighted anatomical images were normalized to a standard MNI template, and parameters from the normalization were applied to contrast images in the second-level analysis to optimize their transformation to MNI space. We thresholded the summary statistical map at p < 0.001 (uncorrected for multiple comparisons). These maps were overlaid on the group's averaged high-resolution structural image in MNI orientation.
Results
Behavioral analysis: behavioral experiment
Initial instrumental learning
Participants readily learned to squeeze the power grip for reward. Supplemental Figure S1 (available at www.jneurosci.org as supplemental material) shows that the number of grips and the force exerted stabilized after the first two blocks. The Greenhouse-Geiser correction was applied when necessary in all the following ANOVAs to correct for violations of the sphericity assumption. Analysis of blocks 2–24 showed that the number of grips [F(21,315) = 2.48, mean squared error (MSE) = 7.48, p < 0.05, partial η2 = 0.14; linear trend: F(1,15) = 8.28, MSE = 36.82, p < 0.05, partial η2 = 0.36] and force [F(21,315) = 2.18, MSE = 42.18, p < 0.05, partial η2 = 0.13; linear trend: F(1,15) = 6.25, MSE = 222.66, p < 0.05, partial η2 = 0.29] declined linearly with time.
Pavlovian conditioning
Pavlovian conditioning was apparent both as a significant difference in latency to respond to the cue preceding the CS and in forced choice and preference rating scores (Fig. 3a). Participants were faster in the CS+ blocks than in the CS− blocks (t(15) = 4.225, p = 0.001). The effect of condition (CS+, CS−, baseline) was significant for both forced choice (F(2,30) = 17.07, MSE = 1.85, p < 0.001, partial η2 = 0.53) and rating (F(2,30) = 10.55, MSE = 0.78, p < 0.001, partial η2 = 0.41). Participants preferred the CS+ over the other two stimuli (F(1,15) = 48.55, MSE = 1.76, partial η2 = 0.76, Cohen's f = 1.78), which did not differ significantly (F(1,15) = 2.43, MSE = 5.05, p > 0.10, partial η2 = 0.14). Participants rated the CS+ as more pleasant than the baseline stimulus (F(1,15) = 4.84, MSE = 1.56, p < 0.05, partial η2 = 0.24, Cohen's f = 0.56), and the baseline stimulus as more pleasant than the CS− (F(1,15) = 6.43, MSE = 1.40, p < 0.05, partial η2 = 0.30, Cohen's f = 0.65). Participants failed to respond in a minority of trials, 7.52% of the cases (SD = 18.0%).
Pavlovian conditioning. Bars (left axis) depict forced-choice preference and pleasantness rating of each of the stimuli. The line graph (right axis) depicts latency to respond to the cue that preceded the CS+ or CS−. A, Data from the behavioral experiment. B, Data from the fMRI experiment. Error bars represent SE.
PIT test
The number of grip squeezes per block that passed a criterion of 50% of participants' maximal force was entered into a 3 (CS: CS+, CS−, baseline) × 12 (blocks) repeated-measures ANOVA. The effect of CS was significant (F(2,30) = 4.81, MSE = 74.14, p < 0.05, partial η2 = 0.24) (Fig. 4a). Planned contrasts showed that participants' grip frequency was higher in the presence of CS+ than in the presence of CS− (F(1,15) = 6.05, MSE = 110.25, p < 0.05, partial η2 = 0.29, Cohen's f = 0.64) or in the baseline condition (F(1,15) = 5.07, MSE = 157.5, p < 0.05, partial η2 = 0.25, Cohen's f = 0.58). Grip frequency in the presence of CS− or baseline did not differ significantly (F(1,15) < 1). Extinction was apparent in the significant effect of blocks (F(11,165) = 3.32, MSE = 98.52, p < 0.05, partial η2 = 0.18), which exhibited a significant linear trend (F(1,15) = 7.27, MSE = 43.4, p < 0.05, partial η2 = 0.33). Blocks and CS did not interact significantly. Grip force was subjected to the same analysis, although we sought to minimize this effect through the experimental instructions and the visual feedback (“thermometer” bar). The effect of CS on grip force was duly not significant.
Pavlovian–instrumental transfer. The number of times participants squeezed the power grip during the transfer test is depicted as a function of the pavlovian stimulus presented during the block. A, Data from the behavioral experiment. B, Data from the fMRI experiment.
Declarative awareness of PIT
Participants were classified as being “aware” or “unaware” of the PIT effect according to their response to the following debriefing question: “Sometimes there were sounds playing with the grip task and images of different colors presented in the background. How did that affect you, if at all?” To be classified as aware, participants had to indicate that the stimuli influenced their behavior in the predicted way (e.g., “In the high pitch tone [this participant's CS+], you knew you will win money, it was easier to grip, you were faster, it affected you”). Responses to the debriefing question were scored for exploratory purposes, although this assessment of postexperimental awareness of PIT cannot determine a causal role for awareness at the time of test. Nine participants (56%) were classified as aware. Aware participants exhibited larger PIT than unaware participants (r = 0.62, p = 0.01). The difference between grip frequency in the presence of the CS+ and CS− was significant for aware participants (t(8) = 3.19, p < 0.05) but not for unaware participants (t(6) = 0.0).
Behavioral analysis: fMRI behavioral task
Initial instrumental learning
Participants readily learned to squeeze the power grip for reward. The number of grips did not significantly change across task blocks (F < 1) (supplemental Fig. S1, available at www.jneurosci.org as supplemental material). Force declined with time (F(21,315) = 1.96, MSE = 59.77, p < 0.01, partial η2 = 0.12; cubic trend: F(1,15) = 2.66, MSE = 95.94, p < 0.05, partial η2 = 0.31).
Pavlovian conditioning
Pavlovian conditioning was apparent in a significant difference in latency to respond to the cue preceding the CS (Fig. 3b). Participants were faster in the CS+ blocks relative to the CS− blocks (t(16) = 1.98, p < 0.05, one-tailed, Cohen's d = 0.22) (Cohen, 1988). There was a trend for preference and rating scores to differ for the CSs in the direction seen in experiment 1, but it did not reach significance. Participants failed to respond in a minority of trials, 1.03% of the cases (SD = 1.46%).
PIT test
The number of squeezes per block that passed a criterion of 50% of participants' maximal force was entered into a 3 (CS) × 6 (blocks) repeated-measures ANOVA. Extinction was apparent in the significant effect of blocks (F(5,80) = 7.56, MSE = 35.5, p < 0.001, partial η2 = 0.32), which exhibited a significant linear trend (F(1,16) = 19.67, MSE = 35.36, p < 0.001, partial η2 = 0.55). The main effect of CS was not significant (F < 1), but it significantly interacted with blocks (F(10,160) = 2.8, MSE = 71.91, p < 0.05, partial η2 = 0.15). This significant interaction led us to focus on the earlier blocks, in which we expected the effect of PIT to be maximal, because extinction could diminish the differences between CSs and, consequently, their motivational impact. On the basis of the results from the behavioral experiment (experiment 1), we used one-tailed post hoc t tests to examine the impact of the CS on average grip frequencies in blocks 1–4 (experiment 2). As Figure 4b shows, in the first four blocks participants gripped more frequently in the presence of CS+ than in the presence of CS− (t(16) = 2.08, p < 0.05, Cohen's d = 0.28) or in the baseline condition (t(16) = 1.91, p < 0.05, Cohen's d = 0.22), which did not differ (t < 1).
Declarative awareness of PIT
Six participants (35%) were classified as aware. The difference between grip frequency in the presence of the CS+ and CS− was marginally significant for aware participants (t(5) = 2.16, p = 0.078) but not for unaware participants (t(10) < 1).
Neuroimaging results
The only significant brain region activated in the within-subject PIT contrast was the NAcc (T = 5.75, x = 4, y = 0, z = −2, 34 voxels) (Fig. 5). The β values corresponding to the correlation of NAcc activity with grip frequency were significantly higher than zero in the CS+ blocks (T = 2.28, p < 0.05), but negative and significantly lower than zero in the CS− blocks (T = −2.3, p < 0.05). An 8 mm sphere around the peak activated voxel in this cluster was used as a mask to probe the pre-PIT grip frequency regressor. In this analysis, no significant activations were found even with a lenient p = 0.05. This suggests that change in grip frequency in the absence of predictive CSs does not significantly modulate the NAcc.
NAcc activity associated with PIT. The difference in the relation of NAcc to instrumental responding in the CS+ and CS− conditions was significant, as shown in the contrast image. The glass brain inset shows this was the sole activation in the brain associated with this contrast. The bar graph shows parameter estimates for the CS+ and CS− conditions, averaged across participants. NAcc activity was positively correlated with grip frequency in the presence of CS+ and negatively correlated with it in the presence of the CS−. Error bars represent the 90% confidence interval. *p < 0.05.
Using the same mask to examine activation in the between-subject PIT contrast, we found that activation in this region not only differentiated the effects of the CSs on grip frequency, but also correlated with the magnitude of the global PIT effect (T = 4.96, x = 4, y = 8, z = −2, 7 voxels). In addition to the NAcc, global PIT was also associated with right amygdala activation (T = 6.16, x = 20, y = −6, z = −18, 18 voxels). The left amygdala was also activated in this contrast, but the peak activation in this cluster was located at the junction of anterior hippocampus and amygdala (T = 8.11, x = −30, y = −10, z = −18, 84 voxels). Figure 6 depicts this activation profile, with the bar graphs plotting parameter estimates for the correlation of global PIT with each CS parameter estimate for the peak voxel in the NAcc and the right amygdala. These graphs show that the correlations with global PIT were significant and positive in the CS+ condition (NAcc: T = 3.07, p < 0.001; right amygdala: T = 2.25, p < 0.001) but did not differ from zero in the CS− condition (p > .05). Supplemental Figure S3 (available at www.jneurosci.org as supplemental material) shows that similar amygdala activations were obtained in participants who trained inside and outside the scanner. In summary, the global, between-subject, PIT contrast showed that participants who exhibited stronger PIT activated their amygdala and NAcc more strongly in the CS+ block than in the CS− block.
Amygdala activity associated with PIT. Participants who showed a larger global PIT expressed enhanced bilateral amygdala activation. The bar graph shows, for the right amygdala and NAcc, mean parameter estimates for the correlation, across participants, of global PIT with the parameter estimate in each CS condition. Error bars represent the 90% confidence interval. *p < 0.001.
For completion, we asked whether the amygdala or the NAcc regions of interest also showed significant activation associated with the CS context after covarying out both within- and between-subject PIT effects. Eight millimeter spheres around the peak activated voxels in the right amygdala and the NAcc clusters, and a significantly activated voxel in the left amygdala, were used as masks, and we examined activation in the CS context contrast with a lenient p = 0.005, in light of small search volumes and a clear a priori prediction. Supplemental Figure S2 (available at www.jneurosci.org as supplemental material) shows that as in the between-subject PIT contrast, the CS context activated all three regions (NAcc: T = 2.52, x = 4, y = −6, z = −6, 7 voxels; right amygdala: T = 2.56, x = 24, y = −6, z = −18, 43 voxels, and T = 2.13, x = −22, y = 0, z = −20, 15 voxels; left periamygdala cortex: T = 6.29, x = −28, y = −12, z = −20, 133 voxels). Together, the activation pattern in the amygdala is consistent with the notion that significant mean differences in response to CS context was correlated with PIT as an index of conditioning, namely, being larger in participants who exhibited larger PIT.
Discussion
Our data demonstrate that, as for other animals, the vigor with which humans execute instrumental actions is subject to motivational influences arising from incidental pavlovian conditioned stimuli. Acquisition of pavlovian conditioning was expressed in differential latencies to, and preferences and ratings of, the CS+ relative to the CS−. Participants were faster to respond to the prereward cue in the presence of the CS+ relative to the CS−. Participants also preferred the CS+ compared with the CS−, rated the CS+ as more pleasant than the CS−, although these effects only attained statistical significance in the behavioral study. Participants readily acquired the instrumental response and squeezed the handgrip to obtain a reward, a response that, under extinction, was more vigorous in the presence of the CS+ relative to the CS−.
Our key neurobiological findings were that the NAcc and the amygdala both showed activity associated with PIT. Activity in the NAcc was parametrically modulated as a function of PIT on a trial-by-trial basis. Animal studies have shown that destruction of this region abolishes PIT (Corbit et al., 2001; Hall et al., 2001; de Borchgrave et al., 2002). Conceptually, the NAcc is often viewed as a “limbic–motor interface” (Mogenson et al., 1980; Mogenson and Yang, 1991; Cardinal et al., 2002), allowing associative information (pavlovian incentive value) to influence instrumental responding. Indeed, theoretical considerations favor PIT as a basic mechanism by which the vigor of habitual actions can be regulated; goal-directed actions, which can in any case be controlled directly by the desirability of their consequences, appear to be less susceptible (Dickinson and Balleine, 1994, 2002; Holland, 2004; Dayan et al., 2006; Niv et al., 2006; Yin and Knowlton, 2006).
Our finding that activation in this structure was positively correlated with response frequency in the presence of CS+, negatively correlated with frequency in the presence of CS−, and positively correlated with the magnitude of the global PIT effect across individuals is consistent with the idea that this region mediates incentive modulation of instrumental action. Given the evidence that dopamine release in the NAcc increases BOLD signal (Knutson and Gibbs, 2007), this correlation pattern dovetails with the findings that presentation of appetitive CS increased DA in this structure (Bassareo and Di Chiara, 1999) and that increased DA enhances, and DA depletion reduces, PIT magnitude (Smith and Dickinson, 1998; Dickinson et al., 2000; Wyvell and Berridge, 2001).
The differential activation of the amygdala in response to the pavlovian CSs was stronger in participants who showed larger PIT effects, although in a way that was not correlated with block-by-block differences in grip frequencies. This is what would be expected if the amygdala reports information about the pavlovian motivational significance of the CSs (Tye and Janak, 2007) but is upstream of the processes actually controlling moment-by-moment choices of whether or not to grip; indeed, just such an organization is suggested by data on rats (Hatfield et al., 1996; Killcross et al., 1997) (for review, see Gallagher and Holland, 1994; Balleine, 2005; Balleine and Killcross, 2006. It would be interesting to design an experiment optimized for dynamic causal modeling to assess the functional link between amygdala and accumbens during PIT in humans.
Appetitive PIT actually comes in two flavors: specific and general (Corbit and Balleine, 2005; Dickinson and Balleine, 2002; Holland, 2004). In general PIT, the pavlovian stimulus enhances any appetitive instrumental response, even when the instrumental response is associated with a different outcome. In contrast, in specific PIT, the pavlovian stimulus associated with a certain outcome only enhances an instrumental response associated with the same outcome. When pavlovian and instrumental outcomes are the same, as in the present experiment, the design is more powerful, because both specific and general mechanisms may play a role in the overall PIT effect. However, we note that without an assay for the specificity of the PIT effect (e.g., with a transfer test using a second instrumental response associated with a different outcome), we cannot determine which type of PIT obtained here. It is also worth noting that, at least in rats, the central and basolateral nuclei of the amygdala (Holland and Gallagher, 2003; Corbit and Balleine, 2005) and the core and the shell of the NAcc (Corbit et al., 2001; Hall et al., 2001) are selectively involved in general and specific PIT, although there is not complete agreement about which does what. Unfortunately, the spatial resolution of our imaging did not allow us to distinguish between the substructures within these regions in any event.
Comparison of effect sizes from the behavioral results in the two experiments shows that pavlovian conditioning and PIT were more robust outside the scanner environment, whereas extinction progressed more rapidly in the scanner. The scanner environment is not optimal for eliciting subtle behavioral effects, being loud and to varying degrees intimidating for some participants, all of which draw on resources that may otherwise be dedicated to the task. Importantly, PIT and extinction may be particularly vulnerable if the environment retards the development of habitual responding, because it is larger when behavior is controlled by the habitual relative to the goal-directed instrumental system (Holland, 2004) (see also Dayan et al., 2006).
In an exploratory test of awareness of PIT, we found that PIT was larger and only significant in participants who were classified as aware. This finding should be interpreted with caution, because of the informal nature of the postexperimental debriefing question and the small number of participants in each group. We interpret this finding to mean that participants who exhibited larger PIT reported it veridically. Participants' reports cannot determine their state of awareness of the transfer during the test itself but suggest an interesting avenue for future research.
Ours is the first neuroimaging investigation of PIT in humans, and extends previous behavioral demonstrations of PIT by using a formal paradigm, established in animal research. One previous behavioral study (Paredes-Olay et al., 2002) focused on transfer associated with causal reasoning, but did so in a way that renders comparison with the animal phenomenon inconclusive. In that study, both the pavlovian conditioning and the PIT stages were instructed, implying a possible confound, and involved choice between alternatives [which may preserve goal-directed instrumental control (Daw et al., 2005; Holland, 2004)]. Furthermore, the pavlovian conditioning was only assessed using self-report, and, unlike the animal studies, the rate of instrumental responding in the presence of the CS+ was not higher than the baseline rate. Similar methodological vulnerabilities also characterize the usage of this paradigm to demonstrate conditioned suppression (Arcediano et al., 1996; Havermans et al., 2005; Neumann, 2006). Other studies that examined the influence of experimentally acquired pavlovian value on instrumental responding (Remington and Strongman, 1970; Di Giusto et al., 1974; Punch et al., 1976; Reed, 1996) have not tested transfer under extinction.
PIT is important because it provides a compelling account of data on apparently non-normative influences on behavior. In keeping with the rodent studies, we focused on the vigor of the execution of hard-learned instrumentally appropriate actions, but the same idea may be used to account for such phenomena as impulsive choice in the face of immediate appetitive outcomes (Ainslie, 1999) (but see Dayan et al., 2006), which also depends on the NAcc (Cardinal et al., 2001; King et al., 2003; Cardinal, 2006). PIT has perhaps been most influential in studies of drug addiction (Everitt et al., 2001), as an account of the effect of environmental drug-associated cues in triggering drug-seeking behavior and relapse in recovering addicts (O'Brien et al., 1998). It would be interesting to extend the explanatory schema PIT offers to account for other common human behaviors, such as framing effects in economic decision making (Tversky and Kahneman, 1981; De Martino et al., 2006) and economic behaviors such as shopping (Knutson et al., 2007).
The understanding of instrumental behavior as motivated both by pavlovian and by instrumental controllers offers a subtly different explanatory schema from two-systems accounts, which contrast some forms of emotional and rational processing (Loewenstein, 1996; Sloman, 1996; Kahneman and Frederick, 2002; McClure et al., 2004). Our findings of the incentive effects of pavlovian cues and the possible roles of NAcc and amygdala enrich the animal studies, and open many avenues to future investigation.
Footnotes
-
This work was supported by a Wellcome Trust Programme grant (R.J.D.) and the Gatsby Charitable Foundation (P.D.). We thank John Parkinson for discussions in relation to PIT, E. Featherstone for excellent technical support, G. C. Tan for assistance with image processing, and M. Pessiglione and two anonymous reviewers for their constructive comments.
- Correspondence should be addressed to Deborah Talmi, Wellcome Trust Centre for Neuroimaging, Institute of Neurology, UCL, 12 Queen Square, WC1N 3BG London, UK. d.talmi{at}fil.ion.ucl.ac.uk
