WWW.JNEUROSCI.ORG
-
The Journal of Neuroscience Advertisement
QUICK SEARCH:   [advanced]


     
-


HOME  |   SEARCH  |   ARCHIVE  |   SUBSCRIBE  |   CONTACT  |   HELP
The Journal of Neuroscience, March 9, 2005, 25(10):2733-2740; doi:10.1523/JNEUROSCI.3360-04.2005

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an eLetter
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via ISI Web of Science (25)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Cox, S. M. L.
Right arrow Articles by Johnsrude, I. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Cox, S. M. L.
Right arrow Articles by Johnsrude, I. S.

 Previous Article  |  Next Article 

Behavioral/Systems/Cognitive
Learning to Like: A Role for Human Orbitofrontal Cortex in Conditioned Reward

Sylvia M. L. Cox,1,2,3 Alexandre Andrade,1,4 and Ingrid S. Johnsrude1,5

1Medical Research Council Cognition and Brain Sciences Unit, Cambridge CB2 2EF, United Kingdom, 2Experimental Psychopharmacology Unit, Brain and Behavior Institute, University of Maastricht, 6200 MD Maastricht, The Netherlands, 3Department of Psychiatry, McGill University, Montreal, Quebec, Canada H3A 1A1, 4Institute of Biophysics and Biomedical Engineering, University of Lisbon, 1700 Lisbon, Portugal, and 5Department of Psychology, Queen's University, Kingston, Ontario, Canada K7L 3N6


   Abstract
Top
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 References
 
A great deal of human behavior and motivation is based on the intrinsic emotional significance of rewarding or aversive events, as well as on the associations formed between such emotional events and concurrent environmental stimuli. Recent functional neuroimaging studies have implicated the ventral striatum, orbitofrontal cortex (OFC), and amygdala in the representation of reward values and/or in the anticipation of rewarding events. Here, we use functional magnetic resonance imaging to compare brain activation during the presentation of reward with that during presentation of (conditioned) stimuli that have been paired previously with reward. Specifically, we aimed to investigate conditioned reward in the absence of explicit reward anticipation. Twenty-two healthy volunteers were scanned while monochrome visual patterns were incidentally associated with reward or negative feedback in the context of a simple card game. In the subsequent session, visual patterns, including the conditioned stimuli, were presented without reward or negative feedback, and the affective valence of these stimuli was assessed behaviorally. The presentation of reward compared with negative feedback activated the ventral striatum and OFC. Activation in the same OFC region was observed when, in the subsequent session, subjects passively viewed the stimuli that had been paired with reward, without the administration of reward and with subjects being essentially unaware of the conditioning manipulation. These findings suggest that the OFC in humans plays an important role in the representation of both rewarding stimuli and conditioned stimuli that have acquired reward value.

Key words: appetitive conditioning; nucleus accumbens; prefrontal cortex; amygdala; preference judgments; financial reward


   Introduction
Top
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 References
 
Novel stimuli in the environment can acquire affective and motivational significance via their association with rewarding events. This process of pavlovian conditioning has consequences for behavior, because these conditioned stimuli can act as rewards in their own right, eliciting behavior and affecting mood. Conditioned reward is not a unitary phenomenon, however (Cardinal et al., 2002Go), and at least two kinds of learning can be distinguished. First, the conditioned stimulus (CS) comes to share affective properties with the reward: it takes on a reward value of its own (here referred to as intrinsic reward value). Second, the CS comes to explicitly predict the occurrence of a rewarding outcome. In this study, we examine the brain substrates of conditioned reward. We begin to decouple the rewarding and predictive values of a CS by using a procedure that minimizes the degree to which volunteers can explicitly predict reward when presented with a CS.

Behavioral procedures in animals have shown how a CS acquires incentive value and have implicated the amygdala and orbitofrontal cortex (OFC) in this process (Holland and Gallagher, 1999Go; Parkinson et al., 2000Go, 2001Go; Chudasama and Robbins, 2003Go; Pears et al., 2003Go). In addition, neurophysiological work in animals has demonstrated that the ventral striatum plays an important role in processing rewarding stimuli and in the anticipation of predicted, as well as conditioned, rewards (Everitt et al., 1999Go; Schultz, 2000Go).

Recent functional neuroimaging studies in humans have yielded results that are consistent with a wealth of animal research (Gallagher et al., 1999Go; Schoenbaum et al., 1999Go; Schultz, 2000Go; Parkinson et al., 2001Go; Baxter and Murray, 2002Go) in demonstrating that the amygdala, ventral striatum, and OFC are involved in aspects of reward (Delgado et al., 2000Go; Elliott et al., 2000Go; J. O'Doherty et al., 2001aGo; Small et al., 2001Go) and conditioned reward (Gottfried et al., 2002bGo, 2003Go; Arana et al., 2003Go; Kirsch et al., 2003Go; J. P. O'Doherty et al., 2003Go). However, it is unclear in these studies of conditioned reward to what extent the results are explained by (explicit) reward anticipation or by other components of reward conditioning, such as the representation of the affective value of the CS.

Our previous work demonstrates that preferences can be conditioned for previously neutral stimuli after repeated pairings with rewarding events in the absence of explicit reward anticipation (Johnsrude et al., 1999Go, 2000Go; Cox et al., 2002Go). However, the brain system supporting this preference learning has not yet been explored in detail. Here, we use a procedure, based on that of Johnsrude et al. (1999Go), during which abstract visual patterns are consistently associated with monetary and auditory reward. In a subsequent session, the patterns are presented alone and their (conditioned) affective valence is assessed behaviorally. Functional magnetic resonance imaging (fMRI) scans are acquired continuously throughout both of these experimental sessions. Subsequent debriefing ensures that volunteers are mostly unaware of reward contingencies. This design allows us to study neural responses in response to reward as well as to conditioned reward in the absence of explicit reward anticipation.


   Materials and Methods
Top
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 References
 
Subjects
Twenty-two right-handed, healthy volunteers (12 males, 10 females; 18-30 years of age) took part in this study. The participants, all of whom were students, were recruited from the Medical Research Council Cognition and Brain Sciences Unit volunteer panel. All of the participants gave informed consent. This study was approved by the Local Research Ethics Committee (Cambridge, UK).

Experimental design
Participants were scanned while performing two separate tasks, which were presented in a fixed order. The first task (conditioning) consisted of two components: (1) a cover task, which was a simple card game, and (2) a conditioning procedure, concurrent with the card game, in which visually discriminable abstract patterns were associated with reward or negative feedback. The card game served to mask the relationships between the visual patterns and reward to prevent the subjects from being aware of the conditioning manipulation.

During the second task (judgment), these abstract patterns were presented on their own, without reward or negative feedback, and subjects were asked to indicate their preference for individual patterns via a paired-comparison procedure.

Conditioning
Cover task. A schematic drawing of a set of trials in the conditioning phase is shown in Figure 1. Subjects were presented with virtual playing cards, shown face up, one at a time (duration, 500-2000 ms), on a computer screen. Subjects were told that each card had a number written on the "back" of it. They were to guess whether the number on the back was higher or lower than the value shown. After the response (and an interstimulus interval of 1500 ms) made with either the right (higher) or left (lower) index finger to press a key on a button box, the back of the card was shown (duration, 2000 ms), and feedback (reward or negative feedback) was given simultaneously. On "correct" trials, accompanied by reward, the number superimposed on the back of the card was presented in green. On "incorrect" trials, accompanied by negative feedback, the number was presented in red. Reward consisted of £0.25 and a melodic flourish (duration, 2670 ms) played through the headphones. Negative feedback was a loss of £0.22 and a buzzer sound (duration, 960 ms). After an intertrial interval of 1000 ms, the following trial was initiated. A "reward bar," continuously visible on the right side of the computer screen, represented the cumulative winnings. This bar rose or fell simultaneously with the presentation of reward or negative feedback. The playing cards varied from numbers 4 to 9. The outcome numbers on the back varied from 1 to 12.



View larger version (22K):
[in this window]
[in a new window]
  		Figure 1. The three stages of the experimental procedure. A, Schematic drawing of the trials in the conditioning task. In the first trial, a correct trial, a card showing the number 5 was presented. The subject made the response "high," after which the outcome number (7) appeared in green, overlaid on one of the five patterns, accompanied by a melodic flourish and an increase in the value depicted by the reward bar (representing +£0.25), presented on the right. In the second trial, an incorrect trial, the subject chose "high," on which the outcome (the number 4 in red) was shown, accompanied by a buzzer sound and a fall of the reward bar (reflecting -£ 0.22). B, Schematic drawing of the trials in the judgment task. On each trial, a set of two patterns was presented: first individually and then followed by the pair. When the pair was shown, the subject chose the one they most preferred. C, Appearance of the screen during debriefing.

 
Conditioning procedure. Unknown to the subjects, the outcome of a trial (correct or incorrect) was prespecified, and the outcome numbers were (pseudorandomly) chosen after the subject's response to conform to this prespecified outcome. Thus, if the face of the card was a 4, the subject pressed "higher," and the trial outcome was supposed to be correct, then the number written on the back of the card would be between 5 and 12. Conversely, if that trial was specified to be incorrect, the number on the back would be 1, 2, or 3. Five visually discriminable, abstract patterns (Petrides and Milner, 1982Go; Johnsrude et al., 1999Go) were presented on the backs of the cards, and the outcome number was superimposed on these. These patterns were paired with reward and negative feedback in predetermined ways at different contingencies. The patterns were not mentioned in the instructions and were irrelevant to the card game cover task. The "positive pattern" was associated with reward on 90% of the trials in which it appeared and with negative feedback on the other 10% of trials. The "negative pattern" was accompanied by reward on 10% of the trials in which it appeared and by negative feedback on the other 90%. The other three patterns were each associated with reward on 50% of the trials in which they appeared and with negative feedback on the other 50% ("bivalent" patterns). These bivalent patterns were included to obscure the pattern-reward/pattern-negative feedback contingencies. To avoid stimulus-specific effects, five versions of the task were prepared, and subjects were randomly assigned to a particular version. In each version, different patterns were assigned to each of the reinforcement contingencies. Thus, the individual patterns that served as positive, negative, and bivalent patterns were different for different subjects.

Two runs of 120 trials (120 correct and 120 incorrect trials) were presented. The positive and negative patterns and one of the three bivalent patterns were presented 60 times each. The other two bivalent patterns were presented 30 times each. Trial order was pseudorandom and fixed. Trial length was dependent on subjects' reaction times and varied from 5000 to 6500 ms. The total length of the task was ~30 min.

Judgment
Eight different patterns were used in this part of the experiment: the five patterns from the conditioning task plus three novel patterns (Johnsrude et al., 1999Go). Each trial consisted of a set of two patterns. Within each trial, the patterns were first presented one at a time, in the center of the screen, in succession (stimulus duration, 1000 ms; interstimulus interval, 500 ms), and then both were presented simultaneously, with one on either side of the screen (Fig. 1 B). When shown together, subjects had to indicate, with a button press, which of the two patterns they preferred (they were instructed "indicate the one you like better: don't think about it too hard, just go with your first impression"). Stimuli remained on the screen until a response was made, again using either the right or left key on the button box. After an intertrial interval of 1000 ms, the next pair of stimuli was presented. Importantly, no reward or negative feedback was given during the judgment task. A total of 72 trials were presented. Each pattern was shown in 18 trials: 9 times on the right and 9 times on the left. In 30 trials, the positive, negative, and bivalent patterns were paired with each other, which allowed us to investigate conditioning effects independent of difference in exposure. In 10 of these 30 trials, the positive pattern was paired directly with the negative pattern. Thirty trials were presented in which each of the novel patterns (those stimuli only presented during judgment) was presented with each of the five patterns shown during the conditioning task. Finally, 12 trials were presented in which each of the novel patterns was paired with another novel pattern. Trial length was dependent on subjects' reaction times and varied from 4500 to 6000 ms. The total length of the task was ~7 min.

Debriefing
Participants were questioned carefully after scanning to determine the degree to which they were aware of the experimental contingencies and the purpose of the experiment. All eight of the patterns that were seen during the judgment condition were presented simultaneously on the screen (Fig. 1C). A number on top of each pattern indicated how many times (of a possible maximum of 18) it had been chosen during the judgment condition. Subjects were asked the following questions to assess their perceptions of their preferences and their awareness of the conditioning manipulation: (1) for the two most-preferred patterns, "You chose this pattern most. Why?"; (2) for the two least-preferred patterns, "You chose this pattern least. Why?"; (3) "Did you see any of these pictures during the card game?" If the subjects answered "yes," they were then asked "Which ones?"; and (4) "Did you notice anything in particular when you saw them?" If the subjects did not say anything spontaneously, "for example, with a particular number or color?" was used as a prompt.

Finally, participants were fully informed about the procedure and purpose of the experiment, reimbursed for their time, and were also paid the money that they had "won" during the card game (£3.60).

Experimental stimulus delivery
Custom software was written in Visual Basic version 6.0 (Microsoft, Redmond, WA). Stimuli were back projected onto a translucent screen positioned within the bore of the magnet behind each subject's head, visible in a mirror placed just above each subject's head. Subjects also wore magnet-compatible, high-fidelity electrostatic earphones, mounted in sound-attenuating ear defenders (Palmer et al., 1998Go).

Image acquisition
Imaging data were collected using a 3 tesla Bruker MedSpec MR system (Ettlingen, Germany). Gradient echo T2*-weighted echo planar images (EPIs) were acquired continuously during conditioning and judgment, with a repetition time of 3 s. Whole-brain volumes of 21 slices (4 mm thick, with a 1 mm interslice gap and an in-plane resolution of 1.95 x 1.95) were collected in an axial-oblique orientation, angled slightly away from the eyes. In addition, gradient-echo non-EP magnetic-field map images were acquired to calculate the deviations of the magnetic field from the desired homogenous field. This information was used to geometrically undistort the EPIs. Finally, a T1-weighted high-resolution anatomical scan of each volunteer was acquired. The start of each run was synchronized with the onset of scanning. The first six EPIs in each session were discarded to avoid T1 equilibrium effects.

Image analysis
fMRI data were preprocessed and analyzed using statistical parametric mapping 99 (SPM99) (Wellcome Department of Cognitive Neurology, London, UK) and statistical nonparametric mapping (SnPM) (Nichols and Holmes, 2002Go).

Before analysis, images were corrected for slice timing, using the first slice of each scan as a reference. To correct for motion, images were realigned with respect to the first image, and a mean realigned image was created. EPIs were then geometrically undistorted using field maps (Cusack et al., 2003Go), followed by spatial normalization to the standard Montreal Neurological Institute (Montreal, Quebec, Canada) (International Consortium for Brain Mapping 152) EPI template. To prevent the normalization from being influenced by areas of signal dropout, a mask was applied to areas in which dropout was observed in the mean EPI images during the calculation of normalization parameters. Finally, our data were spatially smoothed using a 10 mm (full-width at half-maximum) Gaussian kernel.

Data from the conditioning and judgment tasks were modeled and analyzed separately. For both sessions, a general linear model was applied to fit each voxel with a combination of functions derived from convolving the standard hemodynamic response with the time series of the events. Low-frequency "noise" in the time series was removed using a high-pass filter (cutoffs: conditioning, 32 s; judgment, 120 s).

Data modeling
Conditioning. The hemodynamic response function during conditioning was modeled for 12 different events: (1) the start of each trial, which is the presentation of the playing card, (2) the response ("high" or "low" button press), and (3) the onset of 10 different kinds of outcome (five back-ground patterns, each combined with reward and negative feedback).

Six movement regressors were added to the model to take variation attributable to movement into account. The presentation time of the face of the card varied over trials, because this was dependent on how quickly the subjects responded on that trial (mean reaction time, 1245 ms; SD, 657 ms). After an interstimulus interval of 1500 ms, the back of the card, outcome number, and feedback (reward or negative feedback) were presented for 2000 ms. This was followed by an intertrial interval of 1000 ms. To reduce the chance of observing an effect resulting from a correlation between events, the start of the trial and the response were orthogonalized with respect to the outcome of the trial. This meant that variance, which could be attributable to the trial outcome, the high/low response, or the start of the next trial (because these three events were correlated in time), was in fact attributed to the trial outcome (Andrade et al., 1999Go). The events of interest were those in which feedback was presented; blood oxygenation level-dependent (BOLD) signal in response to the presentation of reward was compared with that in response to the presentation of negative feedback. The single-subject contrast images, generated by pairwise comparisons among the parameter estimates for these different events, were then taken to a second-level random-effects analysis using SnPM (www.fil.ion.ucl.ac.uk/spm/snpm). A standard nonparametric randomization/permutation testing procedure was used, because this approach makes minimal assumptions about the distribution of the data (Nichols and Holmes, 2002Go). As a result, the voxel variance can be smoothed over neighboring voxels to calculate pseudo-t statistics. This type of variance smoothing reduces noise from the statistic image and adds degrees of freedom, resulting in increased power of the design. Here, pseudo-t-statistic maps were calculated using 8 mm full-width at half-maximum smoothing of the variance images (this results in smoothing the noise, which is different from spatial smoothing of voxel signal, as described above) and were thresholded at p < 0.05. Because this nonparametric analysis provides a measure of exchangeability of labels across all of the possible comparisons, the calculated p value of this permutation distribution represents an exact, as opposed to estimated, measure and is thus corrected for multiple comparisons across the whole brain. Activation clusters that survived significance testing were then used as regions of interest (ROIs) in the analysis of data from the judgment condition.

Judgment. Ten different event types were modeled during this condition: 1-8, the presentation of the eight patterns individually (the positive, negative, three bivalent, and three novel patterns); 9, the presentation of the two patterns simultaneously; and 10, the response. Six movement regressors were added to the model to take into account variation attributable to movement. The individual patterns were presented for 1000 ms each with an interstimulus interval of 500 ms. The presentation time of the pair of stimuli varied, depending on the reaction time of the participants (mean reaction time, 814 ms; SD, 431 ms).

Contrast images were generated using the events that indexed individually the presentation of each of the eight stimuli. More specifically, activation while subjects were passively viewing the positive pattern was compared with that during viewing of the negative pattern. To exclude any BOLD signal explained by the variance of adjacent events, the onset of the presentation of the pairs of stimuli and the response were orthogonalized with respect to the presentation of the individual stimuli (Andrade et al., 1999Go).

The contrast images of each individual subject were taken into a random-effects analysis using SPM. Statistical maps were thresholded using an ROI analysis using MarsBaR (Brett et al., 2002Go), which applied the statistical model to the mean signal of each ROI. ROIs were defined by the significant activation clusters observed in the reward versus negative feedback contrast during the conditioning phase.


   Results
Top
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 References
 
Behavioral results
As a result of technical problems during the judgment condition, data on two subjects had to be excluded. Reward conditioning was assessed by computing a preference score, which compared the judgment score for the pattern paired most with reward (positive pattern) with that of the pattern paired least with reward (negative pattern). To exclude any confounding effects in this measure, such as the possible effects on preference of unequal exposure to patterns (i.e., novel vs seen before), the judgment score was extracted from only those trials in which the positive pattern was directly paired with the negative pattern, and the subject had to choose which of these two patterns he or she preferred. The preference score was calculated as the number of times that the positive pattern was chosen over the negative pattern, of a maximum of 10; 5 indicated chance performance. The mean across subjects was 6.6. A sign test was used to assess whether preference scores across subjects were, on average, reliably >5. The results show that subjects significantly preferred the positive pattern to the negative pattern (p = 0.048).

Other factors, such as a priori preferences, may have played a role in preference formation, and these may be more subjectively salient. When asked why they had chosen these patterns most/least (debriefing question 1 and 2), all of the participants attributed their liking/disliking to the physical characteristics of the patterns and not to previous experience of the patterns with reward/negative feedback during the conditioning phase (for example, responses for a positive and subsequently preferred pattern included "looks like a roller coaster"; responses for a negative and subsequently not preferred pattern included "looks like broken bones").

In fact, none of the participants was completely aware of the experimental contingencies (able to correctly identify the two patterns most closely associated with reward and negative feedback). Five of 20 people did not recognize any patterns in response to debriefing question 3 ("Did you see any of these pictures during the card game?"). Two people were able to recognize some of the patterns when asked this question but did not include either their positive or negative pattern. Seven people recognized either their positive or negative pattern, but not both, among which four people noticed the association of the positive pattern with positive feedback, concluded from debriefing question 4 ("Did you notice anything in particular when you saw the stimuli during the card game?"). None of the people who had recognized their negative pattern was able to report the relationship between it and negative feedback in response to question 4. Six people recognized both their positive and negative patterns in response to question 3. Of these six, one person claimed that both patterns were associated with negative feedback, and the other five did not report any association when prompted to do so at question 4. Thus, only 5 of 20 volunteers were partially aware of the experimental contingencies: they were able to identify either the positive or the negative pattern associated with reward or negative feedback but not both. The mean preference score among these five partially aware participants was 5.4, which was not significantly different from that of the nonaware volunteers (mean, 6.93; t = 0.72). None of the participants was aware of the purpose of the experiment.

The majority of subjects appeared to be unaware of the experimental contingencies in the conditioning phase, and all of the subjects were unaware of the effects of their experience in the conditioning phase on their own subsequent behavior. The conditioning effect clearly contributes to observed preference but in a relatively subtle way, because most subjects preferred at least one other pattern to the positive pattern. When preference scores were calculated per pattern and ranked within subjects in order of preference, the positive pattern was one of the two most-preferred patterns in 10 of 20 subjects (and it obtained the highest score in only 6 of 20 subjects). The negative pattern was one of the two least-preferred patterns in 10 of 20 subjects (and the lowest-scoring pattern in only 3 of 20 subjects).

Subjects took, on average, 814 ms (SD, 431 ms) to make a judgment response. There was no cutoff for subjects' reaction time (maximum reaction time, 4094 ms), and there were no missed trials.

Imaging results
Conditioning phase: reward versus negative feedback
This contrast reveals the areas involved in response to the presentation of reward compared with the presentation of negative feedback during the conditioning phase.

Significant activations at a level of p < 0.05, surviving a whole-brain correction for multiple comparisons, are listed in Table 1 (Fig. 2). These coordinates represent the peak activation of an entire cluster. One cluster includes all of the adjacent voxels that survive significance. Only one significant peak per cluster is presented in the table.


View this table:
[in this window]
[in a new window]
  		Table 1. Talairach coordinates (Talairach and Tournoux, 1988Go) and pseudo-t values of activated regions during the presentation of reward versus negative feedback in the conditioning task

 



View larger version (47K):
[in this window]
[in a new window]
  		Figure 2. Responses during the presentation of reward compared with negative feedback in the conditioning task, including the OFC, the ventral striatum, and the amygdala. BOLD responses are thresholded at p<0.05, corrected for multiple comparisons. All of these activation clusters were used as an ROI in the analysis of the data from the judgment task.

 
In this contrast, we observed activation in the ventral striatum, primarily in the putamen and nucleus accumbens, extending into the amygdala in the left hemisphere (-26, -10, and -16) (Fig. 2). Significant effects were also detected in the OFC on the midline, the left OFC, anterior cingulate cortex, dorsal thalamus, and bilaterally in the superior temporal gyrus, in what is probably auditory cortex. These auditory activations may result from acoustic differences between the sounds used to signal the outcome (melodic flourish vs buzzer); these sounds were of different duration and were different in spectrotemporal structure.

Judgment condition: activation related to conditioned patterns
In the judgment task, we compared BOLD signal during passive viewing of the positive pattern (presented on its own in the middle of the screen) with BOLD signal during passive viewing of the negative pattern (presented on its own in the middle of the screen). Because of technical problems during the judgment condition, data from two subjects had to be excluded from the analysis (the same two subjects that were excluded from the behavioral analysis). Differences in activation between these two conditions in the remaining 20 subjects, using a whole-brain analysis with p < 0.001 uncorrected, are shown in Figure 3. These areas did not reach significance after a whole-brain correction for multiple comparisons. An ROI analysis was applied based on the significant activations (p < 0.05) observed in the reward versus negative feedback contrast from the conditioning task, as shown in Table 1. Only activation in OFC (peak 0, 44, and -8) was significant. Activation in the retrosplenial cortex reached significance at an uncorrected level but did not survive correction for multiple comparisons across all of the resels of the ROI clusters of activation.



View larger version (49K):
[in this window]
[in a new window]
  		Figure 3. BOLD responses in the OFC to the presentation of the positive pattern compared with the negative pattern during the judgment task. Responses are thresholded at p < 0.001, uncorrected for multiple comparisons. Activation in the OFC survives correction for multiple comparisons using a region of interest that is based on the activated clusters obtained in the reward-negative feedback contrast from the conditioning task (Fig. 2).

 

   Discussion
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
References
 
Although post-test debriefing revealed that a minority of participants may have been partially aware of the relationship between particular patterns and reward and negative feedback, no subject was fully aware. All of the subjects attributed their preferences to the physical characteristics of the patterns and not to previous experience with the stimuli during the card game. We conclude that our participants were mostly unaware of the conditioning manipulation during testing. We observed activation in the OFC region during both reward presentation and also when subjects viewed conditioned patterns without the administration of a physical reward. Because subjects could not explicitly anticipate a rewarding outcome while viewing the CS, we conclude that it is the intrinsic affective properties evoked by the CS that must primarily underlie the BOLD response in the OFC.

Our observation of OFC activation when subjects viewed stimuli associated previously with reward is consistent with theories suggesting that the OFC codes information regarding the reward value of a stimulus. Neurophysiological studies in primates indicate that neurons in the OFC detect and discriminate among rewards of different value and thus may play an important role in identifying or perceiving reward (Schultz et al., 2000Go). Furthermore, neurons in the monkey OFC cease to respond to food rewards when a monkey is satiated, so that the food no longer holds a rewarding value (Rolls, 2000Go). Compatible results have been reported in functional neuroimaging studies in humans. These implicate the OFC in the representation of relative reward value (Elliott et al., 2003Go; Knutson et al., 2003Go) and incentive value of stimuli (Arana et al., 2003Go).

The results also fit in with an alternative view of OFC function: that it is involved in the generation of "somatic markers" for affective states and the integration of this information into subsequent behavior (Schoenbaum et al., 1998Go; Bechara et al., 2000Go). According to the somatic marker hypothesis (Damasio, 1996Go), stimuli evoke a somatic state (measured by an increased skin-conductance response) that is associated with pleasurable or aversive somatic markers (e.g., the anticipation of affective outcomes).

Paulus and Frank (2003Go) suggest that the brain substrate of responses related to somatic markers, which includes regions overlapping with our OFC activation, is also involved in preference judgments. They conducted an fMRI study in which subjects made a preference judgment on pictures of soft drinks, using a paired-comparison procedure, and this was compared with a visual-discrimination task. The authors suggest that competition between appetitive and aversive somatic markers, processed in the ventromedial prefrontal cortex (VMPFC), is critical for decision making and may guide preference judgments even in the absence of rewarding or aversive outcomes associated with the response. Thus, the VMPFC appears to play a role in the representation of complex rewarding values involved in preference judgments (Paulus and Frank, 2003Go) (but see Erk et al., 2002Go). Our results are consistent with and extend these findings. In the

judgment condition, the affective information of the CS was important for the behavioral response later in the trial. After the presentation of the CS, subjects may have been preparing for a response by extracting affective information to be able to subsequently use this information for their response. However, no response selection was required while subjects were viewing the CS (the response was modeled separately from stimulus presentation). In fact, response selection at the time of CS presentation was impossible for approximately half of the trials, when the CS was presented as the first in the pair, because no decision could be made until both stimuli were known. This finding provides additional information that the OFC is specifically implicated in coding and perhaps extracting the affective information from conditioned stimuli to prepare for subsequent decision making, in the absence of response selection. Thus, conditioned stimuli may trigger the cascade of somatic responses via the OFC independently of any decision being required. However, additional studies are required to investigate the exact relationship between somatic responses and neural markers in response to rewarding stimuli. A role for the human OFC in the representation of reward value, as well as guiding behavior based on this information, is consistent with functional neuroimaging studies that investigated responses to rewarding stimuli in the presence or absence of response selection (Arana et al., 2003Go; J. O'Doherty et al., 2003Go). The present study, however, is the first to show similar OFC responses to stimuli that have acquired affective value via associative conditioning.

Rewarding properties of conditioned stimuli have been investigated in rats and nonhuman primates, using conditioned reinforcement and autoshaping procedures. These studies suggest a role for the OFC in processing the incentive value of a conditioned reward (Chudasama and Robbins, 2003Go; Pears et al., 2003Go). The activation in the OFC in response to conditioned reward observed in the present study provides compelling evidence for a similar role for the human OFC in the representation of intrinsic affective value of the CS after CS-unconditioned stimulus (US) pairings.

OFC activation in humans has been observed in only a few other neuroimaging studies of reward conditioning (Gottfried et al., 2002bGo, 2003Go; J. P. O'Doherty et al., 2003Go). For example, Gottfried et al. (2002bGo) paired faces with a pleasant, unpleasant, or neutral odor or with no olfactory stimulus. Participants had to decide the sex of each face, and the odor was administered while each face was on the screen. Activation in the OFC was observed when faces associated with a pleasant odor were compared with neutral faces never paired with an odor. A direct comparison between positively and negatively conditioned faces implicated the ventral striatum and the amygdala. Recently, the same authors reported an fMRI study using a postconditioning devaluation procedure and implicated both the OFC and amygdala in the representation of the current value of reward (odor) predicted by the CS (abstract visual stimulus), as shown by decreased activation in these areas in response to the CS after reward devaluation (Gottfried et al., 2003Go). A crucial difference between these procedures and the paradigm used here concerns anticipation of the CS. Although subjects in the study by Gottfried et al. (2002bGo) had not explicitly been told about the CS-US contingencies, there was no attempt to make them difficult to perceive, and the authors did not report any measures of awareness. Adjusted responses to the CS after postconditioning reward devaluation may reflect both a change in reward prediction and a change in intrinsic reward value of the CS. Finally, J. P. O'Doherty et al. (2003Go) reported activation in the ventral striatum and the OFC in response to reward-prediction errors in the context of appetitive pavlovian conditioning. Reward-prediction errors index the magnitude of the discrepancy between reward expectancy and outcome. Although this clearly plays an important role in the acquisition of motivational significance of the CS via CS-US pairings, this aspect emphasizes the explicit predictive value of the CS. In contrast, the current study reveals OFC activation in the absence of explicit reward prediction, suggesting a role for this region in processing the intrinsic reward properties of stimuli that have acquired affective significance via pavlovian conditioning.

A network of areas, including the ventral striatum, amygdala, and anterior cingulate, was activated in response to the presentation of reward compared with negative feedback, consistent with results from both neurophysiological studies in animals and other human functional neuroimaging studies of reward (Delgado et al., 2000Go; Elliott et al., 2000Go, 2003Go; Knutson et al., 2000Go; Schultz et al., 2000Go; Zalla et al., 2000Go; J. O'Doherty et al., 2001aGo,bGo, 2003Go; Small et al., 2001Go; Gottfried et al., 2002aGo). However, unlike other studies of appetitive conditioning (Gottfried et al., 2002bGo, 2003Go; J. P. O'Doherty et al., 2003Go), we did not observe activation in the amygdala or ventral striatum during the presentation of conditioned reward. This may be because the affective valence is too subtle for these regions to evoke a persistent BOLD response or because the smoothing filter that we used was not optimally sensitive to subcortical activation foci. An alternative explanation may be the lack of explicit anticipation of a physical reward during the judgment condition, which relates to other studies in which amygdala and ventral striatal activation was found in response to the anticipation of both abstract and primary rewards (Knutson et al., 2001aGo,bGo; J. P. O'Doherty et al., 2002Go; Hommer et al., 2003Go). In these studies, reward anticipation was induced by the presentation of differential cues that explicitly signaled subsequent administration of a rewarding, neutral, or aversive outcome. Although a cue predictive of a rewarding outcome may also acquire intrinsic affective value, a CS need not be endowed with the affective properties of the US for the animal to use it to anticipate the occurrence of the US.

In conclusion, we showed distinct patterns of neural responses after the presentation of reward compared with negative feedback, implicating the ventral striatum, amygdala, and OFC. An overlapping OFC region was also activated by the presentation of stimuli associated with reward, without the administration of an immediate reward and without participants anticipating a reward. These results suggest that the OFC plays an important role in coding the intrinsic affective properties of both rewarding events and conditioned stimuli that have acquired a rewarding value in the absence of explicit anticipation of a reward.


   Footnotes
 
Received Aug 16, 2004; revised January 3, 2005; accepted January 30, 2005.

This work was supported by the Medical Research Council and the University of Maastricht (S.M.L.C.). A.A. was funded by a Marie Curie Individual Fellowship. We thank the radiographers from the Wolfson Brain Imaging Center (Cambridge, UK) for their help with data acquisition and Matthew Brett and Tom Nichols for advice on statistical analysis. We are also grateful to John Parkinson for helpful comments and discussion.

Correspondence should be addressed to Sylvia M. L. Cox, Department of Psychiatry, McGill University, 1033 Pine Avenue West, Montreal, Quebec, Canada H3A 1A1. E-mail: syl{at}bic.mni.mcgill.ca.

Copyright © 2005 Society for Neuroscience 0270-6474/05/252733-08$15.00/0


   References
Top
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 References
 

Andrade A, Paradis AL, Rouquette S, Poline JB (1999) Ambiguous results in functional neuroimaging data analysis due to covariate correlation. NeuroImage 10: 483-486.[CrossRef][ISI][Medline]

Arana FS, Parkinson JA, Hinton E, Holland AJ, Owen AM, Roberts AC (2003) Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci 23: 9632-9638.[Abstract/Free Full Text]

Baxter MG, Murray EA (2002) The amygdala and reward. Nat Rev Neurosci 3: 563-573.[CrossRef][ISI][Medline]

Bechara A, Damasio H, Damasio AR (2000) Emotion, decision making and the orbitofrontal cortex. Cereb Cortex 10: 295-307.[Abstract/Free Full Text]

Brett M, Anton JL, Valabregue R, Poline JB (2002) Region of interest analysis using an SPM toolbox. NeuroImage 16: 497.

Cardinal RN, Parkinson JA, Hall J, Everitt BJ (2002) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26: 321-352.[CrossRef][ISI][Medline]

Chudasama Y, Robbins TW (2003) Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci 23: 8771-8780.[Abstract/Free Full Text]

Cox SML, Stefanova E, Johnsrude IS, Robbins TW, Owen AM (2002) Preference formation and working memory in Parkinson's disease and normal ageing. Neuropsychologia 40: 317-326.[CrossRef][ISI][Medline]

Cusack R, Brett M, Osswald K (2003) An evaluation of the use of magnetic field maps to undistort echo-planar images. NeuroImage 18: 127-142.[CrossRef][ISI][Medline]

Damasio AR (1996) The somatic marker hypothesis and the possible functions of the prefrontal cortex. Philos Trans R Soc Lond B Biol Sci 351: 1413-1420.[ISI][Medline]

Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072-3077.[Abstract/Free Full Text]

Elliott R, Friston KJ, Dolan RJ (2000) Dissociable neural responses in human reward systems. J Neurosci 20: 6159-6165.[Abstract/Free Full Text]

Elliott R, Newman JL, Longe OA, Deakin JF (2003) Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci 23: 303-307.[Abstract/Free Full Text]

Erk S, Spitzer M, Wunderlich AP, Galley L, Walter H (2002) Cultural objects modulate reward circuitry. NeuroReport 13: 2499-2503.[CrossRef][ISI][Medline]

Everitt BJ, Parkinson JA, Olmstead MC, Arroyo M, Robledo P, Robbins TW (1999) Associative processes in addiction and reward. The role of amygdala-ventral striatal subsystems. Ann NY Acad Sci 877: 412-438.[CrossRef][ISI][Medline]

Gallagher M, McMahan RW, Schoenbaum G (1999) Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci 19: 6610-6614.[Abstract/Free Full Text]

Gottfried JA, Deichmann R, Winston JS, Dolan RJ (2002a) Functional heterogeneity in human olfactory cortex: an event-related functional magnetic resonance imaging study. J Neurosci 22: 10819-10828.[Abstract/Free Full Text]

Gottfried JA, O'Doherty J, Dolan RJ (2002b) Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci 22: 10829-10837.[Abstract/Free Full Text]

Gottfried JA, O'Doherty J, Dolan RJ (2003) Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104-1107.[Abstract/Free Full Text]

Holland PC, Gallagher M (1999) Amygdala circuitry in attentional and representational processes. Trends Cogn Sci 3: 65-73.[CrossRef][ISI][Medline]

Hommer DW, Knutson B, Fong GW, Bennett S, Adams CM, Varnera JL (2003) Amygdalar recruitment during anticipation of monetary rewards: an event-related fMRI study. Ann NY Acad Sci 985: 476-478.[ISI][Medline]

Johnsrude IS, Owen AM, Zhao WV, White NM (1999) Conditioned preference in humans: a novel experimental approach. Learn Motiv 30: 250-264.[CrossRef]

Johnsrude IS, Owen AM, White NM, Zhao WV, Bohbot V (2000) Impaired preference conditioning after anterior temporal lobe resection in humans. J Neurosci 20: 2649-2656.[Abstract/Free Full Text]

Kirsch P, Schienle A, Stark R, Sammer G, Blecker C, Walter B, Ott U, Burkart J, Vaitl D (2003) Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system: an event-related fMRI study. NeuroImage 20: 1086-1095.[CrossRef][ISI][Medline]

Knutson B, Westdorp A, Kaiser E, Hommer D (2000) FMRI visualization of brain activity during a monetary incentive delay task. NeuroImage 12: 20-27.[CrossRef][ISI][Medline]

Knutson B, Adams CM, Fong GW, Hommer D (2001a) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21: RC159(1-5).[Abstract/Free Full Text]

Knutson B, Fong GW, Adams CM, Varner JL, Hommer D (2001b) Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport 12: 3683-3687.[CrossRef][ISI][Medline]

Knutson B, Fong GW, Bennett SM, Adams CM, Hommer D (2003) A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. NeuroImage 18: 263-272.[CrossRef][ISI][Medline]

Nichols TE, Holmes AP (2002) Nonparametric permutation tests for functional neuroimaging: a primer with examples. Hum Brain Mapp 15: 1-25.[CrossRef][ISI][Medline]

O'Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001a) Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4: 95-102.[CrossRef][ISI][Medline]

O'Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F (2001b) Representation of pleasant and aversive taste in the human brain. J Neurophysiol 85: 1315-1321.[Abstract/Free Full Text]

O'Doherty J, Critchley H, Deichmann R, Dolan RJ (2003) Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci 23: 7931-7939.[Abstract/Free Full Text]

O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33: 815-826.[CrossRef][ISI][Medline]

O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38: 329-337.[CrossRef][ISI][Medline]

Palmer AR, Bullock DC, Chambers JD (1998) A high output, high-quality sound system for use in auditory fMRI. NeuroImage 7: S359.

Parkinson JA, Robbins TW, Everitt BJ (2000) Dissociable roles of the central and basolateral amygdala in appetitive emotional learning. Eur J Neurosci 12: 405-413.[CrossRef][ISI][Medline]

Parkinson JA, Crofts HS, McGuigan M, Tomic DL, Everitt BJ, Roberts AC (2001) The role of the primate amygdala in conditioned reinforcement. J Neurosci 21: 7770-7780.[Abstract/Free Full Text]

Paulus MP, Frank LR (2003) Ventromedial prefrontal cortex activation is critical for preference judgments. NeuroReport 14: 1311-1315.[CrossRef][ISI][Medline]

Pears A, Parkinson JA, Hopewell L, Everitt BJ, Roberts AC (2003) Lesions of the orbitofrontal but not medial prefrontal cortex disrupt conditioned reinforcement in primates. J Neurosci 23: 11189-11201.[Abstract/Free Full Text]

Petrides M, Milner B (1982) Deficits on subject-ordered tasks after frontal- and temporal-lobe lesions in man. Neuropsychologia 20: 249-262.[CrossRef][ISI][Medline]

Rolls ET (2000) The orbitofrontal cortex and reward. Cereb Cortex 10: 284-294.[Abstract/Free Full Text]

Schoenbaum G, Chiba AA, Gallagher M (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1: 155-159.[CrossRef][ISI][Medline]

Schoenbaum G, Chiba AA, Gallagher M (1999) Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19: 1876-1884.[Abstract/Free Full Text]

Schultz W (2000) Multiple reward signals in the brain. Nat Rev Neurosci 1: 199-207.[ISI][Medline]

Schultz W, Tremblay L, Hollerman JR (2000) Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex 10: 272-284.[Abstract/Free Full Text]

Small DM, Zatorre RJ, Dagher A, Evans AC, Jones-Gotman M (2001) Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124: 1720-1733.[Abstract/Free Full Text]

Talairach J, Tournoux P (1988) Co-planar stereotactic atlas of the human brain. New York: Thieme.

Zalla T, Koechlin E, Pietrini P, Basso G, Aquino P, Sirigu A, Grafman J (2000) Differential amygdala responses to winning and losing: a functional magnetic resonance imaging study in humans. Eur J Neurosci 12: 1764-1770.[CrossRef][ISI][Medline]


This article has been cited by other articles:


Home page
 J. Neurol. Neurosurg. PsychiatryHome page
J. L Miller, G A. James, A. P Goldstone, J. A Couch, G. He, D. J Driscoll, and Y. Liu
Enhanced activation of reward mediating prefrontal regions in response to food stimuli in Prader-Willi syndrome
J. Neurol. Neurosurg. Psychiatry, June 1, 2007; 78(6): 615 - 619.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurophysiol.Home page
S. Bray and J. O'Doherty
Neural Coding of Reward-Prediction Error Signals During Classical Conditioning With Attractive Faces
J Neurophysiol, April 1, 2007; 97(4): 3036 - 3045.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
K. Blair, A. A. Marsh, J. Morton, M. Vythilingam, M. Jones, K. Mondillo, D. C. Pine, W. C. Drevets, and J. R. Blair
Choosing the Lesser of Two Evils, the Better of Two Goods: Specifying the Roles of Ventromedial Prefrontal Cortex and Dorsal Anterior Cingulate in Object Choice.
J. Neurosci., November 1, 2006; 26(44): 11379 - 11386.
[Abstract] [Full Text] [PDF]

Home page
 Am. J. Clin. Nutr.Home page
D. S. N. Le, N. Pannacciulli, K. Chen, A. Del Parigi, A. D Salbe, E. M Reiman, and J. Krakoff
Less activation of the left dorsolateral prefrontal cortex in response to a meal: a feature of obesity.
Am. J. Clinical Nutrition, October 1, 2006; 84(4): 725 - 731.
[Abstract] [Full Text] [PDF]

Home page
 J. Neurosci.Home page
A. Galvan, T. A. Hare, C. E. Parra, J. Penn, H. Voss, G. Glover, and B. J. Casey
Earlier development of the accumbens relative to orbitofrontal cortex might underlie risk-taking behavior in adolescents.
J. Neurosci., June 21, 2006; 26(25): 6885 - 6892.
[Abstract] [Full Text] [PDF]

Home page
 ScienceHome page
S. R. Chamberlain, U. Muller, A. D. Blackwell, L. Clark, T. W. Robbins, and B. J. Sahakian
Neurochemical Modulation of Response Inhibition and Probabilistic Learning in Humans
Science, February 10, 2006; 311(5762): 861 - 863.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Submit an eLetter
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow