 |
 Previous Article | Next Article 
The Journal of Neuroscience, April 15, 2001, 21(8):2793-2798
Predictability Modulates Human Brain Response to Reward
Gregory S.
Berns1,
Samuel M.
McClure2,
Giuseppe
Pagnoni1, and
P.
Read
Montague2
1 Department of Psychiatry and Behavioral Sciences,
Emory University School of Medicine, Atlanta, Georgia 30322, and
2 Center for Theoretical Neuroscience, Division of
Neuroscience, Baylor College of Medicine, Houston, Texas 77030
 |
ABSTRACT |
Certain classes of stimuli, such as food and drugs, are highly
effective in activating reward regions. We show in humans that activity
in these regions can be modulated by the predictability of the
sequenced delivery of two mildly pleasurable stimuli, orally delivered fruit juice and water. Using functional magnetic resonance imaging, the activity for rewarding stimuli in both the nucleus accumbens and medial orbitofrontal cortex was greatest when the stimuli
were unpredictable. Moreover, the subjects' stated preference for
either juice or water was not directly correlated with activity in
reward regions but instead was correlated with activity in sensorimotor
cortex. For pleasurable stimuli, these findings suggest that
predictability modulates the response of human reward regions, and
subjective preference can be dissociated from this response.
Key words:
reward; dopamine; fMRI; reinforcement; neural network; nucleus accumbens; striatum
| |
INTRODUCTION |
The pursuit of natural rewards such
as food, drink, and sex is a major external influence on human
behavior. Nevertheless, the issue of how rewards affect human behavior
remains primarily unresolved. There are many factors that contribute to
this gap in our knowledge; however, one roadblock has been the
difficulty of defining and measuring isolated effects of rewards on
human behavior or brain activation. In animals, reward is defined as an
operational concept: a stimulus is deemed rewarding if it positively reinforces a behavior (Hull, 1943 ; Rescorla and Wagner, 1972; Robbins
and Everitt, 1996), that is, reliably increases the likelihood of the
behavior. The same concept applies to humans; however, humans have the
ability to exert all sorts of executive control over their actions, and
so behavioral assays alone are an incomplete way to probe reward
processing. Similarly, explicit reports of likes and dislikes, i.e.,
preferences, are confounded by an individual's subjective perception
of what they like and what they choose to report. To overcome these
experimental difficulties, one would like to monitor concurrently
behavioral output, subjective preference, and brain response during a
well defined task. Taking such an approach, we report here that
activity in human reward regions is more closely correlated with the
predictability of a sequence of pleasurable stimuli than with
explicitly stated preferences.
In humans, activation of reward areas can be visualized with functional
magnetic resonance imaging (fMRI) after administration of drugs, such
as cocaine (Breiter et al., 1997); however, such infusions may not be
representative of normal reward processing because of both direct and
indirect pharmacological effects of cocaine. Furthermore, drugs such as
cocaine may act on different parts of the reward system than so-called
natural rewards such as food and water (Bradberry et al., 2000; Carelli
et al., 2000). Conditioned rewards, e.g., money, may also act on
different parts of the reward system (Thut et al., 1997; Elliott et
al., 2000; Knutson et al., 2000) and may not be an appropriate probe of
primary reward circuits in humans. An alternative approach is suggested by experiments that demonstrate that the predictability of a primarily rewarding stimulus is a critical parameter for activation of reward pathways (Schultz et al., 1992, 1997; Schultz, 1998; Garris et al.,
1999). Physiological recordings in nonhuman primates have demonstrated
that neurons in regions such as the ventral tegmental area (VTA),
nucleus accumbens, and ventral striatum respond in an adaptive manner
to rewarding stimuli such as fruit juice or water (Shidara et al.,
1998). Thus, the predictability of a sequence of stimuli may itself
recruit reward-related neural structures in a manner detectable with
fMRI. Moreover, theoretical models of dopamine release suggest that
unpredictable rewards should elicit greater activity in these regions
(Schultz et al., 1997). We sought to test this hypothesis by using fMRI
to measure the effect of predictability on human brain responses to
sequences of punctate, pleasurable stimuli.
| |
MATERIALS AND METHODS |
Subjects. Twenty-five normal adults underwent fMRI
scanning while being administered small amounts of either oral fruit
juice or water. The subjects ranged in age from 18 to 43, and all
subjects gave informed consent for a protocol approved by the Emory
University Human Investigations Committee.
Experimental task. While in the scanner, subjects received
small amounts of orally delivered fruit juice and water in either a
predictable or unpredictable manner. We chose a sequenced delivery of
fruit juice and water for three reasons: (1) humans find both juice and
water to be subjectively pleasurable; (2) both stimuli are routinely
used as reinforcing stimuli while training nonhuman primates on
behavioral tasks; and (3) midbrain dopaminergic neurons, and presumably
the neurons to which they project, show phasic changes in firing rate
as a function of the temporal predictability of sequential stimuli
(Schultz et al., 1992). Participants received both the juice and the
water in either a predictable or unpredictable manner in two scanning
runs (Fig. 1). During the predictable
run, juice and water boluses were alternated at a fixed interval of 10 sec. During the unpredictable run, the order of juice and water was
randomized, and the stimulus interval was also randomized by sampling a
Poisson interval distribution with a mean of 10 sec. Each run lasted 5 min, and the order of the two runs (predictable or unpredictable) was
randomized across subjects. Because the time to adapt to either
predictability or unpredictability was unknown and because frequent
switching of conditions might cause an interaction with each other,
i.e., the "predictability of predictability," we chose to separate
the conditions across scan runs rather than use smaller condition
blocks within scan runs. Because all aspects of the experiment hinged
on manipulating predictability, we chose to not repeat conditions
within subjects and instead focused on studying a larger number of
subjects.
View larger version (9K):
[in this window]
[in a new window]
|
Figure 1.
Design of the fMRI experiment. A 2 × 2 factorial design was used, with factors of preference (juice or water)
and predictability (predictable or unpredictable). Subjects received
0.8 ml boluses of juice and water in either a predictable or
unpredictable sequence. Using event-related fMRI, brain activation was
analyzed in terms of preference and predictability, as well as the
interaction between them.
|
|
Subjects received 0.8 ml oral boluses of both fruit juice and water via
two plastic tubes. A mouthpiece held the ends of the tubing in place
over the tongue, with the fruit juice infused from the left side of the
mouthpiece and the water from the right. The tubes were ~10 m long
and were connected to a computer-controlled dual-syringe pump (Harvard
Apparatus, Holliston, MA) outside the scanner room. Subjects did
not perform any other task during scanning and were instructed to
simply swallow the fluid each time it was administered. After the scan
session, subjects were debriefed for their fluid preference.
Acquisition of MRI data. Scanning was performed on a 1.5 Tesla Philips NT scanner. After acquisition of a high-resolution T1-weighted anatomical scan, subjects underwent two whole-brain functional runs of 150 scans each (echo-planar imaging, gradient recalled echo; repetition time, 2000 msec; echo time, 40 msec; flip angle, 90°; 64 × 64 matrix, 24 5 mm axial slices acquired parallel to anteroposterior commissural line) for measurement of the
blood oxygenation level-dependent (BOLD) effect (Kwong et al.,
1992; Ogawa et al., 1992). Head movement was minimized by padding and restraints.
Analysis. The data were analyzed using Statistical
Parametric Mapping (SPM99; Wellcome Department of Cognitive Neurology, London, UK) (Friston et al., 1995b). Motion correction to the first
functional scan was performed within subjects using a six-parameter rigid-body transformation. Because swallowing unavoidably causes significant head movement, the motion-correction parameters were also
used to determine whether head motion differed significantly between
the conditions. The mean of the motion-corrected images was then
coregistered to the individual's 24-slice structural MRI using a
12-parameter affine transformation. The images were then spatially
normalized to the Montreal Neurological Institute (MNI) template
(Talairach and Tournoux, 1988) by applying a 12-parameter affine
transformation, followed by a nonlinear warping using basis functions
(Ashburner and Friston, 1999). Images were subsequently smoothed with
an 8 mm isotropic Gaussian kernel and band-pass filtered in the
temporal domain. A random-effects, event-related, statistical analysis
was performed with SPM99 (Friston et al., 1995a, 1999). The experiment
was analyzed as a 2 × 2 factorial design. First, a separate
general linear model (GLM) was specified for each subject, with four
conditions representing the four possible event types:
predictable-preferred fluid, predictable-nonpreferred fluid,
unpredictable-preferred fluid, and unpredictable-nonpreferred fluid.
Four vectors of delta functions with times corresponding to each event
were created for each of the four conditions. These were convolved with
a generic hemodynamic response function and entered into a four-column
design matrix. The mean of each scan run was removed on a voxelwise
basis. We calculated three two-sided contrast images that
corresponded to the main effects of preference [contrast vector
(1-11-1)], predictability [contrast vector (11-1-1)], and the
interaction term [contrast vector (1-1-11)]. The interaction describes how predictability modulates the effect of preference. These
individual contrast images were entered into a second-level analysis,
using a separate one-sample t test (df = 24) for
each side of each term in the GLM (a total of six contrasts). We
thresholded these summary statistical maps at p < 0.001 (uncorrected for multiple comparisons). These maps were overlaid
on a high-resolution structural image in MNI orientation.
Theoretical model. As a tool for both designing and
interpreting the fMRI experiment, we used an existing neural network
model of dopamine release to simulate the brain response to different temporal patterns of rewarding stimuli (Fig.
2). This model was based on the method of
temporal differences (TD), which postulates that a synaptically
reinforcing substance, e.g., dopamine, is released in response to
errors in reward prediction (Schultz et al., 1997). This model has been
used in a wide variety of applications, including complex learning
tasks such as backgammon (Sutton, 1988; Tesauro and Sejnowski, 1989),
as well as successfully predicting the activity of dopamine neurons in
numerous conditioning paradigms (Houk et al., 1995; Montague et al.,
1995) and motor sequencing tasks (Berns and Sejnowski, 1998).
View larger version (21K):
[in this window]
[in a new window]
|
Figure 2.
Neural network model of the experiment and the
brain regions associated with information processing. A,
Diagram indicates our hypothesis for how the sequence of stimuli could
influence dopaminergic output. In this hypothesis, we have indicated
that changes in dopaminergic output could influence target neural
structures in a manner detectable in a fMRI BOLD measurement. The juice
and water are shown to have both sensory (projection from finite time
window box) and reward (the r pathways)
representations in their influence on dopaminergic activity. To
generate an expected hemodynamic response from this hypothesis, we made
a finite time window (small boxes for juice and water),
which determined the value of the immediate reward
r(t) (1 if juice occurred, 0.5 if
water occurred, and 0 if no stimulus occurred). This maneuver
arbitrarily set juice to twice the value of water. This is not
important for the main expectation generated by the model.
B, Predicted dopamine effect for predictable and
unpredictable sequences of juice and water delivery. Horizontal
axis is scan number. Vertical axis is the
expected hemodynamic response predicted by a temporal difference model.
The scale on the vertical axis is arbitrary. The
important point to note is that the predictable run progresses to 0, whereas the unpredictable run remains high-amplitude throughout. The
traces were generated by convolving a hemodynamic response kernel with
the output of a temporal difference model. This suggested that the
average BOLD response would be greater when the stimuli were
unpredictable.
|
|
Briefly, TD learning depends on two primary assumptions. First,
short-term adaptation in a given neural circuit occurs with the goal of
predicting a discounted sum of all future rewards. The definition of a
reward depends on the context in which it is received. If a putative
reward increases the occurrence of a particular behavior, then it is
deemed a positive reinforcer. Depending on the internal state of the
animal, the same reward may not reinforce a behavior, e.g., when the
animal is satiated. In the context of an fMRI experiment, which is
generally unnatural, a familiar appetitive substance such as water or
fruit juice is subjectively experienced as pleasant and therefore
rewarding. Second, reward predictions depend only on the current
representation of a stimulus set. The stimulus representation is
somewhat arbitrary in the model, and it includes some representation
backward through time, i.e., a stimulus trace. For substances such as
water or fruit juice, there exist both sensory dimensions (e.g.,
temperature and tactile sensation on the tongue) and the actual reward,
which is subjectively experienced as pleasure. Therefore, it is
reasonable to consider the tactile dimensions of fluid delivery as both
neutral and distinct from the rewarding dimension. Similarly, these
distinct dimensions are presumed to be processed by different brain
circuits, which can be imaged with fMRI. To map the model output onto a dimension analogous to the measurement obtained with fMRI, we summed
the outputs of both the neutral and rewarding pathways, which we
assumed converged in the ventral striatum and nucleus accumbens. We
acknowledge that there is no direct evidence for this, and depending on
the specific receptor, dopamine can have variable effects on neuronal
activity. The exact experimental design was input to the model, which
was simulated with Matlab 5.3 (MathWorks, Natick, MA). The outputs
corresponding to both the putative dopamine neurons and their
projection sites were calculated for the predictable and unpredictable
runs (Fig. 2).
We should be careful to point out to readers that our use of the
temporal difference model to explain our design and subsequent interpretation (below) is based on its previous success in describing changes in spike output in dopaminergic neurons in primates undergoing related behavioral tasks. There are other plausible computational descriptions that could also suffice.
| |
RESULTS |
After the scans, subjects were queried about their preference for
the two stimuli. Eighteen of 25 subjects (72%) preferred juice, and
the remainder preferred water. Most subjects had a distinct preference
for one or the other, although we did not ask them to quantify this.
Although there was significant head motion during the scans, all of the
translations and rotations around each stimulus were generally small
and were not significantly different between any of the conditions. For
example, the mean ± SD translation associated with each stimulus
was 0.041 ± 0.069 mm in the predictable condition and 0.044 ± 0.069 mm in the unpredictable condition (paired t test;
p = 0.853).
The brain response to the preferred fluid displayed surprisingly little
differential activity relative to the nonpreferred fluid (Table
1). We did not observe any significant
activity difference in classical reward regions such as the nucleus
accumbens, hippocampus, or medial prefrontal cortex. The primary
activity change for preferred > nonpreferred occurred in the
somatosensory cortex in an area near the mouth and tongue region
(t = 4.19, MNI coordinates,  60, 12, 16).
View this table:
[in this window]
[in a new window]
|
Table 1.
Brain regions displaying significant changes in measured
activity (p < 0.001 uncorrected; cluster size >10
voxels, except where noted)
|
|
The main effect of predictability was substantially greater than the
main effect of preference (Fig. 3). For
the unpredictable run relative to the predictable run, bilateral
activation was observed in a large expanse of medial orbitofrontal
cortex that included the nucleus accumbens (Table 1). Additional areas
of activation included a large area of parietal cortex bilaterally and
paracentrally and small focal activations in both the left mediodorsal
nucleus of the thalamus and right cerebellum. Because none of these
regions overlapped with the main effect of preference, they were
maximally activated by unpredictable stimuli, regardless of preference.
For the predictable run relative to the unpredictable run, an area of
the right superior temporal gyrus was activated, as well as focal
activations in the left precentral gyrus and right lateral
orbitofrontal cortex.
View larger version (109K):
[in this window]
[in a new window]
|
Figure 3.
The main effect of predictability showed
that reward-related regions had a greater BOLD response to the
unpredictable stimuli. A, Planes centered at (0, 4, 4)
show that bilateral nucleus accumbens/ventral striatum
(NAC) and bilateral superior parietal cortex were more
active in the predictable condition. B, A small region
in the right superior temporal gyrus was relatively more activated by
the predictable stimuli. Significance was thresholded at
p < 0.001 and an extent >10 contiguous
voxels.
|
|
The interaction between preference and predictability identified areas
in which one effect modulated the other independently of both main
effects. The right insula, left posterior cingulate, and right
cerebellum displayed a significant interaction for the contrast
(preferred-nonpreferred) × (predictable-unpredictable). The
opposite contrast, (preferred-nonpreferred) × (unpredictable-predictable), did not reveal any activations
significant at the p < 0.001 level; however, a small
region in the left superior temporal gyrus (MNI coordinates, 48, 4,
16) was significant at the p < 0.01 level (t = 3.15).
The computer simulation suggested that unpredictable rewards should
evoke more dopamine release than predictable ones (Fig. 2B). When the rewards are predictable, each stimulus
perfectly predicts the subsequent one, and the error signal, which is
presumed to be mediated by dopamine, gradually decreases. When the
rewards are unpredictable, there is no opportunity for the system to
learn, and the response to each stimulus is greater.
| |
DISCUSSION |
Our results demonstrated an interesting separation in the brain
response to predictability and to subjective reports of preference. The
brain response to preference was exclusively cortical, but the response
to predictability showed specific activation of reward systems also
known to be the target of midbrain dopaminergic neurons. If we presume
that activation of these reward areas is pleasurable to humans, then
this finding suggests that the subjective report of preference may be
dissociated from neural circuits known to be powerful determinants of
conditioned behaviors.
Both the water and the fruit juice caused significant activations
throughout the brain, and although some of this response was
attributable to the motoric aspects of the task, specific subsets of these regions were decomposed into dimensions of preference and predictability. The effect of preference was restricted to cortical
regions associated with sensory processing, and the preferred stimulus
resulted in greater activation in these regions. These regions lie near
sensorimotor cortex known to be activated during tongue movements
(Corfield et al., 1999) and swallowing (Hamdy et al., 1999). In
previous work on the brain response to tongue movement, there was
substantial activation of the cerebellum, a finding notably absent in
the main effect of preference. The differential brain response, i.e.,
preferred-nonpreferred, removes common regions of activation;
therefore, the absence of cerebellar activation suggests that
differential tongue movements were unlikely to be the cause of the
pattern of cortical activation for subjective preference. The fact that
a somatosensory region was correlated with stated preference was
suggestive that some differential neural processing occurred for the
two stimuli. It was surprising that this was manifest in a primary
sensory processing area and not in classical reward areas. Although
subjects were forced to designate one substance over the other as their
preference, both fluids were chosen purposely to be pleasurable, in
contradistinction to one being aversive. Because both fluids were
generally pleasurable, the effect of preference might not have been
strong enough to result in a significant activity difference in reward
regions. This would be consistent with findings that midbrain dopamine neurons are preferentially activated by appetitive rather than aversive
stimuli (Mirenowicz and Schultz, 1996). Nevertheless, our findings
suggest a system differentiation of subjective preference from
simple reward, which supports previous hypotheses that "wanting" is
not the same as "liking" (Robinson and Berridge, 1993).
Unlike the effect of preference, unpredictability correlated as a
significant main effect with activity in the nucleus accumbens, thalamus, and medial orbitofrontal cortex, whereas predictability was
correlated predominately with activity in the right superior temporal
gyrus. The former regions correspond closely with known dopamine
projection sites (Koob, 1992; Cooper et al., 1996). It was somewhat
surprising that unpredictability, and not preference, was correlated
with activity in these reward areas. If increased activity in these
regions was associated with pleasure, then one might conclude that
unpredictable rewards were more pleasurable than predictable ones.
However, most of the subjects did not discern any difference between
the predictable and unpredictable conditions. If the unpredictable
rewards were more pleasurable than predictable ones, or vice versa,
then this must be occurring at a subconscious level. An alternative
explanation presumes that dopamine is released in increased amounts to
unexpected rewards (Montague et al., 1996; Schultz et al., 1997;
Schultz, 1998). Dopamine can decrease neuronal excitability (Cooper et
al., 1996) and may also directly constrict the microvasculature (Krimer
et al., 1998), but increased accumbens activity has also been
associated with the subjective pleasure of cocaine (Breiter et al.,
1997). These findings suggest that our observed increase in activation
with unpredictability could be related to increased dopamine release,
either because the accumbens projects to the VTA or because it receives
a projection from the VTA, either of which would be consistent with the
model results. This interpretation should be tempered by two important
facts: (1) the mechanisms that would couple increased dopaminergic
transmission to changes in the BOLD signal are unknown, and (2) we have
no independent measure of dopaminergic transmission, only changes in
the BOLD response. The possibility that we are observing indirectly changes in dopaminergic activity is exciting but cannot be decided unequivocally in an fMRI experiment. It is, however, consistent with
previous findings using positron emission tomography that dopamine is
released into the ventral striatum under conditions of monetary
incentive (Koepp et al., 1998). Coupled with the amplifying effect of
unpredictability, it is also consistent with hypothesized effects of
dopamine on neuronal "gain" (Cohen and Servan-Schreiber, 1992),
with the end result that some regions will increase and others will decrease.
The specific regions activated relatively by unpredictability
corresponded to brain regions associated with appetitive functions. In
addition to the nucleus accumbens, the medial orbitofrontal cortex
showed a main effect for unpredictability. This region has been shown
in primates to integrate both the rewarding and neutral aspects of
taste sensations and is thought to reflect primarily the motivational
values of these stimuli (Rolls, 2000). This region also contains
neurons that discriminate relative preference for reward (Tremblay and
Schultz, 1999). The orbitofrontal cortex is typically difficult to
image with fMRI because of the susceptibility artifact from the nasal
sinuses (Ojemann et al., 1997). However, the region that we identified
is generally superior and caudal to the usual artifact location. This
region has previously been found responsive to pleasant tastes (Francis
et al., 1999). A second region, in the superior parietal lobe, was
probably not related to the rewarding aspects of the task but rather
the result of changes in attention. This region has previously been
implicated in visuospatial attention, especially during expectation
violations (Nobre et al., 1999). Another region, in the left temporal
cortex, showed a borderline significant modulation by unpredictability. In recent fMRI experiments, the left temporal lobe has been associated with processing the predictability of sequential stimuli
(Bischoff-Grethe et al., 2000). Here, we extend these previous findings
from neutral stimuli to pleasurable stimuli, suggesting that this
region may perform a generic monitoring of predictability independently
of stimulus valence.
The brain regions that we identified as responding to unpredictability
in either a direct or modulatory manner have been implicated in a
number of experiments on financial reward. Money can be rewarding to
humans, but it is reinforcing only because it has acquired these
properties through complex conditioning. Similar to the finding that
cocaine acts on different neurons than natural reinforcers (Carelli et
al., 2000), it is possible that conditioned reinforcers, such as money,
act on different neural systems than natural reinforcers such as food
and water. Activity in both the ventral striatum and midbrain have been
correlated with absolute levels of financial reward (Thut et al., 1997;
Delgado et al., 2000; Elliott et al., 2000; Knutson et al., 2000), a
finding notably absent in our results. As noted previously, both the
juice and water were mildly pleasurable, and so there may not have been
a substantial difference in absolute reward, although we assumed a
slight difference in the theoretical model. Also, we did not use any
aversive stimuli or anything that could be construed as a negative
reward, which may also account for this difference. Interestingly, the
regions we identified as being either directly affected or amplified by
unpredictability corresponded to the regions found previously to be
sensitive to the context dependence of the financial reward (Rogers et
al., 1999; Elliott et al., 2000). In particular, both the subgenual cingulate and medial thalamus were correlated with unpredictability in
our study and were found to be context-dependent by Elliott et al.
(2000).
Because predictability modulated the effect of preference, it is
important to distinguish the potential sources of prediction. In a
classical conditioning experiment, a neutral stimulus precedes the
reward. After training, the previously neutral stimulus becomes the
predictor, or conditioned stimulus. Because there are comparatively few
data on the use of oral stimuli in fMRI, we chose to simplify the
experiment and control for the motoric aspects of the task by using two
different oral stimuli, water and fruit juice. Thus, the source of
prediction in our experiment necessarily came from the sequence of
stimuli themselves. In some ways, this is simpler than introducing
another stimulus modality, such as a visual cue, but because both
stimuli were rewarding, we cannot make any conclusions regarding the
process of conditioning. Both the theoretical model (Schultz et al.,
1997) and neurophysiological data (Schultz et al., 1992, 1993) suggest
that reward predictions are computed during the interval preceding
reward delivery. Because we do not know the time scale over which such
predictions are computed, we chose to analyze the experiment as simply
two conditions, predictable and unpredictable. By maintaining a
psychologically reasonable interval between stimuli, 10 sec, there was
insufficient time to resolve differences in interstimulus processing.
Presumably such processing does occur, and this could be resolved with
a differently designed experiment.
In summary, activity in human reward regions can be modulated by the
temporal predictability of primary rewards such as water and juice.
These results provide important support for computational models that
postulate that errors in reward prediction can drive synaptic
modification and extend these conclusions from nonhuman primates to
humans. The regional specificity of this modulation also suggests that
information, as embodied by the relative predictability of a stimulus
stream, may be a form of neural currency that can be detected with fMRI.
| |
FOOTNOTES |
Received Nov. 11, 2000; revised Jan. 17, 2001; accepted Jan. 26, 2001.
This work was supported by National Institute on Drug Abuse Grants K08
DA00367 (to G.S.B.) and RO1 DA11723 (to P.R.M.), the National Alliance
for Research in Schizophrenia and Depression (G.S.B.), and the Kane
Family Foundation (P.R.M.). We thank H. Mao, R. King, and M. Martin for
their assistance with data collection.
Correspondence may be addressed to either Gregory S. Berns, Department
of Psychiatry and Behavioral Sciences, Emory University School of
Medicine, 1639 Pierce Drive, Suite 4000, Atlanta, GA 30322, E-mail:
gberns{at}emory.edu, or P. Read Montague, Division of Neuroscience,
Baylor College of Medicine, 1 Baylor Plaza, Houston, TX 77030, E-mail:
read{at}bcm.tmc.edu.
| |
REFERENCES |
-
Ashburner J,
Friston KJ
(1999)
Nonlinear spatial normalization using basis functions.
Hum Brain Mapp
7:254-266[ISI][Medline].
-
Berns GS,
Sejnowski TJ
(1998)
A computational model of how the basal ganglia produce sequences.
J Cognit Neurosci
10:108-121[Abstract].
-
Bischoff-Grethe A,
Proper SM,
Mao K,
Daniels KA,
Berns GS
(2000)
Conscious and unconscious processing of nonverbal predictability in Wernicke's area.
J Neurosci
20:1975-1981[Abstract/Free Full Text].
-
Bradberry CW,
Barrett-Larimore RL,
Jatlow P,
Rubino SR
(2000)
Impact of self-administered cocaine and cocaine cues on extracellular dopamine in mesolimbic and sensorimotor striatum in Rhesus monkeys.
J Neurosci
20:3874-3883[Abstract/Free Full Text].
-
Breiter HC,
Gollub RL,
Weisskoff RM,
Kennedy DN,
Makris N,
Berke JD,
Goodman JM,
Kantor HL,
Gastfriend DR,
Riorden JP,
Mathew RT,
Rosen BR,
Hyman SE
(1997)
Acute effects of cocaine on human brain activity and emotion.
Neuron
19:591-611[ISI][Medline].
-
Carelli RM,
Ijames SG,
Crumlin AJ
(2000)
Evidence that separate neural circuits in the nucleus accumbens encode cocaine versus "natural" (water and food) reward.
J Neurosci
20:4255-4266[Abstract/Free Full Text].
-
Cohen JD,
Servan-Schreiber D
(1992)
Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia.
Psychol Rev
99:45-77[ISI][Medline].
-
Cooper JR,
Bloom FE,
Roth RH
(1996)
In: The biochemical basis of neuropharmacology, Ed 7. New York: Oxford UP.
-
Corfield DR,
Murphy K,
Josephs O,
Fink GR,
Frackowiak RSJ,
Guz A,
Adams L,
Turner R
(1999)
Cortical and subcortical control of tongue movement in humans: a functional neuroimaging study using fMRI.
J Appl Physiol
86:1468-1477[Abstract/Free Full Text].
-
Delgado MR,
Nystrom LE,
Fissel C,
Noll DC,
Fiez JA
(2000)
Tracking the hemodynamic responses to reward and punishment in the striatum.
J Neurophysiol
84:3072-3077[Abstract/Free Full Text].
-
Elliott R,
Friston KJ,
Dolan RJ
(2000)
Dissociable neural responses in human reward systems.
J Neurosci
20:6159-6165[Abstract/Free Full Text].
-
Francis S,
Rolls ET,
Bowtell R,
McGlone F,
O'Doherty J,
Browning A,
Clare S,
Smith E
(1999)
The representation of pleasant touch in the brain and its relationship with taste and olfactory areas.
NeuroReport
10:453-459[ISI][Medline].
-
Friston KJ,
Frith CD,
Turner R,
Frackowiak RSJ
(1995a)
Characterising dynamic brain responses with fMRI: a multivariate approach.
NeuroImage
2:166-172[ISI][Medline].
-
Friston KJ,
Holmes AP,
Worsley K,
Poline JB,
Frith CD,
Frackowiak RSJ
(1995b)
Statistical parametric maps in functional brain imaging: a general linear approach.
Hum Brain Mapp
2:189-210.
-
Friston KJ,
Holmes AP,
Worsely KJ
(1999)
How many subjects constitute a study?
NeuroImage
10:1-5[ISI][Medline].
-
Garris PA,
Kilpatrick M,
Bunin MA,
Michael D,
Walker QD,
Wightman RM
(1999)
Dissociation of dopamine release in the nucleus accumbens from intracranial self-stimulation.
Nature
398:67-69[Medline].
-
Hamdy S,
Mikulis DJ,
Crawley A,
Xue S,
Lau H,
Henry S,
Diamant NE
(1999)
Cortical activation during human volitonal swallowing: an event-related fMRI study.
Am J Physiol
277:G219-G225[Abstract/Free Full Text].
-
Houk JC,
Adams JL,
Barto AG
(1995)
A model of how the basal ganglia generate and use neural signals that predict reinforcement.
In: Models of information processing in the basal ganglia (Houk JC,
Davis JL,
Beiser DG,
eds), pp 249-270. Cambridge, MA: MIT.
-
Hull CL
(1943)
In: Principles of behavior. An introduction to behavior theory. New York: Appleton-Century-Crofts.
-
Knutson B,
Westdorp A,
Kaiser E,
Hommer D
(2000)
FMRI visualization of brain activity during a monetary incentive delay task.
NeuroImage
12:20-27[ISI][Medline].
-
Koepp MJ,
Gunn RN,
Lawrence AD,
Cunningham VJ,
Dagher A,
Jones T,
Brooks DJ,
Bench CJ,
Grasby PM
(1998)
Evidence for striatal dopamine release during a video game.
Nature
393:266-268[Medline].
-
Koob GF
(1992)
Drugs of abuse: anatomy, pharmacology and function of reward pathways.
Trends Pharmacol Sci
13:177-184[Medline].
-
Krimer LS,
Muly EC,
Williams GV,
Goldman-Rakic PS
(1998)
Dopaminergic regulation of cerebral microcirculation.
Nat Neurosci
1:286-289[ISI][Medline].
-
Kwong KK,
Belliveau JW,
Chesler DA,
Goldberg IE,
Weisskoff RM,
Poncelet BP,
Kennedy DN,
Hoppel BE,
Cohen MS,
Turner R,
Cheng HM,
Brady TJ,
Rosen BR
(1992)
Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation.
Proc Natl Acad Sci USA
89:5675-5679[Abstract/Free Full Text].
-
Mirenowicz J,
Schultz W
(1996)
Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli.
Nature
379:449-451[Medline].
-
Montague PR,
Dayan P,
Person C,
Sejnowski TJ
(1995)
Bee foraging in an uncertain environment using predictive Hebbian learning.
Nature
376:725-728[Medline].
-
Montague PR,
Dayan P,
Sejnowski TJ
(1996)
A framework for mesencephalic dopamine systems based on predictive Hebbian learning.
J Neurosci
16:1936-1947[Abstract/Free Full Text].
-
Nobre AC,
Coull JT,
Frith CD,
Mesulam MM
(1999)
Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention.
Nat Neurosci
2:11-12[ISI][Medline].
-
Ogawa S,
Tank DW,
Menon R,
Ellerman JM,
Kim SG,
Merkle H,
Ugurbil K
(1992)
Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging.
Proc Natl Acad Sci USA
89:5951-5955[Abstract/Free Full Text].
-
Ojemann JG,
Akbudak E,
Snyder AZ,
McKinstry RC,
Raichle ME,
Conturo TE
(1997)
Anatomic localization and quantitative analysis of gradient refocused echo-planar fMRI susceptibility artifacts.
NeuroImage
6:156-167[ISI][Medline].
-
Rescorla RA,
Wagner AR
(1972)
A theory of Pavlovian conditioning and the effectiveness of reinforcement and non-reinforcement.
In: Classical conditioning. 2. Current research and theory (Black AH,
Prokasy WF,
eds), pp 64-69. New York: Appleton-Century-Crofts.
-
Robbins TW,
Everitt BJ
(1996)
Neurobehavioral mechanisms of reward and motivation.
Curr Opin Neurobiol
6:228-236[ISI][Medline].
-
Robinson TE,
Berridge KC
(1993)
The neural basis of drug craving: an incentive-sensitization theory of addiction.
Brain Res Rev
18:247-291[Medline].
-
Rogers RD,
Owen AM,
Middleton HC,
Williams EJ,
Pickard JD,
Sahakian BJ,
Robbins TW
(1999)
Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex.
J Neurosci
19:9029-9038[Abstract/Free Full Text].
-
Rolls ET
(2000)
The orbitofrontal cortex and reward.
Cereb Cortex
10:284-294[Abstract/Free Full Text].
-
Schultz W
(1998)
Predictive reward signal of dopamine neurons.
J Neurophysiol
80:1-27[Abstract/Free Full Text].
-
Schultz W,
Apicella P,
Scarnati E,
Ljungberg T
(1992)
Neuronal activity in monkey ventral striatum related to the expectation of reward.
J Neurosci
12:4595-4610[Abstract].
-
Schultz W,
Apicella P,
Ljungberg T
(1993)
Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task.
J Neurosci
13:900-913[Abstract].
-
Schultz W,
Dayan P,
Montague PR
(1997)
A neural substrate of prediction and reward.
Science
275:1593-1599[Abstract/Free Full Text].
-
Shidara M,
Aigner TG,
Richmond BJ
(1998)
Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials.
J Neurosci
18:2613-2625[Abstract/Free Full Text].
-
Sutton RS
(1988)
Learning to predict by the methods of temporal differences.
Mach Learn
3:9-44.
-
Talairach J,
Tournoux P
(1988)
In: Co-planar stereotaxic atlas of the human brain. Stuttgart, Germany: Thieme.
-
Tesauro G,
Sejnowski TJ
(1989)
A parallel network that learns to play backgammon.
Artif Intell J
39:357-390.
-
Thut G,
Schultz W,
Roelcke U,
Nienhusmeier M,
Missimer J,
Maguire RP,
Leenders KL
(1997)
Activation of the human brain by monetary reward.
NeuroReport
8:1225-1228[ISI][Medline].
-
Tremblay L,
Schultz W
(1999)
Relative reward preference in primate orbitofrontal cortex.
Nature
398:704-708[Medline].
Copyright © 2001 Society for Neuroscience 0270-6474/01/2182793-06$05.00/0
This article has been cited by other articles:
|
|
|
|
 
K. M. COX, H. J. AIZENSTEIN, and J. A. FIEZ
Striatal outcome processing in healthy aging
Cogn Affect Behav Neurosci,
September 1, 2008;
8(3):
304 - 317.
[Abstract]
[PDF]
|
|
|
|
|
|
|
N. J. Diederich and C. G. Goetz
The placebo treatments in neurosciences: New insights from clinical and neuroimaging studies
Neurology,
August 26, 2008;
71(9):
677 - 684.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
Y. Minagawa-Kawai, S. Matsuoka, I. Dan, N. Naoi, K. Nakamura, and S. Kojima
Prefrontal Activation Associated with Social Attachment: Facial-Emotion Recognition in Mothers and Infants
Cereb Cortex,
May 30, 2008;
(2008)
bhn081v1.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. M. Gilman, V. A. Ramchandani, M. B. Davis, J. M. Bjork, and D. W. Hommer
Why We Like to Drink: A Functional Magnetic Resonance Imaging Study of the Rewarding and Anxiolytic Effects of Alcohol
J. Neurosci.,
April 30, 2008;
28(18):
4583 - 4591.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. Van Duynslaeger, F. Van Overwalle, and E. Verstraeten
Electrophysiological time course and brain areas of spontaneous and intentional trait inferences
Soc Cogn Affect Neurosci,
September 1, 2007;
2(3):
174 - 188.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. M. Williams and E. Gordon
Dynamic Organization of the Emotional Brain: Responsivity, Stability, and Instability
Neuroscientist,
August 1, 2007;
13(4):
349 - 370.
[Abstract]
[PDF]
|
|
|
|
|
|
|
M. G. Veldhuizen, G. Bender, R. T. Constable, and D. M. Small
Trying to Detect Taste in a Tasteless Solution: Modulation of Early Gustatory Cortex by Attention to Taste
Chem Senses,
July 1, 2007;
32(6):
569 - 581.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
G. Hasler, S. Fromm, R. P. Alvarez, D. A. Luckenbaugh, W. C. Drevets, and C. Grillon
Cerebral Blood Flow in Immediate and Sustained Anxiety
J. Neurosci.,
June 6, 2007;
27(23):
6313 - 6319.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. M. McClure, K. M. Ericson, D. I. Laibson, G. Loewenstein, and J. D. Cohen
Time Discounting for Primary Rewards
J. Neurosci.,
May 23, 2007;
27(21):
5796 - 5804.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P.R. Corlett, G.D. Honey, and P.C. Fletcher
From prediction error to psychosis: ketamine as a pharmacological model of delusions
J Psychopharmacol,
May 1, 2007;
21(3):
238 - 252.
[Abstract]
[PDF]
|
|
|
|
|
|
|
J. J. Day and R. M. Carelli
The Nucleus Accumbens and Pavlovian Reward Learning
Neuroscientist,
April 1, 2007;
13(2):
148 - 159.
[Abstract]
[PDF]
|
|
|
|
|
|
|
C. Landmann, S. Dehaene, S. Pappata, A. Jobert, M. Bottlaender, D. Roumenov, and D. Le Bihan
Dynamics of Prefrontal and Cingulate Activity during a Reward-Based Logical Deduction Task
Cereb Cortex,
April 1, 2007;
17(4):
749 - 759.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
K. K. Watson, B. J. Matthews, and J. M. Allman
Brain Activation during Sight Gags and Language-Dependent Humor
Cereb Cortex,
February 1, 2007;
17(2):
314 - 324.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. W. Kalivas and N. D. Volkow
The Neural Basis of Addiction: A Pathology of Motivation and Choice
Focus,
January 1, 2007;
5(2):
208 - 219.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Pecina, K. S. Smith, and K. C. Berridge
Hedonic Hot Spots in the Brain
Neuroscientist,
December 1, 2006;
12(6):
500 - 511.
[Abstract]
[PDF]
|
|
|
|
|
|
|
R. B. Postuma and A. Dagher
Basal Ganglia Functional Connectivity Based on a Meta-Analysis of 126 Positron Emission Tomography and Functional Magnetic Resonance Imaging Publications
Cereb Cortex,
October 1, 2006;
16(10):
1508 - 1521.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. Yacubian, J. Glascher, K. Schroeder, T. Sommer, D. F. Braus, and C. Buchel
Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain.
J. Neurosci.,
September 13, 2006;
26(37):
9530 - 9537.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Windmann, P. Kirsch, D. Mier, R. Stark, B. Walter, O. Gunturkun, and D. Vaitl
On Framing Effects in Decision Making: Linking Lateral versus Medial Orbitofrontal Cortex Activation to Choice Outcome Processing.
J. Cogn. Neurosci.,
July 1, 2006;
18(7):
1198 - 1211.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. E. Guyer, E. E. Nelson, K. Perez-Edgar, M. G. Hardin, R. Roberson-Nay, C. S. Monk, J. M. Bjork, H. A. Henderson, D. S. Pine, N. A. Fox, et al.
Striatal functional alteration in adolescents characterized by early childhood behavioral inhibition.
J. Neurosci.,
June 14, 2006;
26(24):
6399 - 6405.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
E. Tricomi, M. R. Delgado, B. D. McCandliss, J. L. McClelland, and J. A. Fiez
Performance feedback drives caudate activation in a phonological learning task.
J. Cogn. Neurosci.,
June 1, 2006;
18(6):
1029 - 1043.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. F. Taylor, B. Martis, K. D. Fitzgerald, R. C. Welsh, J. L. Abelson, I. Liberzon, J. A. Himle, and W. J. Gehring
Medial Frontal Cortex Activity and Loss-Related Responses to Errors
J. Neuros | |
TOP
ABSTRACT
INTRODUCTION
