The value of events that predict future rewards, thereby driving behavior, is sensitive to information arising from external (environmental) and internal factors. The ventral prefrontal cortex, an anatomically heterogeneous area, has information related to this value. We designed experiments to compare the contribution of two distinct subregions, orbital and ventromedial, of the ventral prefrontal cortex to the encoding of internal and external factors controlling the perceived motivational value. We recorded the activity of single neurons in both regions in monkeys while manipulating internal and external factors that should affect the perceived value of task events. Neurons in both regions encoded the value of task events, with orbitofrontal neurons being more sensitive to external factors such as visual cues and ventromedial neurons being more sensitive to internal factors such as satiety. Thus, the orbitofrontal cortex emphasizes signals for evaluating environment-centered, externally driven motivational processes, whereas ventromedial prefrontal cortex emphasizes signals more suited for subject-centered, internally driven motivational processes.
Motivation, that is, what causes an organism to act, is regulated by information arising from internal and external factors (Balleine and Dickinson, 1998; Berridge, 2004; Minamimoto et al., 2009). Internal, subject-related information is related to primary needs such as hunger or thirst, but also to spontaneously initiated cognitive processes. For instance, this is the case both when we go shopping simply because we are hungry and when we plan to go to the store at convenient times to maintain a sufficient food storage. The information about external, environment-related factors arises from sensory cues in the external world, for instance when we purchase the snack that just appeared in an advertisement.
Normal motivation relies on the ventral prefrontal cortex, a heterogenous area (Damasio, 1994; Cardinal et al., 2002; Cox et al., 2005; Ressler and Mayberg, 2007; Wallis, 2007; Berridge and Kringelbach, 2008). Based on cytoarchitectonics and connectivity, ventral prefrontal cortex appears to be segregated into two differentiable circuits: medial [ventromedial prefrontal cortex (VMPFC)], and orbital [orbitofrontal cortex (OFC)]. VMPFC is heavily interconnected with limbic and autonomic structures, and OFC is heavily interconnected with sensory areas (Ongur and Price, 2000). Behavioral and functional imaging results suggest that the anatomical segregation is accompanied by functional differences (Balleine and Dickinson, 1998; Egan et al., 2003; Gottfried et al., 2003; Kringelbach, 2003; Kable and Glimcher, 2007; Ostlund and Balleine, 2007; Rushworth et al., 2007; Behrens et al., 2008; Glascher et al., 2009; Hare et al., 2009). We hypothesized that value information that regulates motivational intensity is evaluated differently in OFC and VMPFC, with OFC neurons emphasizing information about external factors, and VMPFC neurons emphasizing information about internal factors.
We compared the activity of single OFC and VMPFC neurons in behaving monkeys while manipulating external and internal factors controlling the perceived value of task events. We assessed the perceived value of task events by measuring the intensity of two behavioral responses: an operant response (bar release) and an appetitive pavlovian response (lipping) (Bouret and Richmond, 2009). We measured neuronal activity in trials where behavior was guided by visual cues (external factor) or self-initiated (internal factor). We also monitored the influence of satiety, a key internal factor influencing motivation, on behavior and neuronal activity. The neuronal activity in both regions is closely related to the value of task events. The neurons in OFC are more sensitive to external, environment-related information (visual cues), whereas the neurons in VMPFC neurons are more sensitive to internal, subject-related information (self-initiated behavior and satiety).
Materials and Methods
Two male rhesus monkeys, D (9.5 kg) and T (6.5 kg) were used. The experimental procedures followed the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by the National Institute of Mental Health Animal Care and Use Committee.
Each monkey squatted in a primate chair positioned in front of a monitor on which visual stimuli were displayed. A touch-sensitive bar was mounted on the chair at the level of the monkey's hands. Liquid rewards were delivered from a tube positioned with care between the monkey's lips but away from the teeth. With this placement of the reward tube the monkeys did not need to protrude their tongue to receive rewards. The tube was equipped with a force transducer to monitor the movement of the lips (referred to as “lipping” as opposed to licking, which we reserve for the situation in which tongue protrusion is needed) (Bouret and Richmond, 2009). Before each experiment, the amplitude of the signal evoked by delivering of a drop of water through the spout was checked to ensure that it matched the observed lipping response. Monkeys were trained to perform the task depicted in Figure 1.
Cued active trials.
Both monkeys had experience with operant tasks involving a sequential color discrimination task in which they were rewarded for detecting when a target, consisting of a small dot, changed from red to green. Each trial began when the monkey touched the bar. One of three visual cues appeared, followed 500 ms later by a red target (wait signal) in the center of the cue. After a random interval of 500–1500 ms, the target turned green (go signal). If the monkey released the touch bar 200–800 ms after the green target appeared, the target turned blue (feedback signal) and a liquid reward was delivered 400–600 ms later. In cued active trials the reward sizes of one, two, or four drops of liquid were related to the cues. If the monkey released the bar before the go signal appeared or after the go signal disappeared, an error was registered. No explicit punishment was given for an error in either condition, but the monkey had to perform a correct trial to move on in the task. That is, the monkey had to repeat the same trial with a given reward size until the trial was completed correctly. Performance of the operant bar release response was quantified by measuring reaction times and error rates.
Cued passive trials.
Once monkeys adjusted their operant performance as a function of reward-predicting cues (1–2 d), they were exposed to cued passive trials. In these passive trials, monkeys still had to touch the bar to initiate a trial, but once the cue had appeared on the screen, releasing or touching the bar had no effect. Two seconds after cue onset, the blue point also used as a feedback signal in cued active trials was presented and water was delivered 400–600 ms later as a reward. The 2 s delay between cue onset and feedback signal was chosen to match the average interval between these two events in cued active trials. After 2–3 d of training with passive trials alone, monkeys had virtually stopped releasing the bar in cued passive trials. Monkeys were then exposed to a block version of the cued trials for another 2–3 d as follows: blocks of ∼100 trials of each category (cued active or cued passive) alternated without interruption or explicit signaling. In the final version, the six trial types with a combination of reward size (one, two, or four drops) and action contingency (active or passive) alternated randomly. Monkeys were trained for a week in this final version before we started electrophysiological recordings.
Animals were placed in the same environment as before, except that the background of the screen was changed to a large green rectangle. Monkeys rapidly (1 d) learned to hold and release the bar to get the reward without any conditioned cue signaling reward size or timing of actions. To facilitate comparison with cued trials, a blue point was also used as a feedback signal upon bar release and reward was delivered within 400–600 ms. Before the neurophysiological recordings were taken, monkeys were trained for a week with alternating blocks of different reward sizes (one, two, or four drops). Each block comprised ∼50–70 trials with a given reward size, and blocks alternated randomly and abruptly without explicit signaling.
This procedure was conducted in separate sessions. After a neuron had been isolated for recording and the monkey had completed ∼120–160 cued trials, we interrupted the task and delivered ∼100 cc of water through the spout. We then resumed the task for as long as the monkey would work or for an equivalent number of trials as collected before the “free” water delivery.
After initial behavioral training, a magnetic resonance (MR) image at 1.5 T was obtained to determine the placement of the recording well. Then, a sterile surgical procedure was performed under general isoflurane anesthesia in a fully equipped and staffed surgical suite to place the recording well and head fixation post. The well was positioned at the level of the genu of the corpus callosum, with an angle of ∼20° in the coronal plane (Fig. 2).
Electrophysiological recordings were made with tungsten microelectrodes (FHC or Microprobe; impedance: 1.5 MΩ). The electrode was positioned using a stereotaxic plastic insert with holes 1 mm apart in a rectangular grid (Crist Instruments; 6-YJD-j1). The electrode was inserted through a guide tube. After several recording sessions, MR scans were obtained with the electrode at one of the recording sites; the positions of the recording sites were reconstructed based on relative position in the stereotaxic plastic insert and on the alternation of white and gray matter based on electrophysiological criteria during recording sessions.
The lipping signal was monitored continuously and digitized at 1 kHz (Fig. 3 A). For each trial, the latency of lipping responses after cue appearance and feedback (blue spot) signals was defined as the first of three successive windows in which the signal displayed a consistent increase in voltage of at least 100 mV from a reference epoch of 250 ms taken right before the event of interest (cue or feedback).
Single unit activity.
All data analyses were performed in the R statistical computing environment (R Development Core Team 2004). The data were first screened using a sliding window procedure. For each neuron, we counted spikes in a 300 ms test window that was moved in 20 ms increments around the onset of the cue (from −400 to + 1300 ms), around the feedback signal (from −500 to + 1300 ms), and around reward delivery (from −800 to + 1200 ms). At each point, a two-way ANOVA was completed with spike count as the dependent variable. The two factors were reward size (three levels: one, two, four drops) and action (two levels: active, passive). At each time point, we measured the encoding of information about a given factor (percentage of variance explained) for each neuron, as well as the percentage of neurons showing a significant effect (p < 0.05, corrected for multiple comparisons using false discovery rate (“p.adjust” function in R). In self-initiated trials, we used the same approach to study the encoding of reward size around the feedback signal and reward delivery.
Using this screening procedure, we found the epochs that showed a peak in encoding (i.e., variance explained) and focused the analysis on these epochs (n = 5 and 3 in cued and self-initiated trials, respectively; see Result). In each epoch of the cued trials, spike counts were compared across conditions using a two-way ANOVA, with reward size (again, one, two, four drops) and action (active vs passive) as factors. We defined responding neurons as those with a significant effect (p < 0.05) of either factor or their interaction. In self-initiated trials, we used a one-way ANOVA to quantify responses to the reward size factor.
Response latency was defined as the beginning of the first of three successive windows showing a significant effect in the sliding window analysis (p < 0.05). We also measured the time of maximum variance explained after cue onset and around the feedback. It was defined for each neuron as the start time of the window for which the variance explained by a given effect was maximal, whether or not there was a significant response. We considered a 1000 ms cue period starting at cue onset and a 800 ms feedback period centered on the onset of the feedback signal.
To determine the influence of “progression through a session” on neuronal activity, we measured the proportion of responding neurons in each of the five epochs in cued trials (three epochs in self-initiated trials). The means of the proportions of responding neurons across all the epochs of a trial were compared using ANOVA with “brain region” as one factor (two levels: OFC and VMPFC) and progression through a session as another factor (three levels: beginning, middle, and end). Neurons were sorted according to the order in which they were recorded in a session: first (beginning), intermediate (halfway through), and last (last complete recording before the monkey stopped). Neurons that did not belong to any of these three categories were not included in this analysis.
To quantify the effect of the active “satiation” procedure (giving the monkey ∼100 cc of water) on individual neuronal responses, we measured spike counts in epochs where a significant response was detected before the animal was given the bolus of free water. We performed a three-way ANOVA with satiety as the first factor (two levels before and after the bolus delivery), the second being reward size (three levels: one, two, and four drops), and the third being action (active or passive trials). Responses displaying either a main effect of satiety or an interaction between satiety and either of the other two factors were analyzed using a post hoc Tukey honestly significant difference test. The effect of satiation was then classified as “increase” when the encoding of a factor increased, i.e., when a given factor accounted for significantly more variance after than before the satiation, “decrease” when the encoding of a factor decreased, or “change” when the type of response changed, e.g., if the neuron was encoding reward size before and action after the satiation.
The stimulus–reward and action–reward contingencies were manipulated using three different trial types: cued active, cued passive, and self-initiated trials (Fig. 1). In any one trial of each of these trial types, the amount of reward could be one, two, or four drops of fluid. In cued trials, the cue appearing at the beginning of each trial indicated both the amount of fluid reward that would be delivered and whether the trial was active or passive. In cued active trials, the monkeys had to perform an operant bar release response when a red point turned green. A feedback signal (blue point) replaced the green point immediately after each correct response. In cued passive trials, a cue also appeared at the beginning of each trial, and the feedback signal appeared 2 s later, independently of the monkey's behavior (2 s is the average interval between cue and feedback onset in active trials). Cued active and cued passive trials were randomly interleaved during a session. In self-initiated trials the monkeys only had to touch and release a bar; there was no visual cue at the beginning of a trial, but the feedback signal appeared immediately after bar release. Self-initiated trials were presented in randomly alternating blocks, each with a constant reward size (one, two, or four drops). After ∼60 trials, the reward size was changed abruptly. In all trials, cued or self-initiated, the reward was delivered ∼500 ms after the feedback signal.
We trained two monkeys (T and D). To measure the value of task events in all task conditions, we monitored a pavlovian appetitive lipping reaction to the cues and the feedback signal (Fig. 3 A). The percentage of trials with lipping responses to cue appearance increased with reward size but was indistinguishable between cued active and cued passive trials (Fig. 3 B, left) [two-way ANOVA: significant effect of reward size factor (monkey D: F (2) = 26, p < 10−10; monkey T: F (2) = 36, p < 10−10); no effect of action factor, cued active vs cued passive (monkey D: F (1) = 1.8, p = 0.2; monkey T: F (1) = 3.5, p = 0.06)]. This indicates that the perceived value of cues depended upon expected reward size and not upon whether an action would be needed to obtain the reward. In contrast, lipping at the feedback was stronger in cued active than in cued passive trials, and there was relatively little effect of the reward size factor (Fig. 3 B, center) [two-way ANOVA: action (monkey D: F (1) = 91, p < 10−10; monkey T: F (1) = 5.6, p = 0.01); reward size (monkey D: F (2) = 11, p = 2 × 10−5; monkey T: F (2) = 1.9, p = 0.1)]. This indicates that the value of the feedback in cued trials depended much more upon the way in which the trial was completed (active or passive) than on the size of the expected reward. In self-initiated trials, lipping at the feedback increased significantly with reward size (Fig. 3 B, right) (ANOVA for reward size: monkey D: F (2) = 19, p = 4 × 10−7; T: F (2) = 46, p < 10−10). Thus, in the self-initiated trials where there is no action factor, the value of the feedback is strongly related to reward size.
We also assessed perceived value by monitoring an operant response, bar release (Fig. 4), known to be driven by incentive motivation in similar tasks (Bouret and Richmond, 2009; Minamimoto et al., 2009). In cued active trials, error rates decreased significantly with increasing reward sizes (ANOVA: monkey D: F (2) = 33, p < 10−10; monkey T: F (2) = 8, p = 4 × 10−4). In cued passive trials, the monkeys virtually never released the bar. In self-initiated trials, release intervals decreased with increasing reward sizes (ANOVA: monkey D: F (2) = 33, p < 10−10; monkey T: F (2) = 8, p = 4 × 10−4). Thus, the incentive influence of value on operant actions increases with expected reward size in both self-initiated and cued active trials, but not in cued passive trials.
We recorded 167 and 188 neurons from the ventral prefrontal cortex of monkey T and D, respectively. All neurons encountered along the track were included in the analysis, as long as the units were well isolated using a time–voltage threshold discrimination criterion. The activity profiles were similar in the two animals, so the neuronal data were pooled. We reconstructed the locations of the neurons using MR imaging (MRI) (Fig. 2). In cued trials, 112 and 121 neurons were recorded from OFC and VMPFC, respectively. In self-initiated trials, 70 and 74 neurons were recorded from OFC and VMPFC, respectively. Based on a visual inspection, neuronal activity in both regions was affected by both the reward size (one, two, or four drops) and the action (passive vs active trials) factors in cued trials (Fig. 5 A–D) and by the reward size factor in self-initiated trials (Fig. 5 E,F). We performed a screening procedure using ANOVA in sliding windows (with a repeated measure correction, see Materials and Methods) to identify epochs with a strong encoding of reward size and/or action. There were five epochs in cued trials (“cue,” from 0 to 450 ms after cue onset; “wait,” from 500 to 950 ms after cue onset; “pre-feedback.,” from 450 to 0 ms before the feedback; “feedback,” from 0 to 450 ms after the feedback; and “reward,” from 0 to 450 ms after reward delivery). There were three epochs in self-initiated trials (pre-feedback, feedback, and reward). In each epoch, we identified responding neurons using a two-way ANOVA (reward size × action) in cued trials and a one-way ANOVA (reward size) in self-initiated trials (see Materials and Methods).
Neurons in OFC and VMPFC encode the perceived value of task events
We inferred that neuronal activity encoding the value of task events should follow the same pattern as the lipping behavior (strong effect of reward size at cue onset, strong effect of action at the feedback in cued trials, and strong effect of reward size in self-initiated trials) (Fig. 3). We compared the effects of reward size and action factors on spike counts for each neuron using two-way ANOVAs in successive 300 ms windows moved in 20 ms steps (sliding window analysis). At cue onset, the encoding of the reward size factor engaged a larger proportion of neurons and accounted for more variance than the encoding of the action factor (Figs. 6, 7, left panels). The encoding of action became more prominent during the course of a trial, with a sharp increase in the proportion of neurons encoding this factor at the feedback (Figs. 6, center panels, 7 C). In self-initiated trials, the information about reward size arises from the structure of the task (block design) rather than from visual stimuli. A large proportion of neurons encoded this factor (Figs. 6, right panels, 7 E). Thus, neuronal activity in both areas followed the same pattern as lipping responses to cues and feedback signals, in line with the idea that neurons in ventral prefrontal cortex encode the value of these events.
Neuronal activity in these areas was not related to the overt behavior in any simple way that we could identify. We looked for correlation between neuronal activity (firing rates at the cue and around the feedback) and several measures of behavior (reaction time, lipping at the cues, and lipping at the feedback) on a trial-by-trial basis. The number of neurons displaying a significant correlation between firing rate and either reaction time or lipping did not reach significance (i.e., the number of neurons displaying a significant correlation remained lower than the number expected by chance for a sample of this size). Neuronal activity was not related to the physical properties of the stimuli either. In cued trials, eight neurons were tested with two cue sets with which monkeys were equally familiar, and response patterns were indistinguishable between the two cue sets, showing that the responses depended on their associations with the predicted outcomes. Thus, the activity of these neurons is not simply encoding basic motor or sensory processes.
Value-related activity differs between OFC and VMPFC
In cued trials, neurons in both regions were more sensitive to reward size at cue onset and more sensitive to action at the feedback. Nonetheless, response patterns in OFC and VMPFC were different. In OFC, the proportion of neurons encoding reward size was greater than the proportion of neurons encoding action or the interaction between these two factors (Fig. 8 A, top). In VMPFC, the overall proportions of neurons responding to reward size and action were similar, and they were both greater than the proportion of neurons encoding the interaction between the two factors (Fig. 8 A, bottom). In addition, the encoding of the action factor around the feedback differed between OFC and VMPFC. In VMPFC the increase in proportion of neurons encoding action occurred before the feedback (Fig. 8 A, bottom), whereas in OFC the encoding of action peaked after the feedback (Fig. 8 A, top). In self-initiated trials, the encoding of reward size was indistinguishable across the three epochs (χ2 p > 0.05) (Fig. 8 B).
We also examined the timing of responses to the cues and the feedback signal in cued trials by measuring response latencies (when the encoding of an effect started) (Fig. 9 A) and the time of maximum variance explained (when the encoding of an effect peaked) (Fig. 9 B). At cue onset, the median response latency for reward size was significantly shorter in OFC [median = 60 ms, interquartile range (IQR) = 0–140 ms] than in VMPFC (median = 160 ms, IQR = 40–260 ms; Wilcoxon test: p = 0.04) (Fig. 9 A, left). The time at which selectivity peaked (time of maximum variance explained) was shorter for the reward size than for the action factor, but there was no difference between the two regions (Fig. 9 B, left). Thus, at the cue, the encoding of reward size began earlier in OFC than in VMPFC, but the time at which the effect peaked was indistinguishable between the two areas. At the feedback, response latencies for reward size, action, or their interaction were indistinguishable in OFC. In VMPFC, response latencies to action were significantly shorter than responses to reward size or to the interaction between these two factors (Fig. 9 A, right). In addition, in VMPFC the encoding of action (percentage of variance explained) peaked earlier (at the feedback) than the encoding of reward size (after the feedback), whereas in OFC the encoding of reward size peaked before that of action (Fig. 9 B, right).
In short, at cue onset the encoding of reward size is more prominent and arises earlier in OFC than in VMPFC. At the feedback in cued trials, neurons become more sensitive to the action factor and the transition begins earlier in VMPFC (before the feedback) than in OFC (after the feedback signal).
Ventral prefrontal neurons do not encode the incentive influence of value on operant actions
We reasoned that to encode the incentive influence of value on operant actions, i.e., the amount of energy invested in goal-directed behavior, neuronal activity should follow the same pattern as the bar release responses. That is, firing should be affected by reward size in cued active but not in cued passive trials. We searched neurons displaying that specific pattern among neurons displaying a significant interaction between reward size and action. Less than 5% of all the neurons showed this specific pattern across the five epochs of a trial (means: 2.8 ± 1% in OFC and 1.4 ± 1% in VMPFC). Thus, the activity of ventral prefrontal neurons does not reflect the incentive effect of event value on operant actions.
Differential influence of internal and external factors on OFC and VMPFC activity
To assess the relative influence of information about external and internal factors, we compared neuronal activity between cued and self-initiated trials. We reasoned that in cued trials the value of events was determined mostly based on external information (visual stimuli), whereas in self-initiated trials value depended more upon internal knowledge. A direct comparison between the two regions showed that the reward size factor was predominantly encoded in VMPFC neurons during self-initiated trials and predominantly encoded in OFC during cued trials (Fig. 10) (two-way ANOVA on percentages of neurons encoding the factor reward size; significant effect of region: F (1) = 116, p = 1.6 × 10−7; no effect of trial type: F (1) = 2, p = 0.2; and significant interaction: F (1,12) = 77, p = 1.5 × 10−6). Thus, VMPFC neurons are more heavily involved when monkeys spontaneously engage in reward-directed behavior, whereas OFC neurons are more heavily involved when motivational value relies on information provided by visual stimuli.
To assess the influence of satiety, a critical internal factor affecting motivational values, we examined changes in behavior and neuronal activity as monkeys accumulated water during the course of a session (Minamimoto et al., 2009). Monkeys displayed a progressive decrease in lipping responses and bar release performance as they progressed through a session (Fig. 11 A, insets), showing that the value of task events decreases as monkeys accumulate water. In self-initiated trials, in both VMPFC and in OFC the proportion of selective neurons (i.e., showing a significant discrimination) decreased over the course of a session (Fig. 11 A, left) (two-way ANOVA; significant effect of progression: F (2) = 4.5, p = 0.03; significant effect of region: F (1) = 13, p = 3.5 × 10−3; and no interaction: F (2,12) = 0.9, p = 0.4). In cued trials, the proportion of selective neurons decreased in VMPFC but not in OFC, where the proportions of selective neurons across the three periods of a session were indistinguishable (Fig. 11 A, right) (two-way ANOVA; no main effect of progression: F (2) = 0.8, p = 0.4; significant effect of regions: F (1) = 54, p = 1.4 × 10−7; and significant interaction: F (2,24) = 3.5, p = 0.04). To examine whether the decreases could be at least partly due to satiety (fatigue would be another obvious contributor to this effect), we recorded another set of neurons (n = 14 and 12 in VMPFC and OFC, respectively) while the monkeys were given a large bolus of water (∼50% of their usual daily intake) early in the session (before they had completed 200 trials). There was an immediate, significant decrease in performance (average error rate increased from 1.8 ± 0.4% to 13 ± 2%; t(1) = 5.3, p = 2 × 10−6). There was also a significant decrease in selectivity (i.e., in the amount of variance explained by reward size and action factors or their interaction; three-way ANOVA) for 8/9 selective VMPFC neurons, whereas the proportion of responding neurons was unchanged in OFC (Fig. 11 B). This confirms that the effects observed during the course of a session could at least in part be due to satiety. Thus, VMPFC responses appear to depend upon the amount of fluid received up to the time of the recording. In OFC, the proportion of selective neurons was affected by satiety in self-initiated trials, but not in cued-trials, suggesting that when an external stimulus is present its influence is powerful enough to mask any effect related to an internal factor.
Neuronal activity in both OFC and VMPFC is closely related to the perceived value of task events. OFC neurons emphasize value information arising from visual stimuli, whereas VMPFC neurons emphasize value information arising from intrinsic knowledge or satiety levels. The differences in neuronal activity between OFC and VMPFC provide physiological support for the hypothesis, originally based on anatomy, that OFC and VMPFC play different roles in calculating motivational values.
Our data are compatible with previous reports in monkeys and rats showing that OFC neurons are very sensitive to the value of sensory cues, with little influence of the type of operant response animals must perform to get the reward (Schoenbaum et al., 1998; Tremblay and Schultz, 2000; Roesch and Olson, 2004; Padoa-Schioppa and Assad, 2006; Wallis, 2007; Kennerley and Wallis, 2009). This is in line with anatomical data showing strong interactions between OFC and sensory cortices (Ongur and Price, 2000). After the feedback, OFC neurons became more sensitive to the action factor (cued active vs passive trials), in line with a recent report showing that OFC neurons encode the behavioral response at the time of the feedback (Tsujimoto et al., 2009). Here, we show that OFC neurons encode information about the perceived value of both cues and feedback signals, measured using a pavlovian response. This procedure also revealed that the OFC neurons encode the value of task events in a nonchoice situation, even when no movement (or absence of movement) is explicitly required to obtain the reward.
VMPFC neurons are significantly less involved in externally driven motivational processes but heavily engaged when value information arises from internal factors such as spontaneous initiation of actions and thirst. This is in line with anatomical data showing strong interactions between VMPFC, limbic areas. and brainstem nuclei involved in autonomic regulation (Ongur and Price, 2000). These data are consistent with functional imaging studies showing that in humans blood oxygenation level-dependent signals in VMPFC correlate with the feeling of thirst, subjective decision value, or “self-relatedness” (Gusnard et al., 2001; de Araujo et al., 2003; Egan et al., 2003; Kable and Glimcher, 2007; Valentin et al., 2007; Behrens et al., 2008; Northoff and Panksepp, 2008; Glascher et al., 2009; Hare et al., 2009). This is also compatible with earlier experiments showing that electrical stimulation of the VMPFC can elicit drinking in sated monkeys (Robinson and Mishkin, 1968).
The value of cues arises mainly from their association with different reward sizes (Cardinal et al., 2002; Berridge, 2004). In line with the idea that OFC neurons are more sensitive to external information about event value, the encoding of reward size is stronger and appears earlier in OFC than in VMPFC. The value of the feedback signal depends much more upon the action factor, i.e., upon whether completing the trial required a bar release. This effect could be mediated by inputs from structures controlling the action and/or structures monitoring the movement. In any case, the event defined by the appearance of the feedback signal is more valuable to the animal when it follows an operant response. In line with the idea that VMPFC neurons are more sensitive to internal information about event value, the encoding of the action factor increased earlier in VMPFC (at the feedback) than in OFC (after the feedback).
In contrast to measuring the value of task events and their associated outcomes as is often done using choice paradigms (Tremblay and Schultz, 2000; Padoa-Schioppa and Assad, 2006; Kable and Glimcher, 2007; Glascher et al., 2009; Kennerley and Wallis, 2009), here we measure the perceived value of task events using the intensity of an operant bar release and a pavlovian lipping response. The comparison of these responses in passive and active trials allowed us to distinguish activity related to the value of events in all conditions (measured using lipping) from activity more specifically related to operant, goal-directed behavior (Balleine and Dickinson, 1998; Cardinal et al., 2002; Berridge, 2004; Berridge and Kringelbach, 2008; Minamimoto et al., 2009). Although we cannot ask animals about the subjective aspects of these processes, pavlovian conditioning is widely used as a reflection of emotional processes in animals (Cardinal et al., 2002; Berridge, 2004; Berridge and Kringelbach, 2008; Bouret and Richmond, 2009). It is reasonable to think that lipping occurs when an event has a positive (hedonic) affective value. The lipping patterns that we observed are consistent with this interpretation. Neuronal activity in both OFC and VMPFC was closely related to the pattern of lipping responses, but firing did not merely encode motor aspects of lipping. Thus, these data support the idea that the neuronal responses in these two regions of the ventral prefrontal cortex are related to the hedonic value of task events (Cardinal et al., 2002; Kringelbach et al., 2003; Cox et al., 2005; Kable and Glimcher, 2007; Behrens et al., 2008; Berridge and Kringelbach, 2008).
These two areas, OFC and VMPFC, are thought to be involved in assessing information about outcome values during decision processes (Damasio, 1994; Izquierdo et al., 2004; Rushworth et al., 2007; Wallis, 2007). However, our data indicate that relatively few neurons directly encode the incentive influence of the reward on the operant bar release response. In other words, ventral prefrontal neurons do not seem to carry information directly relevant to the modulation of goal-directed actions as a function of the expected reward value. The roles that ventral prefrontal areas play in operant aspects of motivation and decision-making could be exerted via their projections to other structures such as ventral striatum, ventral pallidum, premotor cortex, or anterior cingulate cortex, where information about value would be integrated with motor information to drive reward-directed actions (Shidara and Richmond, 2002; Matsumoto et al., 2003; Amiez et al., 2006; Kennerley and Wallis, 2009).
We propose that OFC and VMPFC have different roles in motivation: they both seem sensitive to the perceived value of events, with VMPFC critical for subject-centered, internally driven motivational processes, whereas OFC is critical for environment-centered, externally driven motivational processes.
This work was supported by the Intramural Research Program of the National Institute of Mental Health. We are grateful to Janine Simmons, Andrew Clark, John Wittig Jr, Narihisa Matsumoto, Walter Lerchner, and Mortimer Mishkin for their helpful comments on this work.
- Correspondence should be addressed to Dr. Barry J. Richmond, Laboratory of Neuropsychology, National Institute of Mental Health, National Institutes of Health, Department of Health and Human Services, Building 49, Room 1B80, Bethesda, MD 20892-4415.