Abstract
The orbitofrontal cortex (OFC) and its interactions with the basolateral amygdala (BLA) are critical for goal-directed behavior, especially for adapting to changes in reward value. Here we used a reinforcer devaluation paradigm to investigate the contribution of OFC to this behavior in four macaques. Subjects that had formed associations between objects and two different primary reinforcers (foods) were presented with choices of objects overlying the two different foods. When one of the two foods was devalued by selective satiation, the subjects shifted their choices toward the objects that represented the nonsated food reward (devaluation effect). Transient inactivation of OFC by infusions of the GABAA receptor agonist muscimol into area 13 blocked the devaluation effect: the monkeys did not reduce their selection of objects associated with the devalued food. This effect was observed when OFC was inactivated during both satiation and the choice test, and during the choice test only. This supports our hypothesis that OFC activity is required during the postsatiety object choice period to guide the selection of objects. This finding sharply contrasts with the role of BLA in the same devaluation process (Wellman et al., 2005). Whereas activity in BLA was required during the selective satiation procedure, it was not necessary for guiding the subsequent object choice. Our results are the first to demonstrate that transient inactivation of OFC is sufficient to disrupt the devaluation effect, and to document a role for OFC distinct from that of BLA for the conditioned reinforcer devaluation process in monkeys.
Introduction
The orbitofrontal cortex (OFC) and its interactions with the amygdala are critical for flexible goal-directed behavior, especially for adapting to changes in reward value. A task that is especially effective for probing this behavioral adaptation in rats (Hatfield et al., 1996), monkeys (Malkova et al., 1997), and humans (Gottfried et al., 2003) is the reinforcer devaluation task. In the procedure used in monkeys, subjects that have formed associations between objects (or other visual cues) and two different primary reinforcers (food rewards) are tested for object preference in a multitrial choice test where an object rewarded with one food is presented concurrently with an object rewarded with the other food. When one food is devalued by selective satiation (feeding that food ad libitum), subjects typically reduce their choice of the objects associated with that food; this shift in object preference is referred to as the “devaluation effect.” Once the initial object–food association is well established, an inability to shift object choice following devaluation could reflect the loss of one or more of the following component functions: (1) registering and encoding a change in food value, (2) integrating the new value of the food with the cognitive representation of the objects, and/or (3) maintaining and using the updated value of the object representations to guide choices. Although damage to either OFC or amygdala, or their interconnections, disrupts the devaluation effect (Malkova et al., 1997; Balleine et al., 2003; Pickens et al., 2003, 2005; Izquierdo et al., 2004; Machado and Bachevalier, 2007a,b), it is possible that these component functions differ with respect to their dependence on the integrity of OFC and amygdala.
The role of a particular brain region for each of the component functions can be assessed by using reversible pharmacological inactivation of the brain region of interest during distinct phases of the task. We have previously used this approach to determine that inactivation of the basolateral amygdala (BLA) during the period of selective satiation blocked the devaluation effect, whereas inactivation of BLA limited to the postsatiety choice period did not disrupt the devaluation effect (Wellman et al., 2005). This suggests that while BLA plays a critical role in registering and integrating the new value of the food with the cognitive representation of the objects associated with that food, it is not required for maintaining and subsequently applying this information. As previously proposed, OFC is well suited for storing and/or maintaining the representations of the devalued cues and using them to guide object selection (Gallagher et al., 1999; Pickens et al., 2003, 2005; Wellman et al., 2005). We therefore predicted that the devaluation-induced shift in object selection during the postsatiety choice period would be sensitive to impairment of OFC function, in contrast to its insensitivity to BLA inactivation. We tested this hypothesis by inactivating the OFC (using focal intracerebral infusions of the GABA-A agonist muscimol) during the object choice period following selective satiation with one of two food choices.
Materials and Methods
Subjects
Four animals were used in the study: two male pigtail macaques (Macaca nemestrina), SV and TR; one male rhesus macaque (Macaca mulata), OT; and one female rhesus macaque, SA. They were 3–6 years of age and weighed 5.6–11 kg at the beginning of this study. All monkeys were housed in pairs in a room with regulated lighting (12 h light/dark cycle) and maintained on primate LabDiet (number 5049; PMI Nutrition International) supplemented with fresh fruit. Water was available ad libitum in the home cage. The study was conducted under a protocol approved by the Georgetown University Animal Care and Use Committee and in accordance with the Guide for Care and Use of Laboratory Animals adopted by the National Institutes of Health.
For the present experiments, each monkey was implanted with a stereotaxically positioned chronic infusion chamber, which allowed a removable injector, fitted with an infusion cannula of adjustable length to be inserted into predetermined sites in the brain through the guiding channels of a grid placed inside the chamber for intracerebral drug infusions (see below).
Infusion chamber and injector
The chamber and injector were a modification of those previously described by Wellman et al. (2005). A rectangular chamber (38 mm length × 44 mm width × 18 mm depth) was made of polyoxymethylene (Helm Tech Machine). It was covered with a removable top that was secured with four screws. For drug infusions, the top was removed and a grid was inserted to provide guiding channels for the placement of the injector and cannula. The grid contained 168 guiding channels set in 12 rows and 14 columns spaced 2 mm apart. The columns were arranged into two groups of seven, separated at the midline by a 12-mm-wide section.
A custom-built telescoping injector made of polyethylene terephthalate polyester (Elmeco Engineering) was designed to fit snugly into the infusion grid and allow an easy adjustment of infusion cannula length. The injector consisted of individual discs that were serially interlocked with stainless steel rods, with the central tubes acting as a guide. A 27 ga tube was inserted into the topmost disc serving as the infusion cannula and was connected via polyethylene tubing (Small Parts) to a 50 μl Hamilton syringe. To perform an infusion, the telescoping apparatus was slowly inserted through the infusion grid until the discs were flush with each other and the grid.
Surgery for implantation of the infusion chamber
After first sedating the monkey with ketamine (10–15 mg/kg, i.m.), a surgical level of anesthesia was established and maintained with isoflurane gas (1–2%, to effect). Atropine (0.02–0.05 ml/kg, i.v.) was administered to counteract the decrease in heart rate caused by the anesthetics and to maintain cardiac output close to normal. Throughout the aseptic procedure, the monkey received an intravenous drip solution of isotonic fluids; heart rate, respiration rate, blood pressure, expired CO2, and body temperature were monitored. For the surgery, the monkey was placed in a stereotaxic head holder (Crist Instrument) and a sterile field was established. The skull was exposed over the area of the cranium. The chamber was positioned stereotaxically, according to the macaque brain atlas (Paxinos et al., 2000), over the region of intended infusions with respect to the ear bars of the head holder. Four small craniotomies were made, each placed outside one of the four sides of the chamber. Four H-shaped anchor bolts made of polyoxymethylene (Elmeco Engineering) were inserted and secured in place with a small amount of bone cement (Palacos R; BioMet Orthopedics). Additional bone cement was used to secure the entire chamber in place. All monkeys received postoperative analgesics as determined in consultation with the facility veterinarian. Approximately 2 weeks after the surgery, the channels intended for drug infusions were opened by hand-held drill under anesthesia and aseptic conditions.
Magnetic resonance imaging
Postoperatively, each monkey received at least one T1-weighted scan to determine and/or verify the coordinates for the infusion sites. Additional scans were performed throughout the experiment as needed. For each structural scan, the monkey was anesthetized with a mixture of ketamine (ketamine hydrochloride, 10–15 mg/kg, i.m.) and xylazine (0.1 mg/kg, i.m.) and placed in a human head coil of a Siemens Trio 3T MRI scanner. An MPRAGE pulse sequence (TR = 1600 ms, TE = 4.38 ms, TI = 640 ms, flip angle = 15°, averages = 1, FOV = 256 × 256 mm2) was used to acquire 3D volume of the monkey brain. Slices (160 total) had a 1.0 mm thickness and effective resolution of 1.0 × 1.0 × 1.0 mm3. To verify the location of the predetermined infusion sites, tungsten microelectrodes (FHC), which were visible on the scan, were inserted through the infusion grid into the specified locations in each hemisphere, with the tip of the electrode placed at ∼10 mm above the intended infusion site. Based on the position of the tip of the electrode on the MRI scan, the final coordinates for the drug infusions were adjusted with respect to the coordinates of the infusion grid.
Intracerebral drug infusions
As shown in Figure 1, drug infusions were targeted bilaterally at Walker's area 13 of OFC (Walker, 1940), more specifically at the caudal portion of the cortex located between the medial orbital sulcus and the lateral orbital sulcus, representing areas 13m and 13l. This region of the OFC receives strong reciprocal projections from the BLA (Porrino et al., 1981; Amaral et al., 1992; Carmichael and Price, 1995; Ghashghaei and Barbas, 2002; Ghashghaei et al., 2007). In addition, this region was included within the area of the intended lesion in the previous lesion studies that examined the role of OFC in reinforcer devaluation (Baxter et al., 2000; Izquierdo and Murray, 2004, 2010; Machado and Bachevalier, 2007a,b; Rudebeck and Murray, 2011).
A, Coronal sections from a standard macaque brain. The numbers on the left (+30, +29, +28) indicate the distance in millimeters from the interaural plane. The symbols show the placement of bilateral infusion sites reconstructed from MRI for each monkey (SA, SV, OT, and TR). The arrows mark the extent of the area of intended infusions within the OFC (area 13). 13l and 13m represent subdivisions of area 13 within the OFC. los, Lateral orbital sulcus; mos, medial orbital sulcus. B, MRI section through OFC in monkey TR corresponding approximately to level +29 showing an electrode positioned in the infusion site (arrow). C, MRI section through OFC in monkey OT corresponding approximately to level +30 showing an electrode positioned in the infusion site (arrow).
The GABAA-receptor agonist muscimol (MUS) was used to transiently suppress synaptic activity in the vicinity of the focal infusion site. Previous work in rats (Martin and Ghez, 1999) showed that a 1 μl infusion of MUS in a concentration of 9 mm affected a sphere of tissue with a radius of 1.5 mm surrounding the site of infusion. In the previous experiment in monkeys from our laboratory (Wellman et al., 2005), 1 μl of MUS in a concentration of 9 mm (9 nmol) per hemisphere was used to inactivate BLA. To inactivate a larger volume of tissue within OFC, we infused 2 μl of MUS in a concentration of 18 mm (36 nmol; Sigma). This dose was comparable to that found effective by Kuwajima and Sawaguchi (2007; 44 nmol) when infused unilaterally into dorsolateral prefrontal cortex in the monkey.
MUS or sterile saline (0.9% NaCl solution) of the same volume were infused bilaterally at a rate of 0.2 μl/min using the removable injector described above, connected to an injection pump (New Era Pumps Systems). The drug infusions were performed using an aseptic technique while the monkey was seated in a primate chair (Crist Instrument) with its head restrained manually. The entire infusion procedure lasted ∼25 min (10 min infusion, 15 min transport time).
Behavioral training and testing
All monkeys were trained on a task described previously by Malkova et al. (1997) and Wellman et al. (2005).
Apparatus and materials.
The monkeys were trained in a Wisconsin General Testing Apparatus located in a darkened sound-shielded room. The test compartment was illuminated with a 15 W fluorescent bulb (Philips F15T8-CW; Philips Electronics), but the monkey's compartment was unlit. The test tray, which was located at the level of the floor of the monkey transport cage, contained three food wells spaced 18 cm apart (center to center) on the midline of the tray. The wells were 25 mm in diameter and 5 mm deep. For the present task, only the two outer wells were used. The stimuli were 120 junk objects that differed widely in shape, size, color, and texture.
Food reinforcers
In monkeys, a consistent preference for one type of food reinforcer over another in the reinforcer devaluation tasks has been reported in studies from several labs, including ours (Malkova et al., 1997; Thornton et al., 1998; Baxter et al., 2000; Izquierdo et al., 2004; Wellman et al., 2005; Mitchell et al., 2007). In an attempt to minimize an imbalance in food preference, we pretested the first monkey (SA) on a food preference test, similar to that used by Baxter et al. (2000) and Izquierdo et al. (2004). In each trial of this test, two of 10 different foods were presented simultaneously for choice. All combinations of foods were tested against each other eight times each, for a total of 360 trials at a rate of 60 trials per day. Based on the results of this test, for the reinforcer devaluation task, we selected two foods that were approximately equally preferred: raisins (Sun-Maid) and Honey Nut Cheerios (General Mills). However, in subsequent stages of the task, the monkey developed a strong preference for raisins over Cheerios, despite the fact that these two foods were equally matched for preference during the initial pretest. The tendency for some monkeys to show a shift in food preference over the training period was also reported by Baxter et al. (2000). We therefore decided not to conduct food preference pretesting on the remaining animals, and instead selected two foods that are commonly used as reinforcers across laboratories: fruit snacks and peanuts (Malkova et al., 1997; Baxter et al., 2000; Wellman et al., 2005).
Thus, for all animals except SA, food 1 was half of a fruit snack, a chewy candy of ∼10 mm size made from fruit juice (Sharkbites; General Mills or Fruity Gems; Bio-serv) and food 2 was half of a honey roasted peanut (Planters; Kraft Foods). Before initiation of the behavioral training, we confirmed that all three monkeys readily retrieved and ate both reinforcers.
Object discrimination training.
The animals were first trained on a set of 60 object-discrimination problems. The objects were placed over the food wells; the monkey could only see and retrieve the food by displacing an object. In each of the 60 object pairs, one object (S+) was baited with a food reinforcer and the other was unbaited (S−). Half of the S+ objects (30) were baited with food 1 and the other half baited with food 2, intermixed within a session. The S+ and S− assignment of the objects, the order of the object pairs, and the food reinforcer paired with particular objects remained constant across days; however, the left–right positions of the S+ object were pseudorandomized. The intertrial interval between the presentations of two consecutive pairs was 20 s. The monkeys were trained at a rate of one session per day, 5 d a week, until they reached criterion, which was set at a mean of 90% correct responses or better over 5 consecutive days (i.e., 270 or more correct responses of 300). The aim of the object discrimination training was to allow the animals to instantiate an association between objects with specific food reinforcers.
Probe sessions.
Upon reaching criterion in object discrimination training, the animals' choices of objects were assessed in probe sessions, in which only baited objects (S+) were used. For each probe session, the 60 S+ objects were randomly assigned to create 30 pairs of probe trials, each offering a choice between an object baited with food 1 and an object baited with food 2, with left–right positions of the two foods pseudorandomized. By repeating this choice with all 30 reinforced object pairs, it was possible to generate a preference score. For example, a preference score of 23 for fruit snack objects would be recorded if the animal chose 23 objects covering fruit snacks and seven objects covering peanuts. Some sessions were baseline probe sessions, in which the animals' choices of objects were assessed with no other experimental manipulations. Other probe sessions (see below) were preceded by a devaluation procedure, in which the monkey was allowed to consume one of the two food rewards to satiation (selective satiation), and intracerebral drug infusions.
Selective satiation.
For the devaluation, conducted ∼24 h after the last feeding, a food box attached to the monkey's home cage was filled with one of the two food reinforcers (either food 1 or food 2) of a measured quantity. The monkey was allowed to eat the food without being directly observed for 15 min. The experimenter then entered the room and checked the amount of food eaten. If the monkey consumed all the food, the experimenter gave the monkey more food. The experimenter then observed the monkey from outside of the housing room until the monkey did not take any more food for 5 min. At that time, the food box was removed, and the amount of remaining food was measured. In all cases, 30 min was a sufficient time to complete this procedure. After selective satiation, animals were tested in the probe sessions as described above.
Under control conditions, many of the objects in the probe session associated with the sated (devalued) food are rejected in favor of the objects associated with the nonsated (non-devalued) food, resulting in a clear shift in the preference score toward the objects associated with the non-devalued food. No object was presented more than once during the probe session, so that the assessment was uncontaminated by new learning. Therefore, the rejection of an object associated with the sated food is an indication that the devaluation of the food was cognitively transferred to the object.
Difference score.
The number of food 1 and food 2 objects chosen in each probe session following devaluation was recorded for each monkey. The number of objects reinforced with the devalued (sated) food the monkey chose was then subtracted from the number of objects reinforced with that same food chosen in the preceding baseline probe session, resulting in a difference score. For example, if the animal chose 23 objects covering fruit snacks during the baseline probe session and 10 objects covering fruit snacks following selective satiation with fruit snacks, the subtraction yielded a difference score of 13. The difference score was assessed for each satiation (food 1 and food 2) separately, and the two values were then added to obtain a cumulative difference score in a similar way as in previous studies in monkeys (Malkova et al., 1997; Thornton et al., 1998; Baxter et al., 2000, 2009; Izquierdo et al., 2004; Wellman et al., 2005; Machado and Bachevalier, 2007a,b; Mitchell et al., 2007; Izquierdo and Murray, 2010). A larger difference score reflects a larger devaluation effect.
Timing of treatments and testing
We assessed the effects on reinforcer devaluation of OFC inactivation at each of two time points: before satiation and after satiation (before the probe session). The timing schedule for the two different infusion probe sessions (MUS before satiation and MUS after satiation) is presented in Figure 2. In both conditions, there was an equivalent duration of time between drug infusion and the probe session. As found in a previous study (Kuwajima and Sawaguchi, 2007), MUS infusions in lateral prefrontal cortex in monkeys resulted in an effective inactivation of the targeted area, as measured by error rate of targeted eye saccades that lasted for at least 2 h. Based on this finding, we expected that the infused area within OFC would be effectively inactivated throughout the probe session in both MUS conditions (MUS before and MUS after satiation).
Timing of treatments and testing. A, B, Timing of the infusion session when MUS or saline was infused before (A) or after (B) satiation.
Weekly schedule of testing sessions
As presented in Figure 3, on day 1 of each week, the monkey was given a practice session on the object discrimination training with the original 60 pairs of objects. On day 2, the monkey was tested in a baseline probe session, in which the monkey made choices in each of the 30 probe trials between an object baited with food 1 and an object baited with food 2, to determine the baseline preference score. On day 3, the monkey was again given a practice session on the object discrimination training with 60 pairs of objects. The practice sessions (days 1 and 3) with the original 60 pairs of objects were administered to ensure that the effects of the satiation procedure did not carry over to the subsequent probe sessions.
Weekly schedule of testing sessions. Days 1–7 represent a sequence of daily behavioral training and testing that was identical for each of the infusion probe sessions conducted on day 4.
On day 4, according to a randomized schedule, the monkey was infused with MUS or saline either before or after selective satiation (before the probe session) to determine the number of choices of food 1 and food 2 objects. It was important to ensure that the repeated devaluations of each food did not alter the monkey's choices of objects during the baseline probe sessions. Therefore, each infusion probe session (day 4) was compared with its preceding baseline probe session (day 2). In addition, to avoid the possibility that the repeated devaluation might affect the monkey's choices and also its willingness to eat a large amount of the same food repeatedly, the saline infusions were limited to two infusions total, one before satiation with one of the foods and one after satiation with the other food. The monkeys were then given at least 2 d of rest (days 5–7) without any testing before this schedule began again. Each monkey completed at least six of these testing schedules (Fig. 3), typically over a period of 6–12 weeks.
As described in Figure 3 (bottom), all monkeys received MUS infusions before satiation with both food 1 and food 2 and MUS infusions after satiation with both food 1 and food 2. Two monkeys (OT and SA) received saline infusions before satiation with food 1 and after satiation with food 2 and two monkeys (TR and SV) received saline infusions in a reversed order, before satiation with food 2 and after satiation with food 1. Within the randomized schedule, selective satiation with food 1 and food 2 alternated between weeks so that at least 2 weeks intervened between repeated satiation with the same food.
Consummatory probe
To determine whether or not the experimental manipulations affected choices of the primary reinforcers (food 1 and food 2) when they were presented without the objects, immediately following the choices of the 30 reinforced object pairs in the probe session, the monkey was given additional 30 trials, each consisting of a choice between the two primary reinforcers (food 1 and food 2), with the left–right positions of the two foods pseudorandomized. Three monkeys (OT, SV, and TR) were given the consummatory probe after baseline probe sessions (six sessions per monkey), and infusion probe sessions following each of the six experimental conditions (saline, MUS before, and MUS after satiation). The fourth monkey (SA) was given the consummatory probes separately, i.e., the monkey was tested only to assess the primary reinforcer choices within a session, two times to assess baseline and two times following MUS/saline infusions. Because this monkey was not tested under all experimental conditions of the consummatory probe, its scores were not included in the statistical analysis for this measure.
Data analysis
All statistical comparisons were within subjects. For MUS infusion before selective satiation, the difference score was assessed for each satiation (food 1 and food 2) separately, and the two values were then added to obtain a cumulative difference score. Similarly, the cumulative difference score was obtained for MUS infusion after satiation (before object probe session). For the purpose of data analysis, all results of saline infusions for each animal (both before and after selective satiation) were pooled to generate a cumulative difference score; this ensured that both foods were represented in the control condition. The differences between the MUS infusions before selective satiation, after selective satiation (i.e., before object probe session), and saline were analyzed by within-subject ANOVA with repeated measures. Similarly, cumulative difference scores were calculated for consummatory probes. The cumulative difference scores were analyzed by within-subject ANOVA with repeated measures comparing saline, MUS infusions before, and MUS after selective satiation.
Results
Infusion site verification
All animals were unavailable for histological processing because they were undergoing additional behavioral testing at the conclusion of the study. Because we have previously demonstrated that MRI localization closely corresponds with histological verification (Wellman et al., 2005), we used MRI evaluation to confirm that infusions in all four animals were placed as intended. The localization of the infusion sites using a tungsten electrode inserted in the sites where infusions were placed is shown in two representative cases, TR (Fig. 1B) and OT (Fig. 1C).
Object discrimination
The monkeys reached criterion in an average of 26 sessions (range, 20–34). Analysis of the responses to each object showed that although each monkey reached criterion when the performance was averaged across all objects, each monkey missed one or two particular objects (different objects for each monkey) consistently, i.e., three to five times each over the last 5 d of training. When one of these objects was subsequently presented together with another baited object during the probe session, the monkeys continued to avoid the object in favor of the other even if the other object was baited with the devalued food (under saline condition). This might have reduced the cumulative difference score by one or two points in each condition.
Baseline object preference
In the baseline probe sessions, all monkeys chose more objects reinforced with food 1 than objects reinforced with food 2 (26:4). There was a significant difference between the number of food 1 objects selected and the number of food 2 objects selected, as shown by a repeated-measures ANOVA (F(3,5) = 21.9 p < 0.05). However, the monkeys' baseline choices did not significantly change across the six baseline sessions (F(3,5) = 0.986; p > 0.10), indicating that the baseline choice was not affected by either satiation or MUS infusions in OFC.
Saline infusions
Each animal received two saline infusions into OFC, one before satiation with one food and one after satiation with the other food. Comparison of the difference scores obtained under these two conditions yielded no significant difference across animals (Student's paired t test, t = −0.654, p > 0.05); therefore, we added the difference scores for each condition to attain a cumulative score.
Effect of MUS in OFC
For these experiments, we predicted that if OFC is necessary for using the updated information to guide selection of objects when presented with a choice, then inactivation during the probe session would impair the shift in object preference. In contrast, if OFC is not necessary for using the updated information and guiding the selection of objects, but only for registration or integration of the devaluation, then inactivation limited to the probe session would be without effect. The results we obtained are consistent with our prediction: MUS inactivation in OFC blocked the devaluation effect compared with saline infusions. The monkeys continued to select the objects that predicted the devalued food.
Cumulative difference scores in each experimental condition (saline, MUS before satiation, MUS after satiation) are presented in Table 1 and Figure 4. There was a significant effect of experimental condition when cumulative difference scores were compared by a repeated-measures within-subject ANOVA (F(3,2) = 10.97; p < 0.05). Post hoc pairwise comparisons (Bonferroni) showed a significant difference between each of the MUS conditions (before and after satiation) compared with saline (p < 0.05), but no difference when the two MUS conditions were compared with each other. In the presence of MUS in OFC, the number of selected objects baited with the devalued food was not different from the number of objects baited with the same food selected during the baseline probe session (paired t test: t = 1.3; p > 0.05 and t = 1.9; p > 0.05, for MUS before and after satiation, respectively). In contrast, in the presence of saline in OFC, the monkeys showed a strong devaluation effect: the number of objects baited with the devalued food decreased significantly compared with the baseline condition (t = 4.8; p < 0.05). Thus, MUS infusions in OFC resulted in a near-complete blockade of the devaluation effect.
Cumulative difference scores in the reinforcer devaluation task
Each bar represents the mean (+SEM) difference score between object choices in baseline and infusion probe session in each infusion condition: saline (white), MUS infusion before satiation (dark gray), and MUS infusion after satiation (light gray). Saline data were averaged for both before and after conditions. There was a significant difference between each of the MUS conditions (before and after satiation) compared with saline (*p < 0.05), but no difference when the two MUS conditions were compared with each other.
Food consumption during selective satiation
To rule out the possibility that a decrease in the devaluation effect could be explained by OFC inactivation affecting food consumption during selective satiation, we measured the amount of food (in grams) consumed during selective satiation in all conditions. There was no effect of experimental condition on food consumption during selective satiation. A two-way repeated-measures ANOVA, with food choice (food 1 or food 2) and experimental condition (saline, MUS before satiation, or MUS after satiation) as repeated measures, yielded no significant differences (F(2,2) = 0.195, p > 0.10). There was a trend toward significance of food choice (F(1,3) = 8.35, p = 0.063), but no significant interaction of these two factors (F(2,2) = 0.573 p > 0.10). In addition, there was no significant difference between the three experimental conditions in the amount of food eaten for peanuts, fruit snacks, or both cumulatively, indicating that the monkeys ate a comparable amount of food in each of the experimental conditions. Thus, OFC inactivation did not affect the amount of food consumed during selective satiation.
Primary reinforcer consumption during probe session
Following OFC inactivation, when the monkeys selected the object overlying the devalued food in the probe session, they continued to retrieve the food revealed by displacing the object. Three monkeys ate almost all the devalued food reinforcers, while one monkey (TR) consistently put the food reinforcer in its mouth but then threw it away.
Consummatory probe
To determine whether MUS in OFC impaired the devaluation of the primary reinforcer (food), monkeys were presented with food choices in the absence of objects. All monkeys with MUS in OFC avoided the devalued food when presented in the absence of objects. Cumulative difference scores for the number of food reinforcers eaten in each experimental condition (saline, MUS before satiation, MUS after satiation) are presented in Table 2 and Figure 5. There was no significant effect of experimental condition when cumulative difference scores were compared by repeated-measures within-subject ANOVA (F(3,2) = 2.0, p > 0.05). These findings indicate that MUS infusion in OFC did not affect the choices of primary reinforcers after devaluation. These observations are similar to those reported for monkeys with OFC lesions (Izquierdo et al., 2004; Machado and Bachevalier, 2007a).
Cumulative difference scores in the consummatory probe session
Each bar represents the mean (+SEM) difference score between food choice in baseline and infusion probe sessions in each infusion condition: saline (white), MUS infusion before satiation (dark gray), and MUS infusion after satiation (light gray). No significant differences between these conditions were found.
Additional control experiment
As an additional control experiment, we examined the effect of MUS infusion in OFC when the probe test was conducted immediately after the MUS infusion, without allowing time for the drug to spread throughout the targeted area. To test this, we infused MUS in OFC after satiation and tested the monkeys immediately thereafter. In this condition, the devaluation effect was not affected by the MUS treatment compared with saline. Difference scores in each experimental condition (saline, MUS after satiation immediately before testing) are presented in Figure 6. In addition, Figure 6 shows the corresponding results from the same animals obtained from the condition with a 30 min delay between MUS infusion and the probe test (i.e., a subset of the data that were used to generate Fig. 4). Three monkeys were used for this experiment (SA, TR, and OT). In monkeys SA and TR, the satiation was done only for food 1, for which both monkeys showed a strong preference. In monkey OT, which showed a more balanced choice of foods, satiation was done for both foods and the average of the two difference scores was used. For statistical analysis, the data from the treatment condition in which MUS was infused 30 min before testing were included. There was a significant effect of experimental condition when the difference scores were compared by repeated-measures within-subject ANOVA (F(2,2) = 28.34, p < 0.01). Post hoc pairwise comparisons (Bonferroni) showed that the condition MUS after satiation with no delay before testing was not different from saline but was significantly different from MUS after satiation with a 30 min delay before testing. These results confirm that a delay is necessary to obtain a drug-induced impairment of the devaluation effect, presumably because the delay allows for drug diffusion to inactivate a critical portion of OFC.
Each bar represents the mean (+SEM) difference score between object choices in baseline and infusion probe session in each infusion condition: saline (white bar), MUS after satiation with no delay between MUS infusion and the probe session (0 min; striped bar), and MUS after satiation with a 30 min delay between MUS infusion and the probe session (30 min; gray bar). In the MUS after satiation with no delay condition, the monkeys were infused with MUS immediately before probe session; the data presented in the graph are difference scores following satiation with food 1 for monkeys SA and TR and the mean of difference scores following both food 1 and food 2 for monkey OT. The data for MUS after satiation with a 30 min delay are the results from the same animals following satiation with the corresponding foods and represent a subset of the data that were used to generate Figure 4. The mean difference score following infusion of MUS after satiation with no delay was not different from saline infusions, whereas the mean difference score for MUS after satiation with a 30 min delay was significantly reduced compared with saline (*p < 0.05).
Discussion
We found that OFC inactivation during the object choice phase disrupted the devaluation effect, i.e., the monkeys did not reduce their selection of objects associated with the devalued food. This supports our hypothesis that OFC activity is required during the postsatiety object choice period to guide the selection of objects. As predicted, we observed the disruption of the devaluation-induced shift in object selection regardless of whether the infusion of MUS took place before or after the satiation period. This finding sharply contrasts with the previous characterization of the role of BLA in the same devaluation process in our laboratory (Wellman et al., 2005). Whereas activity in BLA was required during the selective satiation procedure, it was not necessary for guiding the subsequent object choices. Together, these results indicate that whereas BLA activity is critical only during the satiation phase, OFC activity is critical during the subsequent object choice phase. Our results are the first to demonstrate a difference between the roles of BLA and OFC in the conditioned reinforcer devaluation in monkeys. Our findings are also the first to show that the devaluation effect can be disrupted by pharmacological inactivation of OFC.
The dissociation we observed between the roles of the amygdala and OFC during the choice process is similar to that observed by Arana et al. (2003), who used PET imaging to measure activity in OFC and amygdala in human subjects while they made judgments of the appetitive incentive value of foods presented on a restaurant menu, and then selected goals based on those values. Whereas both amygdala and OFC were activated as a function of incentive value, only OFC was activated when the subjects were making choices informed by the perceived incentive values in the absence of the actual foods (Arana et al., 2003). Together with our data, this supports the concept that the OFC maintains the representations of various cues, e.g., objects, for the purpose of making comparisons between their relative incentive values. The OFC may represent a point of convergence of the sensory features of cues, primary reinforcers, and mnemonic information about their past associations and changes in value. This enables OFC neurons to align behavioral responses with shifts in reward value, leading to strategic decision making that maximizes desirable outcomes (Padoa-Schioppa and Assad, 2006; O'Doherty, 2007; Wallis, 2007; Padoa-Schioppa, 2009; Roesch et al., 2010; Schoenbaum and Esber, 2010; Wallis and Kennerley, 2010).
While our results demonstrate a dissociation between the roles of BLA and OFC during the object choice phase, they do not allow us to conclude that OFC, like BLA, is required during the satiation period. Due to the long duration of action of intracerebral MUS (Kuwajima and Sawaguchi, 2007), MUS given before the satiation period will also be present throughout the choice testing period. Because we have shown that the presence of MUS in OFC during the choice testing period is sufficient to disrupt the devaluation effect, this effect could also account for the disruption of devaluation observed when MUS is given before satiation. To determine whether inactivation limited to the satiation period would also be sufficient to produce this disruption, a new testing protocol would be needed in which the satiation period and the choice testing were separated by a several hour interval to allow for complete drug clearance before testing.
As previously discussed (Wellman et al., 2005), BLA may serve to facilitate the storing of information in OFC and/or other regions to which the amygdala projects (Amaral et al., 1992; Baxter et al., 2000). OFC activity measured by fMRI was decreased following devaluation when human subjects viewed an arbitrary picture that previously had been associated with the odor of that food (Gottfried et al., 2003) or when they were making choices between arbitrary visual patterns that signaled instrumental actions associated with a devalued versus non-devalued food (Valentin et al., 2007). In these cases, as in our experiments, the new (reduced) value of the sated food is transferred cognitively to representations of images or objects that have been previously associated with the food (Tremblay and Schultz, 1999; Rolls, 2000), allowing for the devaluation of the internal representations without the need to learn new associations (Murray et al., 2007).
By demonstrating that transient inactivation of OFC with a GABA-A agonist is sufficient to disrupt the devaluation effect, our results extend previous findings from lesion studies in monkeys that revealed a crucial role for OFC in the reinforcer devaluation task (Izquierdo et al., 2004; Machado and Bachevalier, 2007a,b). Our drug infusions, which targeted area 13 in OFC, were within the region damaged in the lesion studies, which was restricted to areas 11 and 13 (Machado and Bachevalier, 2007a,b); lesions in these same areas also impaired learning rapidly changing relationships between stimuli and reinforcement in a probabilistic setting (Rudebeck and Murray, 2011). The results of earlier lesion studies with more extensive lesions, including portions of areas 10 and 14 (Izquierdo et al., 2004), were comparable to those with more restricted lesions, suggesting that the critical substrate for mediating the devaluation effect is within areas 11 and 13. Because of the differing time frames for testing animals after lesions versus drug microinfusions (weeks/months vs minutes/hours), the two approaches differ with respect to the potential for adaptation and compensation; this makes it difficult to make direct comparisons between lesion and drug inactivation experiments when determining the minimal substrate necessary for disruption of devaluation. Nevertheless, it would be interesting to determine whether drug inactivation targeted to area 11 can impair the devaluation to the same extent that we have seen with inactivation of area 13. In other studies, damage to area 14, but not 11/13, impaired extinction of responding to a previously rewarded object, while neither lesion impaired object discrimination reversal, indicating dissociable contributions of distinct OFC subregions to goal-directed behaviors (Kazama and Bachevalier, 2009; Rudebeck and Murray, 2011).
OFC inactivation did not impair the devaluation of the primary (food) reinforcers when they were presented during a consummatory probe session in the absence of the objects; the treated subjects showed the normal reduction in the selection of the devalued food. This observation, similar to that obtained with OFC lesions (Izquierdo et al., 2004; Machado and Bachevalier, 2007a; Rudebeck and Murray, 2011), indicates that even though the firing of neurons in OFC respond differentially to the sight and taste of foods that have been fed to satiety compared with non-devalued foods (Rolls and Baylis, 1994; Critchley and Rolls, 1996), OFC activation is not required for making choices between the foods. This is not surprising because the monkeys continually see the food during the satiation period, allowing for direct associative conditioning between the stimulus properties of the food and the reduced value (i.e., new associative learning). However, we also found that when a devalued food was presented in association with an object, i.e., after the object was displaced, the OFC-inactivated subjects retrieved the devalued food, and three of four subjects ingested it, just as had been observed after OFC lesions (Machado and Bachevalier, 2007a). This indicates that the devalued food still retains some incentive value compared with no food (i.e., it is better than nothing), because following the selection of an object, only one food is accessible.
Our finding that OFC is critical for expressing the devaluation effect specifically during the probe choice test is consistent with a previous study in rats, in which impairment was found when OFC lesions were made before the probe session, but after the rats had formed light–food associations and the food was devalued by pairing it with a toxin (Pickens et al., 2005). Given the multiple differences between the rat and the monkey tasks, these findings indicate that the importance of OFC extends across species and devaluation procedures with different task demands. This is further supported by in vivo electrophysiological recordings in rats and monkeys showing that both the values of reward-predicting stimuli (Tremblay and Schultz, 2000; Wallis and Miller, 2003; Padoa-Schioppa and Assad, 2006; Kennerley and Wallis, 2009; Padoa-Schioppa, 2009) and the representations of expected outcomes (Feierstein et al., 2006; Roesch et al., 2006; Furuyashiki et al., 2008; Tsujimoto et al., 2009) are maintained in OFC. Thus, OFC activity maximizes behavioral flexibility and adaptability by maintaining a central representation of stimuli that predict rewards so that these internal representations can rapidly be assigned new values based on changes in the values of the rewards they predict, without the need for new associative learning.
Footnotes
The work was supported by NIDA Grants F31DA026705 and T32DA007291, and NINDS Grant T32NS04123 (E.A.W.); Howard Hughes Medical Institute Undergraduate Research Program (J.T.D.); Georgetown University Medical Center Institutional Research Grant (L.M.), and, in part, by NIMH Grant R21MH078277 (L.M.). We thank Carrie Silver, Angela Holmes, Lauren Pepe, Jessica Ihne, Taylor Leath, and Melanie Burgos for technical assistance and Patrick Forcelli and Richard Saunders for comments on this manuscript.
The authors report no financial conflicts of interest.
- Correspondence should be addressed to Dr. Ludise Malkova, Department of Pharmacology, Georgetown University Medical Center, 3900 Reservoir Road NW, Washington, DC 20007. malkoval{at}georgetown.edu
