Abstract
Clinical evidence indicates that damage to ventromedial prefrontal cortex disrupts goal-directed actions that are guided by motivational and emotional factors. As a consequence, patients with such damage characteristically engage in maladaptive behaviors. Other research has shown that neurons in the corresponding orbital region of prefrontal cortex in laboratory animals encode information regarding the incentive properties of goals or expected events. The present study investigates the effect of neurotoxic orbitofrontal cortex (OFC) lesions in the rat on responses that are normally influenced by associations between a conditioned stimulus (CS) and the incentive value of reinforcement. Rats were first trained to associate a visual CS with delivery of food pellets to a food cup. As a consequence of learning, rats approached the food cup during the CS in anticipation of reinforcement. In a second training phase, injection of LiCl followed consumption of the food unconditioned stimulus (US) in the home cage, a procedure used to alter the incentive value of the US. Subsequently, rats were returned to the conditioning chamber, and their responding to the CS in the absence of the food US was tested. Lesions of OFC did not affect either the initial acquisition of a conditioned response to the light CS in the first training phase or taste aversion learning in the second training phase. In the test for devaluation, however, OFC rats exhibited no change in conditioned responding to the visual CS. This outcome contrasts with the behavior of control rats; after devaluation of the US a significant decrease occurred in approach to the food cup during presentation of the CS. The results reveal an inability of a cue to access representational information about the incentive value of associated reinforcement after OFC damage.
Goal expectancy is an important source of guidance in adaptive behavior. Recent findings suggest that a deficit in this function contributes to the clinical presentation of patients with damage to ventromedial prefrontal cortex. Clinical accounts highlight poor judgment and socially inappropriate behavior that can occur despite the patients’ knowledge about social customs and the likely outcome of their actions (Damasio, 1994; Rolls et al., 1994; Bechara et al., 1997). A disturbance in motivational and emotional factors that normally control behavior is often cited to account for this profile, with patients variously described in different contexts as passive or impulsive, distractible or perserverative. Of particular interest is the notion that such patients suffer from goal neglect, a concept that can encompass the seemingly contradictory features of their behavior, e.g., both passive and impulsive (Duncan et al., 1996). As in other forms of neglect, patients are not incapable of awareness but fail to use available information to guide their actions. The relevant information in goal neglect consists of the incentive (or disincentive) properties of expected outcomes. This view of humans with prefrontal damage is consistent with other evidence from recent studies examining the encoding properties of prefrontal neurons in laboratory animals.
Neurons in the orbital region of the ventral prefrontal cortex encode the motivational significance of cues and the incentive value of expected outcomes (Thorpe et al., 1983; Schoenbaum and Eichenbaum, 1995; Critchley and Rolls, 1996; Rolls et al., 1996; Schoenbaum et al., 1998, 1999; Lipton et al., 1999). Schoenbaum et al. (1998, 1999) found that a substantial proportion of cells recorded in the orbital region of the rat exhibited such properties during performance of an odor-guided task. In particular, after rats had sampled an informative odor cue, cells fired differentially depending on whether the outcome was positive (sucrose) or negative (quinine) (Schoenbaum et al., 1998). Evidence in that study indicated that this encoding was prospectively related to the incentive value of the impending event.
Associations in which cues, responses, or contextual information are linked to the incentive properties of outcomes provide an important basis for expectancy in goal-directed behavior. To model this aspect of associative learning, we used a task in which normal performance depends on the ability of a conditioned stimulus (CS) to gain access to the motivational properties of an upcoming unconditioned stimulus (US). Our study used a procedure in which rats were first trained in standard Pavlovian conditioning, using a light CS paired with food delivery. After conditioned responses were established to the CS, rats received the original US (food) in another setting in which it was paired with an aversive event (injection of LiCl). This second phase of “US devaluation” causes normal rats to subsequently decrease their conditioned responses on initial re-exposure to the original CS. Although the CS is absent during devaluation, its earlier association with the US provides a basis for anticipating that event. If orbitofrontal cortex (OFC) function is important for guiding behavior through the use of such associations, then rats with OFC damage should be impaired in the CS test for devaluation.
MATERIALS AND METHODS
Subjects and surgical methods. Forty-three male Long–Evans rats (Charles River Laboratories, Wilmington, MA) were 300–350 gm at the time of surgery. Anesthesia was induced by isoflurane (Isovet; Mallinckrodt, Mundelein, IL) inhalation. After induction of anesthesia, the rat was positioned in a Kopf stereotaxic apparatus, and standard methods were used to make bilateral microinjections of NMDA (Sigma, St. Louis, MO) at the following coordinates according to the Paxinos and Watson (1986) atlas: 4.0 mm anterior to bregma at both 2.2 and 3.7 mm lateral to midline and 4.2 mm ventral from the skull surface. A second set of bilateral injections was made at 3.0 mm anterior to bregma at both 3.2 and 4.2 mm lateral to the midline and 5.2 mm ventral to the skull surface. At each site, NMDA (20 mg/ml) or the Krebs’-Ringer’s solution phosphate vehicle was delivered in a 0.1 μl vol over a 1 min interval, and the injector was left in place for an additional 3 min. The incision was then sutured, and rats were monitored post-operatively on a daily basis. One week after surgery, rats were gradually reduced to 85% of ad libitum weights by limiting access to food; water was always available. Rats were weighed and fed daily to maintain their 85% weights for the remainder of the experiment. Behavioral testing was conducted during the light portion of the light/dark cycle between 7:00 A.M. and 2:00 P.M.
Apparatus and behavior. The testing apparatus consisted of four individual chambers (Coulburn Instruments), each 27.9 × 25.4 × 30.5 cm. The food cup was recessed in the center of one end wall 2 cm above the floor; a 4 W normally off light, which was the source of the visual CS, was located 20 cm above the recessed food cup. A 25 W red bulb placed 2.1 m from the chambers provided continuous dim background illumination. A low-light television camera was placed 2.0 m from the experimental chambers. Videocassette recorders were programmed to record behaviors that occurred during the 10 sec intervals before, during, and after CS presentation.
The behavioral protocol was identical to that described in detail elsewhere (Hatfield et al., 1996). Rats were first trained to eat from the food cups. Ten deliveries of two 45 mg food pellets (which served as the US) were given at random times within a single 64 min session. Rats were then trained with Pavlovian pairings of a light CS (10 sec duration) that terminated with the delivery of two food pellets, which served as the US. Rats received five trials in each of eight daily sessions. A random variable intertrial interval that averaged 8 min was used. After the completion of this conditioning phase, the food pellet US was devalued for half of the rats in each group by pairing with injections of the toxin lithium chloride (LiCl); the remaining rats in each group received unpaired presentations of the food pellet US and LiCl. This procedure was conducted by placing 100 food pellets (450 mg) in a glass dish in the animal’s home cage for 10 min, then giving an intraperitoneal injection of 0.3 m LiCl either immediately or 6 hr later. The devaluation training trials were each separated by a rest day. Finally, the devaluation test was conducted in the original training apparatus. In a single session, each rat received five presentations of the original CS without delivery of the food pellet US.
The measure of learning in appetitive conditioning and in the test for devaluation was food cup behavior. Food cup behavior includes standing motionless in front of the food cup, with the nose or head inserted within the recessed area, and head-jerk behavior (short, rapid horizontal, and/or vertical movements of the head). Behavioral observations were made for each rat at 1.25 sec intervals from videotapes, and paced by auditory signals recorded on the tapes. At each observation, the observer recorded a single behavior. The index of food cup behavior used was percentage total behavior, obtained by dividing the frequency of that behavior in any observation interval by the total number of observations made in that interval. Note that because the number of observations was constant within each observation interval, this measure is an absolute frequency measure, not a relative one. Two observers scored the behavioral data reported in each experiment; the observers were not aware of the rats’ lesion conditions when the data were scored. Food cup behavior is reported for the final 5 sec of the 10 sec CS, before US delivery. A measure of food devaluation in the conditioned taste aversion procedure was the amount of food consumed in that phase of training.
Histology. After completion of behavioral testing, the rats were deeply anesthetized with Nembutal (100 mg/kg) and were perfused transcardially with 0.9% saline followed by 10% formal saline. Brains were removed and stored in 10% formalin for 1 week. The brains were then sectioned (50 μm) in a cryostat, mounted on slides, and Nissl-stained. Coronal sections were taken through the prefrontal region. Histological examination was performed with the aid of theSwanson (1992) atlas using an Olympus (Tokyo, Japan) BH-2 microscope.
Data analysis. In the statistical analyses of the food cup behavior, we used two-tailed, nonparametric statistics. Food consumption data during devaluation training were analyzed using two-way ANOVA. We adopted the 0.05 level of significance.
RESULTS
Bilateral lesions were accurately placed in 24 of the 26 rats that received microinjections of excitotoxin. These rats had a marked loss of neurons in OFC, including the medial, ventrolateral, and lateral orbital regions and both dorsal and ventral agranular insular cortex. Damage was confined to portions of OFC rostral to the genu of the corpus callosum to avoid gustatory input to posterior agranular insular cortex (Saper, 1982; Krushel and Van Der Kooy, 1988). On average, lesions encompassed 75% of OFC bilaterally, ranging from 40 to 100%. A photomicrograph of a lesion is shown in Figure1 along with drawings depicting the range of lesion size included in the experimental group. The largest lesions (Fig. 1B) included some damage to claustrum and frontal and parietal cortex dorsal to OFC (Fig.1A,B). Damage outside of OFC was typically unilateral, and rats with additional cortical damage did not differ behaviorally from those that had no discernible loss of cells in those regions.
Photomicrographs and schematics showing the region of OFC damage. A, Photomicrographs showing a coronal section (anteroposterior, +2.7) from a control brain (left panel) and a lesioned brain (right panel). Note the neuronal loss, accompanied by gliosis, in the ventrolateral and lateral orbital regions and in agranular insular cortex in the lesioned brain. The region of cell loss ends at the medial orbital area and extends dorsally somewhat into parietal cortex. B, Drawings show the approximate extent and range of the lesions that were included in the experimental group. The boundaries of the minimal lesion (black fill) and the maximal lesion (diagonal fill) are shown to indicate of the range of damage. The approximate size of the lesion (crossed fill) in a subject that is representative of average lesion size is also shown [drawings adapted from Swanson (1992)]. Damage was confined to portions of OFC rostral to the genu of the corpus callosum to avoid gustatory input to posterior agranular insular cortex (Saper, 1982; Krushel and Van Der Kooy, 1988).
The acquisition of food cup responses during the initial conditioning phase is shown in Figure 2. Irrespective of lesion, all rats learned to approach the food cup during the CS that signaled delivery of the food pellet before the devaluation procedure, as shown in the left panel. Indeed, statistical analysis (Kruskal–Wallis) revealed no differences among the groups during this phase of the experiment.
Data are shown for each phase of the behavioral experiment. Squares represent control groups, and circles represent the lesioned groups. All groups acquired conditioned food cup responses in phase 1, as shown in the panel on the left. In phase 2, the groups represented byfilled symbols received unpaired food and LiCl;open symbols represent groups that had paired presentations of these events. As shown in the presentation of the phase 3 devaluation test data, the control group for which food was devalued (open bar on the left) reduced conditioned responding relative to the unpaired control group (solid bar). In contrast, the lesioned groups exhibited conditioned responses that did not differ as a function of devaluation. See Materials and Methods for details of statistical analyses.
As a result of US–LiCl pairings, both control and OFC lesion groups formed an aversion to food pellets and would not consume them during consumption tests (Fig. 2, middle panel). Note that control rats and rats with OFC lesions that received unpaired food–LiCl continued to consume food pellets, as expected. ANOVAs for each training condition, however, indicated no significant differences between the control and lesioned groups (F(1,23) = 2.05; p > 0.1 for paired and F(1,20) = 0.015;p > 0.1 for unpaired).
The data of primary interest were obtained during the subsequent test of responding to the light CS in the absence of the food US. The control group that underwent devaluation training showed a spontaneous drop in CRs relative to the control group for which the US was not devalued (control groups, U = 22; p < 0.01). No effect of devaluation, however, was apparent in comparison of the OFC-lesioned groups (U = 62.5; p = 0.60). In addition, the CRs for the OFC devalued group did not differ from the unpaired control group (U = 59;p = 0.94) but differed significantly from the devalued control group (U = 38; p < 0.05).
DISCUSSION
Responding to a CS can be sensitive to post-training alterations in the value of a US (Holland and Straub, 1979, Colwill, 1993). This phenomenon is consistent with the view that performance of conditioned responses (CRs) is mediated, at least in part, by access to properties of an associated US; when the US is not present, the CS activates some representation of the reinforcement. In the current study the behavior of intact rats was influenced by the access of the CS to the altered value of the US, whereas rats with OFC lesions failed to use such information to guide their responses. This deficit cannot be attributed to a general inability to inhibit food-related behavior; when food was presented to OFC rats during devaluation training and after the devaluation test session, they suppressed consumption of the pellets. An impairment in the devaluation test would be anticipated if OFC were essential for establishing or using associations that allow cues to gain access to representations of reinforcement.
A commonly observed impairment after OFC lesions is an inability to extinguish or reverse associations that have been established to reinforcing events (Jones and Mishkin, 1972; Baylis and Gaffan, 1991;Rolls et al., 1994; Rolls, 1996). For example, although initial learning often emerges without difficulty, when reinforcement no longer occurs in extinction, or contingencies are changed in reversal training, laboratory animals and patients with such damage continue to respond as though no change had occurred. The current findings are consistent with those earlier studies, indicating that the behavior of rats with OFC damage is relatively unaffected by changes in the value of reinforcement.
It is interesting that human patients with ventromedial prefrontal damage can often accurately describe task contingencies, but this knowledge is not sufficient to guide behavior. For example, Rolls et al. (1994) reports that such patients could verbally describe how task contingencies had changed during either extinction or reversal procedures, but nonetheless were not able to alter their behavior appropriately in a simple visual discrimination task. Similarly, in a more complex gambling task in which subjects could choose from different decks of cards to incur rewards and penalties, patients with ventral prefrontal damage would continue to choose from decks that they correctly identified as disadvantageous (Bechara et al., 1997). A related finding in that study was that when patients made those choices they failed to exhibit anticipatory autonomic responses that were characteristic of the control subjects. Thus, a lack of motivational/emotional response appropriate to the expected outcome might account for the mismatch between knowledge and the patient’s actions. That observation could reflect either an inability to generate such a response or a lack of access to the appropriate representation of incentive value that evokes autonomic reactions. As indicated by recent electrophysiological recordings in OFC, information about the incentive value of anticipated reinforcers is normally encoded by neurons in this region in rats (Schoenbaum et al., 1998), providing a substrate that could be lacking after damage to this system.
An impairment in the ability of cues (or responses) to gain access to the incentive value of an associated reinforcer would account for impairment in the devaluation paradigm. If that is the case, then how does initial learning proceed in animals or humans with prefrontal damage? Recall that impairments in extinction and reversal learning are observed, although initial learning is often unimpaired in those tasks. Similarly, rats in the current study acquired conditioned food cup responding in the first training phase; they also acquired a normal conditioned taste aversion. Much evidence indicates that associative learning can invoke multiple representations of the CS–US relation. Thus, the associative representation that depends on OFC may occur in parallel with other associative functions that provide a basis for initial learning. For example, a CS that is associated with a rewarding US can itself acquire incentive value. This associative function can be independently demonstrated by the ability of the CS to serve as the US in support of new learning, i.e., instrumental learning for secondary reinforcement. It is interesting that learning based on the acquired reinforcer properties of a CS, such as second order conditioned responses, is relatively insensitive to devaluation (Holland and Rescorla, 1975). Thus, learning that is acquired on such a basis might be relatively immune to changes in the status of the original US, as observed in extinction, reversal training, and in the setting of US devaluation.
A critical dependence on OFC for associative information that links cues to the value of reinforcers or outcomes provides a potential framework for both the experimental observations in laboratory animals and the consequences of damage to this region in humans. By this view, an associative deficit may underlie an inability to effectively use the motivational guidance provided by an expected outcome, thereby providing an information processing account for goal neglect. In serving such a function, it is also evident that OFC is an important component of circuitry that includes other structures involved in motivational processes and associative learning. Of particular importance are direct interconnections between OFC and the basolateral amygdala complex (Krettek and Price, 1977; Kolb, 1984; Price et al., 1987; McDonald, 1991). The report providing evidence for OFC neural encoding of anticipated outcomes (rewarding or aversive) also indicated that a substantial population of neurons in basolateral amygdala (ABL) had those same correlates (Schoenbaum et al., 1998). Approximately 36% of cells in ABL (44 of 121 neurons) had differential activity in an interval before the delivery of the rewarding or aversive outcome similar to the correlates that characterized 22% of cells recorded in OFC (74 of 328 neurons). As might be predicted from these data, ABL also appears to be important for the normal ability to gain access to US representations (Hatfield et al., 1996).
Using the same devaluation procedures used in the current study, a comparable pattern of results was earlier obtained after lesions of the basolateral amygdala (Hatfield et al., 1996). Similar to the current findings with OFC-lesioned rats, bilateral neurotoxic lesions of ABL did not alter acquisition of the initial food cup response and did not affect taste aversion learning in the second devaluation phase. Lesions of ABL, like those of OFC, abolished the effect of devaluation on subsequent responding in the presence of the original CS. Recent evidence has also shown a lack of US devaluation in guiding the behavior of monkeys in a visual discrimination task after neurotoxic amygdala lesions (Malkova et al., 1997). At the same time, lesions of the amygdala central nucleus (Hatfield et al., 1996) and of medial temporal lobe structures, either the hippocampus or entorhinal/perirhinal cortex (Morell, 1997), do not alter the behavior of rats in this setting; rats with such lesions, like normal rats, spontaneously decrease CRs in the presence of cues that signal the devalued US.
These findings implicate connections between OFC and ABL in processes that depend on stimulus–reinforcer associations. Much other evidence supports a role for ABL in such learning processes; it is an important site for convergence of information needed in the acquisition of associations. Interconnections with OFC may then be critical for governing the use of that information to guide goal-directed behavior. Further research is needed to understand interactions between these subcortical and cortical systems that are important in motivational and emotional learning processes.
Footnotes
This work was supported by National Institute of Mental Health Grant RO1 MH53667, Research Scientist Award KO5-MH01149 to M.G., and Mentored Clinical Scientist Award K08-AG00882-01 from the National Institute of Aging to G.S.
Correspondence should be addressed to Dr. Michela Gallagher, Johns Hopkins University, Department of Psychology, 3400 North Charles Street, Ames Hall, Baltimore, MD 21218.
