Abstract
Extinction is a unique learning process that requires the alteration of stimulus-response associations such that the organism ceases to respond to a previously rewarded stimulus. Extinction is mostly studied with fear conditioning and is impaired by lesions of the prefrontal cortex as well as by blockade of NMDA receptors in the amygdala. Because previous tasks could not clearly disambiguate extinction from behavioral disinhibition, the underlying process was difficult to define. In this study, we examined the possible role of NMDA receptors and the pigeon “prefrontal cortex,” the neostriatum caudolaterale (NCL), for extinction of appetitive instrumental conditioning. We used a new design that discerns extinction from behavioral disinhibition. Our results demonstrate that NCL lesions cause deficits neither in extinction learning nor in extinction recall. However, blockade of NMDA receptors in the pigeon NCL by dl-AP-5 drastically impairs extinction learning without producing behavioral disinhibition or deficits in extinction recall. We suggest that NMDA receptors in the NCL contribute to the establishment of a learning process that selectively signals the change in value of the instrumental stimulus. Although NCL plays a key role for extinction learning, other structures can subsume similar functions after postlesional regeneration.
Introduction
Learning pertains not only to the acquisition of new associations, but also to rearrangements of existing ones. Altering previously acquired associations usually involves extinction learning. Results from lesion experiments in mammals point to the involvement of the prefrontal cortex (PFC) and amygdala in the extinction of conditioned responses: in macaques, the extinction of appetitive instrumental conditioning is retarded by frontal cortex lesions (Butter, 1969; Jones and Mishkin, 1972). In rats, the maturation of PFC regions is necessary for instrumental extinction learning (Nair et al., 2001). Extinction of classically conditioned fear responses is sometimes found to be impaired (Morgan et al., 1993; Quirk et al., 1998) or unimpaired (Gewirtz et al., 1997; Morgan & LeDoux, 1999; Quirk et al., 2000) by PFC lesions in rats, but in any case seems to require amygdalar processes. The role of the amygdala in extinction was investigated mainly with regard to NMDA receptors, using local or systemic injections of a NMDA antagonist in classical fear conditioning paradigms. Local NMDA receptor blockade in the amygdala blocks extinction learning in rats (Falls et al., 1992; Davis, 2002; Walker & Davis, 2002). The systemic injection of a NMDA receptor antagonist can have the same effect (Baker & Azorlosa, 1996) or specifically impairs only extinction recall (Santini et al., 2001).
To our knowledge, the specific role of NMDA receptors in PFC for extinction learning has not been examined, and our question was whether the results of these fear-conditioning experiments are specific for aversive stimulation or whether NMDA-dependent processes are also implicated in the extinction of appetitive instrumental conditioning. However, in a fear extinction paradigm, two variables are confounded: resumption of a previously suppressed instrumental behavior after fear extinction may be attributable to mere behavioral disinhibition and not necessarily to associative changes regarding the conditioned stimulus. Therefore, impaired extinction of instrumental responses in a non-aversive extinction paradigm might as well be attributable either to a general behavioral disinhibition or to deficits in the acquisition of extinction learning. Therefore, we devised a special extinction task that enabled us to measure and analyze each of these two aspects independently.
Our animal model is the “prefrontal cortex” of the pigeon: the neostriatum caudolaterale (NCL). The NCL is a forebrain area in birds that is considered a functional equivalent to mammalian PFC because of multiple converging evidence from behavioral (Mogensen and Divac, 1982, 1993; Gagliardo et al., 1996, 1997; Güntürkün, 1997; Diekamp et al., 2001, 2002), electrophysiological (Kalt et al., 1999), and neuroanatomical (Divac et al., 1994; Wynne & Güntürkün, 1995; Leutgeb et al., 1996; Metzger et al., 1998; Kröner & Güntürkün, 1999) data.
In a previous study, local injections of the NMDA receptor antagonist dl-AP-5 into the NCL during color reversal learning had revealed a perseveration on the old S+ combined with an unimpaired first-time acquisition of the new S+ (Lissek et al., 2002). These data suggested that not acquisition of the new instrumental stimulus, but the extinction of the old, previously rewarded response, was adversely affected by the NMDA receptor blockade. However, this study was also unable to clarify if the deficits were attributable to impaired acquisition of extinction learning or to behavioral disinhibition. Therefore, the aim of the present study was to investigate the role of NMDA receptors in the NCL during the extinction of an instrumental response, using a paradigm enabling the dissociation of these two processes.
Materials and Methods
Subjects. Subjects were 24 unsexed and experimentally naive pigeons (Columba livia), obtained from local breeders. All animals were individually caged in a temperature- and humidity-controlled room on a 12 hr light/dark schedule. During experiments, they were maintained at 80% of their free-feeding weight and received water and grit ad libitum.
Apparatus. A conventional Skinner box (36 cm long × 34 cm high × 36 cm wide) was used for training and experiments. The Skinner box was equipped with one pecking key and a solenoid-operated food hopper in the back wall and was computer-controlled by means of a digital input/output board. On the pecking key (2.5 cm in diameter and situated in the center of the wall), white light was displayed during autoshaping, and red or green light was displayed for training and experimental sessions of the extinction task. The Skinner box was illuminated by a house light.
Pretraining and color discrimination task. After an autoshaping procedure, in which pigeons acquired the association between responding to a single pecking key illuminated by white light and subsequent food reward, pigeons were trained in a color discrimination task, which we designed to separate the effects of disinhibition and extinction in the experimental sessions: In each training trial, pigeons were confronted with a single pecking key displaying first the color red for 30 sec, then the color green for 5 sec. Pecking on the green key yielded 2 sec access to the food tray after a delay of 1 sec, whereas pecking on the red key had no effect at all. So there was one stimulus for which responding was never rewarded (red), and a second stimulus for which responding was always rewarded (green). The rewarded green stimulus was later to become the extinction stimulus, whereas the red light served as a measure for behavioral disinhibition during extinction. A training session lasted 80 trials; the learning criterion was set to 80% correct responses in each of three subsequent sessions. After reaching a learning criterion, pigeons were randomly assigned to three treatment groups: a NMDA antagonist group (AP-5, n = 10), an NCL lesion group (LES, n = 6), and a saline control group (SAL, n = 8).
Surgery. For surgery, pigeons were anesthetized with ketaminerompun (40 mg/kg and 8 mg/kg, respectively, i.m.).
Implantation of guide cannulas: AP-5 and SAL group. Aiming at the NCL, two stainless steel cannulas per hemisphere were vertically inserted under stereotaxical guidance (Karten and Hodos, 1967) to reach the following coordinates: anterior (A), 5.25; lateral (L), 5.00; and A 5.25; L 7.50. Cannulas were inserted to 1 mm below the brain surface and were secured with dental acrylic.
Electrolytic lesions: LES group. Tungsten electrodes (0.2 mm in diameter) insulated to within 0.5 mm of the tip were lowered to the following coordinates (Karten and Hodos, 1967): A 4.00, L 5.00 and 6.50; A 5.00, L 4.50, 6.00, and 7.50; A 6.00, L 4.50, 6.00, and 7.50; and A 7.00, L 5.00, 6.50, and 7.75. Each lesion was made by lowering the tip of the electrode 1.5 mm below the brain surface and passing 25 mA of anodal current (positive electrode in brain) for 10 sec.
After 5-6 d of recovery for the AP-5 and SAL groups and 7-10 d for the LES group, pigeons were tested for retention of the color-discrimination task (criterion: a minimum of 80% correct responses in the retention session).
Extinction procedure. Three extinction sessions were conducted: one initial and two recall sessions. The last ones were used to control the memory for the previously acquired extinction. These three sessions were performed on three successive days, with each session lasting 220 trials. The trials were identical to the color-discrimination task, with the one exception that responding to the green key did not result in reinforcement or in any other consequences.
Immediately before each of the extinction sessions, subjects belonging to the AP-5 group received bilateral infusions of the competitive NMDA receptor antagonist dl-AP-5 locally into the NCL. AP-5 was dissolved in saline solution (total volume, 2 μl, containing 10 μg of dl-AP-5; 0.5 μl, i.e., 2.5 μg of dl-AP-5 per cannula). Infusions were made through interior cannulas protruding 1 mm from the tip of the guide cannulas into the brain tissue. We used a microinfusion pump equipped with two 1 μl Hamilton (Reno, NV) syringes to deliver the volume at a flow rate of 0.2 μl/min. Afterward, the infusion cannulas remained in place for another 2 min to allow for diffusion of the infused volume. To infuse through all four cannulas, we performed this procedure twice. Subjects belonging to the SAL group were submitted to the same procedure, receiving vehicle (saline solution) only. Immediately after the infusion procedure, which took ∼12-15 min, the pigeons had to perform the task. All pigeons of these two groups received three infusions of either AP-5 or vehicle during this study. Subjects belonging to the LES group did not receive any additional treatment before being submitted to the extinction sessions.
Histology. To enable reconstruction of the locations of the guide cannulas as well as of the lesion volume, we perfused the pigeons intracardially with 0.9% (w/v) saline (40°C) and a 4% (w/v) paraformaldehyde solution (4°C). The brains were removed, postfixed, and cut into 40 μm frontal slices on a freezing microtome. After staining the slices with cresyl violet, the positions of the cannula tips as well as the lesions were reconstructed at intervals of 500 μm from A 4.00 to A 9.00 and transferred onto standard sections from the pigeon brain atlas (Karten and Hodos, 1967).
Statistical analyses. During training and extinction sessions, we collected the following behavioral data: responses to the green key and responses to the red key. Responses to the green key (i.e., extinction performance) of the three groups were compared by means of ANOVA, followed by a Bonferroni post hoc test. From responses to the green and red keys during training compared with extinction we calculated pre/post response ratios for both keys and each subject. Those ratios were again compared by means of ANOVA followed by a Bonferroni post hoc test, if applicable.
Results
Histology
All cannula injection sites were located within the NCL. Eighty percent of the sites were located within a range of ±0.5 mm from A 5.25. The remaining 20% were situated in a range of ±1 mm from A 5.25 (Fig. 1A). Bilateral lesions were located within the NCL, in a range from A 4.5 up to A 8.5. Figure 1B,C shows the lesioned brain areas for minimum lesions [i.e., areas where lesions in at least two subjects overlap (B)] and maximum lesions [i.e., areas covered by lesions from all subjects together (C)].
Injection sites and lesion locations. A, Schematic frontal sections of the pigeon brain showing the injection sites for AP-5 and saline solution. Dots represent the lower tips of the cannulas; numbers represent the distance (in millimeters) anterior to the center of the ear bars; boldface indicates the frontal plane level at which cannulas were aimed. The NCL area according to Waldmann and Güntürkün (1993) is depicted in light gray. B, Schematic frontal sections of the pigeon brain showing in black the minimum lesion volumes (i.e., areas where lesions in at least two subjects overlap). C, Schematic frontal sections of the pigeon brain showing in black the maximum lesion volumes (i.e., areas covered by lesions from all subjects together). This figure was adapted from graphs in Stereotaxic Atlas of the Brain of the Pigeon (Karten and Hodos, 1967).
Retention session
In the retention session, all animals reached the criterion of a minimum of 80% correct responses and participated in the following experimental sessions.
Extinction learning (session 1)
Responses to the green key
Decreased responding to the green key was the measure for successful extinction: responding to this formerly rewarded stimulus is supposed to cease after a few trials of non-reinforcement. Both the SAL and LES groups showed rapid extinction of the instrumental response with a mean of 37.16 (LES) responses and 37.12 (SAL) responses during the complete session lasting 220 trials, whereas the AP-5 group showed a mean of 93.3 responses.
The univariate ANOVA showed a highly significant effect of group (F(2) = 11.771; p < 0.001) (Fig. 2A). A Bonferroni post hoc test revealed a significant difference between the AP-5 group and both the SAL group (p < .001) and the LES group (p < 0.01). The SAL and LES groups did not differ in their responding to the green key during extinction.
Responses during extinction. A, Mean ± SEM responses to the green key during the extinction session made by AP-5-treated (solid columns), lesioned (hatched columns), and vehicle-treated (open columns) pigeons. B, Means ± SEM values for the pre/post response ratio to the green (solid bars) and red (hatched bars) keys, respectively.
Responses to the red key: comparison with presurgery level
Increased responding to the red key (compared with the training level) was the measure for disinhibition, for which we calculated a pre/post pecking ratio. The “pre” value was constituted by the sum of all pecks onto the red key during the last five training sessions (5 × 80 = 400 trials), the post value was constituted by the sum of all pecks onto the red key during the extinction session of 220 trials. These values were recalculated to responses per 100 trials, and the ratio was calculated according to the formula: post - pre/post + pre. No change in pecking behavior would yield a result of 0, increased responses in the extinction session results in positive values, decreased responding during extinction in negative values. All three groups showed only a slight reduction of pecking behavior on the red key (Fig. 2B). ANOVA did not reveal significant differences in pre/post ratios between groups (F(2) = 0.730; p not significant). This result shows that there was no general behavioral disinhibition in any of the groups that could have caused extinction deficits.
Pre/post Ratio for the Green Key
According to the same formula, we calculated a pre/post ratio for the green key, which gave a significant group effect (F(2) = 9.614; p < 0.001). Although the SAL and LES groups both showed a considerable reduction in responding to the green key, AP-5 animals exhibited the smallest reduction, reflecting their perseverative responding to the green key (Fig. 2B).
A Bonferroni post hoc test showed a significant difference only between the AP-5 and LES groups (p < 0.01) and the AP-5 and SAL groups (p < 0.01), but not between the SAL and LES groups.
Extinction curves
Figure 3 shows combined extinction curves for the AP-5, saline, and lesion groups. We calculated these extinction curves by pooling response data from 10 trials each during the extinction session for each group, resulting in 22 data points in time for the complete session with a maximum value of 10 and a minimum of 0 responses during 10 trials. The figure illustrates the differences in the course of extinction between the SAL and LES groups on the one hand and the AP-5 group on the other hand. The individual extinction curves, from which we derived the pooled data (data not shown), demonstrate that all subjects started their responding at a very high level of 8 -10 responses in 10 trials. But only in the SAL and LES groups did the continuous experience of non-reinforcement result in a fast drop in their responding to 1-2 responses in 10 trials after ∼40 trials, whereas the AP-5 group showed this decrease only after ∼160 trials.
Extinction curves from all three groups. Mean response values to the green key per bin of 10 trials each.
Although the combined curve may give the impression that the AP-5 group started at a lower response level than the remaining two groups, this is because some of the subjects started responding only after some trials had passed unattended. This effect, although occurring in all groups, was most prominent in the AP-5 group.
Pecking behavior onto the red key
Figure 4 shows the pecking behavior onto the red key over the extinction session separately for all three groups; for comparison purposes the pecking behavior onto the green key is also plotted. Response data from 10 trials were pooled, every individual peck during these 10 trials was counted, resulting in a total of 22 data points in time for each color for the complete session. An univariate ANOVA showed that the pecking behavior onto the red key did not differ between groups (F(2) = 0.708; p not significant). Furthermore, there was no systematic relation between lesion extent and red key pecking behavior.
Pecking behavior. Mean pecking responses onto the red and the green key during the extinction session in bins of 10 trials each, shown separately for each experimental group: A, AP-5 group. B, SAL group. C, LES group. Filled circles, Pecks onto the green key; open triangles, pecks onto the red key.
Recall of extinction (sessions 2 and 3)
Responses to the green key
An ANOVA with repeated measures for the extinction recall sessions 2 and 3 revealed no significant differences between the performance of the three groups (F(2) = 0.567; p not significant). The effect of session was also not significant (F(1) = 3.112; p = 0.092, although there was a tendency toward fewer responses in session 3 compared with session 2. This trend was particularly clear in the LES group. In general, NMDA receptor blockade did not impair the recall of a previously acquired extinction. This implies that the delayed acquisition in extinction session 1 did not impair consolidation (Fig. 5).
Responses during recall of extinction. Mean ± SEM responses of the three groups to the green key during the recall of extinction sessions. Session 2, Solid bars; session 3, hatched bars.
Discussion
The main results of this study are: (1) Blocking of NMDA receptors in NCL results in severe deficits in extinction of an instrumental response, whereas recall of a learned extinction remains unimpaired. (2) The extinction learning deficit in AP-5 subjects is not caused by a general behavioral disinhibition. (3) On the other hand, lesioning the NCL causes deficits neither in extinction learning nor in extinction recall.
Dissociating disinhibition and deficits of extinction
The present results show that the extinction of an instrumental response was impaired after NMDA receptor blockade because AP-5 subjects continued to respond to the previously rewarded stimulus despite the fact that no reinforcement was obtained anymore. This perseverative behavior has been observed in experimental animals after PFC lesions (Butter, 1969, Jones and Mishkin, 1972, Dias et al., 1996; Collins et al., 1998), and after dopamine D1 and NMDA receptor blockade in the pigeon NCL (Diekamp et al., 2002; Lissek et al., 2002). It is also a typical symptom of prefrontal dysfunctions in humans (Milner, 1964; Fuster, 1989; Vilki, 1989; Rolls et al., 1994). Although perseveration is often considered to result from a lack of behavioral inhibition (Rolls et al., 1994; Hauser, 1999), in most tasks a disambiguation of disinhibition and impaired extinction learning is not possible. However, this was permitted by the experimental design of our study.
Therefore, our results enable us to show that the extinction learning deficit in AP-5 subjects is not caused by behavioral disinhibition: although responding to the previously rewarded stimulus (green) was continued, there was even a slight reduction in responding to the never-rewarded stimulus (red). The collapsed extinction curves of the three groups (Fig. 3) clearly illustrate the differences: saline and lesion groups reduced their responding to the no-longer-rewarded stimulus after only a few trials, whereas AP-5 animals continued to respond at a very high level for much longer, until finally they too decreased their responding. In addition, the differential responses to the green and the red stimulus reveal that the blockade of NMDA receptors did not impair long-term memory retrieval of the properties of the red key. Instead, the impairment was selective for uncoupling either the association between the formerly rewarded stimulus and the learned response or for altering a representation of the incentive value of the green key (Schoenbaum et al., 2002). Indeed, single unit properties of the NCL reveal response patterns tuned to the representation of relevant stimuli and responses as well as to the evaluation and anticipation of reward (Kalt et al., 1999; Diekamp et al., 2002).
This is consistent with studies showing that the establishment of long-term memory for extinction involves an activation of NMDA receptors (Santini et al., 2001). A recent study demonstrated that the constitution of extinction memory results in cellular prefrontal activations correlated with the recall of extinction (Milad and Quirk, 2002). This result strongly argues for the notion that extinction is not caused by the erasure of an association, but instead it forms a new engram. Milad and Quirk (2002) assume that the neurons constituting this memory trace might indirectly inhibit amygdalar processes that modulate freezing behavior. Our results possibly extend this interpretation because they make it likely that a selective NMDA-dependent acquisition of extinction is also able to alter an appetitive instrumental paradigm, in which extinction is accompanied by a reduction of an instrumental response and not, as in fear conditioning, by an increase.
AP-5 injections did not interfere with the recall of a finally acquired extinction. Thus, once NMDA-dependent synaptic rearrangements accompanying extinction are established, an ignition of the altered assemblies does not depend on an activation of NMDA receptors. A similar dissociation between learning and recall with respect to the effects of NMDA receptor antagonists in various brain regions has been observed in a number of studies using a variety of learning paradigms (Morris, 1989; Miserendino et al., 1990; Shapiro and Caramanos, 1990; Campeau et al., 1992; Xu & Davis, 1992).
Unimpaired extinction and recall after NCL lesions
Lesioning the NCL caused deficits neither in extinction learning nor in extinction recall. Subjects whose NCL had virtually vanished showed extinction at a rate comparable to unimpaired controls and were also statistically indistinguishable in terms of extinction recall. One reason for the differential effects of lesions and receptor blockades could be that the area covered by the lesions was smaller than the effective range of the receptor antagonist. In a pilot study, the spread of AP-5 was evaluated by injecting into the NCL 0.5 μl of the fluorescent tracer rhodamine isothiocyanate, known for its wide diffusion area, resulting in an average spread of 1 mm in diameter around the tip of the cannula. Therefore, infusions of this volume through guide cannulas on positions L 5.00 and L 7.50 should cover the lateral-medial range of the NCL, but diffusion should extend neither anteriorly and posteriorly beyond the NCL area nor into the adjacent ventricle (for a detailed description, see Lissek et al., 2002). Thus, the presumed spread of the NMDA antagonist AP-5 is not likely to extend beyond the range covered by the lesions in our subjects, rendering the above explanation for differential effects unlikely.
A possible solution for the lack of postlesional deficits is the assumption that although extinction learning importantly encompasses NCL processes, it can in principle also be mediated by other structures. This is consistent with speculations that prefrontal functions are subsumed by other structures with practice and suggests that these systems operate in parallel with NCL (Miller, 2000; Schoenbaum et al., 2002). In the case of NCL lesions, brain regeneration processes that are launched in the recovery period of 7-10 d after the lesion would enable subjects to solve the task despite a lesioned NCL. However, a blockade of NMDA receptors within the NCL during extinction learning represents an experimental intervention that takes place before the forebrain areas outside NCL had the possibility to overtake prefrontal functions. As a result, extinction learning is seriously perturbed during the effective time period of the antagonist. Thus, we assume a shift of functions to other areas in the postlesional days to be the main reason for the counterintuitive result that NCL lesions produce fewer deficits than NMDA receptor blockades. This view is consistent with literature showing seemingly contrasting effects of prefrontal lesions on extinction: Although a study with primates finds extinction deficits of an instrumental response after lesioning the PFC (Butter, 1969), other experiments report unimpaired extinction of conditioned emotional responses after PFC lesions (Gewirtz et al., 1997; Morgan and LeDoux, 1999). Moreover, some reports even reveal dissociations between the effects of lesions and receptor blockades in the same area (Wolf et al., 1995). Thus, prefrontal areas might be critical to learn the extinction of previously rewarded associations. However, their function is replaceable by other systems if time for neuronal reorganization is provided.
In summary, the findings of this study demonstrate for the first time that NMDA receptors in the NCL, an associative area in the avian forebrain that is functionally equivalent to the mammalian PFC, are involved in extinction learning of an instrumental response. The perseveration behavior that occurred after NMDA receptor blockade was not caused by a general behavioral disinhibition, but rather by a deficit in acquiring extinction learning. We conclude that the activation of NMDA receptors in the avian “prefrontal cortex” is a necessary prerequisite for the establishment of a memory trace that enables extinction behavior.
Footnotes
This work was supported by Deutsche Forschungsgemeinschaft Grant Gu 227/5. The methods used in this experiment comply with the specifications of the German law for the prevention of cruelty to animals.
Correspondence should be addressed to Dr. Silke Lissek, Institute for Cognitive Neuroscience, Department of Biopsychology, Faculty of Psychology, Ruhr-Universität Bochum, Universitätsstrasse 150, 44801 Bochum, Germany. E-mail: silke.lissek{at}ruhr-uni-bochum.de.
Copyright © 2003 Society for Neuroscience 0270-6474/03/238119-06$15.00/0
