Abstract
Incentive learning is the process via which animals update changes in the value of rewards. Current evidence suggests that, for food rewards in rats, this learning process involves the amygdala. However, it remains unclear whether this learning undergoes protein synthesis-dependent consolidation and “reconsolidation” processes in the lateral and basal nuclei of amygdala. Accordingly, we examined this hypothesis by local infusion of protein-synthesis inhibitor after devaluation of a food reward induced by a shift from a food-deprived to a food-sated state in an instrumental conditioning paradigm. Our results show that intra-amygdala infusions of anisomycin, whether given after the initial devaluation or after a second devaluation session, abolished the changes in the value of the food reward produced by incentive learning. This study provides direct evidence that instrumental incentive learning depends on protein synthesis within the amygdala for both consolidation and reconsolidation and extends the demonstrations of protein synthesis-dependent reconsolidation to reward-related memories.
- basolateral amygdala
- incentive motivation
- memory reconsolidation
- protein synthesis
- instrumental conditioning
- rat
Introduction
The performance of goal-directed instrumental actions depends on the knowledge of both the action-outcome (A-O) relationship and the current incentive value of that outcome (Rescorla, 1998; Dickinson and Balleine, 2002). Considerable evidence suggests, however, that posttraining changes in outcome value, whether induced by a shift in primary motivational state, by conditioned taste aversion (CTA), or by a specific-satiety treatment (Balleine and Dickinson, 1998; Balleine, 2001), only affect performance if rats are given the opportunity to learn about the impact of these treatments on the value of the reward via direct consummatory experience. For example, a motivational shift from hunger to satiety will often only reduce lever pressing if rats have been given the opportunity to consume the reward when sated (Balleine, 2001). The process whereby animals learn about changes in reward value is called incentive learning (Balleine, 2001; cf. Dickinson and Balleine, 2002).
Recent evidence has implicated the lateral and basal nuclei of the amygdala (LBA) in incentive learning. Although lesions of the LBA have no apparent effect on the acquisition of lever pressing per se, they do leave performance insensitive to outcome devaluation (Balleine et al., 2003). Several other findings support the notion that the LBA makes a rather fundamental contribution to learning about the incentive value of anticipated outcomes (Hatfield et al., 1996; Blundell et al., 2003; Pickens et al., 2003; Holland and Gallagher, 2004). Although the specific nature of its involvement remains poorly understood, there is some evidence that the LBA plays a role in the consolidation of changes in incentive value produced by incentive contrast (Salinas et al., 1993; cf. Flaherty, 1996).
According to the memory consolidation hypothesis (McGaugh, 2000; Dudai, 2004), newly formed memories initially exist in a “labile” state sensitive to disruption. Over time, the memory becomes consolidated and resistant to disruption. It has been proposed that translation and transcription are necessary universal conditions for new memories to consolidate (Kandel, 2001). However, this account has been challenged by several demonstrations that the consolidation process is not restricted to initial training but can be engaged when a memory is reactivated (Misanin et al., 1968; Nader et al., 2000). Recent studies on memory “reconsolidation” have primarily used aversive pavlovian conditioning paradigms, including contextual (Debiec et al., 2002) and auditory fear conditioning (Nader et al., 2000; Kida et al., 2002) and conditioned taste aversion (Eisenberg et al., 2003). Although an impairment in appetitive reconsolidation has been reported after systemic manipulations, such as propranolol (Przybyslawski et al., 1999), it remains unclear whether the maintenance of appetitive memories also depends on a protein synthesis-dependent reconsolidation process. The one study that has investigated this issue reported no effect of postreactivation protein synthesis inhibition in the nucleus accumbens on performance of lever pressing for food (Hernandez et al., 2002). These findings do not, however, preclude the possibility that other processes controlling goal-directed instrumental actions undergo reconsolidation. Accordingly, in the current study, we investigated whether the local protein synthesis in the LBA was required for the consolidation and reconsolidation of incentive learning.
Materials and Methods
Subjects. The subjects were 14 experimentally naive male Long-Evans rats. The rats were housed individually and were handled daily for 1 week before surgery.
Apparatus. Training and testing took place in 14 Med Associates (East Fairfield, VT) operant chambers, each equipped with a pump fitted with a syringe that delivered 0.1 ml of a 20% sucrose solution and a pellet dispenser that delivered a 45 mg Noyes pellet (formula A/I; Research Diets, New Brunswick, NJ). Two retractable levers were inserted into the chamber on either the left or right side of the food magazine. Microcomputers equipped with the MED-PC program (Med Associates) controlled the equipment and recorded lever presses and magazine entries.
Surgery and drug infusion. At the time of surgery, animals weighed between 580 and 790 g. Under Nembutal anesthesia (45 mg/kg), rats were implanted bilaterally with 22 gauge stainless-steel cannula into the LBA (3.0 mm posterior to bregma, 5.3 mm lateral to the midline, and 8.0 mm ventral to the skull surface). The rats were allowed to recuperate for 7 d after surgery. Anisomycin (ANI) (125 μg/μl; Sigma, St. Louis, MO) was dissolved in equimolar HCl, diluted with artificial CSF vehicle (VEH), and adjusted to pH 7.4 with NaOH. ANI or VEH (0.5 μl) was infused slowly (0.25 μl/min) into the LBA using a microinfusion pump (Harvard Apparatus, Holliston, MA). The injectors were left in place for 1 min for additional drug diffusion.
Histology. At the end of the experiment, using standard histological methods, animals were perfused transcardially, and their brains were sectioned at a thickness of 50 μm. The sections were stained using thionin and examined by light microscopy for cannula penetration into the LBA.
Instrumental training. After recovery from surgery, the rats were food deprived by restricting their maintenance diet to keep them at ∼85% of their free-feeding weight. After 3 d of adaptation to the food-deprivation regimen, the rats received two 30 min sessions of magazine training, during which pellets and sucrose were delivered on independent random-time 60 s schedules. On each of the subsequent 12 d, the rats received a separate session of training with each response (left and right lever press). Each session was terminated after 30 outcomes were earned. One-half of the rats were reinforced with pellets for pressing the left lever and were reinforced with sucrose for pressing the right lever, whereas the remaining half received the opposite response-outcome (R-O)pairings. The reinforcement schedule gradually shifted from continuous reinforcement to random ratio 5, 10, and 20, with each stage of training lasting 3 d (Fig. 1).
Incentive learning and drug challenge. After the last training session, rats were provided with ad libitum access to their maintenance diet for the remainder of the experiment. After 24 h, the rats were given ad libitum access to one of the training outcomes [outcome 1 (O1), either pellets or sucrose] in their home cage for 1 h as an incentive learning (i.e., devaluation) procedure. This exposure period was intended to provide consummatory experience with O1 exclusively, such that the animal would have the opportunity to learn selectively the reduced incentive value of O1 in a food-sated state. Immediately after the outcome exposure, the rats received LBA infusions of either ANI (n = 7) or VEH (n = 7). Group assignments were made according to training performance to control for response biases across levers and outcomes. The identity of the exposed outcome was balanced across groups and instrumental contingencies. After 24 h, the nondeprived rats were given a 10 min choice extinction test (test 1, consolidation test) during which both responses could be performed but not rewarded. Because O1 was associated with only one of the two actions [response 1 (R1)], the integrity of incentive memory after the exposure is reflected in the subsequent choice performance of the rats (i.e., in a relative reduction in responding on the lever trained with O1; R1 < R2). Therefore, we predict that, if consolidation of incentive learning requires protein synthesis in the amygdala, then VEH-treated rats should show the predicted incentive learning effect, whereas, in the ANI-treated rats, this difference should be attenuated (Fig. 1) (see Fig. 3A).
To test whether incentive learning undergoes reconsolidation, rats were given a second opportunity for a consummatory experience with O1. During this phase, however, each group received the opposite drug treatment after the reexposure session [i.e., the group that initially received VEH infusions was now treated with ANI (group V/A)], whereas the group that initially received ANI now received infusions of the VEH (group A/V). Memory for the reduction in outcome value was assessed 24 h later in a second 10 min choice extinction test (test 2, reconsolidation test). The objective of the reversed design was to first test whether the consolidated incentive memory mediating the performance of group V/A in test 1 undergoes a protein synthesis-dependent reconsolidation process when reactivated by an additional outcome exposure. This would be demonstrated by indifferent responding in group V/A on R1 and R2. Second, test 2 allowed us to assess whether the initial impairment of consolidating incentive learning in group A/V could be reversed via an additional exposure to the outcome (i.e., incentive retraining). This finding would indicate that the amygdala had remained functional and that the deficit in test 1 was, in fact, attributable to the anisomycin treatment but was not attributable to permanent structural damage resulting from the infusion procedure (Fig. 1) (see Fig. 3A). Because it has been shown previously that the incentive learning effect is not mitigated by multiple-outcome exposures (Balleine and Dickinson, 2000), it was assumed that the VEH group would serve as an appropriate control for the ANI group in test 2.
Results
Incentive learning undergoes consolidation
Figure 2 shows the location of the injector tips aimed at the LBA. The results of test 1 are presented in Figure 3B. An ANOVA revealed that the VEH and ANI groups performed a similar number of total responses during this test (F < 1). However, an analysis of the simple main effect of response (R1 vs R2) for each group found significantly less responding on the lever that was associated with O1 in the VEH group (R1 < R2; F(1,12) = 7.63; p < 0.05) but not in the ANI group (F < 1). The selective performance of the VEH group demonstrates that the single outcome exposure is sufficient to produce incentive learning, replicating previous findings (Balleine, 2001). However, the failure to observe this effect in the ANI group suggests that no incentive learning occurred. In addition, the total rate of responding was similar in both groups, suggesting that ANI had no unintended effects that might have interfered with instrumental performance. These findings are consistent with the possibility that anisomycin blocked the consolidation of incentive learning.
Incentive learning undergoes reconsolidation
Although the first outcome exposure provided an opportunity for incentive learning, reexposure to O1 should have served two basic functions. First, for group A/V, which failed to display a selective devaluation effect during test 1, reexposure acts as a retraining session for incentive learning. The ability to reacquire incentive learning would demonstrate that the amygdala function was not compromised by the initial ANI infusion. Second, for group V/A, which has a consolidated memory of incentive learning, this reexposure treatment should both reactivate the consolidated incentive memory of O1 and act as a second training session to learn further the lowered value of that outcome when sated. Consolidation theory predicts that the new learning occurring during the reexposure should be blocked by ANI. Therefore, group V/A should show the same pattern of responding on both tests (i.e., R1 < R2) because any new learning as a result of reexposure will be blocked from consolidating. In contrast, reconsolidation theory predicts that reactivation of the consolidated memory will return that incentive memory to a labile state, which, together with the new learning, should now be susceptible to ANI challenge. Thus, the differential responding on test 1 seen in group V/A should be eliminated on test 2.
The results of the second test are presented in Figure 3C. Again, the drug treatment did not have a significant effect on the overall level of responding; the number of lever presses was similar in both groups (F < 1). However, the groups differed in the way they distributed their responses across the two levers. Critically, group V/A, which showed previously the differential responding indicative of incentive learning, now showed no differential responding between the levers (F < 1). This finding indicates that the incentive learning effect that this group displayed previously during test 1 was disrupted by the ANI infusion they received after O1 reexposure. Hence, the results from group V/A clearly demonstrate that incentive memory can return to an anisomycin-sensitive state in the amygdala when the memory is reactivated. Second, group A/V now displayed significantly less responding for O1 than for O2 (F(1,12) = 4.75; p = 0.05). This finding indicates that, although intra-LBA ANI infusions after the first outcome exposure had interrupted the consolidation of initial incentive learning, subsequent O1 reexposure without ANI was sufficient to establish the devaluation effect. Moreover, their capacity to show sensitivity to outcome value during test 2 suggests that their initial deficit did not result from permanent damage to the LBA.
Because the rats were not retrained on the levers between the two extinction tests, the overall level of responding was somewhat lower during test 2. To compare the selectivity of responding across drug conditions for each group, performance was plotted as the percentage of total responses made for the devalued outcome [i.e., R1/(R1 + R2) × 100] and is presented in Figure 4 (black bars). A two-way ANOVA, using test (test 1 vs test 2) and drug (VEH vs ANI) as factors, on this measure found a main effect of drug (F(1,12) = 6.55; p < 0.05) but neither an effect of test nor a drug by test interaction (both, F < 1). The main effect of drug was tested further by comparing separately the percentage of responses on the devalued action after the VEH and ANI treatments against indifference (R1 = R2). These tests confirmed that the rats reliably altered their choice and responded less on the devalued action after the VEH treatment (F(1,12) = 9.6; p < 0.05), whereas they responded similarly on both actions after ANI (F < 1).
Discussion
The results of this experiment demonstrate that both the consolidation and reconsolidation of incentive memory are blocked by anisomycin injections into the LBA. Regardless of whether infusions were made after initial exposure or reexposure to the outcome, ANI disrupted the impact of this devaluation treatment on subsequent instrumental performance. During the first choice test, rats treated previously with VEH after exposure to the devalued outcome made fewer responses for that outcome, whereas rats treated with ANI responded indifferently across the levers. In the subsequent reexposure session, a treatment that should have reactivated the original incentive memory and allowed for additional incentive learning, the drug conditions were reversed. During the second choice test, rats treated with VEH after reexposure suppressed selectively their performance for the devalued outcome, whereas rats treated with ANI responded similarly for each outcome.
Rats given infusions of ANI immediately after the initial outcome exposure displayed a clear impairment in directing their behavior according to outcome value. Although this result is consistent with the interpretation that the consolidation of incentive learning requires intra-LBA protein synthesis, four alternative accounts should be considered. First, because it has been shown that neurotoxic lesions of the LBA disrupt the control of instrumental performance by outcome value (Balleine et al., 2003), this finding might have been expected if ANI infusions, in addition to blocking the synthesis of proteins, had otherwise permanently disrupted normal LBA function. Second, because goal-directed instrumental performance depends on encoding, not only of the incentive value of the instrumental outcome but also of the A-O relationship, it is possible that ANI had its effect by disrupting the integrity of the A-O memory. However, neither of these accounts anticipates the results of the second test. The finding that group A/V could reacquire incentive learning demonstrates that the effect of ANI was reversible and left the A-O association intact. In addition, there was no evidence that ANI infusions had any general effect on instrumental performance, because the groups responded at similar rates during each test. Third, although a CTA might have resulted from O1-ANI pairings, this does not seem to be the source of the effect of the drug on incentive learning, because this account predicts R1 < R2, whereas the observed result was R1 = R2. Last, ANI could not have had its effect by facilitating extinction consolidation, because there were no extinction contingencies present during the reexposure session. This is because the reexposure treatment represents another session for animals to learn the reduced value of O1 when sated. Recently, it has been shown that infusion of ANI into the LBA leads to widespread inhibition of protein synthesis in the surrounding areas (Maren et al., 2003). This finding raises questions about the possible regional specificity of the results reported here. Speaking in favor of anatomical specificity of these effects, however, are the results of experiments that used the same coordinates and infusion parameters as the current study and showed that infusion of ANI into the LBA, but not just dorsal to it, blocked the consolidation of auditory fear conditioning (Schafe and LeDoux, 2000). Similarly, this dose infused into the LBA, but not dorsal to it, blocked reconsolidation of auditory fear conditioning (S. Duvarci and K. Nader, personal communication). These findings indicate that the effect of local ANI infusions on memory is reasonably restricted to the immediate site of the infusion. Whether the surrounding areas of LBA may also contribute to memory consolidation of incentive learning needs additional research.
The current findings are consistent with previous reports implicating the LBA in the consolidation of memory for changes in the incentive value of rewards (Salinas et al., 1993) and indicate further that its involvement includes the formation of proteins. ANI infusions made after reexposure to the devalued outcome in group V/A abolished the sensitivity of instrumental performance to the reduction in outcome value despite the fact that these subjects had demonstrated clear incentive learning during the first test. Although this finding suggests that the reconsolidation of incentive learning depends on protein synthesis, several alternative accounts should be considered. For example, because this impairment was observed after group V/A had previously received testing, outcome exposure, and infusion treatment, it remains possible that one of these manipulations, and not the ANI infusion, was responsible for the deficit. There is, however, evidence against each of these alternatives. For example, using procedures comparable with those used in the current study, instrumental incentive learning has been shown to be robust and to persist across both multiple tests and outcome exposures (cf. Balleine and Dickinson, 2000). Moreover, the selective performance of group A/V in the second test after they had already received two infusions indicates that the expression of incentive learning was not disrupted by the infusion treatment alone.
Although memory reconsolidation has been demonstrated across species, from sea slugs to humans (Sara, 2000; Pedreira et al., 2002; Child et al., 2003; Sangha et al., 2003; Walker et al., 2003), with its underlying molecular mechanisms being intensively investigated in recent years (Berman and Dudai, 2001; Taubenfeld et al., 2001; Nader, 2003a; Lee et al., 2004), our study makes two major advances. First, it extends the paradigms used to demonstrate reconsolidation from aversive pavlovian conditioning (Nader et al., 2000; Debiec et al., 2002), object recognition (Bozon et al., 2003), and procedure learning (Nader, 2003b; Walker et al., 2003) to learning about the goals that control deliberate action. Although early studies using electroconvulsive shock have provided evidence of memory reconsolidation in maze learning (Lewis and Bregman, 1973), unlike in the current study, they did not identify the anatomical or molecular substrates underlying this effect. Second, this study delineates the role of LBA in the goal-directed instrumental actions. In a previous study, lidocaine injected into the LBA was found to block the decrement of straight alley runway performance produced by a reduction in reward value (Salinas et al., 1993). However, it has remained unclear whether the LBA mediates incentive learning or modulates other brain areas that are responsible for this learning. The results of the current study provide direct evidence that incentive learning is consolidated within the amygdala and are consistent with the view that the LBA is critically involved in the control of performance by outcome or goal expectancies (Balleine et al., 2003; Holland and Gallagher, 2004), a view that has been confirmed by lesion studies (Balleine et al., 2003; Izquierdo and Murray, 2004), neural recording (Schoenbaum et al., 2000) in rats and monkeys, and brain imaging studies in humans (Gottfried et al., 2003).
In summary, the current findings indicate that both the consolidation and reconsolidation of incentive learning depend on protein synthesis within the LBA, suggesting that this structure is involved in the acquisition and maintenance of the reward representations used to guide instrumental performance.
Footnotes
This research was conducted at the University of California, Los Angeles and was supported by National Institute of Mental Health Grant MH56446 (B.W.B.) and an EJLB Foundation Award (K.N.).
Correspondence should be addressed to either of the following: Szu-Han Wang, Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, Quebec, Canada H3A 1B1, E-mail: wangsh{at}ego.psych.mcgill.ca; or Sean B. Ostlund, Department of Psychology, University of California, Los Angeles, Box 951563, Los Angeles, CA 90095-1563, E-mail: sostlund{at}ucla.edu.
Copyright © 2005 Society for Neuroscience 0270-6474/05/250830-06$15.00/0
↵* S.-H.W. and S.B.O. should be regarded as joint first authors of this work.