Abstract
Coding of prediction error by midbrain dopamine neurons has been examined extensively in the framework of associative learning theory. Most of this research has focused on the role of prediction error in determining the reinforcement value of unconditioned stimuli: poorly predicted (“surprising”) outcomes are more effective reinforcers and produce a greater dopamine response than well predicted outcomes. However, surprise also enhances attention to cues that signal poorly predicted outcomes. Previous reports from our laboratories demonstrated that circuitry, including the amygdala central nucleus (CeA), the cholinergic neurons of the substantia innominata/nucleus basalis region, and their innervation of the posterior parietal cortex, is critical to these surprise-induced enhancements of attention in associative learning. The present study considered the origin of prediction error information important for the operation of this system by examining the effects of disrupting communication between the midbrain substantia nigra pars compacta (SNc) and the CeA. Rats received unilateral lesions of the SNc and lesions of the CeA in either the contralateral or ipsilateral hemisphere. Contralateral lesions eliminated the surprise-induced enhancement of attention and learning that was displayed by rats with ipsilateral control lesions. These results show that SNc–CeA communication is critical to mechanisms by which the coding of prediction error by midbrain dopamine neurons is translated into enhancement of attention and learning modulated by the cholinergic system.
- amygdala central nucleus
- substantia nigra
- attention
- associative learning
- prediction error
- surprise
Introduction
The midbrain dopamine (DA) cells, mainly located in the substantia nigra zona compacta (SNc) and the ventral tegmental area (VTA), are shown to code prediction error in reward (Schultz et al., 1997). DA cells do not respond to well predicted rewards but increase activation in the event of unpredicted rewards and decrease activation when predicted rewards are omitted. Because prediction error constitutes a major factor in contemporary associative learning theories (Rescorla and Wagner, 1972; Pearce and Hall, 1980), coding of prediction error by DA cells has been discussed in the context of associative learning (Schultz and Dickinson, 2000). However, the mechanisms by which the coding of prediction error by DA cells influences associative learning are not well understood.
In a neural systems analysis, components of a circuit underlying the effects of prediction error on learning have been identified. According to Pearce and Hall (1980), surprise generated by prediction error enhances attention to the cue that signals the poorly predicted outcome, which in turn enhances subsequent learning about the cue. Holland and Gallagher (1993) first showed a neural system critical to this enhancement of attention and learning. Unlike intact rats, rats with bilateral lesions of the central nucleus of the amygdala (CeA) did not show enhanced attention to a light when the tone that it usually predicted was omitted and subsequently failed to show enhanced learning to the light. Later studies demonstrated that circuitry, including the CeA, the cholinergic neurons of the substantia innominata/nucleus basalis region (SI/nBM), and their projections to the posterior parietal cortex (PPC), was important for this enhanced attentional processing (Chiba et al., 1995; Bucci et al., 1998 Han et al., 1999). The present study considers the origin of prediction error information important for the enhancement of attention and subsequent learning in this system.
Given the dense interconnections between the SNc and CeA (Fallon and Moore, 1978; Swanson, 1982; Gonzales and Chesselet, 1990; Fudge and Haber, 2000), this circuitry seems to be an ideal substrate by which DA prediction error signals could influence the enhancement of associative learning that involves the CeA–cholinergic pathway. Thus, the current study examined the effects of SNc and CeA disconnection on performance in a behavioral task used in previous studies of neural systems that underlie surprise-induced enhancement of attention.
Materials and Methods
Subjects.
Experimentally naive, male Long–Evans rats (Charles River Laboratories, Raleigh, NC), initially weighing 350–400 g, were housed individually in a climate-controlled vivarium on a 12 h light/dark cycle (lights on at 7:00 A.M.) with ad libitum access to water. They were fed ad libitum during acclimation and during the postoperative recovery period. Starting 7 d before training until the completion of the study, they were placed on a restricted diet to maintain 85% of free-feeding body weight.
Surgery.
Rats were anesthetized with isoflurane gas (Abbott Laboratories, North Chicago, IL) and placed in a stereotaxic frame (Kopf Instruments, Tujunga, CA), and surgery was conducted under aseptic conditions. Rats received unilateral 6-hydroxydopamine (6-OHDA) lesions of the SNc and unilateral ibotenic acid lesions of the CeA in either the contralateral or ipsilateral hemisphere. Because connections between the SNc and CeA are predominantly ipsilateral, contralateral lesions of the SNc and CeA prevent communication between those two regions. Notably, these lesions spare functions subserved by each region unilaterally, except for those that require CeA–SNc communication. In contrast, ipsilateral lesions destroy the same amount of tissue in each region as the contralateral lesions but leave communication between the SNc and CeA intact in one hemisphere. Thus, the ipsilaterally lesioned rats served as appropriate controls for assessing the effects of SNc–CeA disconnection in the contralaterally lesioned rats. Each SNc lesion was made using 1.0 μl of 6 μg/μl 6-OHDA (Sigma, St. Louis, MO) in a PBS [i.e., 0.1 m phosphate buffer (PB) with 0.9% saline]/0.1% (w/v) ascorbic acid vehicle, infused over a 10 min period at stereotaxic coordinates 5.3 mm posterior to bregma, 2.4 mm lateral from the midline, and 7.4 mm ventral from the skull surface. The CeA lesions were made at stereotaxic coordinates 2.3 mm posterior to bregma, 4.2 mm lateral from the midline, and 8.0 mm ventral from the skull surface. For each CeA lesion, 0.25 μl of 10 μg/μl ibotenic acid (Biosearch Technologies, Novato, CA) in PBS solution was infused with a Hamilton 2.0 μl syringe over a 4 min period. The lesion sites were balanced, so there were equal numbers of each lesion type in each hemisphere. After surgery, all rats received a single subcutaneous injection of 0.01 mg/kg buprenorphine hydrochloride (Sigma) for amelioration of pain and were allowed to recover from surgery for 10 d before behavioral testing.
Behavioral task.
Holland and Gallagher (1993) used a behavioral task that was first developed by Wilson et al. (1992) to assess the enhanced attention and learning driven by prediction error according to the Pearce–Hall (1980) model. In the first phase (Table 1), rats were trained with a serial light→tone compound on a 50% food reinforcement schedule. Thus, although the light→tone compound was followed by food only half of the time, the light was always followed by the tone. The tone was expected to acquire substantial conditioned responding (CRs) because of its contiguity with food, and the light was expected to acquire minimal CRs because of its relatively poor temporal relationship with food. Importantly, as the serial conditioning continued, the light came to predict the tone accurately, and attention to the light was expected to decline. In the second phase, rats in the “consistent” condition received the same procedure as in the first phase, whereas for the rats in the “surprise” condition, the tone was omitted after the light presentations on nonreinforced trials. Thus, although the food reinforcement schedule remained the same, the light no longer accurately predicted the tone, and so, within the Pearce–Hall (1980) model, attention to the light should be enhanced. In the test phase, the rats received direct light→food pairings. If attention to the light was enhanced by the surprising omission of the tone in the second phase, rats in the surprise group should show enhanced learning. However, if SNc–CeA communication was important for detecting and using prediction error information, then rats with SNc and CeA contralateral lesions should not show this enhanced learning.
Behavioral procedure
Using standard conditioning chambers enclosed in soundproof boxes (Holland and Gallagher, 1993), the rats were first trained to eat the food reinforcer to be used in training. Sixteen food reinforcers (two 45 mg pellets; Research Diets, New Brunswick, NJ) were delivered to a food cup recessed in one chamber wall in a 64 min session, with intertrial intervals (ITIs) ranging from 2 to 6 min. Then, all rats received 10 daily phase 1 conditioning sessions. In each 64 min session, rats received eight reinforced and eight nonreinforced trials of a light→tone compound conditioned stimulus (CS), randomly intermixed with variable ITIs averaging 4 min. The compound CS comprised a 10 s illumination of a 6 W lamp mounted over the food cup, followed by a 10 s presentation of a tone (80 dB, 1500 Hz) through a speaker mounted outside of the chamber. Two 45 mg food pellets were delivered immediately after the tone presentation on reinforced trials. In phase 2, one-half of the rats from each lesion condition received training sessions identical to those given in the first phase for an additional 2 d (consistent condition). Meanwhile, the remaining rats from each lesion condition received eight reinforced trials of a light–tone compound CS (like those given in phase 1) randomly intermixed with eight trials with the light presentation alone (surprise condition). In phase 3, all rats received five daily 64 min test sessions, in which 16 presentations of the 10 s light were each followed by the two-pellet food reinforcer.
Learning was assessed by measuring the food-cup CR, the percentage of time during each recording period that the food-cup photocells reported head entry. We report food-cup CRs during the last 5 s period of CSs because that CR occurs primarily during that time period (Holland, 1977). To reduce the contribution of within-group variation in baseline responding, pre-CS responding (responding during the 5 s empty interval before the light) was subtracted from CS responding to form an elevation score. Importantly, the lesion and behavioral treatment procedures had no differential effects on baseline responding.
Histology.
After completion of the behavioral procedure, the rats were deeply anesthetized with isoflurane gas and perfused with 0.9% saline, followed by 4% paraformaldehyde in 0.1 m PB. Brains were removed, postfixed, cryoprotected overnight in 4% paraformaldehyde in 0.1 m PB containing 12% sucrose, frozen with powdered dry ice, and stored at −80°C. Brains were sliced on a freezing microtome, and 40 μm coronal sections through the CeA and SNc were collected in four series. One series of sections was mounted on slides and stained for Nissl to verify lesions made in the CeA. A second series of sections was used for tyrosine hydroxylase (TH) immunoreactivity, to evaluate the lesions made to SNc DA cells. The assessment of TH immunoreactivity followed a standard immunohistochemical protocol for free-floating tissues (Lee et al., 2005).
Results
Histology
Thirty-four rats were judged as having acceptable lesions of both the SNc and CeA. SNc lesions were rejected if there was <70% damage to SNc or if there was more than minimal damage to the VTA, and CeA lesions were rejected if there was <30% damage to the medial portion of the CeA. The brains with acceptable lesions (19 contralateral and 15 ipsilateral) averaged 80% damage to the SNc [levels 36–39 from the study by Swanson (1992)] and 56% damage to the medial CeA [levels 24–28 from the study by Swanson (1992)]. Sparing of the SNc adjoined the VTA, and sparing of the medial CeA neurons was mostly in the anterior regions [levels 24 and 25 from the study by Swanson (1992)]. Figure 1shows histology from a typical rat that received a lesion of the SNc in the left hemisphere and a lesion of the CeA in the right hemisphere.
Photomicrographs showing representative brain sections of the SNc and CeA of an animal that received an SNc lesion in the left hemisphere and a CeA lesion in the right hemisphere. A, B, Sections stained for TH in the lesioned and intact SNc, respectively. C, D, Nissl-stained sections of the intact and lesioned (arrows) CeAs, respectively. SNr, Substantia nigra pars reticulara; BLA, basolateral amygdala; CEAl, lateral CeA; CEAm, medial CeA; IA, intercalated nucleus of the amygdala.
Behavior
In phase 1, all rats acquired substantial CRs to the tone but responded little to the light. Over the last two sessions, the contralaterally lesioned rats displayed 63.1 ± 4.5% CRs to the tone and 1.3 ± 3.6% to the light, and the ipsilaterally lesioned rats showed 64.4 ± 4.7% and 0.6 ± 2.0% to tone and light, respectively. The lesion condition had no effect on CRs to either the tone or the light (F < 1). The same pattern of behavior was maintained in phase 2; across the four treatment/lesion conditions, responding to the tone ranged from 62.1 ± 6.4 to 76.4 ± 4.8%, and responding to the light ranged from 0.1 ± 2.1 to 3.3 ± 3.6%. The treatment and lesion conditions had no effects on CRs to the tone or the light (F < 1).
Figure 2 shows the primary data of this study, the acquisition of food-cup CRs to the light during the test phase. As predicted from previous studies with intact rats (Wilson et al., 1992; Holland and Gallagher, 1993; Chiba et al., 1995; Han et al., 1999), rats in the ipsilateral lesion–surprise condition showed more rapid learning than rats in the ipsilateral lesion–consistent condition early on. By the last two test sessions, rats in the ipsilateral–surprise condition showed more CRs than the rats in the ipsilateral–consistent condition (t(13) = 2.13; p = 0.026; one-tailed). Contralateral lesions eliminated this surprise-induced enhancement of learning. Rats with those lesions showed similar CRs regardless of behavioral treatment (t(17) = 0.13; p = 0.9), and, critically, among rats that received the surprise treatment, those with contralateral lesions showed significantly less food-cup responding than rats with ipsilateral lesions (t(14) = 2.93; p = 0.01; two-tailed). Notably, these differences are attributable to learning. Although they began to emerge within the first test session, the groups began the test phase with similar levels of responding; over the first half of that session, responding ranged from 10 to 15% across these groups. Thus, these data show that SNc–CeA communication is important for processing enhanced attention and learning induced by surprise/prediction error.
Mean (±SEM) time spent in the food cup during the light presentation in the test phase. Left, The acquisition of food-cup conditioned responses over the course of the test phase. Right, Mean responding over the last two test sessions. Within the rats that received the surprise treatment in phase 2 (□), those with the contralateral lesions of the CeA–SNc did not show enhanced conditioning to the light seen in animals with the ipsilateral CeA–SNc lesions. ∗p < 0.05 (1-tailed), surprise ipsilateral (surprise-ipsi) versus consistent ipsilateral (cons-ipsi); #p < 0.01 (2-tailed), surprise-ipsi versus surprise contralateral (surprise-contra). cons-contra, Consistent contralateral.
Discussion
Rats in which communication between the SNc and CeA was disrupted failed to show surprise-induced enhancement of learning that was observed in control rats, in which communication between those regions was preserved in one hemisphere. Thus, this study implicates connections between the SNc and CeA in the use of prediction error coding by midbrain DA neurons in the enhancement of attentional processes that involve circuitry of the CeA, cholinergic cells in the SI/nBM, and their projections to the PPC (Chiba et al., 1995; Bucci et al., 1998; Han et al., 1999).
Most discussions of the role of DA prediction error signals focus on their role in reinforcement processes, such as those specified by the Rescorla–Wagner (1972) model. However, we have shown elsewhere that the enhanced learning observed in the behavioral task used here is attributable to increased attention to the light CS, and not to variations in reinforcement processes (Holland and Gallagher, 1993; Holland et al., 2001). First, only the probability of tone presentation was altered in the surprise condition; the light–food schedule remained at 50% throughout the training phases in both treatment conditions. Second, the training conditions were designed to minimize direct light–food conditioning during phases 1 and 2. The light was placed in an unfavorable temporal relationship with food, relative to the tone–food relationship, and the light was also chosen to be less salient than the tone. Third, this same surprise condition has been shown to enhance both excitatory light→food learning and inhibitory light→no food learning in the test. This pattern of results is consistent with enhanced attention to the light, but not with enhanced processing of the food reward. Thus, our observation that disrupting communication between the SNc and CeA disrupts performance in this task is an important extension of our understanding of the role of prediction error in modulating attention in learning, for example as specified in the model of Pearce and Hall (1980). However, it remains to be seen whether midbrain DA neurons, in fact, show activation changes in response to unexpected delivery or omission of CSs, comparable to the brief phasic increases in activation observed to unpredicted rewards and decreases observed when predicted rewards are omitted (Schultz et al., 1997).
Heavy projections have been identified in both directions between the SNc and CeA (Fallon and Moore, 1978; Swanson, 1982; Gonzales and Chesselet, 1990; Fudge and Haber, 2000). Either or both of these projections may play a role in surprise-induced enhancement of attention. Consider first the projections from SNc DA neurons to the CeA. Omission of the tone after light presentation in the surprise condition would presumably be accompanied by reduced firing of SNc DA neurons, leading to a transient reduction in DA release in their CeA terminal fields. Because there is a heavy presence of GABAergic cells in the CeA (Swanson and Petrovich, 1998), one possible outcome of decreased DA release in the CeA is disinhibition of the CeA. Disinhibition of the CeA then would increase CeA activity, resulting in enhanced attention to the light CS. However, more study is needed to understand the exact nature of DA influence on the CeA cells. For example, although Swanson and Petrovich (1998) identified GABAergic cells throughout the CeA, Sun and Cassell (1993) found those cells to be concentrated in the lateral CeA. Recent studies suggest that projections from the CeA to the SNc may also play an important role in surprise-induced enhancement of attention. Han et al. (1999) proposed that direct projection from the CeA to the SI/nBM was important for this surprise-induced enhancement of attention. However, a recent study by Holland and Gallagher (2006) suggested that the influence of the CeA on the SI/nBM might be indirect, mediated by another brain region. Using transient inactivation procedures and the same behavioral task as was used in the current study, they found that CeA function was only critical at the time of surprise and that SI/nBM function was only critical at the time of expression of the enhanced attention in the test. Thus, neither CeA nor SI/nBM likely maintained the information of enhanced attention to the CS. Given the direct connection between the SNc and SI/nBM (Hasue and Shammah-Lagnado, 2002) and our current data showing the involvement of the SNc for this task, it is plausible that once enhanced attention to the CS is processed by the CeA, the SNc maintains that information, influencing the action of the SI/nBM at the time of expression. Although the current study cannot address the exact nature of the CeA–SNc–SI/nBM circuitry, it suggests a possible influence of CeA–SNc connection on the cholinergic system in translating surprise/prediction error to enhancement of attention and learning.
The critical role of the CeA and SNc for attention is demonstrated in another task that examines attentional modulation of ongoing behavior, conditioned orienting to cues that signal biologically significant events. Disconnection of the CeA and SNc prevented the normal acquisition of conditioned orienting to a visual cue paired with food (Lee et al., 2005). In the same study, the authors showed that the medial CeA cells retrogradely labeled from the SNc expressed an immediate-early gene, FOS, after conditioned orienting responses, demonstrating the direct involvement of the CeA to SNc projection in conditioned orienting. Furthermore, a series of studies demonstrated time dependencies such as those observed by Holland and Gallagher (2006): CeA function was critical only at the time of acquisition (McDannald et al., 2004; Groshek et al., 2005), and function of another brain region, the dorsal lateral striatum, was critical only at the time of expression of that learned response (Han et al., 1997). Most important, SNc function was critical to both the acquisition and expression of that behavior (El Almamy and Holland, 2006). Thus, it is possible that the SNc may be necessary for both detecting surprise/prediction error and using the information about enhanced attention to the CS in subsequent learning. We are currently evaluating these hypotheses.
The midbrain DA system has long been recognized to be important in many functions. In particular, the mesocortico-limbic system, arising from the VTA, is thought to be important for reward, motivation, and attention (Brozoski et al., 1979; Robinson and Berridge, 1993; Wise 2004), and the mesostriatal system, arising from the SNc, is best known for its role in motor functions, highlighted by the pathological and behavioral changes in parkinsonian patients and in animal models of this disease (Burns et al., 1983; Forno, 1992). Furthermore, results such as the present ones and those of Lee et al. (2005) and Baunez and Robbins (1999) demonstrate that the SNc plays an important role in attentional processing, independent of motor functions. Notably, DA neurons in both of these midbrain regions process prediction error, which may be critical for both reinforcement and attentional processes (Schultz and Dickinson, 2000). The possible involvement of the VTA for surprise-induced enhancement of attention cannot be ruled out. Nevertheless, the current study highlights the need for better understanding of the SNc function as well as the VTA function. It is likely that these DA systems and their functions are highly intertwined. Indeed, anatomical studies show complex interconnections of the terminal fields arising from the DA cells in the VTA and SNc (Fallon and Moore, 1978) and suggest information flow between the VTA and SNc and their projecting areas via limbic–cortical–striatal “spiral loops” (Fudge and Haber, 2000; Haber et al., 2000). The CeA may play a pivotal role in modulating information processing within these systems.
Footnotes
- Received March 28, 2006.
- Revision received April 28, 2006.
- Accepted April 28, 2006.
-
This work was supported by National Institutes of Health Grant MH53667. We thank Weidong Hu for assistance in histology.
- Correspondence should be addressed to Dr. Hongjoo J. Lee, Johns Hopkins University, 3400 North Charles Street, 150 Ames Hall, Baltimore, MD 21218. Email: hjlee{at}jhu.edu