The motivation to seek out rewards can come under the control of stimuli associated with reward delivery. The ability of cues to motivate reward-seeking behavior depends on the nucleus accumbens (NAcc). The molecular mechanisms in the NAcc that underlie the ability of a cue to motivate reward-seeking are not well understood. We examined whether extracellular signal-regulated kinase (ERK), an important intracellular signaling pathway in learning and memory, has a role in these motivational processes. We first examined p42 ERK (ERK2) activation in the NAcc after rats were trained to associate an auditory stimulus with food delivery and found that, as a consequence of training, presentation of the auditory cue itself was sufficient to increase ERK2 activation in the NAcc. To examine whether inhibition of ERK in the NAcc prevents cue-induced reward-seeking, we infused an inhibitor of ERK, U0126, into the NAcc before assessing rats' instrumental responding in the presence versus absence of the conditioned cue. We found that, whereas vehicle-infused rats showed increased instrumental responding during cue presentation, rats infused with U0126 showed a profound impairment in cue-induced instrumental responding. In contrast, intra-NAcc U0126 infusion had no effect on rats' food-reinforced instrumental responding or their ability to execute conditioned approach behavior. Our results demonstrate learning-related changes in ERK signaling in the NAcc, and that disruption of ERK activation in this structure interferes with the incentive-motivational effects of conditioned stimuli. The molecular mechanisms described here may have implications for cue-elicited drug craving after repeated exposure to drugs of abuse.
One factor that contributes to an animal's reward-seeking behavior is the presence of cues and contexts that it previously has associated with reward delivery. Reward-paired stimuli can engage an animal's appetitive motivational system, or “wanting,” and invigorate a variety of reward-seeking responses (Bindra, 1974; Dayan and Balleine, 2002; Berridge and Robinson, 2003). The motivational effect of cue presentation is illustrated most dramatically by the reports of drug craving and relapse to drug-seeking behavior by addicts while in the presence of drug-associated cues and contexts (Ehrman et al., 1992; Dackis and O'Brien, 2005). The effectiveness of reward-paired stimuli in invigorating reward-seeking behavior can be measured using a pavlovian-instrumental transfer paradigm. During pavlovian-instrumental transfer, presentation of a previously acquired conditioned stimulus (CS) that predicts reward delivery increases the rate at which an animal performs reward-related instrumental actions (Edgar et al., 1981; Lovibond, 1981; Dickinson and Dawson, 1987).
The excitatory effects of CS presentation on reward-seeking behavior depend crucially on the nucleus accumbens (NAcc). NAcc lesions produce impairments in pavlovian-instrumental transfer, as well as with other tasks that depend on arousal generated by CS presentation (Parkinson et al., 1999; Corbit et al., 2001; Hall et al., 2001; Cardinal et al., 2002; de Borchgrave et al., 2002). The molecular substrates in the NAcc that underlie conditioned motivational processes are not fully understood. The extracellular signal-regulated kinase (ERK), a member of the mitogen activated protein kinase family (MAPK), is activated in the NAcc after neuronal activity, and NAcc ERK activation has been implicated in certain aspects of drug-seeking behavior (Berhow et al., 1996; Valjent et al., 2000, 2005, 2006; Miller and Marshall, 2005b). However, a role for ERK activation in the NAcc in appetitive learning and motivational responses to reward-associated stimuli has not yet been investigated.
To address a role for ERK activation in the NAcc in appetitive learning and motivational processes, we used an appetitive pavlovian conditioning paradigm to train rats to associate a tone with food reward delivery. We then examined p42 ERK (ERK2) activation in the striatum after rats underwent different amounts of pavlovian conditioning, and after presentation of the tone CS. We hypothesized that, as a result of conditioning, presentation of a reward-associated stimulus would increase ERK activation specifically in the NAcc. We also examined the effects of inhibiting ERK activation in the NAcc on performance of pavlovian-instrumental transfer. We hypothesized that, if ERK activation in the NAcc is important for motivational processes, disruption of ERK activation would prevent the excitatory effects of stimulus presentation on instrumental behavior.
Materials and Methods
A total of 106 male Sprague Dawley rats (Hilltop Lab Animals, Scottdale, PA) were used in this study. Rats weighed 275–300 g upon arrival and were housed individually in wire-bottomed cages and supplied with ad libitum food and drinking water. Rats were handled and weighed daily, and within 4–6 d were placed on a restricted diet of 15–25 g of rat chow per day to maintain their body weight at ∼90% of their ad libitum-feeding weight during the course of the experiment. All animal procedures were approved by the University of Pittsburgh Institutional Animal Care and Use Committee, and all efforts were made to minimize the number of animals used and any discomfort the animals might have experienced. The behavioral data were collected in the Rodent Behavior Analysis Core of the University of Pittsburgh.
Behavioral procedures took place in 4 operant chambers (30 cm × 23 cm × 23 cm; Med Associates, St. Albans, VT). Each chamber was equipped with a single house light, a floor with metal bars, and a loudspeaker that delivered a 3 kHz, 80 db tone when activated. A food cup was mounted on the front wall of the testing chamber, and was attached to a pellet dispenser that released a single 45 mg sucrose pellet (Bio-Serv, Frenchtown, NJ) when activated. An infrared photobeam source and detector were mounted on either side of the food cup, and were used to record food cup-approach behavior. Each chamber contained two retractable levers that were situated on the left and right side of the food cup. Each operant chamber was housed in a sound-attenuating cubicle equipped with a background noise-generating fan to overshadow extraneous sounds (BSR, Laurel, MD). The chambers were controlled through a desktop computer running Med Associates proprietary software (Med-PC).
Appetitive pavlovian conditioning.
Rats were transported from their home cages to the testing room and placed in the operant chamber. Before training, rats were habituated to the conditioning chamber in a single 30 min session with the house light on. Each pavlovian conditioning session commenced with illumination of the house light. During each session, rats were presented with a 3 kHz tone that served as a CS. Each tone was 90 s in duration, and during the tone three sucrose pellets were delivered. The pellets were delivered on a random time 20 s schedule, which commenced 30 s after tone onset. Each session consisted of eight tone presentations with a 3 to 5 min intertone interval (ITI; tone offset to next tone onset; mean, 4 min). Rats performed one training session per day for 4 consecutive days. For each session the number of photobeam breaks caused by head entry to the food cup was recorded during the first 30 s of the CS and during the 30 s preceding CS onset. Rats assigned to the control group were presented with an identical number of 90 s tones; however, food pellets were never delivered at any time during the session. This group served as a baseline for ERK activation against which all treatment groups were compared.
Rats were transported from their home cages to the testing room and placed in the operant chamber. The house light was illuminated and both levers were extended into the conditioning chamber. One lever (active lever), when depressed, delivered a single sucrose pellet into the food cup. Depression of the opposite lever (inactive lever) had no programmed consequences. The active lever was randomly assigned to the left or the right lever and was counterbalanced across cohorts of rats.
Rats first were shaped to press the active lever by smearing the surface of the lever with food paste. Rats then received a single session in which each active lever press resulted in delivery of a single pellet. This was followed on the same day by a single session in which each active lever press resulted in delivery of three pellets. Both sessions terminated once 50 pellets had been delivered. The subsequent 5 d, rats performed two daily sessions in which each active lever press resulted in delivery of five pellets (FR5) with a 15 min intersession interval. Each session terminated after the rat made 500 responses on the active lever (100 pellets earned).
Pavlovian-instrumental transfer test.
Before the transfer test, animals received a pavlovian conditioning reminder session that was identical to the original pavlovian training sessions. The transfer test commenced with illumination of the house light and insertion of both levers into the operant chamber. After a 6 min interval during which rats extinguished responding on the active lever, the first of six 90 s tones was presented, with a 3 min ITI separating each tone presentation. The number of active and inactive lever presses and food cup approaches was recorded during the tone and during a 90 s period preceding tone onset (pre-CS).
Presentation of the CS or training context.
Tests were designed to expose previously trained rats to the CS or the training context and measure ERK activation. During these tests, rats were placed in the operant chamber and the house light was illuminated. Half of the rats were presented with six unrewarded 90 s tone presentations, with a 3 min interval separating each successive tone presentation. The total session duration was 27 min. The remaining rats were presented with no tones and remained in the operant chamber for the same duration as the rats that experienced tone presentations (27 min). The number of photobeam breaks was recorded during the first 30 s of each tone presentation and a 30 s interval preceding tone onset. For rats exposed to the context only, the number of photobeam breaks was recorded during two consecutive 30 s intervals occurring every 3 min.
Tissue preparation and Western blots.
Rats were anesthetized with an intraperitoneal injection of chloral hydrate (300 mg/kg; dissolved in 0.9% saline) followed by rapid decapitation. Brains were removed from the skulls and rapidly frozen in isopentane chilled on dry ice. Tissue samples from the NAcc and dorsal caudate were free-hand dissected from 1 mm slabs of frozen brain tissue. The level of dual-phosphorylated (T183 and Y185) activated ERK2 (pERK2) as well as the level of total ERK2 (tERK2; phosphorylated and unphosphorylated) were evaluated in these samples using Western blot analysis. Samples were homogenized in a buffer containing 150 mm NaCl, 1 mm EDTA, 50 mm Tris, pH 7.4, 0.05% SDS, 1% Triton-X, 1 mm dithiothreitol (DTT), 1 mm phenylmethanesulfonyl fluoride, 2 mm sodium fluoride, 1 mm orthovanadate, 2 mm sodium pyrophosphate, 1 mg/ml leupeptin, and 1 mg/ml pepstatin. Homogenate was centrifuged for 15 min at 14,000 rpm, the supernatant was collected, and the samples were diluted with homogenization buffer to a uniform protein concentration. Sample buffer (2.5 m Tris, pH 6.8, 40% glycerol, 8% SDS, and 30 mg/ml DTT) was added to each sample, and the samples were heated to 95°C for 5 min. From each sample, 30 μg of protein was resolved via SDS-PAGE and blotted electrophoretically to an Immobilon membrane for antibody probing. The membrane was blocked for 30 min in a solution containing 5% bovine serum albumin (BSA), and then incubated overnight in a 5% BSA solution containing an antibody that selectively recognizes dual-phosphorylated (T202/183 and Y204/185), activated ERK1/2 (1:2500 dilution; Cell Signaling Technology, Beverly, MA). The membrane was washed in Tris-buffered saline (0.05 m Tris pH 7.9, 0.15 m NaCl, 0.1% Tween 20). Antigen binding was visualized with an HRP-linked secondary antibody (anti-rabbit, 1:5000; Cell Signaling Technology) and an enhanced chemiluminescence reagent (Lumiglo; Cell Signaling Technology). The membranes were stripped of their antigens by incubation at 55°C for 45 min in a solution containing 62.5 mm Tris pH 6.7, 2% SDS, and 0.62% β-mercaptoethanol. Membranes were blocked and reprobed with an antibody that selectively recognizes total (phosphorylated and nonphosphorylated) ERK1/2 using the same methods as described above. Blot images were captured with a CCD camera (Hamamatsu Photonics, Hamamatsu, Japan) and analyzed using densitometry software (UVP Labworks, Upland, CA). Only the p42 band (ERK2) was analyzed, as this isoform has been implicated more strongly in synaptic plasticity and learning and memory (English and Sweatt, 1996, 1997; Atkins et al., 1998; Mazzucchelli et al., 2002).
Rats were anesthetized with an intraperitoneal injection of chloral hydrate (300 mg/kg; dissolved in 0.9% saline) and rapidly perfused transcardially with 0.9% saline, followed by 4% paraformaldehyde in 0.1 m phosphate buffer. The brains were removed after perfusion and placed in 4% paraformaldehyde overnight and then transferred to a 20% sucrose solution until they sank. The brains were frozen and sectioned at a thickness of 40 μm on a cryostat at −16°C and stored in cryopreservative at −20°C until staining. To probe the sections for pERK1/2 immunoreactivity, sections at a frequency of 120 μm were brought to room temperature over 30 min, washed with sequential rinses in 0.1 m phosphate buffer on a rocker table, immersed in 0.1 m phosphate buffer containing 1% H2O2 for 15 min, and rinsed again several times in 0.1 m phosphate buffer. The tissue was treated to prevent unwanted nonspecific staining with 0.1 m phosphate buffer containing 0.4% Triton X-100 and 10% normal goat serum at room temperature for 1 h. Sections then were incubated with an antibody that selectively recognizes dual-phosphorylated (T202/183 and Y204/185), activated ERK1/2 (1:300; Cell Signaling Technology) in 0.1 m phosphate buffer containing 0.4% Triton X-100 and 10% normal goat serum at 4°C for 48–72 h. All perfusion solutions, rinses, and incubation solutions before and during incubation with the primary antibody contained 1 mm NaF and 1 mm orthovanadate, to prevent dephosphorylation of the antigen. After washing in 0.1 m phosphate buffer over 45 min, sections were incubated in biotinylated goat anti-rabbit (Jackson ImmunoResearch, West Grove, PA) in 0.1 m phosphate buffer containing 0.4% Triton X-100 for 60 min, followed by incubation in ABC complex in 0.1 m phosphate buffer containing 0.4% Triton X-100 (Vectastain Elite Kit; Vector Laboratories, Burlingame, CA) for 60 min at room temperature on a rotator. After washing the sections in 0.1 m phosphate buffer, immunostaining was visualized with a substrate solution containing DAB (0.6 mg/ml plus 0.03% H2O2 plus 4 mm NiCl2 in TBS). Sections were washed several times and then mounted on gelatin-coated slides, dried at room temperature, dehydrated in ethanol, cleared in xylene, and coverslipped with Protocol xylene-based medium.
Sections were examined using an Olympus (Tokyo, Japan) light microscope at 100× magnification. To measure task-related pERK expression in the NAcc, cell counts were made from sections taken from approximately +0.96 to +1.8 mm anterior to Bregma. To avoid duplicate counting, successive sections were separated by at least 120 μm. To count cells in the NAcc, a rectangular 400 μm × 200 μm window was placed medial to the anterior commissure for NAcc shell cell counts, and immunopositive cells that fell within the window were counted. A second counting window was placed lateral to the anterior commissure for cell counts in the NAcc core. Neurons were identified based on their dendritic cell morphology (see Fig. 3B,C). Counts were made in both hemispheres.
Rats were anesthetized with an intraperitoneal injection of a combination of ketamine (100 mg/kg; Sigma, St. Louis, MO) and xylazine (10 mg/kg; Ben Venue Laboratories, Bedford, OH). Rats were placed in a stereotaxic apparatus using blunt ear bars, an incision was made on the scalp, and the skin retracted. Two small holes were drilled in the skull at the following coordinates relative to bregma: anteroposterior, +1.5 mm; mediolateral, ±1.7 mm. Two 26-gauge stainless-steel guide cannula (Plastics One, Roanoke, VA) were lowered 5.4 mm ventral relative to the dural surface of the brain. Each cannula was affixed to the skull with cyanoacrylate glue and dental cement. Wire stylets cut to the same length as the guide cannula were inserted into each cannula, and the skin was sutured around the dental cement platform. Rats were given acetaminophen orally (300 mg/kg per day) for 2 d after surgery and allowed 1 week to recover with ad libitum food and water.
During infusion each rat was restrained by hand, the stylets removed, and a 33-gauge infusion cannula (Plastics One, Roanoke, VA) was inserted into the guide cannula. The infusion cannula extended 7.4 mm ventral to the dural surface of the brain (2 mm beyond the ventral tip of the guide cannula). The infusion cannula was attached via PVC tubing to a 10 μl Hamilton syringe. Infusate was delivered over the course of 1 min, after which the infusion cannula was left in the guide for an additional 1 min. The stylet was replaced after the infusion cannula was removed, and the rat was returned to its home cage for 30 min before behavioral testing. To disrupt ERK1/2 signaling, the mitogen-activated protein kinase kinase (MEK) inhibitor U0126 (Upstate, Lake Placid, NY) was infused into the NAcc. U0126 was first dissolved in 100% DMSO and then diluted with 0.9% saline and Tween 80 to a concentration of 2 mg/ml U0126 in 10% DMSO, 10% Tween 80, and 0.9% saline. This dose has been shown to be effective in inhibiting ERK1/2 activation in vivo (Schafe et al., 2000; Miller and Marshall, 2005b). Vehicle infusate contained 10% DMSO and 10% Tween 80 in 0.9% saline. Each subject received 1 μl infusions of either U0126 or vehicle solution before behavioral testing, and the order of infusates was counter-balanced across subjects as described above.
Rats were anesthetized with an intraperitoneal injection of chloral hydrate (300 mg/kg; dissolved in 0.9% saline), and perfused transcardially with PBS followed by 4% paraformaldehyde. The brains were removed after perfusion and placed in 4% paraformaldehyde overnight and then transferred to a 20% sucrose solution for 2–3 d. The brains were frozen and sectioned at 40 μm on a cryostat at −16°C, and the sections mounted on glass slides. The sections were stained with cresyl violet and prepared for microscopy using standard histological techniques. Cannula tracks were identified based on tissue damage.
To examine whether repeated pavlovian conditioning results in NAcc ERK2 activation, rats were killed after either one pavlovian conditioning session or four pavlovian conditioning sessions, and samples of NAcc and dorsal striatum were probed for ERK2 activation. To examine whether the food-associated tone can itself activate ERK2, rats received four sessions of pavlovian conditioning as described above and, on the following day, were presented with either the tone or the training context before anesthesia administration. Rats were either decapitated and NAcc and dorsal striatum tissue samples harvested for Western blot analysis, or perfused transcardially for ERK immunohistochemistry.
To test whether ERK activation in the NAcc plays a role in pavlovian-instrumental transfer, rats first received pavlovian training followed by instrumental training. Before the transfer test rats received bilateral intra-NAcc infusions of either U0126, a selective inhibitor of MEK, the kinase directly and exclusively responsible for activation of ERK, or vehicle. After the transfer test, rats received 4 d of retraining, which consisted of two daily pavlovian and two daily instrumental conditioning sessions. After completion of the retraining sessions, rats received bilateral intra-NAcc infusions of either vehicle or U0126 infusions and were tested for an effect of the pavlovian conditioned cue on instrumental responding, i.e., transfer. The order of infusate delivery for the two tests was counterbalanced (U0126→vehicle; vehicle→U0126) across subjects.
To assess the effects of ERK inhibition on instrumental activity, rats received infusions of either U0126 or vehicle before their performance in an instrumental FR5 session. These tests took place after the rats had completed the pavlovian-instrumental transfer tests and after a day of retraining of the instrumental response with a single FR5 session. Each rat received either U0126 or vehicle 30 min before performing a single FR5 session. Each rat received both infusates, with one noninfusion day separating tests during which rats performed a single FR5 session. The order of infusates was counter-balanced across subjects. After completion of the infusate tests, rats were perfused and their cannula placement verified histologically.
All statistical analyses were performed using the SPSS software package (SPSS, Chicago, IL). For pavlovian approach behavior, the number of photobeam breaks during the tone and pretone intervals was summed for each session. A difference score was calculated for each animal by subtracting tone from pretone beam-break rates for that session. These difference scores provided a measure of discriminated approach and were compared across groups using between-subjects ANOVAs and post hoc pairwise comparisons using t tests with Bonferroni's correction. For the transfer test, the number of lever presses during the tone and equivalent pretone periods were summed for each session and rates (presses per minute) calculated for each subject. Lever press rates during tone and pretone periods were compared for the different infusates using a repeated-measures ANOVA and paired t tests with Bonferroni's correction.
For Western blot analysis, density measures from pERK2 and tERK2 immunoblots were used to calculate a ratio of pERK2 to tERK2 immunoreactivity for each sample. To make comparisons of samples from different membranes and control for differences in overall signal intensity, each sample ratio was normalized to a control sample (whole-brain lysate) that was present on every membrane. The normalized ratio from each sample then was expressed as a percentage of the mean ratio of the control group. Group comparisons were made using between-subject ANOVAs and t tests with Bonferroni's correction.
For analysis of immunohistochemistry data, the number of pERK-positive cells per square mm was derived for each subject based on cell counts. An ANOVA with group as the between-subjects factor and region as the within-subjects factor was performed on cell count totals from each subject, and comparisons of group means were made using t tests with Bonferroni's correction.
Repeated appetitive pavlovian conditioning results in conditioned ERK2 activation in the NAcc
Thirty rats (seven to eight per group) experienced either one or four pavlovian training session(s), at which time approach to the food-cup was measured. A two-factor ANOVA performed on the rate of discriminated approach during training found a significant interaction between the number of training sessions the rat performed and its training group (pavlovian or control) on discriminated food-cup approach (F(1,29) = 6.06, p < 0.05) (Fig. 1A). Pavlovian conditioned rats showed significantly greater discriminated approach during the fourth conditioning session compared with rats from the control group (independent-samples t test, t(12) = 4.56, p < 0.01), and compared with rats that experienced a single pavlovian conditioning session (t(13) = 3.71, p < 0.01).
To examine whether changes in ERK2 activation occur in the NAcc during appetitive pavlovian conditioning, NAcc and DS samples were taken immediately after training and analyzed for ERK2 activation (normalized pERK2 immunoreactivity relative to normalized tERK2 immunoreactivity). A two-factor ANOVA revealed a significant interaction between the number of training sessions the rats experienced and its training group on NAcc ERK2 activation (F(1,29) = 4.92, p < 0.05) (Fig. 1B). Pavlovian conditioned rats showed significantly greater NAcc ERK2 activation during the fourth conditioning session compared with rats from the control group (independent samples t test, t(12) = 2.84, p < 0.05) and compared with rats that experienced a single pavlovian session (t(13) = 3.10, p < 0.05). This increase in activated ERK2 in the NAcc was not attributable to changes in the total amount of ERK2 protein present in samples taken from trained rats, as there was no effect of training session or group on tERK2 levels (p values > 0.1).
In contrast to the NAcc, no changes were observed in ERK2 activation as a result of pavlovian training in the overlying dorsal striatum. A two-factor ANOVA showed no effect of training group or training session on dorsal striatum ERK2 activation (p values > 0.1) (Fig. 1C). Thus, the observed increase in ERK2 activation in the NAcc after repeated conditioning was not the result of a wholesale increase in activated ERK2 throughout the brain; within the striatum, this increase in ERK2 activation appears to be specific to the NAcc.
To determine the duration of the ERK2 signal after the fourth pavlovian conditioning session, 8 rats were killed 3 h after the fourth pavlovian conditioning session. NAcc ERK activation observed 3 h after training did not differ from control animals (independent samples t test, p > 0.1). Rats killed immediately after the fourth training session demonstrated significantly greater NAcc ERK2 activation compared with rats killed 3 h after the fourth session (independent-samples t test, t(13) = 2.44, p < 0.05) (Fig. 1B, right). These data show that the increase in ERK2 activation in the NAcc after the fourth training session is transient, lasting <3 h.
The increase in ERK2 activation after four training sessions could be the result of exposure to a discrete reward-associated stimulus, exposure to the training context, or the result of execution of a conditioned approach response. To distinguish among these possibilities, 16 pavlovian conditioned rats (eight per group) were presented with either the tone or the training context without reward on the day after the last of four training sessions. Rats were divided into a “trained-tone” group that was presented with unrewarded tone presentations, and a “trained-context” group that was presented with only the training context and neither the tone nor the food reward. An additional five rats served as tone controls and received only tone presentations during training. Rats presented with the context alone and rats presented with the tone and the context showed similar overall rates of food-cup approach during the test (trained tone, 5.72 approaches/min; trained context, 5.94 approaches/min). However, among trained rats presented with the tone, a significantly greater number of responses occurred during the tone when compared with the pretone interval, whereas responses were distributed equally across the test session among trained rats that were presented with the context alone (Fig. 2A). This is shown by a significant interaction between group (trained tone or trained context) and interval (pretone or tone) when food cup approach rates were analyzed using an ANOVA with both within- and between-subjects factors (F(1,15) = 5.89, p < 0.05). Immediately after this test, animals were killed to examine ERK2 activation in the NAcc. ERK2 activation in NAcc samples was found to be greater in trained rats presented with the tone (trained-tone group) than in trained rats presented with the context only (trained-context group) or in control rats (single-factor ANOVA: effect of group, F(2,20) = 4.32, p < 0.05; independent samples t tests: trained tone versus trained context, t(14) = 2.68, p < 0.05; trained tone vs tone control, t(11) = 2.66, p < 0.05) (Fig. 2B). No effect of cue presentation was found on tERK2 levels (p values > 0.1), which indicates that the increase in ERK2 activation observed in the trained-tone group was a result of an increase in phosphorylated ERK2 and not caused by a decrease in total ERK2. In contrast to the NAcc, no effect of cue presentation on ERK2 activation was observed in the dorsal striatum (p values > 0.1) (Fig. 2C).
To localize changes in ERK activation within the NAcc after tone presentation, immunohistochemistry for dual-phosphorylated ERK1/2 (pERK) was performed on brain sections taken from trained rats presented with the tone or control rats. Eleven rats experienced four sessions of repeated tone-food pairings (n = 6) or served as tone controls (n = 5) and, 1 d after training, all rats received tone presentations, as described above. Immediately after this test, rats were perfused and their brains prepared for pERK immunohistochemistry. Rats in the trained group showed significantly greater discriminated approach during the test when presented with the tone compared with the control group (trained mean, 2.91 ± 0.93 SEM; control mean, 0.33 ± 0.56; independent samples t test, t(9) = 2.79, p < 0.01). A comparison of cell count totals from the core and shell regions of the NAcc demonstrated an overall effect of training on the number of pERK-positive cells in the NAcc (F(1,10) = 16.77, p < 0.01) (Fig. 3). Comparisons of group means revealed significantly more pERK-positive cells in the NAcc shell (independent-samples t test, t(9) = 2.52, p < 0.05) and the NAcc core (independent-samples t test, t(9) = 2.56, p < 0.05) in trained rats presented with the tone compared with rats in the tone control group. Together with the training-related changes in ERK2 activation in NAcc homogenate described above, these findings suggest that ERK activation in the NAcc core and the NAcc shell is regulated by cues that the animal has associated with reward delivery.
Intra-NAcc infusion of U0126 prevents pavlovian-instrumental transfer
To examine whether ERK activation has a functional role in tone-elicited reward seeking, ERK signaling in the NAcc was disrupted in rats while they performed a pavlovian-instrumental transfer test. Nine rats with guide cannula aimed at the NAcc received pavlovian and instrumental training and then were tested for an effect of pavlovian conditioning on instrumental responding, i.e., positive transfer. Thirty min before the transfer test, rats received infusions of either U0126 or vehicle. Vehicle-treated rats showed positive transfer, i.e., a greater number of active lever responses during the tone compared with responding during an equivalent interval preceding tone onset. In contrast, rats treated with the MEK/ERK inhibitor U0126 showed no evidence of transfer, pressing the lever no more frequently during the tone than in the absence of the tone (Fig. 4A,B). A two-factor repeated-measures ANOVA was performed on active lever rates (responses per minute) with stimulus (tone or pretone) and infusate (U0126 or vehicle) as within-subject factors. A significant effect of stimulus was found (F(1,8) = 10.65, p = 0.01), which indicates a positive transfer effect. There was no effect of infusate (p > 0.1), which indicates that overall lever pressing rate was not affected by the inhibitor. Importantly, there was a significant interaction between stimulus and infusate (F(1,8) = 10.09, p = 0.01), which indicates that when rats were treated with vehicle, the presence of the tone enhanced active lever pressing; however, no enhancement of active lever responding occurred in the presence of the cue when rats were infused with the MEK inhibitor. Pairwise comparisons using paired t tests showed that the number of active lever responses was significantly greater during the tone compared with no-tone periods when rats were treated with vehicle (paired t test, t(8) = 3.72, p < 0.01), and that the number of active lever responses during the tone was significantly greater when rats were treated with vehicle compared with when they were treated with U0126 (paired t test, t(8) = 3.06, p < 0.05).
Performance on the inactive lever was assessed to determine whether U0126 disrupted the rats' ability to discriminate the active from the inactive lever. There was no effect of infusate or tone presentation on inactive lever responding (mean inactive lever response rates per min: vehicle tone, 0.44 ± 0.30; pretone, 0.23 ± 0.23; U0126 tone, 0.44 ± 0.11; pretone, 0.36 ± 0.18; p values >0.1). The overall low rates of inactive lever responding and the lack of an effect of infusion of the MEK inhibitor on inactive lever response rates suggest that U0126 had no effect on the rats' ability to distinguish active from inactive levers, nor did it affect the animals' ability to direct their responses toward the active lever.
The rate of active-lever responding during the 6 min extinction phase preceding the onset of the first tone was examined to determine whether the failure of rats to show an effect of the conditioned cue on instrumental responding after intra-NAcc U0126 infusion was a result of a general decrement in motivation for food. There was no effect of U0126 infusion on active lever response rates during extinction (p > 0.1) (Fig. 4C), which provides indirect evidence that motivation for food was not affected by U0126 infusion. To test directly whether food motivation was disrupted by U0126 infusion, rats received infusions of the respective infusates 30 min before performing under an instrumental FR5 schedule for food pellets. The rate of food responding did not differ whether rats received intra-NAcc U0126 or vehicle infusion before the session, as shown by the lack of an infusate effect on active lever response rates (p > 0.1) (Fig. 4D). These data demonstrate that intra-NAcc infusions of U0126 did not disrupt the rats' general ability to perform instrumental responses, nor did the infusions reduce the rats' motivation to seek out and consume food. Rather, the lack of enhancement of active lever responding during tone presentations when rats were treated with intra-NAcc U0126 strongly suggests a specific disruption in the ability of conditioned cues to modulate instrumental responding in these animals.
In contrast to its effects on pavlovian-instrumental transfer, intra-NAcc U0126 infusion had no effect on conditioned food-cup approach. During the transfer test, rats made significantly more food-cup approaches during the tone than during the pretone interval regardless of infusate condition (mean food-cup approach rate per min: vehicle tone, 6.67 ± 0.33; pretone, 5.06 ± 1.15; U0126 tone, 5.52 ± 0.79; pretone, 3.57 ± 0.76; repeated-measures ANOVA, effect of stimulus, F(1,8) = 16.15, p < 0.01). No effect of infusate was observed on approach rates either during the tone or pretone period (p values > 0.1). The dissociation of effects on pavlovian-instrumental transfer and conditional approach suggests that intra-NAcc infusion of U0126 specifically disrupts the motivational or arousing features of the tone while leaving intact its response-eliciting properties.
ERK activation in the NAcc is increased during pavlovian-instrumental transfer
To examine whether ERK activation in the NAcc is increased during the pavlovian-instrumental transfer test, 13 rats underwent pavlovian and instrumental conditioning as described above. On the day after the last training session, rats were split into two groups. One group (n = 7) was tested for pavlovian-instrumental transfer and the second group (n = 6) was tested for conditioned food cup approach. The pavlovian-instrumental transfer test was performed according to the procedures described above. The test for conditioned approach was identical to the pavlovian-instrumental transfer test, except the levers were retracted during the duration of the test. For rats tested without the levers present there was a significant effect of tone presentation on food-cup approach (repeated-measures ANOVA, F(1,6) = 21.58 p < 0.01) (Fig. 5A). For rats tested with the levers present there was a significant effect of tone presentation on the rate of responding on the previously active lever (F(1,7) = 15.13, p < 0.01) (Fig. 5A).
Rats were perfused after the test and immunohistochemistry for pERK performed on brain sections. A control group (n = 7) was included to provide a baseline of pERK-positive staining with which to compare staining in trained animals. This control group underwent identical training and test procedures as the treatment group except that pellets were never delivered during either the pavlovian or the instrumental training phases. Comparison of the number of pERK-positive cells in the NAcc of the three groups of rats demonstrated an overall effect of test group on the number of pERK-positive cells in the NAcc (F(2,20) = 9.07, p < 0.01) (Fig. 5B). Comparisons of group means revealed significantly more pERK-positive cells, compared with the control group, in rats that underwent the pavlovian-instrumental transfer test (t(12) = 2.97, p < 0.01) and rats that underwent the test for conditioned approach (t(11) = 4.07, p < 0.01). Importantly, the number of pERK-positive cells did not differ between rats that underwent pavlovian-instrumental transfer and those that underwent the test for conditioned approach (p > 0.1). These data demonstrate that the level of active ERK is elevated in the NAcc during pavlovian-instrumental transfer as well as during conditioned approach. Furthermore, these data suggest that CS presentation can increase ERK activation regardless of the particular behavioral responses elicited by the CS.
Verification of infusion sites
Histological verification of infusion sites was made by examining cannula tracks in mounted brain sections. Placement within NAcc was determined based on the termination of cannula tracks and their proximity to the anterior commissure, which passes through the medial portion of the accumbens core (Fig. 4E). All animals showed cannula placement within the accumbens, with the exception of two rats whose placement was posterior to decussation of the anterior commissure, and whose behavioral data were not included in the study.
To verify the efficacy of the U0126 infusion in disrupting ERK phosphorylation in the NAcc, five rats with bilateral cannulas targeting the NAcc received infusions of U0126 in one hemisphere and vehicle in the contralateral hemisphere, and 30 min later were perfused and their brains prepared for analysis for pERK immunoreactivity. In an effort to avoid a floor effect, rats were injected peritoneally with d-amphetamine (1 mg/kg; Sigma) 15 min before perfusion; this treatment was shown previously to elevate pERK levels in the NAcc shell (Valjent et al., 2004). Significantly fewer pERK-positive cells were found in both the core and the shell of the NAcc near the U0126 infusion site compared with the vehicle infusion site (repeated measures ANOVA, shell, F(1,4) = 8.63, p < 0.05; core, F(1,4) = 5.17, p < 0.05) (Fig. 6A). To estimate the regional specificity of effect of U0126, the number of pERK-positive cells was counted in the dorsal striatum and the adjacent lateral septum. No effect of infusion was found on pERK in these structures (p values > 0.1) (Fig. 6A), which indicates that the behavioral effects of U0126 infusion are most likely the result of inhibition of ERK activation within the NAcc.
The aim of these experiments was to determine whether ERK activation in the NAcc has a role in appetitive learning and motivational responses to reward-associated stimuli. We found that repeated pairing of an auditory stimulus with food delivery enables the auditory cue to increase ERK2 activation in the NAcc. This increase in ERK activation occurred during tests of both conditioned approach and pavlovian-instrumental transfer. Furthermore, we found that the ability of the auditory cue to motivate instrumental responding depends on ERK activation in the NAcc. Intra-NAcc infusion of the ERK1/2 inhibitor U0126 prevented the excitatory effects on instrumental responding during cue presentation in the pavlovian-instrumental transfer test. U0126 infusion did not affect approach behavior or the performance of food-rewarded instrumental actions. These data raise the possibility that the motivational/arousal properties of a reward-associated stimulus depend on ERK signaling within the NAcc. These results have important implications for understanding the molecular mechanisms that underlie the motivational control over reward-seeking behavior by stimuli acquired through learning.
Presentation of an appetitive CS activates ERK2 in the Nacc
Our results demonstrate that a motivationally relevant cue can activate ERK2 in the NAcc. The increase in ERK activation in the NAcc after cue presentation is unlikely to represent an acquisition process. We found no increase in ERK2 activation after one training session, when memory acquisition was likely to have begun. Instead, the increase in ERK2 activation in the NAcc emerged only after the animals had acquired the conditioned response (i.e., after the fourth training session). The increase in ERK activation occurred after tests of conditioned approach as well as after the pavlovian-instrumental transfer test, which suggests that this increase is independent of the expression of a particular behavioral response. Rather, the increase in ERK activation appears to occur as a result of presentation of a motivationally relevant reward-associated cue.
We believe that, as a result of learning, the food-associated tone conditionally activates ERK in the NAcc. Such an effect is consistent with the observed increases in neural activity, dopamine release, and immediate-early gene expression within the NAcc that accompany an animal's exposure to a stimulus that has previously been associated with delivery of a rewarding outcome (Bassareo and Di Chiara, 1999; Ito et al., 2000; Schroeder et al., 2001, 2003; Cheng et al., 2003; Phillips et al., 2003; Nicola et al., 2004; Miller and Marshall, 2005a; Roitman et al., 2005; Schiltz et al., 2005; Day et al., 2006). The present data raise the possibility of ERK activation as a possible mechanism for linking CS-driven dopamine (DA) release in the NAcc to changes in immediate-early gene expression (Valjent et al., 2005; Girault et al., 2007). However, the precise mechanism through which CS presentations give rise to increased ERK activation is not known. It may be that each CS phasically increases the level of active ERK in NAcc neurons, or that active ERK protein accumulates gradually in these neurons with each successive CS presentation.
ERK activation in the NAcc is necessary for pavlovian-instrumental transfer
We found that ERK activation in the NAcc plays a crucial role in the ability of a reward-associated stimulus to engage appetitive motivational systems. The motivation to seek out rewards is subject to conditioning, and this conditioned “wanting” reflects the formation of associations between environmental stimuli and affective/motivational features of the appetitive outcome (Dayan and Balleine, 2002; Berridge and Robinson, 2003). The existence of such associations enables appetitive cues to invigorate motor responses aimed at procurement of rewards. The NAcc is critical for this process. NAcc lesions prevent motivational responses during cue presentation, including during pavlovian-instrumental transfer (Parkinson et al., 2000; Corbit et al., 2001; Hall et al., 2001; de Borchgrave et al., 2002). Likewise, DA receptor antagonists and lesions of the ventral tagmental area block pavlovian-instrumental transfer, whereas DA receptor agonists enhance the motivational response to cues and pavlovian-instrumental transfer (Taylor and Robbins, 1984; Cador et al., 1991; Wyvell and Berridge, 2000; Di Ciano et al., 2001; Parkinson et al., 2002; Yun et al., 2004; Murschall and Hauber, 2006).
Our results show that ERK signaling in the NAcc is a critical substrate for mediating the motivational properties of an appetitive cue. The failure of a positive effect of the conditioned cue on instrumental responding after intra-NAcc infusion of U0126 was not a result of an inability of the rats to perceive the auditory cue, because these animals demonstrated conditioned food-cup approach while ERK activation was inhibited. Likewise, the failure of transfer could not be attributed to a general reduction in the animals' motivation for food, because intra-NAcc inhibition of ERK had no effect on food-reinforced instrumental responding. The observed deficit in pavlovian-instrumental transfer appears to reflect a specific disruption in the cue's ability to motivate the rats to engage in behaviors that result in rewarding outcomes. Our results showing increased ERK activation during pavlovian-instrumental transfer raise the possibility that the incentive motivational effect of tone presentation depends on CS-evoked increases in ERK activation. However, the link between changes in ERK activation brought about by CS presentation and the process of positive transfer is not known. It may be that the operations necessary to translate arousal caused by cue presentation into a specific learned motor output are disrupted by ERK inhibition or that the appetitive cue itself has lost its motivational or incentive salience during ERK inhibition.
The mechanism through which ERK activation in the NAcc may result in an enhancement of reward seeking during cue presentation is not known. It may be that ERK activation enables a CS to motivate behavior by altering the excitability of NAcc cells in which this enzyme is activated. For instance, ERK has been shown to phosphorylate the Kv4.2 A-type potassium channel, thereby reducing the outward current mediated by that channel and rendering the cell more responsive to excitatory inputs (Hoffman and Johnston, 1998; Adams et al., 2000; Yuan et al., 2002; Schrader et al., 2006). In this way, ERK activation could enhance the responsiveness of NAcc neurons to excitatory input, thus increasing the likelihood that the animal commits a learned motor response in the presence of an appetitive, motivationally relevant stimulus. Indeed, it has recently been shown that mice expressing a constitutively active form of calcium calmodulin-dependent kinase II in the ventral striatum show reduced excitability in NAcc neurons and an impairment of pavlovian-instrumental transfer (Wiltgen et al., 2007).
Relevance to addiction
The involvement of ERK in cue-elicited reward-seeking behavior has relevance to understanding features of addictive behavior. A major source of drug relapse is exposure to drug-paired cues (Ehrman et al., 1992; Dackis and O'Brien, 2005). One hypothesis for relapse is that drugs of abuse render the brain more susceptible to the motivational effects of reward-predictive cues (Robinson and Berridge, 2003). Our results suggest the possibility that drug-induced alterations in ERK signaling in the NAcc enhance the motivational properties of a CS. Drugs of abuse activate ERK in many brain regions, including in the NAcc (Berhow et al., 1996; Valjent et al., 2000; Brunzell et al., 2003; Valjent et al., 2004). A positive link between ERK activation and drug seeking has been demonstrated in both the NAcc and the central nucleus of the amygdala (Lu et al., 2005, 2006; Miller and Marshall, 2005b; Valjent et al., 2006). Whether ERK activation in the NAcc is involved in reinstatement of drug-seeking behavior by drug-paired cues is not known.
We found that presentation of motivationally relevant reward-paired cues can activate ERK in the NAcc in a regionally and temporally specific manner. Furthermore, we found that ERK activation in the NAcc is necessary for such a cue to motivate reward-seeking behavior. These data begin to illustrate a molecular mechanism within the NAcc for mediating conditioned motivational processes. Furthermore, these data have implications for our understanding of how dysregulation of these molecular mechanisms by repeated exposure to drugs of abuse may result in aberrant behavioral and motivational features that characterize drug addiction.
This work was supported by National Institute of Neurological Disorders and Stroke Grants R01046423 (E.T.) and T3207391 (M.W.S.), and National Institute on Drug Abuse Grant F32019431 (M.W.S.). We thank Sam Golden and Quynh Vo for contributing to this study.
- Correspondence should be addressed to Edda Thiels, Department of Neurobiology, Center for the Neural Basis of Cognition, University of Pittsburgh School of Medicine, 6064 Biomedical Science Tower 3, 3501 Fifth Avenue, Pittsburgh, PA 15260.