Abstract
NMDA receptors (NMDARs) contribute to phasic transmission and synaptic plasticity and are thought to be important for learning. To better understand where NMDAR signaling is necessary for learning, we combined viral genetic strategies with genetic mouse models to investigate the contribution of NMDARs in the dopamine system to appetitive Pavlovian conditioning. NMDAR signaling in dopamine neurons was not required for Pavlovian conditioning; however, NMDARs in D1 dopamine receptor (D1R)-expressing medium spiny neurons (MSNs), which receive input from dopamine neurons, were critical for this type of learning. NMDAR signaling was also required in brain regions that project to dopamine neurons, because removing NMDARs from afferent neurons to the ventral tegmental area (VTA) also prevented learning. This effect was likely attributable to loss of NMDAR signaling in the neurons of the prefrontal cortex (PFC), because learning could be restored in these animals by rescuing NMDAR expression in the PFC. Moreover, removing NMDARs exclusively from the PFC also prevented learning. Our findings suggest that NMDARs in neurons that project to and receive projections from the VTA are necessary for Pavlovian conditioning and specifically implicate the PFC and D1R-expressing MSNs in associative learning.
Introduction
Repeated neuronal activation leads to alterations in synaptic strength that are widely accepted to represent a neuronal correlate of learning (Bliss and Lomo, 1973; Whitlock et al., 2006). Because NMDA receptors (NMDARs) contribute to this process, these receptors are directly implicated in many forms of learning (Malenka and Bear, 2004). Appetitive Pavlovian conditioning is a simple form of learning in which a conditioned stimulus (CS) is repeatedly paired with an unconditioned stimulus (US) such as food. After multiple CS–US pairings, the CS becomes predictive of US delivery and elicits a conditioned response (CR). During appetitive Pavlovian conditioning in mice, a CS (e.g., a lever) is repeatedly paired with food delivery (US), and the CR is an increase in CS-elicited anticipatory behavior, commonly referred to as Pavlovian-conditioned approach (CA) (Kheirbek et al., 2008; Parker et al., 2010). CA reflects the predictive and motivational properties of the CS and serves as a behavioral measure of learning.
The most-implicated neurotransmitter in appetitive Pavlovian learning is dopamine. Rapid and phasic dopamine activity occurs in response to the US and emerges in response to the CS as learning occurs (Pan et al., 2005; Day et al., 2007; Parker et al., 2010). Intact dopamine signaling is necessary for CA in rodents (Parkinson et al., 2002), and evidence suggests that NMDARs in the dopamine system are also important (Di Ciano et al., 2001; Dalley et al., 2005; Stuber et al., 2008). We have demonstrated previously that NMDARs within dopamine neurons are not necessary for the emergence of either CS-elicited phasic dopamine release or CA in mice (Parker et al., 2010). We conclude from these findings that, if NMDAR signaling within the dopamine system is critical for this type of learning, then it must occur within brain regions that project to dopamine neurons, receive dopaminergic innervation, or both.
Midbrain dopamine neurons receive input from many brain regions and send projections to many more (Björklund and Dunnett, 2007; Geisler et al., 2007). The ventral tegmental area (VTA) receives excitatory input from the prefrontal cortex (PFC), the lateral hypothalamus, and the pedunculopontine and laterodorsal tegmental nuclei among others (Geisler et al., 2007). Which of these projections modulate dopamine neuron activity during appetitive Pavlovian learning and whether NMDARs within these afferent brain regions contribute to the emergence of CA is not known. The output of the VTA is divergent and includes projections to the ventral striatum, PFC, and amygdala (Lammel et al., 2008). Whereas the ventral striatum is specifically thought to be important for appetitive Pavlovian conditioning (Parkinson et al., 2000; Di Ciano et al., 2001; Cardinal et al., 2002a; Parkinson et al., 2002), the other output structures of the VTA are also implicated in associative learning (Cardinal et al., 2002b). Moreover, within these output structures there are many different types of neurons (Kreitzer, 2009), and which populations require NMDARs for appetitive Pavlovian conditioning is not known.
Using viral genetic strategies and genetic mouse models, we characterized the requirement of NMDAR signaling in brain regions that project to and receive projections from the VTA during Pavlovian conditioning. By testing this requirement in specific neuronal subtypes, our findings elucidate the circuitry within the dopamine system that underlies associative learning.
Materials and Methods
Animals
All mouse lines used in these experiments were extensively backcrossed to C57BL/6 mice. Both male and female mice were used in all experiments, and all mice were between 8 and 10 weeks of age at the start of experiments. All animal protocols were approved by the University of Washington Institutional Animal Care and Use Committee.
DAT–Cre; NR1–KO mice.
Mice with a conditional allele for Grin1 (Tsien et al., 1996) and Slc6a3Cre/+ (Zhuang et al., 2005) mice were crossed to generate Slc6a3Cre/+; Grin1Δ/lox (KO mice) and Slc6a3Cre/+; Grin1lox/+ (controls) mice as described previously (Zweifel et al., 2008).
GPR88–Cre; NR1–KO mice.
Cre–GFP was targeted to the first coding exon of Gpr88 locus, and the frt-flanked neomycin-resistance gene that was used for positive selection was removed by crossing these mice with Gt(ROSA)26Sor mice expressing Flp recombinase. Gpr88Cre/+;Grin1Δ/+ males were crossed to Grin1lox/lox females to generate Gpr88Cre/+;Grin1Δ/lox KO and Gpr88Cre/+;Grin1lox/+ control mice as described (Beutler et al., 2011).
D1–Cre; NR1–KO mice.
Drd1aCre/+;Grin1Δ/+ males were crossed to Drd1a+/+;Grin1lox/lox females to generate Drd1aCre/+;Grin1Δ/lox KO and Drd1aCre/+;Grin1lox/+ control mice as described (Beutler et al., 2011).
Virus-injected mice.
Grin1lox/lox and Grin1lox/+ animals used in viral experiments were generated by crossing Grin1lox/+ and Grin1lox/lox animals. All animals injected with virus carried at least one copy of a Cre-activated yellow fluorescent protein (YFP) reporter mouse, Gt(Rosa)26Sor–flox–stop–YFP (Srinivas et al., 2001).
Viral injections
For virus-mediated knock-out of Grin1 from VTA afferents, a retrogradely transported CAV–Cre vector was used (Soudais et al., 2004). The virus contains a myc-tagged Cre gene with a nuclear localization signal (NLS), the expression of which is driven by the cytomegalovirus promoter. For virus-mediated rescue of Grin1 in the PFC of animals injected with CAV–Cre in the VTA, an adeno-associated virus (AAV)–fsNR1 virus was used (Beutler et al., 2011). This virus contains a hemagglutinin (HA)-tagged NMDAR1-3a splice variant (Zukin and Bennett, 1995) preceded by a floxed SV40 late poly(A)-addition site and seven in-frame termination codons. Expression of Grin1 is driven by the cytomegalovirus-chicken β-actin (CBA) promoter after Cre-mediated recombination. For virus-mediated removal of Grin1 from the PFC, an AAV–Cre vector was used (Beutler et al., 2011). This virus contains a myc-tagged Cre–GFP fusion gene with a 5′ NLS, whose expression is driven by CBA promoter.
To remove Grin1 from VTA afferents, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 μl of CAV–Cre titered at 2.5 × 109 viral genomes/μl at the following coordinates from bregma: ML, ±0.5 mm; DV, −4.5 mm. The AP coordinate was corrected for the size of the animal according to the following formula: AP = (lambda to bregma distance)/4.21 × −3.5) mm. To restore Grin1 to VTA-projecting PFC neurons of animals injected with CAV–Cre in the VTA, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 μl of AAV–fsNR1 titered at 6.0 × 109 viral genomes/μl at the following coordinates from bregma: ML, ±0.27 mm; DV, −2.35 mm. The AP coordinate was corrected for the size of the animal according to the following formula: AP = (lambda to bregma distance)/4.21 × 1.9) mm. To remove Grin1 from the PFC, Grin1lox/lox and Grin1lox/+ animals were injected bilaterally with 0.5 μl of AAV–Cre at the same coordinates. All animals were allowed 2 weeks to recover before behavioral testing began.
Appetitive Pavlovian conditioning
All training was done in operant conditioning chambers (ENV-307W; Med Associates). All mice were first trained to retrieve food pellets in a single magazine training session in which 10 pellets were delivered at a variable intertrial interval (ITI) of 90 s. After the mice had been trained to retrieve food pellets, they received five sessions of Pavlovian conditioning, which consisted of 25 CS–US pairings in which a 10 s lever presentation was immediately followed by the delivery of a food reward. Importantly, the mice were not required to press the lever to obtain the food reward and did not interact with the lever in this paradigm. This training procedure differed slightly from our previous study in that CS presentation terminated just before instead of just after US delivery (Parker et al., 2010). Head entries to the food receptacle were detected by the disruption of an infrared photobeam. Learning was measured as the increase in CS-elicited head entry rate (CS–HE rate) to the food receptacle relative to baseline (ITI–HE rate). CA was calculated as the difference between CS–HE and ITI–HE rates for each mouse.
Immunohistochemistry
After behavioral experiments, all stereotactically injected animals were killed and perfused as described (Hnasko et al., 2006). Briefly, animals were given a lethal dose of Beuthenasia (Schering-Plough) and transcardially perfused with PBS, followed by 4% paraformaldehyde (PFA) in PBS. Brains were removed, postfixed in PFA overnight, cryoprotected in 30% sucrose, and frozen. Free-floating coronal sections (30 μm) were immunostained. For animals injected with CAV–Cre in the VTA, viral targeting and expression in the VTA and PFC was confirmed by immunostaining with a mouse anti-tyrosine hydroxylase (TH) antibody (1:1000; Millipore) and a rabbit anti-GFP antibody (1:1000; Invitrogen) to confirm expression of the Cre-dependent YFP reporter. Cy3- and Cy2-conjugated secondary antibodies were used (1:200; Jackson ImmunoResearch). For animals injected with AAV1–fsNR1 in the PFC, viral targeting and expression in the PFC was confirmed by immunostaining with a mouse anti-HA antibody (1:1000; Applied Biological Materials) and rabbit anti-GFP antibody as above. For animals injected with AAV–Cre, viral targeting and expression was confirmed in the PFC by immunostaining with rabbit anti-GFP. Images were acquired on a Nikon Eclipse E600 microscope.
Statistics
All statistical comparisons (two-way ANOVA) and all graphical representations were generated using GraphPad Prism (GraphPad Software). In all cases, statistical interactions (CS vs ITI or genotype × day) are reported. Statistical values are not included in the figures when comparisons were not significant.
Results
NMDAR signaling in dopamine neurons is not required for appetitive Pavlovian conditioning
Although NMDAR signaling contributes to the burst firing of dopamine neurons (Zweifel et al., 2009) and phasic dopamine release that occurs during learning, we have previously demonstrated that NMDARs in dopamine neurons are not required for the development of CS-elicited dopamine release or CA (Parker et al., 2010). To validate this experiment, we generated mice lacking NMDARs in dopamine neurons by crossing mice with a conditional allele encoding the essential NR1 subunit of the NMDAR with mice expressing Cre-recombinase (Cre) from the dopamine transporter gene (Slc6a3) locus (DAT–Cre; NR1–KO mice) (Fig. 1A). We used a conditioning paradigm that was slightly modified from our previous study (see Materials and Methods) and more similar to those used by other laboratories (Day et al., 2007; Flagel et al., 2011). As before, learning was measured as an increase in CS-elicited head-entry (CS–HE) rate to the food receptacle relative to basal intertrial interval head-entry (ITI–HE) rate. This pattern of responding has been termed “goal-tracking” (as opposed to “sign-tracking”) and is thought to reflect the learned predictive value of the CS (Flagel et al., 2011). This distinction is noteworthy because these different response patterns are thought to reflect different learning strategies driven by divergent neural substrates (Flagel et al., 2011). Nonetheless, both sign-tracking and goal-tracking responses are valid behavioral metrics for monitoring the process of learning. Despite using a slightly modified training procedure, we again found that both DAT–Cre; NR1–KO and control mice increased their CS–HE rate relative to their basal ITI–HE rate (Fig. 1B,C) (data represent mean ± SEM; all statistical comparisons and subject numbers are reported in the figure legends). This similar pattern of responding to the CS resulted in equivalent increases in their CA scores during training (Fig. 1D). These observations confirm that NMDAR signaling within dopamine neurons is not required for learning in this paradigm.
Mice lacking NMDARs in dopamine neurons learn normally during Pavlovian conditioning. A, Gray dots in the midbrain indicate that only the dopamine neurons in this brain region lack NMDARs in DAT–Cre; NR1–KO mice. B, Control mice (n = 11) increase their CS–HE rate relative to their baseline ITI–HE rate during training (CS vs ITI; F(4,80) = 6.2, ***p < 0.001). C, DAT–Cre; NR1–KO mice (n = 11) also increase their CS–HE rate relative to ITI–HE rate (CS vs ITI; F(4,64) = 6.2, ***p < 0.001). D, Both groups increase their CA during learning.
NMDARs in striatal medium spiny neurons and D1R-expressing neurons are necessary for appetitive Pavlovian conditioning
Because NMDARs are not required in dopamine neurons, we predicted that NMDAR signaling might be important in brain regions that are modulated by dopamine release. During Pavlovian conditioning, neurons from the VTA release dopamine in the ventral striatum, which modulates glutamate transmission in this brain region to generate appropriate action sequences in response to environmental cues (Redgrave et al., 1999). Evidence suggests that NMDAR signaling in the ventral striatum is important for the development of cue-dependent anticipatory behavior during Pavlovian conditioning (Di Ciano et al., 2001). However, there are many types of neurons within the striatum (Kreitzer, 2009), and which populations specifically require NMDARs for this behavior is not known. To assess the specific contribution of NMDAR signaling within medium spiny neurons (MSNs) to the development CA, we generated mice lacking NMDARs in all MSNs by crossing mice with a conditional Grin1 allele to mice expressing Cre from the Gpr88 gene locus (GPR88–Cre; NR1–KO mice) (Fig. 2A). GPR88 expression has been shown to be specific to D1- and D2-expressing striatal MSNs, with no detectable expression in other striatal cell types (Massart et al., 2009), and striatal NR1 expression is significantly reduced in GPR88–Cre; NR1–KO mice (Beutler et al., 2011).
Pavlovian conditioning is disrupted in mice lacking NMDARs in striatal MSNs and D1R-expressing neurons. A, Large gray dots indicate the brain region lacking NMDARs (striatal MSNs) in GPR88–Cre; NR1–KO mice. B, Control mice (n = 9) increased their CS–HE rate relative to their ITI–HE rate (CS vs ITI; F(4,64) = 12.6, ***p < 0.001). C, CS–HE and ITI–HE rates did not differ during training in GPR88–Cre; NR1–KO mice (n = 9). D, CA score increased in control but not GPR88–Cre; NR1–KO mice during training (KO vs control; F(4,64) = 5.4, ***p < 0.001). E, Small gray dots indicate that D1R-expressing neurons lack NMDARs in D1–Cre; NR1–KO mice. F, Control mice (n = 11) increased their CS–HE rate relative to ITI–HE rate during training (CS vs ITI; F(4,80) = 6.5, ***p < 0.001). G, CS–HE and ITI–HE rates did not differ during training in D1–Cre; NR1–KO mice (n = 9). H, CA score was significantly attenuated in D1–Cre; NR1–KO mice during training (KO vs control; F(4,72) = 3.4, *p < 0.05).
Unlike controls, GPR88–Cre; NR1–KO mice did not develop an increase in CS–HE rate relative to their ITI–HE rate (Fig. 2B,C). This observation indicates that GPR88–Cre; NR1–KO did not learn the predictive value of the CS because there was no increase in their CA score (Fig. 2D) and that NMDAR signaling, specifically in MSNs, is critical for Pavlovian conditioning.
NMDAR-dependent signaling and long-term potentiation in striatal neurons is modulated by D1Rs (Levine et al., 1996; Kerr and Wickens, 2001), and the phasic dopamine activity that occurs during Pavlovian conditioning is believed to specifically engage the lower-affinity D1Rs (Richfield et al., 1989; Hong and Hikosaka, 2011). Consistent with D1R involvement in Pavlovian conditioning, D1R–KO mice or wild-type mice given a D1R antagonist do not develop CA during training (Parker et al., 2010). Because D1R signaling is necessary for associative learning and because NMDAR signaling within striatal MSNs is critical for Pavlovian conditioning, we hypothesized that NMDAR signaling specifically within D1R-expressing cells would be important for Pavlovian conditioning. To test this prediction, we created mice selectively lacking NMDARs in D1R-expressing neurons by breeding mice with a conditional Grin1 allele to mice expressing Cre from the Drd1a gene locus (D1–Cre; NR1–KO mice) (Fig. 2E). Similar to the GPR88–Cre; NR1–KO mice, D1–Cre; NR1–KO mice also did not develop CA when compared with controls (Fig. 2F–H). This observation suggests that NMDAR signaling specifically in D1R-expressing neurons is critical for Pavlovian conditioning and that NMDARs in D2R-expressing MSNs alone is not sufficient for this type of learning. The combined results support the hypothesis that D1R-expressing MSNs integrate dopamine and glutamate via D1R and NMDAR signaling (Levine et al., 1996; Kerr and Wickens, 2001), respectively, to facilitate learning during Pavlovian conditioning.
Removing NMDARs from the VTA and its neuronal inputs prevents appetitive Pavlovian conditioning
CS-elicited dopamine release increases during Pavlovian conditioning in mice lacking NMDARs in dopamine neurons (Parker et al., 2010). Therefore, we predicted that NMDAR signaling within brain regions that modulate VTA dopamine neuron activity might also be important for this type of learning. To test this hypothesis, we bilaterally injected canine adenovirus type 2 expressing Cre (CAV–Cre) into the VTA of mice with a conditional Grin1 allele (CAV–VTA; NR1–KO mice). CAV–Cre has been shown previously to preferentially transduce neurons and to be efficiently trafficked via retrograde axonal transport (Soudais et al., 2004; Hnasko et al., 2006). Therefore, we predicted that this strategy would lead to Cre expression and NMDAR inactivation in the VTA as well as in brain regions that project to the VTA (Fig. 3A). As indicated by the expression of a YFP reporter gene in both TH-immunopositive and TH-immunonegative neurons, we observed Cre-mediated recombination in both dopamine and non-dopamine neurons of the VTA, respectively (Fig. 3B,C). To confirm efficient retrograde transport of CAV–Cre, we examined YFP expression in brain regions that are known to project to the VTA. We observed robust YFP expression within the prelimbic and infralimbic regions of PFC (Fig. 3D), a finding that is consistent with the selective transduction of VTA-projecting PFC neurons (Carr and Sesack, 2000; Geisler et al., 2007).
Mice lacking NMDARs in the VTA as well as in VTA-projecting brain regions failed to learn during Pavlovian conditioning. A, Viral injection strategy for CAV–VTA; NR1–KO mice. Solid green shading indicates that most cells within the VTA lack NMDARs, and partially shaded green indicates that only the VTA-projecting neurons lack NMDARs in brain regions outside of the VTA. B, C, CAV–Cre injection is selective for the VTA and leads to viral transduction and activated Cre expression (green) in both TH-positive (red) and TH-negative neurons. Scale bars: B, 500 μm; C, 100 μm. D, CAV–Cre injection into the VTA transduces and activates Cre expression (green) in VTA-projecting PFC neurons. Scale bars: Top, 1 mm; bottom, 200 μm. Blue, DAPI stain. E, Control mice (n = 8) increase their CS–HE rate relative to ITI–HE rate during training (CS vs ITI; F(4,56) = 5.9, **p < 0.01). F, CAV–VTA; NR1–KO mice (n = 6) fail to increase their CS–HE rate relative to ITI–HE rate. G, Control but not CAV–VTA; NR1–KO mice increase their CA score during training (control vs KO; F(4,48) = 5.9, ***p < 0.001).
When we tested CAV–VTA; NR1–KO mice in our Pavlovian conditioning paradigm, we found that the acquisition of CA was significantly attenuated in these animals (Fig. 3E–G). These results suggest that, although NMDARs in dopamine neurons are not required for Pavlovian conditioning, intact NMDAR signaling in neurons that project to VTA dopamine neurons is critical for this type of learning.
NMDARs in the PFC are necessary for appetitive Pavlovian conditioning
If NMDAR signaling within neurons that project to the VTA is critical for learning, we predicted that removing these receptors from one of these input brain regions would disrupt Pavlovian conditioning. To test this prediction, we designed a viral injection strategy to bilaterally remove NMDAR signaling from the prelimbic region of the PFC. We selected this brain region because it sends a dense glutamatergic projection to the VTA (Geisler et al., 2007) and because it had robust CAV–Cre transduction in CAV–VTA; NR1–KO mice. Moreover, stimulation of the PFC activates VTA neurons and causes dopamine release in the ventral striatum, an effect that depends on the VTA (Karreman and Moghaddam, 1996; Moorman and Aston-Jones, 2010).
To selectively remove NMDARs from the PFC, we injected AAV1 expressing Cre (AAV–Cre) into the PFC of mice with a conditional Grin1 allele to generate AAV–PFC; NR1–KO mice (Fig. 4A). Cre-mediated recombination in the PFC of AAV–PFC; NR1–KO mice was evident by cytosolic YFP expression (Fig. 4B,C). In agreement with a contribution of NMDAR signaling in the PFC to Pavlovian conditioning, AAV–PFC; NR1–KO mice did not develop a CA response (Fig. 4D–F). These results indicate that NMDAR signaling in PFC neurons during Pavlovian conditioning is necessary for learning in this paradigm.
Mice lacking NMDARs in the PFC do not learn during Pavlovian conditioning. A, Viral injection strategy for generating AAV–PFC; NR1–KO mice. Solid green shading indicates that most cells within PFC injection site lack NMDARs. B, C, AAV–Cre injection into the PFC transduces and activates Cre-mediated recombination in his brain region. Scale bars: B, 1 mm; C, 200 μm. D, Control mice (n = 8) increase their CS–HE rate relative to ITI–HE rate during training (CS vs ITI; F(4,56) = 2.9, *p < 0.05). E, AAV–PFC; NR1–KO mice (n = 6) fail to increase their CS–HE rate relative to ITI–HE rate. F, Control but not AAV–Cre; NR1–KO mice increase their CA score during training (control vs KO; F(4,48) = 2.7, *p < 0.05).
Rescuing NMDARs in VTA-projecting PFC neurons restores learning in CAV–VTA; NR1–KO mice
The observation that NMDARs in the PFC are required for Pavlovian conditioning raises the possibility that the NMDAR signaling that is critical for this type of learning occurs within the subpopulation of PFC neurons that project to the VTA. To test this hypothesis, we designed a combinatorial viral injection strategy to remove NMDARs from neurons that project to the VTA and then selectively restore NMDAR signaling only to the population of PFC neurons that project the VTA. We accomplished this by injecting AAV1 vector containing HA-tagged, floxed-stop Grin1 cDNA (AAV–fsNR1) into the PFC of CAV–VTA; NR1–KO mice (Fig. 5A). We have demonstrated previously that virus-encoded NR1 expression occurs selectively in the presence of Cre when using AAV–fsNR1 (Beutler et al., 2011). We predicted that, if NMDAR signaling in VTA-projecting PFC neurons is important for Pavlovian conditioning, then these rescued animals (CAV–VTA; PFC–NR1–Rescue mice) should have restored learning. To confirm that we had rescued NR1 expression in the PFC of these animals, we examined the PFC of CAV–VTA; PFC–NR1–Rescue mice for HA-immunoreactive neurons that also expressed YFP (Cre-positive). Consistent with restored NR1 expression in the PFC of these animals, HA and YFP immunostaining overlapped in this brain region (Fig. 5B–D). During Pavlovian conditioning, CAV–VTA; PFC–NR1–Rescue mice learned at a similar rate to control mice subjected to the same surgical procedures. These results indicate that NMDAR signaling in the subpopulation of PFC neurons that project to the VTA is sufficient for appetitive Pavlovian conditioning.
Selectively restoring NMDARs to the VTA-projecting neurons of the PFC rescues learning in CAV–VTA; NR1–KO mice. A, Top, Diagram of AAV–fsNR1 construct; bottom, viral injection strategy for generating CAV–VTA; NR1–PFC–Rescue mice. Solid green shading indicates that most cells within the VTA lack NMDARs, and partially shaded green indicates that only the VTA-projecting neurons lack NMDARs in brain regions outside of the VTA. Partially shaded red region indicates the selective restoration of NMDARs to VTA-projecting PFC neurons. B, CAV–Cre injection into the VTA transduces and activates Cre expression (green) in VTA-projecting PFC neurons. Scale bar, 200 μm. C, AAV–fsNR1 injection into the PFC activates HA–NR1 expression in the PFC (red). D, Overlay of Cre-activated YFP expression (green) and HA–NR1 staining (red) indicates that NR1 is re-expressed in VTA-projecting PFC neurons. E, Control mice (n = 8) increase their CS–HE rate relative to ITI–HE rate during training (CS vs ITI; F(4,56) = 3.8, **p < 0.01). F, CAV–VTA; NR1–PFC–Rescue mice (n = 7) also increase their CS–HE rate relative to ITI–HE rate during training (CS vs ITI; F(4,48) = 2.7, *p < 0.05). G, Both groups increase their CA score during learning.
Discussion
Determining the precise contribution of dopamine to stimulus–reward learning is an area of ongoing research. Although dopamine has been shown to be critical for the acquisition, consolidation, and expression of the appetitive association (Di Ciano et al., 2001; Dalley et al., 2005; Parker et al., 2010), precisely which brain areas and cell types contribute to each of these processes are incompletely understood. Moreover, whether dopamine encodes the predictive value of the CS, its incentive motivational properties, or both, remains unclear (Berridge and Robinson, 1998). Nonetheless, it is well established that dopamine is involved in appetitive Pavlovian conditioning, and, given the general involvement of NMDARs in learning (Whitlock et al., 2006), it follows that NMDAR signaling within the dopamine system is likely to be important for appetitive Pavlovian learning. Because NMDAR signaling can be abolished in distinct neuronal populations by inactivating its obligatory NR1 subunit, we were able to use a combinatorial genetic approach to inactivate NMDAR signaling as a way to better understand the functional circuitry of the dopamine system during appetitive Pavlovian conditioning. It is important to note that, because our behavioral readout represents the integration of all of the processes that underlie learning (e.g., acquisition, consolidation, and expression), we can only conclude from our results that some aspect of appetitive Pavlovian conditioning is disrupted by our genetic manipulations.
Our initial observation that NMDARs in dopamine neurons are not necessary for appetitive Pavlovian conditioning led us to hypothesize that NMDAR signaling within brain regions that project to dopamine neurons, receive dopaminergic innervation, or both would be important for this type of learning. To address these possibilities, we first focused on striatal neurons that receive dense dopaminergic innervation. Given that NMDAR signaling in the ventral striatum is important for appetitive Pavlovian conditioning and given the importance of dopamine (Di Ciano et al., 2001), it follows that NMDARs in the subpopulation of striatal neurons that densely express dopamine receptors (the MSNs) would be important for this type of learning. However, the specific requirement for NMDARs in MSNs for appetitive Pavlovian conditioning had not been directly demonstrated. For instance, there are many other types of neurons within the striatum that express NMDARs, including aspiny GABAergic and cholinergic neurons (Kreitzer, 2009). The cholinergic neurons, in particular, have been implicated in processing information regarding CS–US relationships (Joshua et al., 2008). Our finding that NMDARs in MSNs are necessary for appetitive Pavlovian conditioning highlights the importance of these neurons for integrating information about predictive environmental cues to engage in anticipatory behavior.
MSNs can be subdivided into at least two classes: D1R- and D2R-expressing MSNs (Kreitzer, 2009). Because dopamine has been shown to modulate glutamatergic input to both classes of MSNs (Shen et al., 2008), NMDAR signaling in either or both could be important for appetitive Pavlovian conditioning. However, we already demonstrated that D1R signaling is required for this type of learning (Parker et al., 2010), so we predicted that NMDAR signaling within the D1R-expressing MSNs would be particularly important. Our finding that D1–Cre; NR1–KO mice had a deficit in appetitive Pavlovian conditioning further highlights the importance of D1R-expressing neurons for this type of learning by showing that NMDAR in addition to D1R signaling is required in these neurons. Although our data suggest that NMDARs in D2R-expressing neurons were not sufficient for appetitive Pavlovian conditioning, future studies could determine the necessity of NMDAR signaling within the D2R subpopulation of neurons and better characterize the contribution of these MSNs to this type of learning.
We next sought to expand on the observation that mice lacking NMDARs in dopamine neurons develop CS-elicited dopamine release and learn at the same rate as controls during appetitive Pavlovian conditioning (Parker et al., 2010). These results suggest that the neurons that were driving VTA dopamine neuron activity during learning had formed the CS–US association. Therefore, we predicted that NMDAR signaling within brain regions that project to the VTA would be important for this type of learning. We explored this possibility using CAV–VTA; NR1–KO mice and found that this was indeed the case. When we subsequently asked which VTA-projecting brain regions required NMDAR signaling, we found that NMDARs in the PFC were critical for appetitive Pavlovian conditioning. Moreover, our experiments allowed us to specifically pinpoint the subset of PFC neurons within the prelimbic area that project to the VTA as being important for learning in this paradigm.
The observation that the PFC projects to the ventral striatum in addition to the VTA raises some additional points for consideration (French and Totterdell, 2002). For instance, it is not known whether the VTA-projecting and striatum-projecting PFC neurons are the same population of cells. Therefore, it is possible that PFC–striatum connectivity is also disrupted in CAV–VTA; NR1–KO mice, and this could account for the learning deficit in these animals. It has also been reported that most of the VTA dopamine neurons that are innervated by the PFC actually project back to the PFC rather than to the ventral striatum (Carr and Sesack, 2000). This same study also reported that the VTA neurons that receive input from the PFC and send projections to the ventral striatum are GABAergic rather than dopaminergic neurons (Carr and Sesack, 2000). In the context of these findings, our data suggest that VTA–PFC interconnectivity is important for forming appetitive Pavlovian associations and that the PFC may, in turn, influence behavior by directly or indirectly modulating the activity of the ventral striatum. Future studies will be important to better understand the interconnectivity of the PFC with other brain structures to assess how these connections contribute to learned behavior.
Important afferents to the PFC include glutamatergic projections from the basolateral amygdala and the ventral subiculum of the hippocampal formation (Bacon et al., 1996; Carr and Sesack, 1996). Both of these brain regions are critical for memory formation and both also send projections to the ventral striatum (French and Totterdell, 2002). Moreover, the amygdalar projection to the PFC is implicated in cue-based learning (McGinty and Grace, 2008). It seems likely that the crosstalk between all of these structures is important for modulating dopamine and MSN transmission to generate learned behavior (Cardinal et al., 2002b). Because all of the aforementioned brain regions are implicated in Pavlovian conditioning, our current findings behaviorally corroborate previous anatomical and electrophysiological studies characterizing the involvement of the PFC in a corticobasal ganglia reward network (McGinty and Grace, 2008; Sesack and Grace, 2010).
Although we report here that attenuating NMDAR signaling within the PFC disrupts associative learning, NMDAR hypofunction (Mohn et al., 1999; Coyle, 2006), particularly within this brain region (Akbarian et al., 1996), has been hypothesized to contribute to many of the symptoms observed in schizophrenia. Although it is not precisely known whether the NMDAR hypofunction in the PFC observed during schizophrenia occurs in glutamatergic pyramidal neurons, GABAergic interneurons, or both (Rolls et al., 2008), our results raise the possibility that some of the cognitive deficits in schizophrenic patients may be attributable to NMDAR hypofunction within the subpopulation of PFC neurons that project to the VTA (Sesack and Carr, 2002). Using behavior to better understand the roles of these different populations of PFC neurons based on their anatomical connectivity as well as genetic identity will aid in our understanding of this brain region and provide more insights into therapeutic interventions for the complex psychiatric illnesses that result from its dysfunction.
We conclude from our findings that NMDARs and, presumably, NMDAR-dependent synaptic plasticity within neurons that project to and receive projections from VTA dopamine neurons, are important for appetitive Pavlovian learning. Our data specifically implicate D1R-expressing MSNs and VTA-projecting PFC neurons in this process. These experiments pinpoint the specific brain regions and cell types involved in the acquisition and maintenance of learned associations and offer a more detailed representation of the functional connectivity of the corticobasal ganglia reward network and how this network influences learning and behavior.
Footnotes
This work was supported in part by a National Institute on Drug Abuse Grant T32 DA07278-14 (J.G.P.) and by the Achievement Rewards for College Scientists Foundation (L.R.B.). We thank Paul E. M. Phillips, Nigel S. Bamford, and Matt E. Carter for thoughtful discussion of this manuscript. We also thank Charlie Henschen and Patricia Salles for assistance with mouse breeding and histology.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Richard D. Palmiter, Howard Hughes Medical Institute and Department of Biochemistry, Box 357370, University of Washington, Seattle, WA 98195. palmiter{at}uw.edu
