Abstract
The survival of an organism is dependent on its ability to respond to cues in the environment. Such cues can attain control over behavior as a function of the value ascribed to them. Some individuals have an inherent tendency to attribute reward-paired cues with incentive motivational value, or incentive salience. For these individuals, termed sign-trackers, a discrete cue that precedes reward delivery becomes attractive and desirable in its own right. Prior work suggests that the behavior of sign-trackers is dopamine-dependent, and cue-elicited dopamine in the NAc is believed to encode the incentive value of reward cues. Here we exploited the temporal resolution of optogenetics to determine whether selective inhibition of ventral tegmental area (VTA) dopamine neurons during cue presentation attenuates the propensity to sign-track. Using male tyrosine hydroxylase (TH)-Cre Long Evans rats, it was found that, under baseline conditions, ∼84% of TH-Cre rats tend to sign-track. Laser-induced inhibition of VTA dopamine neurons during cue presentation prevented the development of sign-tracking behavior, without affecting goal-tracking behavior. When laser inhibition was terminated, these same rats developed a sign-tracking response. Video analysis using DeepLabCutTM revealed that, relative to rats that received laser inhibition, rats in the control group spent more time near the location of the reward cue even when it was not present and were more likely to orient toward and approach the cue during its presentation. These findings demonstrate that cue-elicited dopamine release is critical for the attribution of incentive salience to reward cues.
SIGNIFICANCE STATEMENT Activity of dopamine neurons in the ventral tegmental area (VTA) during cue presentation is necessary for the development of a sign-tracking, but not a goal-tracking, conditioned response in a Pavlovian task. We capitalized on the temporal precision of optogenetics to pair cue presentation with inhibition of VTA dopamine neurons. A detailed behavioral analysis with DeepLabCutTM revealed that cue-directed behaviors do not emerge without dopamine neuron activity in the VTA. Importantly, however, when optogenetic inhibition is lifted, cue-directed behaviors increase, and a sign-tracking response develops. These findings confirm the necessity of dopamine neuron activity in the VTA during cue presentation to encode the incentive value of reward cues.
Introduction
Associative learning strategies are used daily by humans and animals alike to make situational decisions. Such strategies often rely on cues, or stimuli, in the environment to guide behavior and can directly impact the survival of an organism. In rodents, individual differences in cue-motivated behaviors can be captured using a Pavlovian conditioned approach (PavCa) paradigm, wherein presentation of a discrete cue (conditioned stimulus [CS]) is followed by delivery of a food reward (unconditioned stimulus [US]) (Flagel et al., 2009). Following PavCA training, two distinct phenotypes emerge: goal-trackers (GTs) and sign-trackers (STs) (Hearst, 1974; Boakes, 1977; Robinson and Flagel, 2009). While both GTs and STs attribute predictive value to the reward cue, STs also attribute incentive value to the cue. The attribution of incentive motivational value, or incentive salience, transforms the cue itself into an attractive and desirable stimulus (Berridge and Robinson, 2003). For STs, both food- and drug-associated cues gain appreciable incentive value and thereby the ability to elicit maladaptive behaviors (Saunders and Robinson, 2010, 2011; Yager and Robinson, 2013; Yager et al., 2015). The ST/GT model, therefore, can be harnessed to elucidate the neurobiological mechanisms that encode the predictive versus incentive value of reward cues. Further, this model can help us better understand the neural processes that contribute to shared symptomatology between psychiatric disorders, as an increased propensity to attribute incentive salience to reward cues (i.e., to sign-track) has been associated with externalizing behaviors and deficits in executive control in both rodents and humans (Flagel et al., 2010; Phillips and Sarter, 2020; Colaizzi et al., 2023).
Dopamine has been implicated in a number of psychiatric disorders, predominantly via its role in learning, attention, and motivation (Nestler and Carlezon, 2006; Howes and Kapur, 2009; Grace, 2016; Volkow et al., 2017). However, the precise role of dopamine remains a subject of debate, especially as it pertains to reward processing and learning about stimuli in the environment (Berridge, 2007; Schultz et al., 2017; Berke, 2018; Lerner et al., 2021). While dopamine has long been considered a prediction error signal (Schultz et al., 1997), used to update the predictive value of reward-cues during associative learning, deficiencies in this theory have been recognized (Saunders et al., 2018; Sharpe et al., 2020; Kutlu et al., 2021; Jeong et al., 2022). Of particular relevance, the ST/GT model previously revealed that intact dopamine signaling is necessary for the attribution of incentive salience to reward cues, or what we refer to here as Pavlovian “incentive learning,” and not the encoding of predictive value alone, or “predictive learning” (Flagel et al., 2011; Saunders and Robinson, 2012; Yager et al., 2015). Both systemic and local (NAc core) blockade of dopamine receptors prevents the acquisition and expression of a sign-tracking conditioned response (CR) with no effect on goal-tracking (Flagel et al., 2011; Saunders and Robinson, 2012). Additionally, using fast-scan cyclic voltammetry, it was shown that dopamine-encoded “prediction error” signals are present in the NAc core of STs, but not GTs (Flagel et al., 2011). Together, these findings led to the conclusion that dopamine encodes the incentive value of Pavlovian reward cues. However, it was not clear from these studies whether dopamine neuron activity precisely at the time of cue presentation is necessary for incentive value encoding and the acquisition of sign-tracking behavior. To address this question, we exploited the temporal resolution of optogenetics. Specifically, we used tyrosine hydroxylase (TH)-Cre rats to selectively inhibit dopamine neurons in the VTA during cue presentation early in Pavlovian training. We found that male TH-Cre Long Evans rats have an inherent tendency to sign-track, and that optogenetic inhibition of VTA dopamine neurons during cue presentation blocks this tendency, without affecting goal-tracking behavior. An in-depth analysis of behavior using DeepLabCutTM (DLC) (Mathis et al., 2018) revealed that the effects of this manipulation were time-locked and specific to cue-elicited incentive motivation.
Materials and Methods
General methods
Subjects
In total, 128 male Long Evans rats were received from a breeding colony maintained by the H.A. laboratory (University of Michigan, Ann Arbor, MI). Only male rats were used for these studies, which are a direct follow-up to prior studies that had been conducted with male rats (Flagel et al., 2011; Saunders and Robinson, 2012). The breeding colony originated in 2013 with 2 TH-Cre male Long Evans rats from the K.D. laboratory (Stanford University, Stanford, CA). The colony has since been maintained by breeding TH-Cre Long Evans male rats with WT Long Evans female rats. Rats were bred and weaned in either the Biological Science Research Building or the Molecular and Behavioral Neuroscience Institute building (University of Michigan, Ann Arbor, MI) and transferred to the S.B.F. laboratory in the Molecular and Behavioral Neuroscience Institute building around postnatal day (PND) 46. Rats were housed under a 12 h light-dark cycle (lights on at 6:00 A.M. or 7:00 A.M. depending on daylight saving time) with climate-controlled conditions (22 ± 2°C) and had ad libitum access to food and water throughout the study. They were pair- or triple-housed before surgery, and single-housed following surgery and for the duration of the experiment. Rats were acclimated to the housing room for a minimum of 1 week before experimenter handling began. Behavioral testing took place during the light-phase between 10:00 A.M. and 4:00 P.M. All procedures followed the Guide for the care and use of laboratory rat (National Academy of Sciences, 2011) and were approved by the University of Michigan Institutional Animal Care and Use Committee.
PavCA
Before assessing PavCA behavior, rats were briefly handled by the experimenters for 2-4 d. As described below, a single pretraining session preceded PavCA sessions. For the 2 d before pretraining, rats received ∼25 banana-flavored grain pellets (each pellet 45 mg; Bio-Serv) in their home cage to acquaint them with the reward.
PavCA testing occurred inside Med Associates chambers (24.1 × 21 × 30.5 cm) located in sound-attenuating boxes equipped with a fan to reduce background noise (see Fig. 2c). The chambers contained a magazine that was connected to a pellet dispenser and placed in the center of one wall 3 cm above the chamber floor. An illuminated retractable lever (i.e., lever-cue) was located either to the left or right of the magazine (Med Associates, ENV-200R2M-6.0), 6 cm above the chamber floor. The magazines used were taller (2 × 6 in) than the standard (2 × 2 in) to allow rats to access food pellets without interference from their headcap and tethers. A white house light was placed at the top of the chamber on the wall opposite to the magazine and lever-cue and remained on for the duration of the session. Magazine entries were recorded by a break of the photograph beam inside the cup. Lever-cue contacts were recorded following a minimum force of 10 g.
Before each session, the rats were transferred to the testing room in their home cage. They were left in the room for a minimum of 30 min to allow them to acclimate. Rats were initially placed into the Med Associates chambers for a single pretraining session. At the start of the pretraining session, the food magazine was baited with two food pellets to direct the rats' attention to the location of reward delivery. During pretraining, the lever-cue remained retracted, and rats received a food pellet in the food magazine on a variable 30 s (range 0-60 s) schedule. There was a total of 25 trials, and the pretraining session lasted ∼12.5 min wherein head entries were recorded, and food pellet consumption confirmed. Following pretraining, rats had a single session of PavCA each day for either 5 (colony characterization) or 6 (optogenetic inhibition) consecutive days. The start of each PavCA session began with a 5-min waiting period followed by the house light turning on which signified the start of the session. As previously described (Meyer et al., 2012; Campus et al., 2019), during PavCA, the illuminated lever-cue (CS) entered the chamber for 8 s; and on retraction, a food pellet (US) was immediately delivered into the adjacent food magazine. PavCA sessions consisted of 25 lever-cue (CS)/food-US trials on a variable 90 s schedule (range 30-150 s). Each session lasted ∼40 min. It was confirmed that all food pellets had been consumed following each session.
Med Associates software recorded the following behaviors during PavCA sessions: (1) number of food magazine contacts made during the 8 s lever-cue presentation, (2) latency to contact the food magazine during lever-cue presentation, (3) number of lever-cue contacts, (4) latency to lever-cue contact, and (5) the number of food magazine entries made during the intertrial interval (ITI, i.e., food magazine contacts made in between lever-cue presentations). Contact and latency data were used to calculate the PavCA Index. As previously described (Meyer et al., 2012), the PavCA Index is a composite measure calculated using the following formula: [Probability Difference Score + Response Bias Score + (–Latency Difference Score)/3]. PavCA Index scores range from −1 to 1, with a score of −1 representing individuals with a CR focused solely on the food magazine during lever-cue presentation (i.e., a “pure” GT) and a score of 1 representing individuals with a CR focused solely on the lever-cue on its presentation (i.e., a “pure” ST) (see Fig. 1a). For colony characterization, the PavCA Index from Sessions 4 and 5 were averaged to assess the frequency distribution of STs (range of 0.5-1) and GTs (range of −0.5 to −1.0), as shown in Figure 1b.
Behavioral characterization of the Long Evans colony
Subjects
To determine the inherent behavioral phenotypes of the transgenic rats, a subset of male Long Evans (n = 57) and TH-Cre (n = 38) rats underwent “baseline” characterization of PavCA behavior (see Fig. 1). The wild type (WT) rats were from 2 generations and 15 litters, and the TH-Cre rats were from 2 generations and 7 litters. Just over half of the WT rats (n = 31) had sham surgery before “baseline” characterization, whereas the remaining WT (n = 26) and TH-Cre rats were left undisturbed before “baseline” characterization.
Sham surgery
Rats were PND ∼90 at the time of surgery. All rats had anesthetic induction with 5% isoflurane (Vet One) delivered via an induction chamber and were given an injection of carprofen (5 mg/kg, s.c.) for analgesia during surgery. Sham surgery consisted of levelling rats in the stereotaxic frame, shaving and cleaning the scalp, and drilling two holes directly above the VTA (bilaterally from bregma, AP −5.76; ML ± 2.98). A 1 µl Hamilton Neuros Syringe was lowered into the holes (from bregma, DV −8.4) and pulled up after 10 min. The surgical site was closed with clips (Stoelting), which were removed 7-10 d following surgery. After the clips were removed, experimenter handling and behavioral testing procedures began.
Statistical analyses
Statistical analyses were conducted with the Statistical Package for the Social Sciences program version 27.0 (IBM). To compare differences in the PavCA Index among the colony (see Fig. 1a), a linear mixed-effects model (LMM) with restricted maximum likelihood estimation was used. This analysis applies multiple covariance structures to the dataset, and the structure with the lowest Akaike information criterion was selected as best fit (Verbeke, 1997; Duricki et al., 2016). LMM was conducted to compare genotypes and groups across Sessions 1-5. Session was used as the repeated variable and genotype/group as the between-subjects variable. For one analysis, the WT group was split into rats that were Naive (WT Naive) and those that received Sham surgery (WT Sham); and for another analysis, the WT groups were collapsed and compared with TH-Cre rats. To assess the differences in the relative proportions of phenotypes between the genotypes in the Long Evans colony, a Fisher's exact test was conducted on the PavCA Index (see Fig. 1b). For all analyses, statistical significance was set at p < 0.05, and Bonferroni post hoc comparisons were made when significant main effects or interactions were detected.
Optogenetic inhibition of the VTA
Subjects
To determine the effects of optogenetic inhibition of dopamine neurons during Pavlovian cue-reward learning (see Fig. 2), 33 male TH-Cre rats from 3 generations and 11 litters were used. Some rats were excluded for not consuming pellets during pretraining (n = 4), poor virus expression/probe placement (n = 8), or head caps coming off prematurely (n = 4). Because of technical issues, Session 4 data were lost for 1 rat in the halorhodopsin group and Session 5 data were lost for 1 rat in the control group. Data from these 2 rats are included in the analyses for other sessions. In total, 17 of 33 rats are included in the behavioral analyses assessing the effects of optogenetic inhibition, with 10 in the halorhodopsin group and 7 in the control group. For DLC analyses, 3 of these rats were excluded because of technical issues with video capturing, resulting in 8 in the halorhodopsin group and 6 in the control group.
Viral vectors
A Cre-dependent inhibitory optogenetic construct halorhodopsin (eNpHR, AAV5-Ef1a-DIO eNpHR 3.0-enhanced yellow fluorescent protein [EYFP] at titer ≥ 1 × 1013 vg/ml, Addgene plasmid #26966) or an empty vector (control, AAV5- Ef1a-DIO EYFP at titer ≥ 1 × 1013 vg/ml, Addgene plasmid #27056) was used. Both plasmids were provided by K.D. and obtained from Addgene.
Virus and optogenetics probe implant surgery
Rats were around PND ∼90 at the time of surgery. All rats had anesthetic induction with 5% isoflurane (Vet One) delivered via an induction chamber and were given an injection of carprofen (5 mg/kg, s.c.) for analgesia during surgery. Rats were placed into a stereotaxic frame, and the scalp was shaved and cleaned. Two holes were drilled directly above the VTA (bilaterally from bregma, AP −5.76; ML ± 2.98). Four additional holes were drilled ± 2 mm ML from bregma; 2.4 mm stainless-steel screws (Plastics One) were secured into the four holes. A Hamilton Syringe (5 µl Model 85 RN, Small Removable Needle, 26s gauge, 2 in, point style 2) was placed into a pump (Harvard Apparatus Pump 11 Elite) and then connected to P50 tubing and a guide cannula (Plastics One) that screwed onto a 10 mm injector (Plastics One). A Cre-dependent inhibitory optogenetic construct (halorhodopsin, eNpHR) or Cre-dependent control virus (EYFP) was bilaterally injected into the VTA at a 10° angle (from bregma, AP −5.76; ML ± 2.98; DV −8.4) at a rate of 100 nl per min over a 10 min period (1 µl total) (see Fig. 2a). The injector remained in place for an additional 10 min. After diffusing, fiber optic implants were inserted 0.3 mm above the injection site at a 10° angle (from bregma, AP −5.76; ML ± 2.98; DV −8.1, see Fig. 2a). Fiber optic implants were made in house and consisted of 200-µm-diameter optic fibers (Thor Labs) inserted into 10.5-mm-long ferrules (Thor Labs). Only fibers >85% laser emittance were used for surgery. The fiber optic implants were secured with acrylic cement (Bosworth New Truliner, Keystone Industries). The plastic screw from a guide cannula (Plastics One) was placed at the most anterior portion of the headcap as the acrylic cement was drying (later used for securing the rats headcap during behavior). A 3 to 4 week period for virus incubation followed surgery (see Fig. 2d). Before testing, the acrylic headcap was covered in black pet safe nail polish (Warren London Pawdicure Dog Nail Polish Pen) to occlude the laser light from illuminating the behavioral chamber.
PavCA sessions and laser parameters
Rats received ∼25 banana-flavored grain pellets (each pellet 45 mg; Bio-Serv) in their home cage for the 2 d before testing. Before each session, the plastic screw at the front of the headcap was connected to a reinforced cannula spring (Plastics One) and the optogenetic probes cleaned with ethanol and bilaterally connected to individual optogenetic cables that were secured with a ceramic mating sleeve (Thor Labs).
Rats had one pretraining session followed by six PavCA sessions. For the first three PavCA sessions (Trials 1-75), rats received photoinhibition of the VTA continuously during the 8 s lever-cue presentation via a 593.5 nm Yellow DPSS Laser (Shanghai Laser & Optics Century) (see Fig. 2b–d). Parameters known to be effective for inhibiting dopamine neurons were used (Gradinaru et al., 2010; McCutcheon et al., 2014). Laser power was calibrated to ∼10 mW/mm2 from the tip of the optogenetic cables before each session. Cables were also tested after each session to ensure laser power was consistent throughout. For Sessions 4-6 (Trials 76-150), rats were connected to the reinforced cannula spring and optogenetic cables as described above, but the laser was turned off (i.e., no photoinhibition occurred on Sessions 4-6). Each rat had three PavCA sessions of “laser on” followed by three PavCA sessions of “laser off,” across six consecutive days.
Video analysis
Videos were obtained from Session 3 of PavCA and additional behavioral metrics were obtained from experimenter observation and using DLC, as described below. DLC allowed us to verify experimenter observations and behavioral data obtained from Med Associates, and enhanced the granularity with which we can assess the effects of optogenetic inhibition on individual rats.
PavCA orienting response
An experimenter assessed whether a rat oriented toward the lever-cue or food magazine for each trial of Session 3. An orienting response was defined as a head movement directed toward the lever-cue or food magazine at any point during the 8.2 s lever-cue presentation. The probability of approaching either the lever-cue or food magazine was then calculated as the number of trials with an orienting response (to either the lever-cue or food magazine)/25. In addition, the percentage of trials on which an orienting response was directed toward the lever-cue, food magazine, or both was determined.
DLC
Session 3 videos were further analyzed using DLC and custom MATLAB (R2021b) scripts. Raw videos were processed in Adobe Premiere Pro to increase contrast and enhance brightness within the behavioral chambers. Videos in this dataset on average contained 35,000 frames, of which 75 frames were extracted for training. Training videos were from Session 1. Labeling of all videos for training was completed by two experimenters who were blind to experimental groups. Each video was manually labeled with the following markers on the rats' body which were used for the planned analyses: tether, nose, left-shoulder, right-shoulder, and tail base (see Fig. 7a). The food magazine in the chamber was also labeled. Training was conducted via DLC 2.1.10.4 downloaded from GitHub (https://github.com/DeepLabCut/DeepLabCut) and installed onto University of Michigan, Great Lakes Computing Cluster. Following training of the network, locations of each marker for images analyzed on separate videos were extracted with a p-cutoff-parameter of 0.8 (see Movie 1). Videos for analysis that failed to meet criteria (>10 outlier frames) were reanalyzed following relabeling of outlier frames and retraining of the network. Three videos were excluded from analyses because of camera angle (1 control rat and 1 halorhodopsin rat) or an occluded camera (1 halorhodopsin rat).
Representative videos from DLC analysis during lever-cue presentation. Representative videos from one rat in the control group and one with lever-cue-paired VTA dopamine inhibition (i.e., eNpHR group) with markers overlaid from DLC. Body part markers with a p-cutoff-parameter of 0.8 were included in the video. Interpolated points for frames without markers are not shown.
Data extracted from Session 3 videos were postprocessed in MATLAB and aligned with Med Associates data. Linear and spline interpolation was applied in addition to DLC filtering to further smooth pose estimation coordinates for frames during which body markers were not visible because of lighting or occlusion. Time points of each lever-cue presentation per trial were extracted from analyzed videos to validate alignment to lever-cue presentation based on Med Associates data. The following behaviors were extracted and analyzed in MATLAB: time in zone (food magazine or lever-cue), head orientation to food magazine or lever-cue, approach behavior (approach bouts) toward food magazine or lever-cue, locomotion, and latency to approach the food magazine. The length of the time bins for illustration and analyses were different across outcome measures to better capture behaviors that were time-locked to cue-presentation or retraction, and/or those that occurred during the ITIs. Further, the reference body part differs for some of the measures as described below. For time in zone, body position was tracked by following the rats' headcap tether throughout the chamber. The tether was chosen as the reference point because of the robustness of this marker tracking, before interpolation.
For heatmap analyses, 8.2 s time bins were chosen to capture each rat's location immediately before lever-cue presentation (see Fig. 7b), during lever-cue presentation (see Fig. 7c), immediately after lever-cue presentation (see Fig. 7d), and during the ITI (see Fig. 7e). This time bin reflects the duration of each lever-cue presentation; the lever is out for 8 s, and it takes 0.1 s for the lever to present and 0.1 s for the lever to fully retract. Each heatmap (time in zone, see Fig. 7b–e) represents the average location of rats across all trials on Session 3, integrated over 123 video frames (8.2 s time bins) for each period of interest. A 2D Gaussian smoothing kernel with a SD of 3 was applied to smooth the heatmap images (see Fig. 7b–e).
Head orientation (see Fig. 8a) was tracked by angle changes between two vectors: (1) from the points between the rats' nose to headcap tether and (2) from the rats' headcap tether to the food magazine. For head orientation, the center of the food magazine was the reference point, and the videos were assessed across 4-s time bins during different phases of the session: 4 s before lever-cue presentation (see Fig. 8b), the first 4 s during lever-cue presentation (see Fig. 8c), 4 s before pellet delivery (see Fig. 8d), the 4 s immediately after pellet delivery (see Fig. 8e), and 4 s during the ITI (see Fig. 8f). These data are illustrated as the average head direction for each individual rat and as group means (see Fig. 8).
For approach behavior (see Fig. 9), body position was tracked by following the rats' nose, the reference body part for this metric, throughout the chamber for the entire session. An approach bout was counted when the rats nose remained within 75 pixels (1 cm) of the lever-cue (see Fig. 9a,b) or food magazine (see Fig. 9c,d) for >15 frames. An approach bout ended when the interbout interval exceeded 15 frames (1 s). Approach directed toward the lever-cue or food magazine was quantified during the 8.2 s lever-cue presentation (see Fig. 9a,c), and during the ITI (i.e., after lever-cue retraction) (see Fig. 9).
Locomotion was captured as the velocity (cm/s) and distance traveled (cm) based on tracking the rats' center of mass, the point between the rat's right and left shoulders. Velocity was quantified for the 4 s period immediately before and during lever-cue presentation (see Fig. 10a). The distance from the lever-cue during the 2 s period preceding lever-cue presentation and the first 2 s during lever-cue presentation was also assessed (see Fig. 10b). Upon lever-cue retraction, the latency to approach the food magazine (see Fig. 10c) was analyzed as an index of activity when dopamine neurons were no longer inhibited.
Code accessibility
DLC settings, desktop parameters, and code for performing post-processing reconstructions and analysis are publicly available on GitHub: https://github.com/alvchiu/THCREOpto-dlc.
Perfusion and tissue processing
Rats were perfused within 5 d following the experiment. Rats were first anesthetized with ketamine (90 mg/kg, i.p.) and xylazine (10 mg/kg, i.p.) and then transcardially perfused with 0.9% saline and 4% formaldehyde, pH 7.4. Following brain extraction, the tissue was postfixed in 4% formaldehyde for 24 h at 4°C and then placed in 30% sucrose at 4°C (sucrose in 0.1 m PBS, pH 7.4) for 3 d. The brains were frozen using dry ice and coated in a Tissue-Plus Optimal Cutting Temperature compound (Fisher HealthCare). Coronal brain slices were taken at 40 µm using a cryostat at −20°C (Leica Biosystems). The whole brain was collected, and slices were placed into well plates containing cryoprotectant and then stored at −20°C. Slices with the VTA were isolated, mounted onto SuperFrost Plus microscope slides (Fisher Scientific), and coverslipped with DAPI as a counterstain (diluted 1:5000 in 90% glycerol). Images were captured using a Zeiss AxioImager M2 motorized fluorescent microscope (Carl Zeiss). Fluorescent images of endogenous virus expression and optogenetic probe placement were evaluated by two experimenters blind to the experimental groups (see Fig. 2d, representative image). Virus expression was evaluated based on distinct localized cell body expression of EYFP (virus tag) within the VTA (see, e.g., Fig. 2d) and probe placements were confirmed if they were visualized bilaterally within the VTA (see Fig. 3b,c).
Histology
Virus expression in the VTA was evaluated following immunofluorescent staining of TH. Immunohistochemical procedures took place at room temperature, and each step was followed by three washes of 0.1 m PBS for 5 min each. Sections were blocked with 2.5% normal donkey serum (Jackson ImmunoResearch Laboratories) + 0.4% Triton X-100 + 0.1 m PBS for 1 h, then incubated overnight in the primary antibody solution (rabbit anti-TH, Abcam, ab16453) diluted 1:500 in 0.1 m PBS + 0.4% Triton X-100 + 1% normal donkey serum. The next day, sections were incubated in the secondary antibody solution containing biotinylated donkey anti-rabbit antibody (Jackson ImmunoResearch Laboratories, 711-065-152), diluted 1:500 in 0.1 m PBS + 0.4% Triton X-100 + 1% normal donkey serum, for 2 h. Sections were incubated with Streptavidin, AlexaFluor-594 conjugated (Fisher Scientific, S11227), diluted 1:1000 in 0.1 M PBS for 1 h. Slides were mounted and coverslipped as described above. Z-stack and images were captured using a FV3000 confocal microscope and the FV31S-SW Viewer software (Olympus Microscopes). Single-channel and triple-labeled fluorescent images of DAPI (cobalt), TH (red), and EYFP (green) to stain nuclear DNA are represented in Figure 3a.
Statistical analyses
A linear mixed model (LMM) with restricted maximum likelihood estimation was used to assess PavCA behavioral outcome measures across sessions. When two sessions were directly compared, a two-way ANOVA or t test was performed, as described below. A simple linear regression analysis was used to test whether the Acquisition Index (Δ PavCA Index between Sessions 3 and 1) predicted the Final Index (the PavCA Index on Session 6). For all analyses, statistical significance was set at p < 0.05, and Bonferroni post hoc comparisons were made when significant main effects or interactions were detected. Effect size (Cohen's d) (Cohen, 1988) was calculated for pairwise comparisons. Effect sizes were considered with respect to the following indices: 0.2, small; 0.5-0.8, medium; 1.2-2.0, large (Cohen, 1988; Sawilowsky, 2009).
LMM was conducted to compare experimental groups across Sessions 1-3 (“laser on”) or Sessions 4-6 (“laser off”). That is, Sessions 1-3 or Sessions 4-6 were used as the repeated variable, and experimental group (control vs halorhodopsin) was used as the between-subjects variable. For lever-directed behaviors, a LMM was also conducted to compare control Sessions 1-3 (“laser on”) to halorhodopsin Sessions 4-6 (“laser off”). A two-way ANOVA was conducted when Session 3 (“laser on”) was directly compared with Session 6 (“laser off”), with session (3 and 6) as the within-subject independent variable and experimental group (control or halorhodopsin) as the between-subject independent variable. Differences in the number of lever-cue contacts (dependent variable) between Sessions 1 and 4 were analyzed using an unpaired t test (control Session 1 vs halorhodopsin Session 4) or a paired t test (control Session 1 vs control Session 4 or halorhodopsin Session 1 vs halorhodopsin Session 4). For both the LMM and ANOVA, lever-directed behaviors (number of lever-cue contacts, probability to approach the lever-cue, latency to approach the lever-cue), food magazine-directed behaviors (food magazine contacts during lever-cue presentation, probability to approach the food magazine during lever-cue presentation, latency to approach the food magazine during lever-cue presentation), and food magazine entries during the ITI (non-CS food magazine head entries) were used as dependent variables.
Behavioral output from video analyses were also statistically analyzed. A two-way ANOVA was conducted to assess orienting responses directed toward the lever-cue or food magazine on Session 3, with experimental group (control or halorhodopsin) and location (lever-cue or food magazine) as the independent variables. For the data generated by DLC, a Kolmogorov–Smirnov (KS) two-sample test compared the distributions for head direction responses (dependent variable) between experimental groups (control or halorhodopsin) at the following 4-s periods: (1) immediately before lever-cue presentation, (2) the first 4 s of lever-cue presentation, (3) the last 4 s of lever-cue presentation), (4) immediately after pellet delivery, and (5) during the ITI. Differences between experimental groups (control or halorhodopsin) in approach bouts (each bout ≥ 1 s, dependent variable) toward the lever-cue or food magazine (1) during lever-cue presentation and (2) after lever-cue retraction were analyzed using unpaired t tests. Differences between experimental groups (control or halorhodopsin) in locomotor activity and latency to approach the food magazine were analyzed using unpaired t tests. A two-way repeated-measures ANOVA was conducted to compare the velocity of movement during the 4-s period before lever-cue presentation and the first 4 s during lever-cue presentation, with time as the within-subject independent variable and experimental group (control or halorhodopsin) as the between-subject independent variable. A two-way repeated-measures ANOVA was also conducted to compare distance from the lever-cue during the 4-s period immediately before lever-cue presentation and the first 2 s during lever-cue presentation, with time as the within-subject independent variable and experimental group (control or halorhodopsin) as the between-subject independent variable.
Results
Behavioral characterization of the Long Evans transgenic rat colony
PavCA distribution
The tendency to sign- or goal-track (without optogenetic manipulation) was assessed in WT and TH-Cre littermates from our in-house breeding colony. There were no significant differences in the PavCA Index across Sessions 1-5 when comparing WT Naive versus WT Sham versus TH-Cre (Table 1). Further, PavCA Index did not differ significantly across sessions between WT rats that did or did not receive sham surgery (Fisher's exact test, p = 0.60); thus, these groups were collapsed for visualization (Fig. 1a,b) and further analyses. While there were not robust differences between WT and TH-Cre rats across sessions, there was a significant group × session interaction (F(4172.373) = 3.175, p = 0.015; Table 1) and post hoc analyses revealed that TH-Cre rats had a higher PavCA Index on Session 5 relative to WT rats (p = 0.040, Cohen's d = 0.46; Fig. 1a). Out of the total population of rats that were screened (N = 95), ∼84% were STs. Of the WT (n = 57 in total) rats ∼81% were STs and of the TH-Cre (n = 38 in total) rats, ∼89% were STs; but the population distribution between WT and TH-Cre rats did not significantly differ (Fisher's exact test, p = 0.40; Fig. 1b). These data suggest that this colony of Long Evans male rats are skewed toward STs, regardless of genotype. This skew in the population provides an opportunity to assess the impact of neuronal manipulations on the attribution of incentive salience to reward cues and thereby the development of sign-tracking behavior.
The population of Long Evans male rats are skewed toward sign-trackers (STs). a, PavCA Index shown as mean ± SEM across five training sessions for WT rats (n = 57) and naive TH-Cre rats (n = 38). b, Frequency histogram represents the number of rats exhibiting a mean PavCA Index (averaged from Sessions 4 and 5) between −1.0 and 1.0 for each of the groups depicted. 84% of the population (n = 95) was skewed toward STs (≥0.5, n = 46 WT and n = 34 TH-Cre), 14% of the population were intermediate responders (0.5 ≤ −0.5; n = 10 WT and n = 3 TH-Cre), and 2% GTs (≤−0.5, n = 1 WT and n = 1 TH-Cre).
Statistical analyses for PavCA Index across sessions 1-5a
Optogenetic inhibition of the VTA
Effects of optogenetic inhibition during lever-cue presentation in PavCA
To assess the role of dopamine in the attribution of incentive salience to a reward cue, we expressed an inhibitory opsin (eNpHR) in dopamine neurons in the VTA of TH-Cre rats (Figs. 2, 3). Disrupting cue-elicited dopamine through optogenetic inhibition of the VTA reduced lever-directed behaviors. During Sessions 1-3, when the laser was turned on concurrently with lever-cue presentation, there was a significant effect of experimental group and/or a group × session interaction for all measures of lever-directed behavior (Table 2). As shown in Figure 4a, while rats in the control group increased the number of lever-cue contacts across Sessions 1-3, rats in the halorhodopsin group did not (group × session interaction: F(2,15.773) = 4.119, p = 0.036; effect of session for control group: F(2,15.279) = 4.418, p = 0.031). Post hoc comparisons revealed a significant reduction in the number of lever-cue contacts among halorhodopsin rats compared with control rats on Session 3 (p = 0.02, Cohen's d = 1.27; Fig. 4a). In agreement, the probability to approach the lever-cue was lower among halorhodopsin rats relative to control rats across Sessions 1-3 (effect of group: F(1,15.405) = 4.954, p = 0.041) and only those in the control group showed a significant increase in the probability to approach the lever-cue across sessions (group × session interaction: F(2,18.848) = 5.237, p = 0.016; effect of session for control group: F(2,24.073) = 5.730, p = 0.009; Fig. 4b). Post hoc analyses revealed that control rats had a significant increase in the probability to approach the lever-cue on Session 3 relative to Session 1 (p = 0.008, Cohen's d = 0.95). Further, the latency to approach the lever-cue significantly decreased across Sessions 1-3 in control rats, but not those in the halorhodopsin group (effect of group: F(1,15.396) = 4.620, p = 0.048; group × session interaction: F(2,15.436) = 4.513, p = 0.029; effect of session for control group: F(2,18.848) = 6.053, p = 0.009; Fig. 4c). Again, post hoc comparisons revealed that the group differences were most apparent on Session 3, when halorhodopsin rats took more time to approach the lever-cue relative to control rats (p = 0.008, Cohen's d = 1.34; Fig. 4c). Thus, for all of these measures, the difference between groups was most apparent on Session 3, after control rats began to exhibit a sign-tracking conditioned response.
Experimental timeline. a, Schematic represents optogenetic viral infusion and fiber placement; 1.0 µl of either eNpHR (halorhodopsin) or EYFP (control) virus was infused and the optogenetic fiber was placed bilaterally in the VTA. b, Purple represents week of experimental timeline. Week 4 of the experiment is expanded below in gray to show PavCA sessions. c, Schematic of behavioral paradigm in which lever-cue (CS) presentation (8.2 s) was paired with laser inhibition of dopamine neurons during PavCA Sessions 1-3. The laser remained off for the last three PavCA sessions (4-6). d, Representative image of virus expression in the VTA at 10× magnification depicting fluorescence of Cre-dependent EYFP (green) and DAPI (cobalt) to stain nuclear DNA. VTA, ventral tegmental area; IPN, interpeduncular nucleus.
Virus expression and optogenetic probe placement in VTA. a, Representative images of virus expression in the VTA at 20× magnification depicting single channels of DAPI (cobalt), TH (red), and EYFP (green). A merged image of all channels represents overlap of TH and EYFP in the VTA. Coronal atlas images from (b) anterior to (c) posterior, relative to bregma −5.20 to −6.48 are depicted with virus expression spread (in green) and probe placement markers (black + sign) for both experimental groups (EYFP and eNpHR). The density of the virus expression is reflected by the different hues of green. Darker color represents greater density. The densest virus expression and successful probe placements were between bregma −6.12 and −6.36.
Statistical analyses for lever- and magazine-directed behaviorsa
Inhibition of dopamine in the VTA attenuates sign-tracking behavior. a–c, Lever-directed and (d–f) magazine-directed behaviors for Sessions 1-3 (laser on) and Sessions 4-6 (laser off). a–f, Right, Comparison of Session 3 (laser on) to Session 6 (laser off). Data are mean ± SEM for (a,d) number of contacts or head entries, (b,e) probability, or (c,f) latency to approach the lever-cue (left) or food magazine (right). a–c, Optogenetic inhibition of dopamine neurons in the VTA decreases lever-directed behaviors in eNpHR rats (n = 10) compared with controls (n = 7) on Sessions 1-3. Both groups had similar responding for lever-directed behaviors between Sessions 4-6. d–f, Optogenetic inhibition during Sessions 1-3 had no effect on magazine-directed behaviors. Both groups had similar responding for magazine-directed behaviors between Sessions 4-6. Bracket indicates significant difference between groups on Session 3. *p < 0.05.
The impact of VTA dopamine neuron inhibition was specific to lever-directed behaviors and did not affect goal-directed behaviors during Sessions 1-3. Figure 4d–f illustrates the lack of differences between experimental groups during the “laser on” sessions for the number of food magazine contacts during lever-cue presentation, the probability to approach the food magazine during lever-cue presentation, and the latency to approach the food magazine during lever-cue presentation (see also Table 2).
Consistent with the data above, experimenter observation of the videos from Session 3 revealed that the probability to orient toward the lever-cue significantly differed between experimental groups, whereas the probability to orient toward the food magazine did not (group × location interaction: F(1,28) = 4.788, p = 0.039; effect of group for lever-cue: F(1,24) = 5.538, p = 0.027; Fig. 5a). As shown in Figure 5b, rats in the control group oriented to both the lever-cue and food magazine on ∼46% of trials, whereas those in the halorhodopsin group did so on ∼36% of trials. Orientation to both the lever-cue and food magazine on a given trial suggests that neither rats in the control group nor the halorhodopsin group were extreme STs or GTs by Session 3, and this is consistent with the data shown in Figure 4b. These data might also suggest that the value of the lever-cue has not been fully learned by Session 3. Nonetheless, a conditioned orienting response is apparent for rats in both groups and inhibition of VTA dopamine neurons selectively affects the response directed toward the lever-cue.
Head orientation to the lever-cue and food magazine during lever-cue presentation. a, Mean ± SEM for probability to orient to the lever-cue or food magazine during lever-cue presentation on Session 3 (final day of VTA dopamine neuron inhibition). Rats in the control group oriented more toward the lever-cue than those in the eNpHR group that received VTA dopamine neuron inhibition. b, Percent of trials that included orientation toward the lever-cue, food magazine, or both for control (purple) and eNpHR (orange) rats. Bracket indicates significant difference between groups on Session 3. *p < 0.05.
Importantly, the effects of VTA dopamine neuron inhibition did not extend to behavior during the ITI, as there were no significant differences in head entries into the food magazine in between trials during Sessions 1-3 (Table 2). Further, every rat consumed all of the food pellets that were delivered each session.
Effects of lifting optogenetic inhibition in later sessions
In the absence of VTA dopamine neuron inhibition and laser presentation (Sessions 4-6), there were no significant differences in lever-directed behaviors between halorhodopsin and control rats. There were no significant differences between experimental groups for lever-cue contacts, probability to approach the lever-cue or latency to approach the lever-cue on Sessions 4-6 (Fig. 4a–c; Table 2). Relative to rats in the halorhodopsin group, however, control rats showed a trend toward a greater probability to approach the lever-cue (effect of group: F(1,14.997) = 4.157, p = 0.059) and a decreased latency to approach the lever-cue (effect of group: F(1,15.095) = 4.074, p = 0.062; Fig. 4b,c). As shown in Figure 4a–c, without VTA dopamine neuron inhibition during lever-cue presentation, the halorhodopsin rats began to exhibit lever-directed behaviors comparable to control rats. In support, the “learning curve” for lever-directed behaviors did not differ between rats in the control group on Sessions 1-3 and those in the halorhodopsin group on Sessions 4-6 (Table 2), when there was no laser inhibition. Further, there were no significant differences in lever-directed behaviors on Session 1 for control rats relative to Session 4 for halorhodopsin rats, and only rats in the control group had a significant increase in lever-cue contacts between Session 1 and Session 4 (t(6) = –2.62, p = 0.04). Together, these data demonstrate that rats in the halorhodopsin group did not attribute incentive salience to the lever-cue when they were receiving cue-paired laser inhibition; but once laser inhibition was removed, they were fully capable of doing so. Thus, VTA dopamine neuron activity is necessary for encoding the incentive value of reward cues.
There were no significant group differences on Sessions 4-6 in head entries, probability, or latency to approach the food magazine during lever-cue presentation (Fig. 4d–f; Table 2). While all rats tended to decrease the number of head entries into the food magazine during the ITI across Sessions 4-6 (effect of session: F(2,30) = 3.593, p = 0.04; Table 2), those in the control group tended to do so less than those with prior VTA dopamine neuron inhibition (effect of group, F(1,15) = 4.476, p = 0.05, Table 2). These data suggest that rats in the halorhodopsin group may be less efficient in retrieving food pellets, and this is consistent with delayed entry into the food magazine on lever-cue retraction, as presented below (see Fig. 10c).
Comparing “laser on” versus “laser off” sessions
To further assess differences in behavior as a function of VTA dopamine neuron inhibition, we directly compared Session 3, the last PavCA session with laser inhibition, to Session 6, the last PavCA session without laser inhibition. Rats in both the control group and halorhodopsin group showed an increase in lever-directed behaviors on Session 6 relative to Session 3 (Fig. 4a–c; Table 3). Relative to rats in the halorhodopsin group, however, rats in the control group had a higher probability to approach the lever-cue (effect of group, F(1,15) = 4.623, p = 0.048; Fig. 4b) and had a tendency to do so more quickly (effect of group, F(1,15) = 3.917, p = 0.066). There were no significant differences in food magazine-directed behaviors between Session 3 versus Session 6 (Fig. 4d–f).
Statistical analyses comparing “laser on” vs “laser off” periodsa
We also assessed whether cue-paired inhibition of dopamine neuron activity limited the ability of behavior during Sessions 1-3 to predict subsequent behavior (Fig. 6). Indeed, we found that the change in behavior from Session 1 to Session 3 (i.e., Acquisition Index) predicted the PavCA Index on Session 6 (i.e., Final Index) in control rats (F(1,5) = 9.629, p = 0.027), but not in halorhodopsin rats (F(1,8) = 0.549, p = 0.480). The Acquisition Index was an excellent predictor (β = 0.811, p = 0.027) and explained 65.8% (adjusted r2 = 0.658) of the variance in the Final PavCA Index among controls. However, inhibition of dopamine neuron activity during lever-cue presentation restrained the predictive power (β = 0.253, p = 0.480), as only 6.4% (adjusted r2 = 0.064) of the variance in Final Index could be explained by the Acquisition Index in halorhodopsin rats.
The change in PavCA during Acquisition predicted the Final PavCA Index in control rats. Individual data points are shown for control (purple, n = 7) and eNpHR rats (orange, n = 10) with their respective regression lines. Acquisition Index reflects (Δ PavCA Index between Sessions 3 and 1), and the Final Index reflects Session 6 PavCA Index. The Acquisition Index significantly predicted the Final PavCA Index in controls, but not in eNpHR rats. *p < 0.05.
Deep phenotyping expands behavioral analysis
Behavioral video analysis with DLC confirms that perturbing cue-elicited dopamine reduces multiple facets of cue-directed behaviors. Video analyses from Session 3, the final session of VTA dopamine neuron inhibition, were partitioned into different periods to assess location and time spent near (Fig. 7), orientation to (Fig. 8), and approach toward (Fig. 9) the lever-cue or food magazine, as well as locomotor activity throughout the session (Fig. 10).
Inhibition of dopamine in the VTA reduces time spent near the lever-cue. a, Representative frame from a video with arena borders overlaid and magazine and lever labeled. Marker labeling from Deep Lab Cut: nose (purple), tether (green), shoulders (orange/yellow), tail (blue), and magazine (red). b–e, Each heatmap represents the rats' average location across all trials during Session 3 of PavCA training (integrated over 123 video frames, 8.2 s time bins). Red represents more time spent in a given location. Blue represents less time. Location of the rats is shown for (b) the 8.2 s before lever-cue presentation, (c) 8.2 s during lever-cue presentation, (d) 8.2 s immediately after lever-cue retraction, and (e) 8.2 s during the ITI. Left, Control rats. Middle, eNpHR rats that received dopamine inhibition during lever-cue presentation. Right, Subtraction of eNpHR with control heat maps zoomed in on the magazine and lever. Red represents that the eNpHR rats spent more time in a given location. Blue represents that control rats spent more time in a given location.
Inhibition of VTA dopamine neurons blunts the orienting responses to the lever-cue. a, Left, Head orientation was calculated from each video frame and is depicted as a unit vector on a polar histogram with the relative location of the magazine, lever, and back of chamber labeled. Right, Representative frame from a video with magazine and lever labeled. Marker labeling from Deep Lab Cut: nose (purple), tether (green), shoulders (orange/yellow), tail (blue), and magazine (red). Head orientation was calculated from the angle between two vectors: (1) from the tether to the nose and (2) from the tether to the magazine. b–f, Polar histograms showing individual rat average head direction (left) and group data head direction (center/right) (b) before lever-cue presentation (4 s), (c) during lever-cue presentation (4 s), (d) before pellet delivery (4 s), (e) after pellet delivery (4 s), and (f) during the ITI (4 s). c, Compared with controls, eNpHR rats that received inhibition of dopamine neurons in the VTA had more variability in their head direction during lever-cue presentation. *p < 0.05.
DLC analysis confirms that inhibition of VTA dopamine neurons suppresses sign-tracking behavior. Data are mean ± SEM for approach behavior directed toward the (a,b) lever-cue (left) or (c,d) food magazine (right) during the (a,c) 8.2 s lever-cue presentation (top) or (b,d) after lever retraction, during the ITIs (bottom). An approach bout was counted when the rat's nose remained within 75 pixels (1 cm) of the lever-cue or food magazine during >15 frames (1 s). A bout ended when the interbout interval exceeded 15 frames. Relative to control rats, eNpHR rats exhibit less approach behavior toward the lever-cue, and this is true (a) during lever-cue presentation and (b) after lever-cue retraction (during the ITIs). Approach behavior directed toward the food magazine does not differ between control rats and those in the eNpHR group. **p < 0.01. ***p < 0.001.
Inhibition of dopamine neurons suppresses lever-cue elicited locomotion. Data are mean ± SEM. a, Average velocity (cm/s) of movement during the 4-s period immediately before and immediately on lever-cue presentation. b, Distance (cm) from the lever-cue for the 2-s period immediately before and immediately on lever-cue presentation. c, Latency to approach the food magazine immediately following lever-cue retraction. a, Control rats moved faster when the lever-cue was presented, relative to the period before lever-cue presentation. Control rats also moved quicker than eNpHR rats that received inhibition of dopamine neurons when the lever-cue was presented. b, Control rats moved closer to the lever-cue when it was presented relative to eNpHR rats that received inhibition of dopamine neurons. c, eNpHR rats that received lever-paired inhibition of dopamine neurons were delayed in approaching the food magazine on lever-cue retraction. *p < 0.05. ***p < 0.001.
Location in chamber
Time spent in locations of the behavior chamber was analyzed in 8.2 s time bins, reflective of the period from lever-cue presentation to retraction. As indicated by the heatmaps shown in Figure 7b, during the 8.2 s period before the lever-cue was presented, control and halorhodopsin rats were found throughout the chamber with a tendency to gather near the food magazine. Once the lever-cue was presented, rats in the control group appeared to spend more time near and around the lever-cue (Fig. 7c), while those in the halorhodopsin group were around the food magazine or in other locations in the chamber. Immediately after lever-cue retraction, rats in both groups spent more time at the location of food delivery (Fig. 7d). A subtraction analysis further illustrates the difference in time spent near the lever-cue and food magazine for rats in the halorhodopsin group compared with those in the control group (Fig. 7, right).
Head orientation
DLC analysis of head orientation revealed a trend toward a significant difference between groups in the 4 s preceding lever-cue presentation (KS test, p = 0.065), with rats in the control group showing a greater tendency to orient to the side of the chamber containing the lever-cue and food magazine (Fig. 8b). During the first 4 s of lever-cue presentation, rats in the control group preferentially oriented toward the lever-cue relative to those in the halorhodopsin group (KS test, p = 0.034; Fig. 8c). During the last 4 s of lever-cue presentation, there was a trend for a significant difference in head orientation between groups (KS test, p = 0.087; Fig. 8d). There were no significant differences in head orientation to the lever-cue location or food magazine after pellet delivery (KS test, p = 0.328; Fig. 8e) or during the ITI (KS test, p = 0.118; Fig. 8f). These data are consistent with those presented above and with the experimenter-scored videos (Fig. 5). We demonstrate that inhibition of VTA dopamine neurons impacts orientation toward the lever-cue upon its presentation, without affecting orientation toward the food magazine.
Approach behavior
Approach toward the lever-cue and food magazine was assessed as an additional metric that is not captured by the Med Associates output, but one that is a hallmark of incentive salience attribution (Berridge, 2001; Cardinal et al., 2002). Approach behavior was counted when the rats' nose was within 1 cm of either the lever-cue or food magazine for >1 s. Relative to rats in the control group, those in the halorhodopsin group showed less approach behavior toward the lever-cue on its presentation (t(12) = 4.911, p < 0.001; Fig. 9a). Interestingly, the same was true during the ITIs (i.e., after the lever had been retracted) (t(12) = 3.753, p = 0.003; Fig. 9b), reflecting a general tendency for rats in the control group to spend more time by the lever-cue location (see also Fig. 7d). There were no significant differences between groups for approach toward the food magazine at any time point: during lever-cue presentation (t(12) = −0.079, p = 0.939; Fig. 9c) or after lever-cue retraction (t(12) = −0.788, p = 0.446; Fig. 9d).
Locomotor activity
Importantly, the experimental groups did not display significant differences in nonspecific locomotion during Session 3 (t(12) = 0.406, p = 0.692; data not shown). Upon lever-cue presentation, however, control rats displayed a significant increase in the speed of movement relative to the time period before lever-cue presentation (p < 0.001, Cohen's d = 1.73; Fig. 10a). This change in velocity was not apparent in halorhodopsin rats (effect of group: F(1,24) = 20.202, p < 0.001, effect of time: F(1,24) = 13.509, p = 0.001, group × time interaction: F(1,24) = 7.444, p = 0.012); and post hoc comparisons revealed that control rats move to the lever-cue faster than halorhodopsin rats (p < 0.001, Cohen's d = 0.26). When lever-cue-elicited locomotion was evaluated as the distance from the lever-cue 2 s before and 2 s during presentation, there was a significant effect of time (F(1836) = 115.680, p < 0.001); but as shown in Figure 10b, the lever-cue-elicited locomotion was more apparent in rats in the control group relative to those in the halorhodopsin group (effect of group: F(1836) = 1135.491, p < 0.001; group × time interaction: F(1836) = 47.958, p < 0.001; Fig. 10b). Post hoc comparisons revealed that rats in the control group moved closer to the lever-cue relative to those in the halorhodopsin group both before (p < 0.001, Cohen's d = 0.01) and after lever-cue presentation (p < 0.001, Cohen's d = 0.045). Interestingly, relative to those in the control group, rats in the halorhodopsin group were delayed in approaching the food magazine once the lever-cue had been retracted (t(12) = −2.409, p = 0.03; Fig. 10c). This delay could potentially be because of reduced salience of the food magazine as a result of VTA dopamine neuron inhibition (Mahler and Berridge, 2009; DiFeliceantonio and Berridge, 2012). Nonetheless, all rats consumed all of the food pellets and there were no significant differences between groups in general locomotor activity. Thus, these data support the notion that cue-elicited dopamine neuron activity in the VTA plays a selective role in encoding the incentive value of reward cues.
Discussion
It is known that VTA dopamine neuron activity is involved in reward processing; however, the precise contributions of dopamine in terms of temporal specificity and value encoding remain a subject of debate (for review, see Berridge, 2007, 2012; Zhang et al., 2009; Berke, 2018; Stauffer, 2018; Triche et al., 2022). Here we capitalized on the temporal precision of optogenetics and used a transgenic rat colony with a tendency to sign-track to further explore the role of dopamine in reward learning. We demonstrate that inhibition of VTA dopamine neuron activity during presentation of a discrete cue that predicts reward delivery prevents incentive value encoding. Specifically, inhibition of VTA dopamine neurons during lever-cue presentation precludes the attribution of incentive motivational value to the reward cue and thereby blocks the development of a sign-tracking conditioned response. Detailed analysis of behavior using DLC reinforced the specificity of these effects, revealing that locomotor activity was not affected by selective inhibition of VTA dopamine neuron activity, nor was orientation or approach directed toward the location of reward delivery. Further, when VTA dopamine neuron activity was restored, rats developed a sign-tracking CR indicative of incentive salience attribution. These data are in agreement with prior studies demonstrating that dopamine is essential for incentive learning and the acquisition and expression of sign-tracking behavior (Flagel et al., 2011; Saunders and Robinson, 2012; Yager et al., 2015).
The role of dopamine in reward processing has been presented within the context of multiple learning theories and frameworks, some of which are in direct opposition (Berridge, 2007, 2012; Langdon et al., 2018; Gershman and Uchida, 2019; Schultz, 2019; Lerner et al., 2021). The long-prevailing view that dopamine encodes the predictive value of reward cues and reflects a universal learning signal (Schultz et al., 1997) has been met with conflicting data. With the use of new technologies to further probe the role of dopamine in reward learning (for review, see de Jong et al., 2022), it has been elegantly demonstrated that dopamine promotes associative learning (Sharpe et al., 2020) and encodes perceived saliency independent of valence (Kutlu et al., 2021), even when conditions are ripe for prediction error signals. Further, using a novel computational approach, it was shown that dopamine conveys causal associations without reward prediction error (Jeong et al., 2022). Our prior work with the ST/GT animal model is consistent with these more recent findings. We demonstrated that dopamine in the core of the NAc encodes the incentive properties of reward cues, not the predictive (Flagel et al., 2011). That is, the classic prediction error shift in dopamine from the reward (unconditioned stimulus) to the reward cue (conditioned stimulus) occurs in STs, but not GTs. If dopamine were merely a predictive learning signal, the shift in dopamine would have been evident in both STs and GTs, as the reward-cue is a predictor and elicits a CR for both. Moreover, blockade of dopamine signaling prevented the learning of a sign-tracking conditioned response, but not goal-tracking (Flagel et al., 2011; Saunders and Robinson, 2012).
Here we demonstrate that inhibition of VTA dopamine neurons selectively during lever-cue presentation prevents the attribution of incentive salience to the lever-cue and thereby precludes the development of a sign-tracking response. Upon restoration of VTA dopamine neuron activity, the same rats attributed incentive value to the lever-cue in a manner that was indistinguishable from rats in the control group during their initial Pavlovian training sessions. Consistent with these findings, dopamine neurons in the VTA are activated to a much greater extent in STs relative to GTs during lever-cue interaction (Ferguson et al., 2020) and VTA dopamine neuron stimulation can itself transform a predictive stimulus into an incentive stimulus (Saunders et al., 2018). Specifically, neurons projecting from the VTA to the core of the NAc were found to be especially important for incentive value encoding (Saunders et al., 2018), but others have reported that dopamine within the NAc shell encodes incentive salience (Saddoris et al., 2015). The current findings add to the growing body of literature that supports a role for dopamine neurons in the VTA in incentive value encoding.
Dissociating the effects of cue-paired inhibition of VTA dopamine neurons on encoding the predictive versus incentive value of reward cues is complex. To better assess the effects of this manipulation on predictive learning, we evaluated the conditioned orienting response, which is indicative that a stimulus–reward relationship has been learned (Buzsaki, 1982). Experimenter observation and DLC analyses revealed that rats in both groups exhibited a conditioned orienting response directed toward the lever-cue and/or food magazine, but only lever-cue oriented responses were affected by VTA dopamine neuron inhibition. These findings are seemingly in contrast to prior studies that have shown that the conditioned orienting response directed toward the lever-cue remains intact in STs following administration of a dopamine antagonist into the core of the NAc, and that only approach and interaction with the lever-cue are attenuated (Saunders and Robinson, 2012; Yager et al., 2015). It is important to note, however, that the manner in which we assessed conditioned orienting differs from these studies. For example, rats in the current study were not habituated to the presentation of the lever-cue in the absence of reward delivery (as in Yager et al., 2015). Further, we assessed conditioned orienting behavior across all trials on Session 3, presumably as the value of the lever-cue was still being learned, whereas other studies assessed it at the time an extreme sign-tracking response was evident (Saunders and Robinson, 2012; Yager et al., 2015) and only during the latter half of the session (Saunders and Robinson, 2012). It is also possible that, in the current study, the lever-directed conditioned orienting response is affected via inhibition of dopamine neuron activity in nonstriatal regions, such as the PFC (Swanson, 1982), which is known to play a role in “cognitive” or model-based learning (Swanson, 1982; Dayan and Balleine, 2002; Dickinson and Balleine, 2002; Kuhn et al., 2018; Morrens et al., 2020; Ioanas et al., 2022). Regardless, our findings are consistent with the conclusion that dopamine neuron activity is necessary for attributing incentive salience and transforming a predictive stimulus into a “motivational magnet,” but not for learning the stimulus–reward relationship.
Based on the current findings and those previously published (Flagel et al., 2011; Saunders and Robinson, 2012), we would not expect cue-paired inhibition of VTA dopamine neurons to affect the prepotent CR in GTs. Importantly, however, we were not able to directly assess the effects of VTA dopamine neuron inhibition on the development of goal-tracking behavior as the rats used in this study have an inherent predisposition to sign-track. Indeed, the low levels of goal-tracking behavior might have precluded us from detecting any significant decreases in this behavior as a function of VTA dopamine neuron inhibition. It is not clear why the current population of Long Evans rats is skewed toward STs, but similar shifts in behavioral phenotypes have been observed in other rat populations bred in house (Chang S, Felix P, Westbrook S, unpublished data). Further, inherent tendencies to sign- or goal-track are known to differ between rats from different vendors (Fitzpatrick et al., 2013; Khoo et al., 2022), and the same is true for mice of different strains (Dickson et al., 2015). The results reported here are also limited by the fact that only male rats were studied. While this research was based on prior findings using male rats, it will be important to determine whether the same neural mechanisms encode the incentive motivational value of reward cues in female rats. To date, there is little evidence to support robust sex differences in the propensity to attribute incentive salience to a food cue (Pitchers et al., 2015), but female rats tend to be skewed more toward STs (Hughson et al., 2019) and the underlying neural mechanisms warrant further investigation.
Although locomotor activity did not appear to be generally affected by inhibition of VTA dopamine neuron activity in the current study, it is possible that general motivation was affected in real time. It is difficult to dissociate the effects of this manipulation on the motivation to approach the lever-cue from incentive value encoding. However, if cue-paired inhibition of VTA dopamine neuron activity were merely affecting motivation, we would expect rats to exhibit higher levels of approach on Session 4, when the inhibition was terminated. Instead, we observed a gradual increase in approach to the lever-cue between Sessions 4-6, presumably as a function of incentive salience attribution. Further, rats that received cue-paired inhibition showed little change in locomotor activity during inhibition. Finally, the fact that a conditioned orienting response directed toward the food magazine did not differ between groups and that all of the rats consumed all of the food pellets that were delivered suggests that cue-paired inhibition of dopamine neuron activity in the VTA did not generally affect learning or motivation under the current conditions.
While the optogenetic parameters used here have been shown to be effective in silencing dopamine neurons (Gradinaru et al., 2010; McCutcheon et al., 2014), local network excitability in opsin-expressing neurons (Raimondo et al., 2012; Alfonsa et al., 2016) as well as reduced excitability in non–opsin-expressing neurons (Parrish et al., 2023) has also been reported following continuous optogenetic inhibition. Further, rebound excitation can induce behavioral changes that are apparent immediately upon laser cessation (Arrenberg et al., 2009). It is therefore possible that the results reported here were affected by rebound excitation, but we do not think this is the case, as there were few differences in behavior between halorhodopsin rats and those in the control group immediately upon laser cessation. Of note, however, halorhodopsin rats were slower to approach the location of reward delivery on cessation of laser inhibition. This delay in approaching the food magazine on lever-cue retraction could suggest that the perceived meaning of lever-cue retraction was still being learned. In support, rats in the halorhodopsin group continued to show increased responding in the food magazine relative to those in the control group during the ITI. Further, others have reported that, relative to GTs, STs show greater engagement of VTA dopamine neurons upon both the presentation and retraction of the lever-cue (Ferguson et al., 2020). It is also possible that inhibition of VTA dopamine neurons attenuated any incentive motivational value placed on the food magazine (Mahler and Berridge, 2009; DiFeliceantonio and Berridge, 2012), which could explain the slight increase in goal-directed behaviors on nonlaser sessions in halorhodopsin rats. Thus, while we recognize the potential impact of rebound excitation on the reported findings, we do not believe it to significantly affect our interpretation or conclusions.
The current findings expand and enhance the existing literature pertaining to the role of dopamine in reward learning. We clearly demonstrate that cue-paired inhibition of VTA dopamine neuron activity prevents the attribution of incentive motivational value to the reward cue and the development of sign-tracking behavior. Using DLC, we were able to thoroughly assess metrics of learning and incentive motivation and rule out nonspecific effects of VTA dopamine neuron inhibition. This study was designed around the fact that the TH-Cre population we used was skewed toward STs; yet, this precluded us from assessing the effects of this manipulation in GTs. The findings were interpreted with this individual variability or lack thereof in mind, and we consider this a valuable example for the field to consider. Further, the complexity of behavior and nuances therein are illustrated here and should be noted for those using cutting-edge techniques to decipher brain–behavior relationships. We will continue to capitalize on “deep phenotyping” approaches to assess the effects of similar manipulations in female rats and to better elucidate the neural substrates underlying reward learning and incentive value encoding. Based on the current findings, however, we conclude that cue-elicited dopamine is critical for the attribution of incentive salience to reward cues.
Footnotes
This work was supported by National Institutes of Health R01 DA038599 and DA054094 to S.B.F.; Hope for Depression Research Foundation to H.A.; National Alliance for Research on Schizophrenia and Depression Young Investigator Grant to C.R.B.; and National Institutes of Health R01 DK129366 to C.R.B. A.G.I. and A.S.C. were supported by National Institute on Drug Abuse T32 Training Program in Neuroscience (National Institutes of Health T32-DA7281). A.G.I. was also supported by National Science Foundation Graduate Research Fellowship DGE 1256260 and University of Michigan Rackham Merit Fellowship. Use of DeepLabCutTM was supported in part through computation resources and services provided by Advanced Research Computing at the University of Michigan (Ann Arbor). We thank Drs. Kent Berridge, Stephen Chang, and Terry Robinson for insightful comments on earlier versions of this manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Shelly B. Flagel at sflagel{at}med.umich.edu