Abstract
The ability to recognize motivationally salient events and adaptively respond to them is critical for survival. Here, we tested whether dopamine (DA) neurons in the dorsal raphe nucleus (DRN) contribute to this process in both male and female mice. Population recordings of DRNDA neurons during associative learning tasks showed that their activity dynamically tracks the motivational salience, developing excitation to both reward-paired and shock-paired cues. The DRNDA response to reward-predicting cues was diminished after satiety, suggesting modulation by internal states. DRNDA activity was also greater for unexpected outcomes than for expected outcomes. Two-photon imaging of DRNDA neurons demonstrated that the majority of individual neurons developed activation to reward-predicting cues and reward but not to shock-predicting cues, which was surprising and qualitatively distinct from the population results. Performing the same fear learning procedures in freely-moving and head-fixed groups revealed that head-fixation itself abolished the neural response to aversive cues, indicating its modulation by behavioral context. Overall, these results suggest that DRNDA neurons encode motivational salience, dependent on internal and external factors.
SIGNIFICANCE STATEMENT Dopamine (DA) contributes to motivational control, composed of at least two functional cell types, one signaling for motivational value and another for motivational salience. Here, we demonstrate that DA neurons in the dorsal raphe nucleus (DRN) encode the motivational salience in associative learning tasks. Neural responses were dynamic and modulated by the animal's internal state. The majority of single-cells developed responses to reward or paired cues, but not to shock-predicting cues. Additional experiments with freely-moving and head-fixed mice showed that head-fixation abolished the development of cue responses during fear learning. This work provides further characterization on the functional roles of overlooked DRNDA populations and an example that neural responses can be altered by head-fixation, which is commonly used in neuroscience.
Introduction
Dopamine (DA) is widely implicated in reward-seeking behavior and reward prediction error (RPE) encoding (Schultz et al., 1997; Bromberg-Martin et al., 2010). Increasing evidence suggests that DA also mediates non-reward functions, showing diverse responses to surprising, novel, or aversive events (Menegas et al., 2017; de Jong et al., 2019; Lutas et al., 2019; Robinson et al., 2019). These observations lead to the hypothesis that DA supports motivational control via at least two functional cell-types: one that encodes motivational value and another that signals motivational salience, which is defined as the absolute of motivational value (Bromberg-Martin et al., 2010).
DA neurons in the lateral ventral tegmental area (VTA) or medial substantia nigra pars compacta (SNc) and those projecting to the lateral nucleus accumbens (NAc) are activated by rewarding events or cues and inhibited by aversive ones, supporting motivational value encoding (Matsumoto and Hikosaka, 2009; de Jong et al., 2019). By encoding appetitive and aversive events in opposite directions, they can provide a teaching signal for seeking positive reinforcement and value-based learning (Bromberg-Martin et al., 2010). In contrast, DA neurons in the lateral SNc and amygdala-projecting VTA cells are activated by both rewarding and aversive events or cues, consistent with motivational salience encoding (Matsumoto and Hikosaka, 2009; Menegas et al., 2017; Lutas et al., 2019). Such neural dynamics, excitation to both rewarding and aversive events with weaker responses to neutral events, can help detection of external or internal stimuli of high significance and promote appropriate behavioral responses to them (Bromberg-Martin et al., 2010).
We and others have characterized dorsal raphe nucleus (DRN)DA neurons, demonstrating that their population activity reflects salience rather than value (Cho et al., 2017; Groessl et al., 2018; Lin et al., 2020); showing activation to diverse stimuli with positive and negative valence, but not with neutral value. Here, we further examine the hypothesis that DRNDA neurons encode motivational salience, at both population and single-cell levels, using associative learning tasks in which the motivational salience and value of innately neutral cues were dynamically modulated by pairing them with positive, neutral, or negative outcomes. We also investigated whether DRNDA responses to the same motivationally salient stimuli are modulated by internal state, expectation, and/or external behavioral context, such as head-fixation setup which is commonly used in neuroscience experiments.
Materials and Methods
Experimental animals
Subjects were Th-ires-cre transgenic mice (Th: tyrosine hydroxylase, a rate-limiting enzyme for DA synthesis; Lindeberg et al., 2004) of both sexes, aged two to four months at the time of surgery. Th-ires-cre mice were used in this study to selectively target DRNDA neurons; the specificity of cre expression (compared with immunohistochemistry of Th+ neurons) in this mouse line has previously been shown to be around 60–75% in the DRN, which is comparable to an alternative line that express Cre recombinase under the DA transporter promoter (Li et al., 2016; Matthews et al., 2016; Cho et al., 2017). Typically, experiments lasted until the mice were six to eight months old. Animals were originally group-housed but were later single-housed after undergoing surgery for photometry or two-photon imaging. Mice were housed in a room on a 12/12 h light/dark cycle (lights off at 6 A.M., lights on at 6 P.M.). All experiments were performed during the dark phase. Mice had ad libitum access to food and water before the start of water restriction (see Water restriction and habituation procedures for head-fixed experiments for details). All animal husbandry and experimental procedures involving animal subjects were conducted in compliance with the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health and approved by the Office of Laboratory Animal Resources at the California Institute of Technology (IACUC protocol number 1730). Animals were excluded from analysis if no photometry or two-photon signals were observed four weeks after surgery. One mouse was excluded during the associative learning experiments (Figs. 1, 2) because of health concerns related to water restriction. At the end of the experiments, the brains from all animals with fluorescence signals were histologically verified to have fibers or gradient index (GRIN) lenses located over the DRN.
Surgical procedures
Stereotaxic surgeries for viral vector injections and implantation of an optical fiber/ferrule for photometry or a GRIN lens for two-photon imaging were performed as previously described (Cho et al., 2017) with slight modifications. After anesthesia (isoflurane gas/carbogen mixture, 5% for induction and 1.5–2% for maintenance), surgical preparation and exposure of the skull, a craniotomy hole was drilled in the skull (anteroposterior axis: −4.7 mm, mediolateral axis: −1.5 mm, relative to bregma). Adeno-associated viruses (AAVs) encoding jGCaMP7f or GCaMP6m in a cre-dependent manner (diluted to 1.0 × 1013 genome copies/ml, both from Addgene) were injected to the DRN (anteroposterior axis −4.7 mm, mediolateral axis: −1.5, dorsoventral axis −3.2 and −2.9 mm, relative to bregma) at a 25° angle. A total of 300 nl of AAV was infused at each site along the dorsoventral axis, at a rate of 50 nl/min. After injection, the needle was held in the same place for an additional 10 min. Finally, the needle was slowly withdrawn over ∼10 min to prevent backflow.
For fiber photometry, an optical fiber/ferrule (fiber: 400-µm diameter, NA 0.48, cut length: 4 mm, ferrule: 1.25-mm diameter, zirconia, glued with low-autofluorescence epoxy, Doric Lenses) was mounted to a stereotaxic cannula holder (SCH_1.25, Doric Lenses), lowered toward the DRN at a 25° angle, stopping 0.25 mm above the site of virus injection. For two-photon imaging, a 25-gauge needle (outer diameter = 0.515 mm) was attached to the stereotaxic holder (1766AP, David Kopf Instruments) and slowly lowered up to 2 mm along the dorsoventral axis (relative to bregma, at 25°) to make a path for the GRIN lens (GLP-0584, Inscopix; 0.5-mm diameter, 8.4-mm length). Then, a small, customized zirconia ferrule (2.5-mm length, 520-µm hole size; Kientec System) was carefully glued to surround the GRIN lens at one end. The same cannula holder was used to hold the GRIN lens, touching the surrounding zirconia ferrule rather than the fragile and sensitive lens. The GRIN lens was slowly lowered into the brain, stopping 0.25 mm above the site of virus injection. After implantation, a thin layer of adhesive cement was applied to the skull surface and around the implant for strong fixation. After the adhesive cement had completely dried, a layer of black dental cement was applied to build a head cap. For mice in the two-photon imaging experiments (Figs. 3, 4) and comparison of DRNDA dynamics in freely-moving versus head-fixed groups (Figs. 5, 6), a customized ring for head fixation (stainless steel, 5-mm inner diameter, 11-mm outer diameter) was super-glued to the cement surface before the dental cement was fully dried, so that the ferrule or GRIN lens tip was located within the ring. More dental cement was applied inside the ring. To protect the GRIN lens from damage, the lens tip was covered with a small piece of Parafilm and low-toxicity silicone adhesive (Kwik-sil, World Precision Instruments) was applied. After the silicone adhesive fully solidified, mice were unmounted from the stereotaxic frame and their recovery was monitored for ∼2 h.
Fiber photometry
Fiber photometry was performed as previously described (Lerner et al., 2015; Cho et al., 2017; Robinson et al., 2019).
Two-photon imaging
In vivo two-photon imaging was performed with a custom-built microscope. Briefly, a pulsed femtosecond laser beam from a Ti:Sapphire laser system (940 nm), coupled with OPA (Insight DS+, Spectra-Physics), passed through a beam expander (75:50) and an iris (SM1D12C, Thorlabs), which was set to 3 mm. An XY galvanometer (6215H, Cambridge Technology) was placed before a pair of scan lenses (LSM54-1050, Thorlabs) and a tube lens (ITL200, Thorlabs). An 805-nm short-pass dichroic mirror (DMSP805SPL, Thorlabs) was used to allow simultaneous near-infrared (IR) visualization along with two-photon excitation. Near-IR visualization for sample localization was achieved by a 75-mm tube lens (AC508-075, Thorlabs), directed to an HDMI-output camera (HD205-WU, AmScope). A 500- to 700-nm reflecting dichroic mirror (T600/200dcrb, Chroma) was used to split the two-photon excitation and emission paths. A 20×/0.5 NA air objective (Olympus, UPLFLN20XP) was used, and the laser power was set to 60–80 mW. Emitted photons were passed through the collective optics (AC508-100-A, f = 100 mm, at z = 100 mm from BA, most convex side facing the sample; and a pair of LA1131, f = 50 mm at z = 150 mm and z = 156 mm from the back BA, convex sides facing each other) and a 680-nm low-pass filter (et680sp-2p8, Chroma) into the photomultiplier tube (Hamamatsu R3896). Laser intensity was controlled by the rotation of a half-λ waveplate (Thorlabs AHWP05M-980) relative to a Glan polarizer (Thorlabs GL10-B) using a motorized rotation stage (Thorlabs PRM1/Z8). Stage XY adjustment and microscope focus was controlled by motorized linear actuators (Z825B, Thorlabs). Imaging data were acquired using an FPGA DAQ board (National Instruments 7855R) and custom-written software in Labview. An electromechanical shutter (Uniblitz VS25, Vincent Associates) was used to ensure laser safety during imaging. The imaging frame size was 194 × 194 pixels with a 4-Hz frame rate. In three mice, two or three fields of view (FOVs) that showed different sets of neurons were obtained by vertically displacing the objective relative to the lens (at least 100 µm apart in the z-direction). We did not attempt to match or keep the same FOVs across different recording days. The microscope (including objective and optical pathway) itself was tilted by 25° to align with the implanted GRIN lens when animals are head-fixed, in other words, animal's head was not rotated, which could have given additional stress or discomfort. During each imaging session, after finding an FOV, two-photon scanning was triggered for each trial 15 s before the CS delivery and terminated 20 s after the CS delivery.
Water restriction and habituation procedures for head-fixed experiments
All animals reported here underwent water restriction procedures (1.5 ml/d, provided at 4 P.M. everyday), starting from two to three weeks after surgery when mice had fully recovered. The water restriction was mainly to motivate the animals to engage in reward learning, but the fear-learning-only cohort (Figs. 5, 6) were also water restricted to maintain consistent experimental conditions and to facilitate habituation training. Once water restriction started, mice were weighed daily and were returned to ad libitum access to water if their weight loss was >10% of their prerestriction weight. Animals were water restricted at least for 5 d before they started freely-moving associative learning tasks or habituation training for head-fixed experiments.
Regarding habituation procedures for head-fixation, we generally followed a previously published protocol (Guo et al., 2014); but extra steps and longer training duration were added to ensure that the mice were slowly acclimated to the setup. On day 1, mice were familiarized with experimenter handling for ∼15 min. After mice became calm on the experimenter's hand (exhibiting grooming behavior and spending less time looking outside the hand), they were given access to up to 0.4 ml of 5% sucrose water, delivered via a 20-μl pipette. After reward consumption, mice were further handled for ∼2 min before being returned to their home cages. On day 2, mice were handled in a similar fashion for 5 min with access to up to 0.1 ml of 5% sucrose water. We then introduced the body tube (made from Plexiglas) with the other hand and let animals explore the tube freely. We performed this step up to 10 times until mice voluntarily entered the body tube. After entering the tube, the experimenter gently held the tail to prevent escape and mice were rewarded with up to 0.1 ml of 5% sucrose for another 5 min while in the body tube. Next, the experimenter quickly took hold of the implanted head ring with fingers and secured it to the head fixation bar (all within 10 s). The body tube was also secured with a lever. Mice remained head-fixed for 5 min and 20 μl of 5% sucrose was provided every 20 s. On day 3, mice were further acclimated to the apparatus, now with 10 min of head-fixation and 5% sucrose reward delivered every 30 s. From day 4 to day 8, mice were introduced to the lickometer used in the experiments and the duration of head-fixation was gradually increased from 15 to 35 min (increasing by 5 min every day), with reward provided every minute. We reasoned that mice showed good habituation and were ready to advance to the behavioral experiments when they consumed the reward throughout the duration of head-fixation and when they produced much less feces than on the first day of training. We trained mice in the head-fixation apparatus for up to 35 min, well above the duration of recordings (∼25 min for two-photon imaging, ∼15 min for head-fixed fear learning). Note that we extended the number of days of habituation training compared with Guo et al. (2014) to make sure that the mice were habituated to the experimental setting slowly, to minimize the level of stress as much as possible.
Associative learning tasks in freely-moving photometry recordings
All behavioral experiments were programmed and executed with ABET II software (Lafayette Neuroscience). After animals were water restricted for 5 d, they were introduced to an operant chamber (Lafayette Neuroscience) and allowed to freely explore for 30 min with a patch cable attached; 5% sucrose was delivered to the lick port at intervals randomly drawn from a uniform distribution of 45–75 s, so that mice could learn the location. Licks were counted when the IR beam at the lick port was broken. This habituation was repeated for 2 d.
After habituation, the reward (appetitive) learning phase started. Mice were introduced to two types of conditioned stimuli (CS-A and CS-B; 5 kHz tone or white noise, 75 dB, 5 s, counterbalanced across animals). A total of 25 μl of 5% sucrose reward [as the unconditioned stimulus (US)] was delivered only after CS-A presentation; there was no outcome after the CS-B presentation. Within a session, 20 CS-A and reward pair trials and 10 CS-B and no outcome pair trials were given. The intertrial interval was drawn at random from a uniform distribution of 45–75 s. There was a total of 21 reward learning sessions for all animals, and photometry signals were recorded on day 1 (“before learning”) and day 21 (“after reward learning”). On day 1, video was also recorded.
To examine whether DRNDA cue responses are influenced by internal state, mice performed half of the trials in a reward learning session (10 CS-A and reward pairs, five CS-B and no outcome pairs) while they were thirsty and completed the other half after satiety. In between these two separate sessions, they were given free access to water for 3 h in their home cages. After this experiment, mice underwent regular water restriction for 2 d. To study whether DRNDA neurons encode positive prediction error, mice performed another experimental session in which the US was presented without the predictive CS-A. In this session, there were 10 “expected” trials (CS-A paired with the US) and five “unexpected” trials (US only).
Subsequently, mice underwent a fear (aversive) learning phase, in which the previously reward-predicting CS-A no longer predicted any outcome and the previously neutral CS-B was paired with an aversive foot-shock (0.5 mA for 1 s). The duration of both CSs was increased to 10 s so that freezing behavior could be quantified. Sixteen CS-B and shock pairs were presented and eight CS-A and no outcome pairs were presented with the intertrial intervals as described above. Photometry and video recordings (to measure freezing behavior) were performed on day 2 of fear learning (“after fear learning”). On day 3, after mice were fully trained on the fear learning task, we performed a similar prediction error experiment, but now with aversive cue CS-B and foot-shock. In this experiment, three unpredicted shocks were presented intermixed with 10 normal CS-B and shock pairings.
Finally, animals underwent an extinction learning phase in which both CS-A and CS-B were presented (10-s duration, 15 times each) but paired with no outcomes. This was repeated for 4 d and recording was performed on day 5 (“after extinction”).
Associative learning tasks in head-fixed two-photon imaging
Procedures for the associative learning tasks for two-photon imaging under head-fixation were similar to the procedures used in the freely-moving condition, with some small differences. ABET II software was also used to execute the associative learning tasks. Before the imaging experiments started, mice underwent habituation training (see Water restriction and habituation procedures for head-fixed experiments) for 8–10 d and were then transferred to the microscope imaging setup. Mice were further acclimated to the imaging setup for 2 d, receiving free reward (5% sucrose) every 90 s for 35 min. Before learning recordings were performed on day 1 of the reward learning phase: two mice with multiple FOVs performed two separate sessions in a single day, separated by a 6-h interval. After reward learning recordings were obtained on days 18–20 after the mice showed clear discrimination between the reward-predicting CS-A and the neutral CS-B on the basis of their anticipatory licking behavior. One FOV was imaged per day. During training, 20 CS-A trials and 10 CS-B trials (5-kHz tone or white noise, 75 dB, counterbalanced) were presented with intertrial intervals of 45–75 s. On imaging days, 10 CS-A trials and 10 CS-B trials were presented per session.
Fear learning was conducted similarly to the methods stated above (Associative learning tasks in freely-moving photometry recordings), except that tail-shock was used as the US and imaging was performed on day 2. Tail-shock (0.5 mA for 1 s) was administered via two pregelled electrodes wrapped around the tail and connected to a stimulus isolator (Isostim A320R, World Precision Instruments), following Kim et al. (2016). Tail-shock was triggered by external transistor-transistor logic pulses generated by ABET II software. Because of the highly aversive nature of the US, we selected only one FOV per mouse and performed recordings once for after fear learning. During training, 10 CS-A trials and 20 CS-B trials were presented with intertrial intervals of 45–75 s. On the imaging day, 10 CS-A trials and 10 CS-B trials were presented.
Fear learning tasks in freely-moving and head-fixed photometry recordings
To examine whether DRNDA responses to aversive cues are affected by the external behavioral context (freely-moving vs head-fixed conditions), we performed photometry recordings in two different behavioral contexts. All mice underwent identical surgery, water restriction, and head-fixation habituation procedures before being randomly assigned to either the freely-moving or the head-fixed group. The fear learning task was slightly different from the ones described above (Associative learning tasks in freely-moving photometry recordings) and was adapted from previous studies (Groessl et al., 2018; Cai et al., 2020). White noise (20 s) was used as the CS. The duration of the CS was set to 20 s to better quantify freezing behavior as an index of learning. For the US, the freely moving group received foot-shocks within an operant chamber and the head-fixed group received tail-shocks. Six CS–US pairs were presented with intertrial intervals of 60–120 s. The next day, a subset of mice from both groups performed a fear recall experiment. Mice were introduced to a novel cylindrical cage and allowed to freely explore for ∼5 min to habituate to the novel context. After the habituation period, four CSs were presented with no US to see whether cue-induced freezing behavior was evoked.
Data analysis
Behavior
For reward learning, we counted the number of anticipatory licks, defined as licks during the CS presentation before the reward is delivered, as a proxy for learning. For fear learning in freely-moving conditions, freezing behavior was used as an index of associative learning and was quantified visually by an observer blind to the experimental condition. For fear learning in head-fixed conditions (Figs. 5, 6), the number of licks was counted throughout the session. As these mice received 5% sucrose during the habituation procedure, all tested mice showed continuous licking as soon as they were head-fixed (Fig. 5D). We measured whether this licking behavior was affected by repeated CS–US pairings (Lovett-Barron et al., 2014). We note that this does not directly reveal whether the animals have learned the association between the CS and the aversive US per se Therefore, on the next day, mice from both freely-moving and head-fixed groups performed a fear recall session. We compared their freezing behavior during the baseline (after 5-min habituation but before presentation of the first CS) with that during CS presentation to test whether mice could recall the shock-paired cues.
Fiber photometry
Acquired photometry data files were processed with custom-written MATLAB code, as in previous studies (Lerner et al., 2015; Cho et al., 2017; Robinson et al., 2019). Signals from 490- and 405 nm excitation wavelengths were low-pass filtered at 2 Hz with zero-phase distortion. To calculate ΔF/F, a least-squares linear fit was applied to the isosbestic signal and aligned to the GCaMP signal. The fitted signal was subtracted from the 490-nm signal and subsequently divided by the fitted 405-nm signal. Fluorescence signals were then converted to robust z-scores in each trial using the median and median absolute deviation (MAD) of the baseline, defined as a 15-s epoch before the CS presentation. Neural activity was quantified either by the area under the curve (AUC) per second (Fig. 1) or the peak fluorescence (Figs. 2, 4). AUC per second was used in Figure 1 because there was a possibility of inhibited or reduced activity (Fig. 1E; reflected as a negative AUC value) for paired cues, and indeed our data show small but negative values for CS-B after extinction learning (Fig. 1K). In other cases, neural activity was compared using the peak fluorescence, since all showed increased fluorescence from baseline on CS or US presentation.
Two-photon imaging
First, the separate imaging files (one file for each trial) were concatenated in MATLAB and saved as a tiff file. The combined movie was motion-corrected using a non-rigid registration algorithm in Suite2p (Pachitariu et al., 2017) and saved as another tiff file. This motion-corrected imaging file was loaded in ImageJ and the regions of interest (ROIs; corresponding to a single neuron) were manually drawn on the basis of the mean and standard deviation projection images (McHenry et al., 2017). Fluorescence time-series were extracted for each ROI by averaging all pixels within the ROI for each frame. To remove potential contamination from neuropil or nearby dendrites/axons, we extracted the fluorescence from a ring-shaped region (after enlarging each ROI 1.5 times and excluding the original ROI) and removing pixels in other ROIs, if any. This neuropil fluorescence was subtracted from the ROI fluorescence after being scaled by a correction factor (cf.). Usually, the cf. is estimated as the ratio of fluorescence intensity between a blood vessel and neighboring non-ROI neuropil region; however, since we were not able to identify any blood vessels in our imaging datasets, we adopted a cf. of 0.6, which is within the range used in previous studies (Chen et al., 2013; Cox et al., 2016). Therefore, the neuropil-corrected fluorescence (Fcorrect) was calculated as: Fcorrect = FROI – cf. × Fneuropil, after FROI and Fneuropil were smoothed with a median filter (length: 4). We further normalized the fluorescence in each trial by calculating the robust z-score using the median and MAD of the baseline, defined as a 10-s epoch before CS presentation. Neural activity was quantified for each cell by calculating the AUC per second during the baseline and the CS presentation.
Statistical analysis
Sample sizes were determined to be comparable to previous similar studies with calcium imaging (Groessl et al., 2018; Lin et al., 2020). Statistical analysis was performed with GraphPad Prism 8 (GraphPad Software, Inc) or MATLAB (MathWorks). All statistical tests performed and results are stated in the figure legends and provided in detail in Source Data 1. Statistical tests were chosen according to the nature of the experiments and datasets. Paired or unpaired t tests were performed for single-value comparisons. When ANOVA (one-way or two-way repeated-measures) was performed for multiple trials or groups, post hoc Sidak's test was used to correct for multiple comparisons. To examine whether the DRNDA cue response was significantly different from baseline at the single-cell level (Fig. 4), the Wilcoxon sign-rank test was used to calculate the p value for each cell. Then, the false-discovery rate (FDR) correction was applied (q < 0.05) to correct for multiple comparisons. When the response of a neuron was statistically significant after the FDR correction, the mean value of the cue response was compared with baseline and declared “significantly increased” if it was larger or “significantly decreased” if it was smaller. No outliers were removed from any of statistical analyses.
Histology
Mice were euthanized with CO2 and transcardially perfused with 15 ml of ice-cold 1× PBS with heparin (10 U/ml), and then 30 ml of ice-cold 4% PFA. Mouse brains were removed from the skull and postfixed in 4% PFA at 4°C overnight. The PFA solution was switched to 1× PBS the next morning. Brains were cut into 50-µm coronal sections with a vibratome (VT1200, Leica Biosystems). Sections were stored in 1× PBS solution at 4°C until further processing. For immunohistochemistry, brain sections were first incubated in a 1× PBS solution with 0.1% Triton X-100 and 10% normal donkey serum (NDS) with primary antibodies at 4°C overnight. The next day, sections were thoroughly washed four times in 1× PBS (15 min each). Then brain sections were transferred to 1× PBS with 0.1% Triton X-100 and 10% NDS with secondary antibodies and left overnight at 4°C. The next morning, sections were washed as described above and mounted on glass microscope slides (Adhesion Superfrost Plus Glass Slides, Brain Research Laboratories). After the sections were completely dry, they were cover-slipped after applying DAPI-containing mounting media (Fluoromount G with DAPI, eBioscience). Fluorescent images were obtained with either a confocal (LSM 880, Carl Zeiss) or a fluorescence microscope (BZ-X, Keyence).
Results
To explore the encoding properties of DRNDA neurons, bulk fluorescence from DRNDA cells expressing jGCaMP7f (Dana et al., 2019) was recorded with fiber photometry as a proxy for population neural activity (Fig. 1A). Implanted fibers were well localized above DRNDA neurons (Fig. 1B,C). Mice underwent three stages of associative learning tasks (Fig. 1D). First, mice were trained in reward or appetitive learning, in which one auditory conditioned stimulus (CS-A) was paired with a sucrose reward (US) and a second stimulus (CS-B) was paired with no reward. Subsequently, mice underwent fear or aversive learning, in which the previously rewarded CS-A predicted no outcome and the previously unrewarded CS-B was paired with mild foot-shock. Finally, mice underwent extinction training, in which both CSs were paired with no outcome. Through these distinct stages of learning, the motivational salience and value of innately neutral cues were dynamically modulated by pairing them with outcomes with different valence.
DRNDA neurons dynamically track the motivational salience of conditioned stimuli. A, Schematic of the fiber photometry setup used for GCaMP (490 nm) and isosbestic (405 nm) excitation and detection of emitted signals in mice freely moving in an operant chamber, which had a speaker for presenting the CS sounds, a lickometer for delivering the reward, and metal grids for delivering foot-shocks. B, A representative histologic image of jGCaMP7f-expressing DRNDA neurons showing the location of the photometry fiber tip. C, Schematic of the anatomic locations for individual fiber implants. D, Three stages of associative learning with two cues (CS-A and CS-B). Reward learning was performed first, followed by fear learning and then extinction learning. E, Hypothetically, neurons that track motivational valence, such as DA neurons in the lateral VTA or those projecting to the NAc lateral shell (Matsumoto and Hikosaka, 2009; Bromberg-Martin et al., 2010; de Jong et al., 2019); should show increased activity to reward-paired cues after reward learning and decreased activity to shock-paired cues after fear learning, compared with baseline or before learning. These changes in activity should both be reduced to close to baseline after extinction learning. On the other hand, neurons that track motivational salience should show increased activity to both reward-paired and shock-paired cues, after reward and fear learning respectively, and return to close to baseline after extinction learning. F, Behavioral data summary. Mice successfully discriminated the CS at each stage: they showed increased anticipatory licks to CS-A (blue) after reward learning (##p = 0.0089, before vs after for CS-A; *p = 0.0336, CS-A vs CS-B after learning) and increased freezing behavior to CS-B (red) after fear learning (###p = 0.0004, before vs after for CS-B; ***p = 0.0006, CS-A vs CS-B after learning). G, Photometry response before learning for CS-A (blue) and CS-B (red), with the CS onset (black dotted line) and US onset (gray dotted line) indicated. Top panel, Individual trials from an example mouse. Bottom panel, Averaged photometry response from all animals. Scale bar here also applies to E–G. H, Same as G, but after reward learning. I, Same as G, but after fear learning. J, Same as G, but after extinction learning. Note the absence of a US onset. K, DRNDA neuronal response, quantified by the area under curve during cue presentation, tracks the change of salience in CS at each stage (####p < 0.0001, before learning vs after reward learning for CS-A; ****p < 0.0001, CS-A vs CS-B after reward learning; ###p < 0.0003, before learning vs after fear learning for CS-B; **p < 0.0048, CS-A vs CS-B after fear learning). Data are presented as the mean ± SEM.
We hypothesized that value-tracking neurons may show minimal activity to cues before pairing or after extinction, but show activation to cues with positive value and inhibition to cues with negative value, similar to DA neurons in the lateral VTA and those projecting to the NAc (Fig. 1E, left; Matsumoto and Hikosaka, 2009; de Jong et al., 2019). On the other hand, salience-tracking neurons may show minimal activity to neutral cues, but demonstrate excitation to both reward-predicting and shock-predicting cues, such as DA neurons in the lateral SNc or those projecting to basal amygdala (Fig. 1E, right; Matsumoto and Hikosaka, 2009; Menegas et al., 2017; Lutas et al., 2019).
Behavioral data showed that mice could discriminate the reward-predicting CS-A from the neutral CS-B, showing increased anticipatory licks to reward-predicting cues after training (n = 5 mice; two-way repeated measures ANOVA; F(1,8) = 4.583, ptime × CS = 0.0647; F(1,8) = 11.54, ptime = 0.0094; F(1,8) = 2.581, pCS = 0.1468; post hoc Sidak's test; CS-A vs CS-B after learning, *p = 0.0336; before vs after for CS-A, ##p = 0.0089; Fig. 1F, left). They also learned the contingency shifts with fear learning and responded appropriately, displaying increased freezing to shock-predicting CS-B over CS-A with no outcome (n = 5 mice; two-way repeated measures ANOVA; F(1,8) = 10.12, ptime × CS = 0.0130; F(1,8) = 33.83, ptime = 0.0004; F(1,8) = 11.09, pCS = 0.0104; post hoc Sidak's test; CS-A vs CS-B after learning, ***p = 0.0006; before vs after for CS-B, ###p = 0.0004; Fig. 1F, right).
Photometry data showed that before learning (day 1 of reward training), CS responses were small for both CSs, followed by increased activity on reward consumption (Fig. 1G; Cho et al., 2017; Lin et al., 2020). After reward learning, the reward-predicting CS-A induced robust excitation whereas the response to the neutral CS-B remained small (Fig. 1H). After fear learning, the CS-A response became smaller as it no longer predicted reward, and the CS-B response became larger, reflecting its pairing with the aversive US (Fig. 1I). After extinction learning, both CS responses were reduced to baseline (Fig. 1J). Collectively, these results suggest that DRNDA population activity dynamically tracks the motivational salience of cues through increases in activity, rather than the valence of the cue (n = 5 mice; two-way repeated measures ANOVA; F(3,24) = 14.98, ptime × CS < 0.0001; F(3,24) = 11.89, ptime < 0.0001; F(1,8) = 3.305, pCS = 0.1066; post hoc Sidak's test; CS-A vs CS-B after reward learning, ****p < 0.0001; CS-A vs CS-B after fear learning, **p < 0.0048; before learning vs after reward learning for CS-A, ####p < 0.0001; before learning vs after fear learning for CS-B, ###p = 0.0003; Fig. 1K; Groessl et al., 2018; Lin et al., 2020).
The motivational salience of cues may depend on the animal's internal state; for example, water-predictive cues may be highly salient to thirsty animals but are perceived as less salient and attractive if satiated. To test this idea, after mice were fully trained in the reward-learning task, they completed 50% of trials while thirsty and the other 50% while satiated (Fig. 2A). After satiety, mice stopped responding to the reward-predicting CS-A, as evidenced by the extinction of anticipatory licking [n = 5 mice; two-way repeated measures ANOVA; F(1,8) = 8.093, ptime × CS = 0.0217; F(1,8) = 43.93, ptime = 0.0002; F(1,8) = 7.688, pCS = 0.0242; post hoc Sidak's test; CS-A (blue) vs CS-B (red) when thirsty, **p = 0.0022; thirsty vs sated for CS-A, ###p < 0.0003; Fig. 2B]. Neural responses to the CS-A were also diminished after satiety while responses to the neutral CS-B remained unchanged (n = 5 mice; two-way repeated measures ANOVA; F(1,8) = 5.699, pstate × CS = 0.0440; F(1,8) = 9.393, pstate = 0.0155; F(1,8) = 11.68, pCS = 0.0091; post hoc Sidak's test; CS-A vs CS-B during thirsty, **p = 0.0015; thirsty vs sated in CS-A, ##p = 0.0097; Fig. 2C–E), suggesting that CS salience signals can be modulated by internal motivational states.
DRNDA neuronal responses are modulated by internal state and expectation. A, To test whether DRNDA CS responses were influenced by the animals' internal state, fully trained mice underwent half of a reward learning session while thirsty and completed the other half while sated. B, Behavioral response to reward-paired CS-A, quantified by anticipatory licks during CS presentation, was reduced after satiety (**p = 0.0022, CS-A vs CS-B when thirsty; ###p < 0.0003, thirsty vs sated after CS-A). C, Averaged CS-A response during thirsty (blue) and sated (green) states. Scale bar here also applies to D. D, Averaged CS-B response during thirsty (red) and sated (yellow) states. E, The CS-A response was significantly diminished after satiety, while the CS-B response showed no change (**p = 0.0015, CS-A vs CS-B after satiety; ##p = 0.0097, thirsty vs sated in CS-A). F, To examine whether DRNDA US responses were modulated by expectation, 5% sucrose or foot-shock were occasionally introduced in the absence of predictive cues after reward and fear learning, respectively. G, Averaged DRNDA response to expected (dark blue) versus unexpected (light blue) reward consumption. Photometry traces were aligned to consumption onset. Offset in the baseline in expected consumption came from CS-induced excitation. H, Unexpected reward consumption evoked higher neural activity than expected consumption, quantified by peak fluorescence (*p = 0.0470). I, Averaged DRNDA response to expected (orange) versus unexpected (yellow) shock delivery. Photometry traces were aligned to shock onset. Offset in the baseline in expected consumption came from CS-induced excitation. J, Unexpected foot-shock induced higher neural activity than expected shock delivery, quantified by peak fluorescence(*p = 0.0240). Data are presented as the mean ± SEM.
Ventral midbrain DA neuronal activity or DA release in the NAc can be modulated by surprise or expectation (Schultz et al., 1997; Patriarchi et al., 2018). To examine whether DRNDA responses to USs were modulated by expectation, mice received unexpected rewards or shocks in the absence of predictive cues, among fully predicted US during regular CS–US pairings (separately after reward or fear training; Fig. 2F). Photometry recordings demonstrated that DRNDA responses were larger for unexpected rewards than for expected consumption (n = 5 mice; paired t test; t(4) = 2.836, *p = 0.0470; Fig. 2G,H), similar to DA release in the NAc (Patriarchi et al., 2018), although the volume or identity of reward was identical. Additionally, DRNDA neurons showed larger responses to unexpected shocks than expected ones (n = 5 mice; paired t test; t(4) = 3.539, *p = 0.0240; Fig. 2I,J). Together, these suggest that DRNDA neuronal response to the same US, both of positive and negative valence, can be modulated by surprise or expectation.
DRNDA neurons track the motivational salience of CSs at the population level, as demonstrated by increases in bulk fluorescence to reward-paired or shock-paired cues (Fig. 1), but it is unclear how individual neurons are tuned to salient cues with distinct valence. To address this question, we performed two-photon imaging (Fig. 3A) to visualize calcium responses in single DRNDA neurons. For this purpose, GRIN lenses were implanted over the DRN to image GCaMP6m-expressing DRNDA neurons (Fig. 3B,C). To minimize stress or discomfort during head-fixed imaging, mice underwent extensive habituation training, up to 35 min/d for 8–10 d before data acquisition (for details, see Materials and Methods). We imaged individual DRNDA neurons from multiple FOVs (Fig. 3D) while mice performed associative learning tasks (Fig. 3E).
Two-photon imaging of DRNDA neurons during associative learning tasks. A, Schematic of the two-photon microscope setup, in which mice were head-fixed under the objective. Reward was provided through a lickometer and tail-shock was used as an aversive US. B, To visualize DRNDA neurons at the single cell level, AAV encoding cre-dependent GCaMP6m was injected into the DRN. GRIN lenses were implanted at a 25° angle, followed by implantation of a head ring for head-fixation. C, Schematic of the anatomic locations for the implanted GRIN lenses. D, Example FOVs, visualized as standard deviation projection images. E, Two stages of associative learning with two cues (CS-A and CS-B). Reward learning was performed first, followed by fear learning.
Before learning, only a small fraction of neurons showed significantly increased CS responses over baseline (significance was determined by Wilcoxon sign-rank test followed by FDR correction to account for multiple comparisons, q < 0.05; Fig. 4A). After reward learning, the majority of single DRNDA cells (∼67%) developed increased responses to the reward-predicting CS-A but not to the neutral CS-B with no outcome (Fig. 4B). Surprisingly, after fear learning, most DRNDA neurons did not show much of significant changes in activity from the baseline, even to the shock-predicting CS-B (Fig. 4C). The absence of aversive cue responses was puzzling, given that our and other previous results from freely-moving photometry or microendoscopic imaging showed robust increases (Fig. 1I; Groessl et al., 2018; Lin et al., 2020).
Single-cell DRNDA responses to CS and US during reward and fear learning, measured by head-fixed, two-photon calcium imaging. A, DRNDA neuronal responses to the CS before learning (day 1 of reward learning). Top panel, Population average of all imaged cells during CS-A (blue, left) and CS-B (red, right). Middle panel, Heatmap of the averaged CS responses of individual DRNDA cells during CS-A (left) and CS-B (right). Neurons are sorted by the AUC of the CS-A response. There were 65 neurons in total, from six FOVs in four mice. Bottom panel, Proportion of neurons that showed a significant increase, a significant decrease, or no change in activity in response to the CS, relative to baseline. Significance was determined by Wilcoxon sign-rank test followed by FDR correction to account for multiple comparisons (q < 0.05). B, Same as E, but after reward learning. There were 95 neurons in total, from eight FOVs in foue mice. C, Same as E, but after fear learning. There were 42 neurons in total, from three FOVs in three mice. D, Heatmap of the averaged CS and US response after reward learning in B of individual DRNDA cells during CS-A (left) and CS-B (right). Neurons are sorted by the AUC of the US-A (reward) response. E, Proportion of neurons that showed a significant change, decrease or no change in activity in response to the US, relative to baseline. F, Proportion of neurons that show significant change to CS-A and US. Data are presented as the mean ± SEM.
For the US responses of individual DRNDA cells, we could only analyze the reward responses since tail-shock used during fear learning introduced uncorrectable motion from body movement. After reward learning, still the majority of single DRNDA neurons (∼70%) showed significantly increased responses to reward consumption over the baseline, while expected omission after CS-B induced no change (Fig. 4D,E). We found that more than half of imaged DRNDA cells (∼57%) showed significant excitation to both CS and US over the baseline (Fig. 4F), while there were other DRNDA neurons with selective activation to either CS (∼11%) or US (∼12%). These data suggest that DRNDA neurons show mixed selectivity to reward-predicting cues and reward itself.
Absence of DRNDA response to aversive cues at the single-cell level was striking, and we reasoned that the change in behavioral context (freely-moving vs head-fixed) may have caused this unexpected variation in DRNDA responses. To test this hypothesis, we performed photometry in freely-moving and head-fixed mice undergoing a similar fear learning procedure (single session, six paired CS–US events; foot-shocks to freely-moving and tail-shocks to head-fixed mice, because of differences in experimental setups; Fig. 5A,B), now again at the population level. All mice were water restricted and habituated for head-fixation in the same manner, and then randomly assigned to one of the two groups. Freely-moving mice were able to learn the association within these trials, showing a progressive increase of freezing in response to the CS (n = 10 mice; one-way repeated measures ANOVA; F = 48.37; ****p < 0.0001; post hoc Sidak's test; trials 3–6 vs trial 1, ####p < 0.0001; Fig. 5C). Head-fixed mice showed licking behavior when head-fixed, as they were regularly given free reward during habituation training, but the initiation of fear learning task induced a rapid decrease in licking as they received repetitive CS–US pairings (n = 6 mice; paired t test; t(5) = 9.817, ***p = 0.0002; Fig. 5D,E). Notice that this does not necessarily reflect animals' learning of paired CS per se, as the extinction of lick can also result from repeated shocks or context.
Fear learning in freely-moving state and head-fixed setup. A, Mice were divided into two groups (freely-moving and head-fixed) and underwent fear learning experiments. Freely-moving mice received foot-shocks as the US in an operant chamber, whereas head-fixed mice received tail-shocks. B, Schematic of the anatomic locations for the implanted optical fibers (black: freely-moving group, magenta: head-fixed group). C, Quantification of freezing during the shock-predicting CS showed that the mice in the freely-moving group learned the CS-US association within six trials (****p < 0.0001; ####p < 0.0001, Trial 3-6 vs Trial 1). D, Raster plot of licks from six head-fixed mice (each row) during fear learning. Red triangles denote the onset of shock-predictive cues. Note that, before fear learning, these mice were already habituated to the head-fixation setup with occasional sucrose delivery, so they licked continuously at the start. This licking behavior reduced dramatically across repeated CS–US pairings. E, The number of licks was significantly decreased after six trials of fear learning compared with the baseline period before the first CS-US presentation (***p = 0.0002). Data are presented as the mean ± SEM.
In freely-moving mice, CS response to the shock-predictive cues gradually increased across paired trial, while head-fixed mice showed no significant change in CS responses across the same number of trials (n = 10 freely-moving mice, n = 11 head-fixed mice; two-way repeated measures ANOVA; F(5,95) = 6.243, ptrial × group < 0.0001; F(3.302,62.73) = 1.087, ptrial; = 0.3645; F(1,19) = 4.482, pgroup = 0.0434; post hoc Sidak's test; freely-moving vs head-fixed group in trial 6, *p = 0.0150; Trial 1 vs Trial 5 in freely moving group, #p = 0.0452; trial 1 vs trial 6 in freely moving group, ##p = 0.0042; Fig. 6A–C). This group difference at the neural level is unlikely to stem from the distinct shock methods, as both induced similar US responses throughout the learning (n = 10 freely-moving mice, n = 11 head-fixed mice; unpaired t test; t(19) = 1.056, p = 0.3041; Fig. 6D,E). Moreover, both groups showed learning to the paired CS by showing increased freezing behavior compared with the baseline during freely-moving recall sessions, albeit with a group difference (n = 4 freely-moving mice; n = 6 head-fixed mice; two-way repeated measures ANOVA; F(1,8) = 1.639, pstim × group = 0.2364; F(1,8) = 122.3, pstim < 0.0001; F(1,8) = 10.87, pgroup = 0.0109; post hoc Sidak's test; freely-moving vs head-fixed group during CS, *p = 0.0149; baseline vs CS in freely-moving mice, ####p < 0.0001; baseline vs CS in head-fixed mice, ###p = 0.0001; Fig. 6F). Altogether, these results indicate that aversive cue signaling by DRNDA cells, but not reinforcement itself, can be modulated by external behavioral context, especially during stressful situations where mice are forced to be immobile and receive repetitive aversive stimuli.
Population DRNDA responses to aversive, shock-predicting CS depend on the external context. A, Heatmap of the averaged photometry responses in the freely-moving group across 6 trials. Each row is the average response of all mice in the group. Note the gradual development of a time-locked CS response across CS–US pairings. B, Same as A, but for the head-fixed group. Note the absence of time-locked CS response even across repeated CS–US pairings. C, Freely moving mice (black) showed a significant increase in the CS response during fear learning, whereas head-fixed mice (magenta) showed no change (#p = 0.0452, Trial 1 vs Trial 5 in freely moving group; ##p < 0.0042, Trial 1 vs Trial 6 in freely moving group; *p = 0.0150, freely-moving vs head-fixed group in Trial 6). D, Averaged photometry response of foot-shocks (black) to freely-moving mice and tail-shocks (magenta) to head-fixed mice. E, Responses to foot-shocks (freely-moving mice, black) and tail-shocks (head-fixed mice, magenta) were not significantly different. F, Both groups showed increased freezing compared with baseline during the freely-moving recall test performed the next day (four CS presentations in a novel arena, averaged), albeit with a group difference (####p < 0.0001, baseline vs CS in freely-moving mice; ###p = 0.0001, baseline vs CS in head-fixed mice; *p = 0.0149, freely-moving vs head-fixed group during CS). Data are presented as the mean ± SEM. n.s., not significant.
Discussion
Detecting motivationally salient stimuli from the complex environment and choosing appropriate behavioral responses to them are indispensable for animal's survival. In this study, we demonstrated that DRNDA neurons contribute to this process by tracking the motivational salience assigned to the previously neutral cues with associative learning and extinction tasks in both appetitive and aversive contexts (Fig. 1; Groessl et al., 2018; Lin et al., 2020). These learning-dependent neural dynamics resemble salience-tracking DA neurons in the ventral midbrain, such as those projecting to the basal amygdala (Lutas et al., 2019). Additionally, responses to the same reward-predicting cues were modulated by internal motivational state (Fig. 2), correlating with animal's behavioral responses depending on their needs to detect and respond to the motivational salient stimuli.
It is well known that DA neural activity and release in response to the same reward can be modulated by expectation (Schultz et al., 1997; Patriarchi et al., 2018), showing increased firing and release when the reward was unexpectedly delivered or consumed. We showed that DRNDA neurons demonstrate similar property by showing higher population neural activity to unexpected reward over fully expected ones (Fig. 2G,H). Moreover, we demonstrate that DRNDA response to the aversive US was also enhanced when they were delivered without predictive cues, although the intensity of foot-shocks was kept identical (Fig. 2I,J). Together, these data indicate that expectation can modulate DRNDA neuronal activity regardless of valence of the US as unexpected stimuli could be perceived more salient than expected ones, possibly to affect downstream neural processing and adjust for proper behavioral responses as well. This is similar to basal forebrain cholinergic neurons that respond to both positive and negative reinforcers, whose activities were also scaled by surprise or the level of expectation, albeit more strongly for the appetitive US (Hangya et al., 2015). Future experiments should investigate whether DRNDA neurons indeed encode “unsigned prediction error,” whether their dynamics are modulated by deviation from expected value, such as different quantity or intensity for the same US.
We are only in the beginning of understanding the encoding dynamics of individual DRNDA neurons, as the in vivo single-cell recording of these neurons is challenging and scarce. Groessl et al., (2018) first presented the microendoscopic imaging of DRNDA neurons at the single-cell level during a freely-moving fear learning task, demonstrating that many of individual cells were activated by foot-shocks and paired cues. Our imaging data with two-photon microscope, to our knowledge, are the first attempt to examine DRNDA dynamics at the single-cell resolution across appetitive reward learning (Figs. 3, 4). Interestingly in fear learning (Groessl et al., 2018); the majority of responsive cells encoded the US itself at the late stage of learning, while our data during reward learning showed that more than half of DRNDA neurons jointly encoded CS and US and that there were separate minor populations that selectively signaled CS or US. At the fear recall session where mice were exposed to the aversive cues with no shocks, only ∼30% of neurons responded to the CS (Groessl et al., 2018). On the other hand, from our data in appetitive context, two thirds of cells showed significant excitation to rewarding cues (Fig. 4). The reason for this difference is unclear, but it may stem from distinct valence or time required for CS-US learning in these tasks, separate neural inputs, different signal-to-noise ratio of imaging modalities, as well as from freely-moving versus head-fixed state. For future studies, it would be of great interest to investigate whether the same DRNDA neurons can encode the motivational salience at the single-cell level, i.e., showing excitation to cues or stimuli with both positive and negative contexts or there are separate populations tuned to each valence, showing salience encoding property only at the population level.
Surgical implantation of GRIN lens over the DRN was a significant challenge for this study, because the vertical insertion of lens over the DRN can often cause lethal damage to the subject by rupturing the transverse sinus. This was the motivation for pursuing 25° angled implantation toward the deep brain structure – with this approach, we found that the lens went through the posterior cortex without damaging any major blood vessels. Another challenge for DRN imaging was its proximity to the aqueduct with continuous flow of cerebrospinal fluid. Indeed, our histology data showed that GRIN lens contacted or broke the aqueduct (also possible from histologic processing), which could have introduced uncorrectable motion when mice received mild tail-shocks. Future studies aiming for more stable imaging of DRN cells should take these into considerations: one potential solution may be to use a GRIN lens with 90° reflection (e.g., “PRISM” lens) to better avoid a contact with aqueduct. Moreover, as recently demonstrated (Gong et al., 2020), attaching tungsten wires around the cylindrical GRIN lens can provide additional mechanical support to achieve stable and high-quality imaging of hard-to-reach brain regions like the DRN during diverse behavior tasks.
The absence of neural responses to aversive cues during head-fixation, both at single-cell and population levels (Figs. 4, 6), was unexpected and striking, given that our results and previous studies from others have shown robust responses to aversive cues when animals are freely moving (Groessl et al., 2018; Lin et al., 2020). From our finding that DRNDA cue responses track the motivational salience attributed to the cues (Fig. 1), it is conceivable that aversive cues across fear learning could be no longer perceived as motivationally salient to the subject, because painful US cannot be avoided or escaped.
To minimize the potential confounding effect of discomfort from the head-fixation setup, mice in this study were extensively habituated to the setup with longer periods of time than a published protocol (Guo et al., 2014). Indeed, mice showed signs of sufficient habituation, as they continued to consume rewards, even after 35 min of immobile head-fixation during training. Besides habituation, two different delivery methods of electrical shock (tail vs foot) as an aversive reinforcement can be another confounding factor. However, tail-shocks are also commonly used in fear learning paradigms in head-fixed setups (Kim et al., 2016; Yu et al., 2017) and, in our photometry data, they evoked similar neural activation as foot-shocks given to freely-moving animals (Fig. 6). It is also possible that the level of arousal could be distinct across freely-moving versus head-fixed animals; this may be possible in the beginning of sessions, as we observed larger responses to the first cue in head-fixed mice. However, repetitive CS-US delivery involving painful shocks should have increased the level of arousal in both groups of animals to similar levels. Indeed, we observed the biggest difference in CS response in the last trial, suggesting that the possibly distinct level of arousal from two setups may not have played significant roles in generating such drastic differences in CS responses.
Head-fixation is widely used in imaging and behavioral experiments because of the need for mechanical stability or convenience, but the effects on animal behavior or neural activity are often assumed to be negligible. However, in rodents, head-fixation affects vocalization behavior (Weiner et al., 2016). At the neural level, acute head restraint reduces the reward and cue responses of VTA DA and DRN serotonergic neurons (Zhong et al., 2017). A recent study demonstrated that chronically head-fixed mice showed higher corticosterone (CORT) level over control subjects, even up to 25 d of daily training (Juczewski et al., 2020). It was previously shown chronic changes in peripheral CORT level can negatively regulate the DA transmission in the cortex and striatum (Lindley et al., 1999). Our findings extend these observations and demonstrate that neural responses to similar behavioral experiments can be affected by chronic head-fixation, especially in highly stressful and inescapable contexts.
This study builds on previous findings that DRNDA activity signals the motivational salience of cues in a learning-dependent manner, increasing in response to CSs that are paired with outcomes of either valence and declining with extinction (Groessl et al., 2018; Lin et al., 2020). We additionally demonstrated that DRNDA responses to the same CS or US can be modulated by internal state, expectation, and even external behavioral context. The dynamic nature of salience encoding by DRNDA neurons may serve as a “gain control” in downstream processing in the extended amygdala (Kash et al., 2008; Groessl et al., 2018) through both fast-acting glutamate and modulatory DA (Matthews et al., 2016; Li et al., 2016). Together, motivational salience signaling by DRNDA neurons can contribute to orient attention toward encountered stimuli of high importance and enable the selection of appropriate behavioral responses, dependent on various factors such as internal state, the level of expectation or external context.
Footnotes
This work was supported by the National Institutes of Health (NIH) Director's New Innovator Grant IDP20D091182-01, Presidential Early Career Award for Scientists and Engineers, the NIH/National Institute on Aging Grant 1R01AG047664-01, the NIH Grant BRAIN 1U01NS090577, Heritage Medical Research Institute, Chen Institute (V.G.), Beckman Institute (V.G. and D.A.W.), Tianqiao & Chrissy Chen Institute Chen Graduate Fellowship (X.C.), Colvin Divisional Fellowship at Caltech (A.K.), and the Children's Tumor Foundation Young Investigator Award 2016-01-00 (to J.E.R.). We thank the entire Gradinaru lab for critical feedback.
The authors declare no competing financial interests.
- Correspondence should be addressed to Viviana Gradinaru at viviana{at}caltech.edu
