Abstract
It has been argued that the dopaminergic system is involved in the attribution of motivational value to reward predictive cues as well as prediction error. To evaluate, dopamine neurons were recorded from male rats performing a Pavlovian approach task containing cues that have both “predictive” and “incentive” properties. All animals learned the predictive nature of the cue (illuminated lever entry into cage), but some also found the cue to be attractive and were motivated toward it (“sign-trackers,” STs). “Goal-trackers” (GTs) predominantly approached the location of reward receptacle. Rats were implanted with tetrodes for neural electrophysiological recordings in the ventral tegmental area. Cells were characterized by spike waveform shape and firing rate. Firing rates and magnitudes of responses in relation to Pavlovian behaviors, cue presentation, and reward delivery were assessed. We identified 103 dopamine and 141 nondopamine neurons. GTs and STs both showed responses to the initial lever presentation (CS1) and lever retraction (CS2). However, higher firing rates were sustained during the lever interaction period only in STs. Further, dopamine cells of STs showed a significantly higher proportion of cells responding to both CS1 and CS2. These are the first results to show that neurons from the VTA encode both predictive and incentive cues, support an important role for dopamine neurons in the attribution of incentive salience to reward-paired cues, and underscore the consequences of potential differences in motivational behavior between individuals.
SIGNIFICANCE STATEMENT This project serves to determine whether dopamine neurons encode differences in cued approach behaviors and incentive salience. How neurons of the VTA affect signaling through the NAcc and subsequent dopamine release is still not well known. All cues that precede a reward are predictive in nature. Some, however, also have incentive value, in that they elicit approach toward them. We quantified the attribution of incentive salience through cue approach behavior and cue interaction, and the corresponding magnitude of VTA neural firing. We found dopamine neurons of the VTA encode strength of incentive salience of reward cues. This suggests that dopamine neurons specifically in the VTA encode motivation.
Introduction
Dopamine neurons have been implicated in many different psychological functions related to reward. It has long been important in learning, prediction error, and reward evaluation. The way in which it may be involved in addiction is uncertain and controversial. More recently, evidence has shown a role for dopamine in incentive motivation. The pattern of anatomic connectivity of the mesolimbic dopamine circuit supports the idea of a role in reward mechanisms clearly. Dopamine neurons in the ventral tegmental area (VTA) project to the nucleus accumbens (NAcc) core and shell. Interestingly, dopamine signals in the NAcc result in differing responses in the core and shell. Dopamine release in the core is biased toward encoding prediction error of cues, specifically that phasic release of dopamine to reward-predictive cues vary as a result of anticipated reward value (Day et al., 2011; Creed et al., 2014; Saddoris et al., 2015; Gillis and Morrison, 2019). Alternatively, the shell region of the NAcc responds to changes in incentive salience (i.e., motivation for rewards) (Wheeler et al., 2011; Saddoris et al., 2015; Saunders et al., 2018). Thus, dopamine release is important to guide learning and response-rewarded behaviors.
The ability of cues to acquire motivational value varies among individuals and has important implications in addictive behavior (Berridge and Robinson, 2003; Flagel et al., 2008; Berridge, 2012). One way to detect individual differences in cue-driven motivation is through Pavlovian conditioning, where a discrete cue is paired with a palatable food reward. Some animals (sign-trackers [STs]) approach and interact with the cue, whereas others (goal-trackers [GTs]) approach food receptacle during cue presentation (Flagel et al., 2007). All individuals learn the predictive nature of the cue (i.e., prediction error). Only in STs does the cue acquire motivational value (i.e., the attribution of incentive salience) (Berridge, 2007) that elicits approach behavior (Saunders and Robinson, 2010, 2011).
STs and GTs differ in dopamine release patterns in the NAcc (Flagel et al., 2007) and neural firing patterns observed in the ventral pallidum (VP) (Ahrens et al., 2016, 2018). Rats with a sign-tracking phenotype show cue-evoked dopamine release patterns in the NAcc core that are not seen in GTs (Flagel et al., 2011). In a direct comparison of incentive salience attribution to prediction-related events, STs showed a relative increase in dopamine release in response to cues in contrast to stable cue-related dopamine release in GTs, supporting a role for the attribution of incentive salience to cues (Berridge, 2007; Flagel et al., 2011). Dopamine release in the NAcc has also shown to be important in acquisition and expression of a cue-directed conditioned response (CR) (Di Ciano et al., 2001; Blaiss and Janak, 2009; Chang et al., 2012) This stands in contrast to the proposed enhanced dopamine activation, as suggested by the proponents of dopamine/learning mechanism perspective (Schultz et al., 1997).
The objective of this study is to investigate differences associated with attribution of incentive motivation to reward-predictive cues. The firing patterns of dopamine and nondopamine cells in the VTA have not been directly examined in STs and GTs; therefore, this study used in vivo electrophysiology to record single-unit activity in the VTA of rats as they performed their preferred ST or GT CRs. We found that dopamine cells, but not nondopamine cells, showed enhanced activity in STs, but not GTs, during exposure to a lever cue. This enhancement was seen as sustained changes in firing during the period of cue exposure when rats performed sign-tracking CRs; however, there were no group differences in the immediate phasic responses to the onset of the cue, which is thought to encode primarily predictive signal (Tindell et al., 2005). These findings support the idea that one of the primary roles of dopamine is to modulate the state of incentive motivation that is triggered in some individuals by reward-paired cues.
Materials and Methods
Animals and care
A total of 28 male Sprague Dawley rats were used with an initial weight of 200-250 g (Charles River). Males were housed in pairs in a reverse light:dark (14:10) cycle with lights off at 10:00. Upon arrival, they received 2 d to habituate to their new surroundings. Testing began 1 week after acclimation and handling. They remained in pairs until electrode implantation when they were housed individually. All testing was performed during the dark cycle, between 10:00 and 18:00 with water and food available ad libitum throughout the study (except while in testing chamber). All procedures were approved by the University of Michigan Committee on the Use and Care of Animals and Institutional Animal Care and Use Committee.
Pavlovian conditioned approach (PCA)
Animals (n = 28) first underwent PCA training to determine phenotype. Day 1 served as magazine training with delivery of 25 banana-flavored food pellets (unconditioned stimulus [UCS]) (BioServ) on a variable time 30 s schedule. PCA training took place on the next 5 d. The Pavlovian trial had a predictive cue, consisting of an illuminated lever (conditioned stimulus [CS]) inserted through the wall into the cage for 8 s. The reward pellet was released from the food dispenser at the moment the lever was retracted, traveled through 18 in. of tubing, and arrived at the food dish 1.2 s later. There were 25 trials per session presented on a variable time 90 s (30-150 s) schedule. Importantly, pellet delivery was independent of subject response.
PCA indexing
The degree of incentive behavior was quantified using lever presses and magazine entries recorded during the 8 s period when the lever was inserted into the cage. Three measures were calculated for each trial: overall counts of lever presses and magazine beam breaks, the probability of performing either a lever-directed or magazine-directed CR, and the latency to contact the lever and/or magazine. These three measures were combined into a single PCA score that rates sign/goal-tracking tendencies on a scale of −1 to 1. The PCA score was determined by the following: (1) latency difference [(time to approach magazine during CS – time to approach lever)/8]; (2) response bias [(# lever deflections – # magazine entries)/(# lever deflections + # magazine entries)], and (3) approach probability difference [(probability of contacting lever – probability of contacting magazine)] (see Meyer et al., 2012). A score of <−0.5 indicated a GT phenotype (n = 11), >0.5 indicated a ST phenotype (n = 6), and −0.5 to 0.5 indicates an intermediate phenotype (excluded from further study, n = 11).
Electrodes and surgery
Electrodes were manufactured in the laboratory. A total of 32 Teflon-coated wires (12.5 µm, Sandvik) were arranged into two individually moveable bundles containing 16 wires each and were implanted bilaterally. Each hemisphere contained four tetrodes, four wires wound and heat-fused together. A grounding wire was also soldered between the electrode board and bone screw. Electrode was connected to an OmniPlex D signal amplifier system (Plexon), which transmitted signals from the wires to a recording computer for analysis.
Electrode bundles were sterilized before implant. Animals were anesthetized with isoflurane (initially 3%, maintained 2%-2.5%). Body temperature was maintained using a temperature therapy pad. Hair was removed, and the incision site was scrubbed with a disinfectant solution (e.g., sterile iodine). An incision was made (∼2 in. in length) in a rostrocaudal direction from just behind the eyes to just behind the ears. To ensure the head was level, the difference in DV measurement between bregma and lambda was not >0.25 mm. The target coordinates for electrode implant were as follows: VTA at AP, 4.9 mm; ML, ±1.0 mm; DV, 8 mm; and a 1 mm craniotomy was made. Recording electrodes were implanted 1 mm above the VTA and lowered that distance on the day of the first recording. Bone screws and dental acrylic (Dentsply, Henry Schein Medical) were used to anchor electrode to skull and to completely cover incision. Animals were treated with a topical analgesic (lidocaine hydrochloride jelly, Thermo Fisher Scientific), Triple Antibiotic Ointment (Thermo Fisher Scientific), flunixin (2.5 mg/kg, FlunixiJect, Henry Schein Medical), and penicillin (Henry Schein Medical) immediately as well as for 2 d after surgery. Subjects were given 7 d to rest before neural recording began. Animals were singly housed after surgery, and enrichment (toilet paper tubes, shredded paper, etc.) was provided.
Neural discrimination and analysis
Dopaminergic neurons from the VTA were targeted using unique characteristics making them distinguishable from other neuron types (Roesch et al., 2007). Dopamine neurons were identified by three defining characteristics (Pan et al., 2008): (1) a low basal firing rate (<10 Hz), (2) long spike duration (>1.2 ms), and (3) a >50% decrease in firing rate following apomorphine injection (Fig. 1A) (Aebischer and Schultz, 1984; Jo et al., 2013). Recorded neurons were discriminated from noise using Offline Sorter (Plexon). Neural reactivity was analyzed during the period 10 s before lever extension to 5 s after lever retraction using a laboratory-prepared custom database application (The Form, University of Michigan) and NeuroExplorer (Plexon). Cross correlations were run to ensure no redundancies in the discriminated neural waveforms.
Neural recording classification and location. A, Cluster analysis defining dopamine and nondopamine neurons. Black dots represent neurons classified as dopamine with corresponding waveform example. Dotted line indicates 1.2 ms, the cutoff for waveform length to differentiate dopamine neurons. B, Placement of electrodes for GTs (red) and STs (blue) was confirmed with cresyl violet staining of coronal slices. Electrodes were lowered 40-80 µm each day. Dashed line indicates total advancement of neurons. Bars represent total area of recording. Only those days when electrodes were completely within VTA were analyzed.
Response to task events
The mean rate of firing for each unit was calculated during four periods defined with respect to task behavioral events. (1) CS Onset was defined as the period from lever extension to 400 ms after lever presentation. This period relates to the audiovisual cue of lever presentation. (2) The Cue Interaction period was defined as the interval from 1 s to 8 s after lever presentation. It is during this period when subjects were engaged in a CR. STs typically interacted with the lever and GTs interacted with the food cup. (3) CS Offset was defined as the 0-400 ms period following the moment of lever retraction, which was simultaneous with feeder click to release pellet. (4) The UCS period included the reward delivery and pellet consumption, a period 600-1600 ms following CS Offset. During this period, STs leave the lever and approach the food cup. Typically, GTs are already there. Events were compared with a baseline rate, 200 ms bins over a 5 s period before CS presentation. A responsive neuron is one with a significant change from baseline (see Statistics). Those trials where the mean baseline was zero were deleted from the analysis (no more than 9 trials per unit/session). Only those neurons with at least 10 good trials (baseline rate > 0 spikes/s) were included in the analysis. As baseline firing rates differ between cells, mean and SD were calculated. This was used to calculate z score during epochs of interest to normalize firing rate changes (magnitude) of individual neurons and perform population analyses of responsive neurons. Because of the unique firing patterns of dopamine neurons, with slow “tonic,” baseline firing characterized by single spikes and short bursts of firing followed by long (>160 ms) quiescent periods (Grace and Bunney, 1983, 1984a,b), the z score during baseline resides at ∼−0.5 for both GTs and STs. As the z score is only used to make comparisons between phenotypes, this deviation from 0 is not considered an issue.
Response coding
The population of responsive neurons was calculated as a proportion of total neurons analyzed. We compared proportions of neurons responding to the specified events (CS Onset, CS Offset, etc.) between ST dopamine and GT dopamine, and ST nondopamine and GT nondopamine neurons.
Response to behavior
Response of dopamine neurons to movement-related events were also compared as changes in neural firing magnitude. A z score was calculated for magnitude changes to (1) sign-orienting and (2) magazine entry (described in behavior procedure). GTs rarely approached and interacted with lever (<1 contact per trial), so comparisons with lever contact were not possible.
The CR of STs and GTs differ, whereby GTs direct their attention to the magazine and exhibit nose poking actions. By contrast, STs direct their attention toward the lever and exhibit bites, paw grabs, and sniffs. The vigor of CR is directly related to level of incentive salience. To further support that changes in firing are because of differences in attribution of incentive salience, a Spearman correlation analysis was performed on phenotypic index scores (related to the attribution of incentive salience, toward −1 for GTs or 1 for STs) and the average response magnitude of all cells from that individual rat. The magnitude of all units was averaged for each subject and each time period tested, and then plotted as a function of index score. In this manner, each subject is only allotted one data point and represents the contribution of all neurons in response to the Pavlovian task. The results further allow us to determine how motivational output relates to firing rates of neurons.
Verification of dopamine-like neurons
Neural correlates of behavioral events during the Pavlovian task were assessed in STs and GTs on a total of 103 dopamine-like (henceforth called “dopamine”) and 141 nondopamine-like (henceforth called “nondopamine”) neurons (Table 1). No neurons were recorded from 2 STs and 4 GTs. A subset of subjects (n = 9) were injected with apomorphine (0.75 mg/kg, subcutaneous) or 9% saline immediately after PCA testing sessions (n = 11) were completed. While recording from the same neurons as during the PCA session, animals were tested with 25 additional PCA trials. A total of 59 neurons, 32 dopamine and 27 nondopamine were analyzed for pharmacological effects of apomorphine injection. Of the dopamine neurons, 77% showed a reduction (>40%) in firing rate following apomorphine injection and were confirmed to be dopamine.
Neurons per subject analyzeda
Behavioral analysis
Subject's behavior was video recorded from three cameras positioned around chamber and synchronized in time to trial start. Cameras were angled to give view of lever, food cup, and overall chamber. Videos were analyzed on a frame-by-frame (1/30 s) basis by computer to determine onset of movement using custom software (DataRat, Aldridge Lab). Sign-orienting was scored once per trial and is the moment that subjects turned toward wall with lever and food cup, after lever presentation. If subjects were already facing the wall, sign-orienting was scored as moment of lever presentation. Following orient behavior, subjects could approach either lever or food cup and were scored as first movement toward stimulus and was scored once per trial. Expression of CRs targeted toward different stimuli was compared between subjects. Lever deflections were part of a sign-tracking CR and were moments when rats pressed lever either with their nose, teeth, or paw. Food cup entries were defined moments when animals placed their nose in the food cup, either as a goal-tracking CR or in pellet retrieval. Animals were tested daily for as long as neurons could be recorded, up to 12 d (three full turns, with one-third turn each day). Most animals (GTs, n = 9; STs, n = 5) were tested for 8 d with a few (GTs, n = 3; STs, n = 3) were tested for the full 12 d.
Histology
Twenty-four hours after the final testing day, a small marking lesion was made to facilitate confirmation of final recording site location. The lesion was made by passing a small current (0.5 mA) through one wire in each tetrode bundle for 30 s. Subjects were killed 48 h later using an overdose of sodium pentobarbital (Fatal Plus, Vortech Pharmaceuticals). Brains were removed and frozen fresh using dry ice and then stored at −20°C before sectioning.
Brains were sliced coronally and stained with cresyl violet. Electrode bundle placement was confirmed for each testing day using the Paxinos and Watson (1997). This allowed for the accurate assessment of the location from which neurons were recorded (Fig. 1B). Electrophysiological recordings were only used on days in which bundles were within the VTA.
Statistics
Where datasets failed tests for normality, nonparametric analyses were performed. To determine responsive neurons, a Kruskal–Wallis and Bonferroni-corrected pairwise Mann U post hoc tests were performed, comparing the mean firing rates during the Pavlovian task to baseline on a trial basis to determine significant rate changes (increase or decrease) during the testing session (α < 0.05/15). The z score was also analyzed between responsive ST and GT neurons (dopamine, nondopamine separately) for each epoch using nested t tests. Further, to ensure that results were not driven by any single rat, as variability of recorded neurons from subjects ranged 2-21, the PCA index was compared with z score for each Pavlovian epoch. A correlation analysis was also performed on units responsive to CS Onset to determine whether intertrial interval related to firing magnitude. The z score was compared with intertrial interval on a per trial basis for each unit.
Excitatory and inhibitory firing rate differences were compared between neural types (dopamine and nondopamine) of STs and GTs for each time interval using a nested t test. Proportions of neurons showing excitatory or inhibitory responses were compared using a χ2 or Fisher's exact test. Total population analyses were performed using Fisher's exact test.
Behavioral differences between STs and GTs were compared using a Bonferroni-corrected t test. Statistics were performed with GraphPad Prism version 8 and R version 3.0.3.
Results
Dopamine neurons respond to incentive cues
Neurons were recorded from groups of animals that respond differently to Pavlovian cues, STs and GTs. Neural responses were evaluated at four time points in each trial: (1) CS Onset, (2) Cue Interaction period, (3) CS Offset, and (4) UCS (Fig. 2A). Baseline firing rates of dopamine neurons were similar between STs and GTs, ranging from 0.37 to 4.58 spikes/s for STs and 0.3 to 2.29 spikes/s for GTs and did not differ between phenotypes (t(118) = 0.49, p = 0.62). Dopamine neurons (n = 103) in both phenotypes were equally responsive overall, showing a significant change in firing during one or more epochs compared with baseline over the entire 13 s of the Pavlovian task (STs: 76%; GTs: 77%; Fig. 2B). To the specific events (CS Onset, Cue Interaction, etc.), proportions of responsive dopamine neurons differed significantly only during Cue Interaction (Fig. 2C). Proportions of nonresponsive neurons did not differ (Fig. 2D). Examples of typical responsive neurons from STs (left) and GTs (right) are shown, including markers of behavioral events determined from video ratings (Fig. 2E).
Normalized response magnitude of dopamine neurons to Pavlovian cue presentation. A, Pavlovian task. Every session contained 25 trials. In each trial, an illuminated lever was inserted into the cage at random intervals between 30 and 150 s. Lever was presented for 8 s, then retracted. At the same time, a feeder released a banana pellet. There was a slight delay between when pellet was released and when it was available in food magazine. Data are presented as mean, SEM B, Dopamine neurons from ST (blue) and GT (red) responsive to Pavlovian task were normalized to a background interval using a z score and averaged across the time period just before and after lever presentation. C, Bar graph represents proportion of neurons responsive to each part (highlighted periods) of Pavlovian task. D, Bar graph of proportion of nonresponsive neurons. E, Example of a dopamine neuron from an ST (left) and GT (right). Red lines indicate time of lever onset and offset. Horizontal blue line represents baseline firing (5s prior to lever onset). Blue triangle represents lever contact. Pink triangle represents magazine entry. Green triangle represents pellet in mouth.
All neurons responsive to any part of the Pavlovian task were analyzed to compare neural activation by firing rate z scores (STs, n = 45; GTs, n = 34; Fig. 3A). The difference was most apparent during the Cue Interaction period, when STs interacted with the lever and GTs approached and interacted with the food cup. Lever (cue) interaction in STs was associated with strong activation in 30% (13 of 45 responsive neurons) of dopamine neurons (Fig. 3B). In contrast, not a single dopamine neuron of 34 responsive GT units was responsive during food cup engagement in the same time period (Fig. 3B). The z score of GTs during this time appears to show an overall inhibitory response, but this is actually a return to baseline. Most of the ST lever-interaction responses (77%) were excitatory in nature resulting in an overall ramping up of dopaminergic cell activity beginning 1 s after lever insertion (Fig. 3A,B). Because the firing rates both increased and decreased compared with baseline, the magnitude of firing changes during the interaction period was not significantly correlated with the degree to which an animal expressed an ST or GT phenotype, that is, PCA index score (r = 0.48, p = 0.12) (Fig. 3C). This was because of the inhibitory response of one ST. Insertion of the lever cue triggered strong CR toward the lever in STs, but it should be emphasized that GTs were also interacting vigorously with the food dish. Despite the similarity in motor activation in the two phenotypes, the focus on the incentive cue by STs was associated with strong VTA dopaminergic activation in contrast to relative quiescence in GT neurons.
Magnitude response of dopamine neurons to Cue Interaction. Neurons were analyzed for their response to Cue Interaction, the last 7 s of lever presentation during which the subjects are engaged in a CR. Responses are either directed toward lever (STs, blue) or location of pellet receipt (GTs, red). A, Magnitude of all responsive neurons (z score) was greater in STs than GTs. B, Proportion of excitatory and inhibitory responses in STs and GTs. Most of the responses from STs were excitatory. None of the neurons from GTs was responsive during this period. C, Correlation of neural firing (all neurons) with PCA index, that is, the propensity to interact with either lever (toward 1.0) or food cup (toward −1.0). ***p < 0.01, STs compared with GTs. Data are mean, SEM.
Response to lever retractions + feeder click (CS2) and delivery + consumption of the reward (UCS) exposed a second ST/GT difference. Both phenotypes (STs and GTs) showed a gradual increase in firing rate during CS2 that peaked at UCS (Fig. 4A). CS Offset, the retraction of the lever and simultaneous pellet feeder click, was the final cue in the sequence that was followed immediately by reward delivery. It activated dopamine neurons in both GTs and STs (Fig. 4A–D). Magnitude of response did not differ (t(4) = 1.80, p = 0.15; Fig. 4C,D). Although the overall neural participation during both periods is high, relatively few cells in either STs (11%) and none in GTs responded to both CS Offset and UCS delivery (Fig. 4B). This pattern suggests that dopamine neurons are coding these behavioral events separately.
Magnitude of dopamine neuron response to Cue Offset and reward delivery. Neurons from STs (blue) and GTs (red) were normalized (z score) and aligned to the lever retraction (CS Offset). A, The magnitude of responsive neurons averaged during the time of CS Offset and delivery of UCS. B, Pie charts represent proportion of neurons responsive to CS Offset (black), UCS (light gray), or both (dark gray) for STs (left) and GTs (right). Most neurons were not responsive to both events. Excitatory and inhibitory responses of STs (C) and GTs (D) to CS Offset. Neural response of STs (E) and GTs (F) to pellet delivery (UCS). Dotted lines indicate moment of lever retraction. Shaded regions represent CS Offset and UCS epochs. Pie charts represent overall percentage of neurons responsive to each task along with proportion of excitatory (E) and inhibitory (I) responses. Neural firing magnitudes were greater in STs compared with GTs. G, Magnitude of neural firing to CS Offset and (H) UCS in relation to PCA index. There was a significant correlation to UCS. Data are mean, SEM. *p < 0.05.
The proportion of CS Offset responsive neurons in STs was slightly greater than in GTs (13 of 45, 29% vs 6 of 34, 17%), but the difference was not significant (Fisher's exact test, p = 0.44). Similarly, no differences were found between number of excitatory versus inhibitory responses, although ∼67% of the neurons responding to CS Offset were excitatory in STs, but inhibitory in GTs (Fisher's exact test, p = 0.35).
In STs, however, the response to pellet delivery was sustained after the lever retraction and was significantly higher than response magnitude of GTs (t(8) = 2.85, p = 0.02; Fig. 4E,F). Although the magnitude of firing rate was greater, the proportion of responsive neurons in STs compared with GTs (ST: 29 of 45, 63%, GT: 19 of 34, 56%) was not significant (Fisher's exact test, p = 0.72). The pellet reached the food dish 1 s after CS Offset, during the UCS phase when it was retrieved and consumed. In both STs and GTs, most neurons (90% of UCS responsive cells, ST: 26 of 29, GT: 17 of 19) were responsive to UCS by excitation (Fig. 4E,F). Magnitude of excitatory response was significantly higher in STs (t(41) = 3.22, p = 0.002). This suggests that those subjects with a propensity to attribute incentive value on reward cues also exhibit greater firing rate changes to reward itself. Inhibitory responses at UCS were small and weaker in magnitude in GTs. These profile differences were not different significantly (t(2) = 0.59, p = 0.61). There were no significant differences between STs and GTs in proportion of excitatory responses (Fisher's exact test, p = 0.85) or inhibitory responses (Fisher's exact test, p = 1).
Although the rates of activation were slightly higher in STs, the magnitude of change to CS Offset was not related to PCA index (r = 30, p = 0.33; Fig. 4G). However, neural activity associated with UCS was correlated to ST or GT phenotype (r = 0.57, p = 0.05; Fig. 4H). Thus, overall, dopamine neurons in STs reflect greater attribution of incentive value to cues.
Reward cues activate dopamine neurons in both STs and GTs
Predictive cues activated dopamine neurons in both STs and GTs. The first cue, consisting of the lever insertion (CS Onset), carries the most information as it predicts all consecutive events in the Pavlovian sequence. The actual occurrence of this onset signal was unpredictable as the intertrial interval varied in length randomly. Dopamine neurons in both phenotypes were highly responsive to this predictive cue (Fig. 5A,B; STs: 38%, 17 of 45; GTs: 32%, 11 of 34). The magnitude of firing rate changes did not differ between STs and GTs (excitation: t(4) = 0.85, p = 0.41; inhibitions: t(5) = 0.16, p = 0.88). Nor did the combined magnitude (t(4) = 0.20, p = 0.85; Fig. 5C), and there was no correlation between normalized rates compared with PCA index (r = −0.12, p = 0.72; Fig. 5D). A correlation was also performed to compare z score with intertrial interval per unit to determine relation to surprise of lever presentation. Firing magnitude relating to surprise also did not show any significant results (ST: R2 values = 0.002-0.14; p values = 0.07-0.83; GT: R2 values = 0.002-0.07, p values = 0.19-0.83). These results indicate that STs and GTs are coding the predictive properties of lever presentation equally.
Magnitude differences of dopamine neurons to Cue Onset. Firing rates of responsive neurons were normalized for STs (blue) and GTs (red) to determine magnitude of change to cue onset (lever presentation, shaded region). A, Excitatory and inhibitory responses of STs, with size of pie chart representing overall number of neurons responsive to CS Onset. E, Excitatory; I, inhibitory. B, Excitatory and inhibitory responses of GTs and respective pie chart of proportion of responses. C, The change in firing rate for all responsive neurons. D, The magnitude of firing of all cells in relation to phenotypic index. E, Proportions of neurons responsive to CS Onset (predictive cue), CS Offset (incentive cue), or both were compared in ST (left) and GT (right) populations (not significant) as well as between STs (left) and GTs (right). Data are mean, SEM. Percentages are of total cells recorded. #p < 0.01.
The proportions of cells responsive to the phasic cue events CS Onset and CS Offset, like the magnitudes of the responses, also did not differ between STs and GTs. However, STs, which attribute incentive value to cues had proportionally more responsive cells than GTs to both cues (Fisher's exact test, p = 0.04; Fig. 5E). In STs, 59% (10 of 17) of the cells responded to both CS Onset AND CS Offset, whereas <25% (2 of 11) responded to both in GTs, indicating different coding mechanisms to the 2-cue Pavlovian task.
Neural encoding: motivation versus movement
There is abundant evidence that dopamine neuron activity reflects multiple aspects of reward and motor behavior, including roles in both action and motivation (Saunders et al., 2018; Mohebi et al., 2019). One action in this Pavlovian task that is similar in both ST and GT phenotypes consists of a brief orientation toward the lever at the moment it is inserted into the chamber (Yager and Robinson, 2013; Ahrens et al., 2016). Like the action itself, neural activity correlated with it was similar in both STs and GTs, although there with a trend of greater activation in STs (t(16) = 2.112, p = 0.051; Fig. 6A). A second action analyzed was nose poke into food magazine over the time period from lever presentation through pellet consumption. Despite the fact that GTs made more magazine food cup entries (average: 5 per trial compared with STs 2 per trial), dopamine neural activation in STs evoked by contacts with the magazine was more intense overall than activation in GTs (t(16) = 3.29, p = 0.005; Fig. 6B). This is consistent with positive correlations in neural firing magnitude with phenotypic index: that STs show greater firing over the time period tested. Further, all animals showed similar number of contacts (STs-lever interactions, GTs-magazine entries) during the testing period, not dependent on phenotype (t(8) = 1.48, p = 0.18) (Fig. 6C) indicating similar motor intensity.
Movement during Pavlovian conditioning task. STs (blue) and GTs (red) were analyzed for their responses to movement specific to the Pavlovian task. A, Neural firing of dopamine (DA) was compared with sign-orienting, a neutral movement and (B) magazine entry, representing motivationally relevant movement. Data are mean, SEM C, Average number of motor responses per trial across all sessions tested for each subject, represented by PCA index score (not significant). **p < 0.001.
The question remains: could the neural activation patterns of the phenotypical CRs simply be encoding the differences in movement between goal-tracking and sign-tracking? During the interaction period, behavioral intensity toward the lever in STs and toward the magazine in GTs was similar; however, dopamine neural activation in GTs was essentially unresponsive and flat, whereas activity in STs ramped up (Fig. 1). If dopamine neurons are playing a role in motor control, that responsibility is not engaged in GTs during the interaction period. Alternatively, this difference in activation in the two phenotypes may represent instead the differential motivational incentive salience properties that distinguish STs (Hamid et al., 2016).
Nondopamine neurons encode incentive salience
Of the 141 nondopamine neurons recorded, 67% were responsive overall, with no significant differences in the proportion of responsive neurons between STs (73%, 50 of 68) and GTs (61%, 44 of 73; χ2(1) = 2.40, p = 0.12), although response patterns showed some differences. In particular, nondopamine neurons encoded characteristic differences to the four events of the Pavlovian task (Fig. 7A). Proportions of neurons responding to these four events were similar between STs and GTs (Fig. 7B), as were proportion of nonresponsive cells (Fig. 7C) and are described in detail below. The direction of neural firing changes, however, were different with STs showing an excitatory response to the 8 s cue presentation, whereas GTs showed a greater inhibitory response (Fig. 7D,E).
Patterns of nondopamine neuron firing. A, Magnitude of response was calculated (z score) for responsive nondopamine neurons in STs (blue) and GTs (red) during Pavlovian task. B, Proportion of responsive neurons to each of event of the Pavlovian task and (C) proportion of nonresponsive nondopamine neurons. D, Example on nondopamine ST (left) and E) GT (right). Data are mean ± SEM. Red bars represent CS Onset and CS Offset. Blue line represents baseline firing rate. Blue triangle represents lever contact. Pink triangles represent magazine entry. Green triangle represents mouth pellet.
Compared with dopamine neurons, there was no difference in overall number of responsive neurons in either STs or GTs (χ2(3) = 5.50, p = 0.14). Proportions of nondopamine neurons responsive to each of the four Pavlovian events of interest were similar in STs (χ2(3) = 1.29, p = 0.73) but different in GTs (χ2(3) = 15.67, p = 0.001). More nondopamine neurons were responsive to Cue Interaction and CS Offset in GTs compared with dopamine neurons. A nested one-way ANOVA showed no difference in firing magnitude between nondopamine and dopamine neurons to CS Onset (F(3,10) = 0.23, p = 0.87), Lever interaction (F(2,33) = 1.55, p = 0.22), CS Offset (F(3,55) = 2.61, p = 0.06), and UCS (F(3,15) = 0.59, p = 0.63).
Cue responses in nondopamine neurons
The Cue Interaction period activated nondopamine neurons in both GTs and STs, which stands in contrast to dopamine neurons where only ST neurons were activated. The Cue Interaction responses were equally excitatory and inhibitory (Fisher's exact test, p values = 0.26-0.41; Fig. 8A,B). Firing magnitude did not differ (inhibition: t(2) = 1.78, p = 0.22; excitation: t(11) = 1.36, p = 0.20; Fig. 8A,B) in GTs compared with STs. Comparing the net absolute changes in magnitude revealed no overall firing magnitude difference in STs compared with GTs (t(21) = 1.49, p = 0.15; Fig. 8C). The degree of phenotype difference (PCA index) was unrelated to neural firing rate magnitudes in nondopamine neurons (r = −0.06. p = 0.86; Fig. 8D), which contrasts with Cue Interaction responses in dopamine neurons.
Nondopamine response to Cue Interaction. Neurons were analyzed for their response to Cue Interaction, the last 7 s of lever presentation during which subjects are engaged in a CR. Responses are either directed toward location of pellet receipt (GTs, red) or toward lever (STs, blue). A, Magnitude (z score) of excitatory and inhibitory responses of ST neurons responsive to Cue Interaction was calculated during this time. Pie chart size represents overall percentage of (B) excitatory and inhibitory magnitude of GT neurons responsive to Cue Interaction. C, Magnitude of all neurons responsive to Pavlovian task. D, Correlation of magnitude with PCA index, that is, the propensity to interact with either food cup (toward −1.0) or lever (1.0). Data are mean ± SEM.
Nondopamine neuronal responses were analyzed to the predictive cue, CS Onset. Similar proportions of neurons had excitatory responses in GTs (66%, 13 of 20 of CS Onset responses) and STs (50%, 9 of 17 of CS Onset responses) (Fisher's exact test p = 0.14; Fig. 9A,B). There was no statistical significance between STs and GTs for magnitude of excitatory (t(20) = 0.02, p = 0.99) nor for inhibitory (t(5) = 0.1.11, p = 0.32) responses (Fig. 9A,B). There were also no differences in neural firing magnitude overall to predictive CS Onset (t(6) = 0.74, p = 0.49; Fig. 9C). There was also no correlation between magnitude of response and propensity to approach lever (PCA index; r = −0.32, p = 0.34; Fig. 9D). These results are similar to those of dopamine neurons.
Nondopamine coding properties to CS Onset. Neurons responsive to Pavlovian task were analyzed for inhibitory and excitatory responses. Size of pie chart relates to proportion of neurons responsive to CS Onset. A, STs. B, GTs. C, Magnitude of individual neurons firing to Pavlovian task was calculated using z score. D, Firing magnitude of all responsive neurons was compared with PCA index. An index of −1.0 to −0.5 indicates GT, whereas an index of 0.5 to 1.0 indicates ST. Data are mean ± SEM.
To the CS Offset (lever retraction/feeder click), there was no difference in overall magnitude between STs and GTs (t(38) = 0.38, p = 0.70; Fig. 10A), nor for excitatory (t(19) = 0.30, p = 0.77) and inhibitory (t(5) = 0.79, p = 0.46) responses (Fig. 10B,C). Proportions of inhibitions were also similar and represented approximately two-thirds of the responses seen in STs and GTs to CS Offset (Fisher's exact test, p = 0.73).
Nondopamine coding properties to CS Offset and reward delivery. A, Nondopamine neurons responsive to Pavlovian task were normalized to baseline (z score) and presented as absolute value. Excitatory and inhibitory firing rate changes were calculated specifically to CS Offset for STs (B) and GTs (C). Neurons specifically responsive to pellet delivery were calculated for STs (D) and GTs (E). Correlations of PCA index and magnitude to CS Offset (F) and reward delivery (G). All neurons were responsive to Pavlovian task. Data are mean ± SEM.
In response to UCS (pellet delivery and consumption), STs and GTs showed similar firing rate changes to pellet delivery (t(7) = 0.08, p = 0.94; Fig. 10A). Between STs and GTs, both excitatory (t(27) = 0.97, p = 0.34) and inhibitory (t(3) = 0.55, p = 0.62) responses were not different (Fig. 10D,E). STs showed equal proportions of excitatory and inhibitory responses, whereas they were mostly excitatory in GTs. There were no differences in proportion of excitatory responses (Fisher's exact test, p = 0.54), but there were significantly more inhibitory responses in STs compared with GTs (Fisher's exact test, p = 0.004). There was no correlation with PCA index scores and either CS Offset (r = −0.06, p = 0.86) or UCS (r = −0.27, p = 0.42) (Fig. 10F,G). Together, these results indicate that individuals attributing incentive value to Pavlovian cues are using different neural coding strategies within the VTA in dopamine neurons, whereas nondopamine neurons often overlap in their response patterns in STs and GTs.
Discussion
Individual differences in approach behaviors to reward-paired cues, and the attribution of incentive salience, were reflected in patterns of neural activity in the VTA. Firing rates of dopamine neurons from STs were higher than GTs, although CRs were of the same intensity, and this trended with the degree of expression of ST character. Although there was GT engagement at the goal, no responsive dopamine neurons in GTs were observed during the interaction phase. It is unlikely the difference could be explained by motor correlates since, despite the energetic movements of GTs, DA neuron activity was flat. The strong activation in STs suggests instead a clear role for dopamine neurons to encode incentive salience attribution, which is a trait apparent in STs (Flagel et al., 2007; Meyer et al., 2012). The faster firing in ST dopamine neurons during the Cue Interaction phase extended to and reached peaks during UCS pellet delivery and consumption exceeding similar peaks in GTs. This difference was also seen in the nose poke action, a motor response exhibited by both STs and GTs, although magnitude was greater in STs. In contrast to dopamine neurons, the proportions of nondopamine neurons responding to Cue Interaction and CS Offset were not differentiated in the two phenotypes. Future work to distinguish nondopamine projection and interneurons may clarify these differences. The somewhat stronger response to cue onset and offset and absent incentive cue responses in GTs suggests that, in contrast to STs, they may use a predictive rather than incentive behavioral strategy preferentially.
The present study extends the distinction between neural coding patterns to predictive (CS Onset) and incentive (CS Offset) cues in VTA seen earlier in the VP (Tindell et al., 2004, 2005; Ahrens et al., 2016). The characterization of incentive salience attribution in animals with ST tendencies and the dopamine neural coding patterns relating to the attribution of motivational value echoes the findings of Saunders et al. (2018) showing Pavlovian cues imbued with motivational value. The complex dynamics and differences of VTA dopamine signaling in the two phenotypes may reflect interplay of reward prediction error learning signals and motivational activation reported by Mohebi et al. (2019). It underscores the importance of the phenotypical characterization of this study for assigning dopaminergic function and potential strategies different phenotypes might use to accomplish behavioral goals.
The first cue, CS Onset, served as an equal predictor of future reward delivery and elicited a similar sign-orienting response in all subjects (Yager et al., 2015). The magnitude of this brief neural activation (rate code) and the proportions of responsive dopamine neurons (population coding) to CS Onset did not differ between STs and GTs, indicating that VTA dopamine neurons encode the predictive properties of the cue and/or sign-orienting action equally. Midbrain dopamine neuron responses to predictive cues have been described by others (Schultz et al., 1997; Pan et al., 2005; Jo et al., 2013). Neural response patterns in the VP, however, in the same paradigm differ from these VTA responses. Predictive cues in the VP were more deeply modulated in STs compared with GTs (Ahrens et al., 2016). This difference may reflect circuit modulation from NAcc or through other inputs with implications for dopamine release in the NAcc and propagation of learning and motivation signals downstream. Indeed, the increased VTA magnitude to the second cue, CS Offset, compared with CS Onset in STs may reflect a role for dispensing incentive value onto predictive cues.
As STs attribute incentive salience to the lever and only dopamine cells from STs show a response to lever presentation, our results support the hypothesis that neural firing patterns of the ventral basal ganglia may encode individual differences in motivation. Others have also found a role specifically for VTA dopamine neurons in incentive motivation, particularly those projecting to the NAcc core (Saunders et al., 2018). A similar pattern is observed in VP neurons (Ahrens et al., 2016), suggesting that incentive value signals are transmitted and maintained through the mesolimbic circuit separately from motor signals. These results clearly delineate a role for dopamine neurons in incentive motivation.
Experiments to differentiate incentive salience attribution from motor responses in VP neural firing patterns showed very little overlap in VP neurons between predictive (2.8%), incentive (6%) cues and sign-orienting behaviors (Tindell et al., 2005). In the same Pavlovian task used here, a comparison of VP firing patterns between lever and magazine interactions showed no rate differences (Ahrens et al., 2016), making a pure motor explanation unlikely. Our observation in VTA that the initial sign-orienting action at CS Onset evoked identical neural responses in both STs and GTs supports this view. The motor response to magazine entry (at CS Offset) was also similar in STs and GTs, but neural firing magnitude in STs was greater than GTs. This suggests that the activation in STs may be imbued with an additional component related to incentive value of the reward; the motor action itself appears to be imbued with greater incentive value in STs than GTs. Thus, while all STs express similar motor behaviors during the Cue Interaction period, dopamine neurons fire to a greater extent in those who attribute incentive value to cues. In agreement with our proposal, Saunders et al. (2018) found a dissociation of dopamine neurons in the VTA and SNc, suggesting that SNc dopamine activation is related to locomotion while VTA is related to motivation.
The majority (60%) of dopamine neurons were responsive to pellet delivery and were significantly higher in ST than GT dopamine neurons. Other studies have shown that dopamine neurons fire to reward delivery during training, but shift to cue onset rather than reward once cue-reward association is learned (Schultz et al., 1997; Schultz, 1998). Such patterns are also seen with dopamine release in the NAcc (Day et al., 2011) and neural firing patters within the NAcc core (Gillis and Morrison, 2019). In contrast, our study suggests that both STs and GTs continue to signal reward receipt long after the learned cue-reward association. Hamid et al. (2016) have also reported ramping up in dopamine neurons after the reward cue, as seen in current paper, with rates dependent on reward value. There may also be a component related to higher motivational value on the pellet reward in STs.
In the present study, we found greater numbers of inhibitory signaling nondopamine neurons in STs to pellet receipt; however, we did not dissociate local from projection neurons, which have opposing effects on VTA dopamine neurons (Alcantara et al., 2003; Brown et al., 2012; Cachope et al., 2012). We also found a greater proportion of nondopamine neurons responding to Cue Interaction and CS Offset in GTs. These results may affect neural coding downstream from the VTA in addition to local VTA dopamine neurons and contribute to the blunted response of GTs to the interaction and UCS phases. Other studies have found that aversive stimuli selectively activate local GABA neurons (Cohen et al., 2012; Tan et al., 2012), indicating a role in learning motivationally relevant properties of associated stimuli.
Prior studies have indicated dopamine neurons signal through both phasic and tonic patterns, each providing unique information to downstream sites (Grace, 1991; Goto et al., 2007; Owesson-White et al., 2009). Phasic, in this paper, refers to a short burst of firing lasting 300-400 ms (up to 1 s), relating to synaptic levels of dopamine and is quickly modulated by reuptake by presynaptic transporters. Tonic refers to periods of extended activity relating to extracellular dopamine levels and is a result of multiple dopamine neurons firing in concert (i.e., population coding) (Floresco et al., 2003). Studies have indicated that phasic dopamine release may be involved in learning of a predictive stimulus (Goto and Grace, 2005; Saddoris et al., 2015), whereas tonic release may be involved in the motor responses that follow. As population coding is more consistent with extended dopamine release and STs showed higher response proportions to CS Onset, CS Offset, and Cue Interaction, it can be predicted they would also show extended dopamine release in the NAcc and propagate signals of incentive salience.
In conclusion, we found differences in neural firing patterns within the VTA dependent on trait differences in responses to Pavlovian cues. A brief burst of dopamine neuron activation was similar in all subjects to predictive cues. In contrast, dopamine neuron firing during the period between cue presentation and reward delivery was enhanced in sign-tracking animals alone. Animals with this trait exhibited an incentive salience strategy to approach and interact with the cue. Dopamine neurons in all animals continued to respond to the reward even after it was well learned, which is in contrast to reports in primates (Schultz, 1998). These results suggest that targeting dopamine neuron activation specifically in animals that attribute incentive value to reward cues may alter or halt behaviors directed at maladaptive signs, such as drug cues.
Footnotes
This work was supported by National Institutes of Health PAF ID: 09-1793 and National Institute on Drug Abuse PAF ID: 11-PAF05335. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.
The authors declare no competing financial interests.
- Correspondence should be addressed to Lindsay M. Ferguson at LMFerguson{at}mednet.ucla.edu
