Abstract
Initiating a reward-seeking behavior involves deciding on an action, how fast to initiate the action (initiation vigor), as well as how much effort to exert. These processes are thought to involve the mesolimbic dopamine system. Dopamine levels in the ventral striatum rise before initiating a reliably reinforced behavior. However, it is unknown whether dopamine is similarly involved with unreinforced actions (inactive lever presses, premature food port entries, insufficient number of active lever presses). Furthermore, does the dopamine response when initiating an action reflect specific aspects of motivated behavior, such as initiation vigor and exerted effort? Here, we analyzed voltammetry recordings of dopamine levels in the nucleus accumbens (NAcc) core and shell in rats working for food under a progressive ratio reinforcement schedule. We examined dopamine levels when rats initiated distinct actions (active lever presses, inactive lever presses, food port entries) that were temporally separated from cue- and reward-evoked dopamine release. Active lever pressing bouts were preceded by elevated dopamine release in the NAcc shell, as well as in the NAcc core, although only when rats exhibited high initiation vigor. Dopamine levels were transiently reduced in the NAcc core following an unreinforced food port entry and were unchanged throughout the NAcc when initiating inactive lever presses. The effort exerted and vigor to initiate a bout of active lever presses were signaled by dopamine transmission in the NAcc core, but not in the NAcc shell. These results demonstrate that the dopamine response when initiating a behavior is both region- and action-specific.
SIGNIFICANCE STATEMENT Exogenous activation of the mesolimbic dopamine system facilitates motivated behavior. However, a direct relationship has not been established between endogenous phasic dopamine transmission and measures of motivation, such as the vigor to initiate an action and the effort exerted in a bout of activity. The present work demonstrates that the dopamine response when initiating an action depends both upon where dopamine is released and what action is performed. Furthermore, dopamine reflects measures of motivated behavior selectively within the nucleus accumbens core.
Introduction
The motivation to engage in reward-seeking behavior involves a series of decisions. First, one needs to decide on an action, then determine how fast to initiate the action (initiation vigor), as well as how much effort one will exert. The mesolimbic dopamine system is involved in these neurobiological processes as activation of the dopamine system decreases the latency to initiate an action (Cole and Robbins, 1989; Weafer and de Wit, 2013) as well as enhances the effort exerted to earn a reward (Zhang et al., 2003; Treadway et al., 2012). Conversely, attenuating dopamine transmission increases the delay before engaging in a task (Cole and Robbins, 1989) and reduces the exerted effort (Floresco et al., 2008b; Salamone and Correa, 2012).
Different patterns of dopamine transmission can elicit distinct behavioral outcomes (Goto and Grace, 2005; Tsai et al., 2009; Bass et al., 2013). Dopamine neurons can fire in a pacemaker manner, which contributes to basal or tonic dopamine levels, or fire in bursts, leading to a phasic elevation in dopamine concentration (Goto et al., 2007). Previous studies implicate tonic dopamine levels in facilitating the vigor to initiate an action (Niv et al., 2007), as well as the effort exerted in a particular action (Beeler et al., 2010), although the role of phasic dopamine release in these processes is largely unknown. Phasic electrical or optogenetic activation of midbrain dopamine neurons is sufficient for initiating conditioned approach behavior (Phillips et al., 2003; Ilango et al., 2014), although these exogenous manipulations of the dopamine system cannot establish whether endogenous dopamine transmission encodes behavioral measures of motivation. Dopamine neuron firing and phasic dopamine release within the ventral striatum are elevated before initiating a reward-seeking action (Phillips et al., 2003; Roitman et al., 2004; Jin and Costa, 2010; Wassum et al., 2012), although it is unclear whether this dopamine response precedes unreinforced behaviors as well. Furthermore, the motivation-related signals contained in the dopamine response when initiating an action are not well characterized because this has primarily been examined when minimal effort is required to earn a reward (i.e., a single lever press) (Phillips et al., 2003; Roitman et al., 2004), or under conditions where there is a fixed action-sequence leading to reward delivery (Jin and Costa, 2010; Wassum et al., 2012).
To address these questions, we analyzed fast-scan cyclic voltammetry recordings of dopamine levels in the nucleus accumbens (NAcc) core and shell when rats worked for food rewards under a progressive ratio (PR) reinforcement schedule. In this self-paced task, rats engage in bouts (single or repeated instances of a given behavior) of different unreinforced actions with varying amounts of vigor and effort. During trials rats can initiate bouts of inactive lever presses, bouts of premature head entries to the food receptacle, and bouts of active lever presses that do not immediately result in the delivery of a reward. Here, we examined the phasic dopamine response just before, and immediately following the initiation of a new behavioral action. The inherent behavioral variability in this self-paced task allowed us to examine whether the dopamine response when initiating a bout reflects (1) the type of behavior that is performed, (2) the initiation vigor (time elapsed since the end of the previous bout), and (3) the effort exerted in a bout (number of consecutive repeats of a particular action). Our results demonstrate that the dopamine response when initiating a behavior is both region- and action-specific. Notably, the effort exerted and the vigor to initiate a bout of active lever presses was reflected by dopamine transmission selectively within the NAcc core.
Materials and Methods
Dataset.
The current study examined dopamine levels during the initiation of action-specific bouts of behavior collected from previous experiments where male Sprague Dawley rats worked for food rewards under a PR reinforcement schedule (Wanat et al., 2010, 2013). Dopamine in the NAcc was detected using fast-scan cyclic voltammetry with chronically implanted electrodes and was isolated from the voltammetric signal using chemometric analysis (Heien et al., 2005) using a standard training set of stimulated dopamine release detected by chronically implanted electrodes (Clark et al., 2010; Wanat et al., 2010, 2013). Dopamine concentration was estimated based upon the average post-implantation sensitivity of electrodes (Clark et al., 2010). Voltammetry data were smoothed using a three-point moving average. The histologically determined location of recording sites is presented in Figure 1A with 18 electrodes in the NAcc core and 10 electrodes in the NAcc shell, collected from a total of 27 rats. One rat had electrodes in both the NAcc core and NAcc shell that were included for analysis. A subset of the rats were implanted with guide cannulae targeting the ventral tegmental area (Wanat et al., 2013), although no pharmacological manipulations were performed before or during the sessions analyzed in this study.
Rats were trained to lever press for food as described previously (Wanat et al., 2010, 2013). Briefly, rats were placed on mild food restriction (∼15 g/d of standard laboratory chow) to target 90% free-feeding weight, allowing for an increase in weight of 1.5% per week. Operant chambers (Med Associates) contained two retractable levers with corresponding cue lights on either side of the food tray on a single wall. Behavioral sessions began with both levers extending (active and inactive), and illumination of the house light and the cue light over the active lever. Completion of the correct number of lever presses on the active lever led to the delivery of a food reward (45 mg food pellets, BioServ), retraction of the levers, and the cue and house lights turning off for a 30 s intertrial interval. Rats were first trained to press for food rewards under a fixed-ratio (FR-1) reinforcement schedule in sessions consisting of 60 trials. After completing FR-1 training on both levers, the operant requirement was increased up to an FR-8 schedule before voltammetry recordings were performed in conjunction with behavioral sessions. During PR sessions, the operant requirement on each trial (T) was the integer (rounded-down) of 1.4(T−1) lever presses, starting at 1 lever press. PR sessions ended after 15 min elapsed without completion of the response requirement in a trial. We analyzed data collected from the first two PR sessions as this ensured an identical training history for all the subjects included in this study (Wanat et al., 2010, 2013). Rats were trained on an FR-4 reinforcement schedule on days preceding PR sessions, although the data from these FR-4 sessions were not examined due to the negligible number of bouts that could be analyzed.
Data analysis.
We examined the dopamine response during trials when rats initiated a bout (single or repeated instances of a given action) of active lever presses, inactive lever presses, or head entries to the food tray. Analysis of dopamine levels was performed on a 2 s window centered on the first action in a bout. To prevent any confounding influence of cue-evoked dopamine release on our analysis, bouts were excluded if initiated within 10 s of the trial-onset cue presentation (Wanat et al., 2010, 2013). Bouts initiated <1 s following the end of the previous bout were also excluded to mitigate the influence of the dopamine response from the previous action on the analyzed bout. We examined all active lever-pressing bouts, including those that were eventually reinforced. However, we did not analyze voltammetry data collected following the reward delivery to prevent any confounding influence of reward-evoked dopamine release. The dopamine response for all bouts of a particular action (active lever, inactive lever, or head entry) was averaged for each electrode, which ensured the data from each electrode were given equal weight in the analysis. A permutation test was performed on the average response across electrodes to establish where dopamine levels significantly differed from null values when initiating a bout of activity (Ko and Zeh, 1988; Nichols and Holmes, 2002; Kobayashi and Schultz, 2008).
The permutation test involves comparing the observed difference between two datasets relative to the difference between permuted datasets (see Fig. 2A–C). To perform this test, we compared the collected data relative to a null dataset that was derived by mean centering the collected data to zero. In this manner, the null dataset has an equivalent number of samples (i.e., the number of electrodes included for the analysis; n = 18 for NAcc core, n = 10 for NAcc shell) and the same error as the collected dataset. Next, we generated two permuted time series datasets (Perm1 and Perm2) that were the same size as the collected and null datasets (n = 18 for NAcc core; n = 10 for NAcc shell). Perm1 and Perm2 datasets were created by a random selection of the time series from both the collected and null datasets. For example, Perm1 could include the collected data from electrode 1, the null data from electrode 5, the null data from electrode 4, etc. Deriving the permuted datasets in this manner ensured no overall group difference between the Perm1 and Perm2 datasets. The maximum difference between the mean time series of Perm1 and Perm2 was then calculated. This random resampling process was repeated 1000 times to generate a confidence interval at all points in the time series (p < 0.05), which was adjusted to account for both positive and negative changes in the dopamine response relative to null values. We did not adjust for multiple comparisons when analyzing the data in a time series, consistent with published reports (Kobayashi and Schultz, 2008; Hamid et al., 2016; but see Hollon et al., 2014; Syed et al., 2016).
To examine the relationship between dopamine levels and measures of motivated behavioral output, we indexed the dopamine response during the bout initiation to the time that had elapsed since the last action (initiation vigor) and the number of instances an action was repeated in a bout (effort exerted). The behavioral data for each rat and for each type of bout (active lever, inactive lever, or head entry) was sorted into tertiles and log transformed to linearize the data (for example analysis, see Fig. 3). For each electrode, we calculated the average dopamine response corresponding to the tertile-sorted behavioral data. This normalization sorting allowed us to visualize and analyze the dopamine response across animals that exert different levels of activity in this PR task (Wanat et al., 2010). For example, assume Rat A initiates 60 active lever bouts and Rat B initiates 90 active lever bouts. The tertile sort of the data based upon the initiation vigor or effort exerted creates three groups (low, medium, or high), comprised of 20 bouts per group for Rat A and 30 bouts per group for Rat B.
After sorting the behavior and voltammetry data into tertiles, we plotted the dopamine response as a function of the behavioral response for each electrode and at each time point in the 2 s time series. We then calculated the slope of this relationship for each electrode at each time point in the series. The resulting slope as a function of time was averaged across electrodes and statistically analyzed using the permutation test as described above. Deviations in this measure (slope of dopamine response vs behavior) from null values indicate when dopamine transmission reflects the assessed measure of motivated behavior (initiation vigor or exerted effort) during bout initiation. These data are graphically presented in a manner where positive deflections on the y-axis reflect a positive relationship between dopamine release and the behavioral measure of motivation. Because initiation vigor is inversely related to the time since the previous action (i.e., greater initiation vigor equals less time), these corresponding graphs plot the inverse slope on the y-axis (see Figs. 3⇓⇓–6). All other statistical analyses were performed in Prism (GraphPad) using Student's t tests. Data are presented as mean ± SEM unless otherwise noted.
Results
In this study, we examined endogenous phasic dopamine release in rats initiating self-driven behavioral actions to determine whether this dopamine signal is uniform across different types of behaviors and to assess whether this signal reflects measures of motivation, such as initiation vigor and exerted effort. To address these questions, we analyzed fast-scan cyclic voltammetry recordings of dopamine transmission in the NAcc core (n = 18 electrodes) and NAcc shell (n = 10 electrodes) from rats (n = 27; Fig. 1A) working for food rewards under a PR reinforcement schedule (Wanat et al., 2010, 2013). There was no difference in the number of trials completed in PR sessions between rats with NAcc core electrodes and rats with NAcc shell electrodes (unpaired Student's t test, p > 0.05; Fig. 1B). Rats engaged in a number of behaviors in this self-paced task, including bouts (single or repeated instances of a given behavior) of active lever presses (n = 2003), inactive lever presses (n = 1071), and head entries into the food receptacle tray (n = 2046; Fig. 1C). These action-specific bouts occurred primarily at the end of PR session, with 84% of bouts occurring within the last three trials (Fig. 1D,E). The motivation to engage in these bouts was reflected by the initiation vigor (time elapsed since the end of the previous bout) and the amount of effort exerted (consecutive repeats of a given action). The initiation vigor and the effort exerted in bouts were no different between rats with electrodes in the NAcc core and rats with electrodes in the NAcc shell (unpaired Student's t test, p > 0.05 for all comparisons; Fig. 1F,H). The distribution of the effort exerted and the initiation vigor for a bout expectedly varied across trials in this self-paced PR task (Fig. 1G,I). When approaching the breakpoint trial, rats initiated bouts after longer delays (Fig. 1G), decreased the number of active and inactive lever presses in bouts, and increased the number of head entries in bouts (Fig. 1I). Although the distribution of the effort exerted and the initiation vigor for a bout varied, there was considerable overlap with these distributions as the rats approached the breakpoint trial.
Rats engage in self-driven bouts of repetitive actions when working for food rewards under a progressive ratio reinforcement schedule. A, Voltammetry recording sites with solid black circles representing NAcc core placement (n = 18) and open blue circles representing NAcc shell placement (n = 10). B, There was no difference in the number of trials completed between animals with electrodes in the NAcc core and shell. C, Example of behavioral activity during this task where a rat initiates bouts of active lever presses, head entries, and inactive lever presses when working for food rewards. Red triangles represent the initiation of a bout. D, E, The number of bouts increased as rats approached the last trial in a session in animals with electrodes NAcc core (D) and NAcc shell (E). F, The amount of time elapsed since the end of the previous bout (initiation vigor) was no different between animals with electrodes in the NAcc core or shell. G, Distribution of the initiation vigor for bouts across trials plotted as the median with the error bars denoting the 25% and 75% quartiles. H, The number of repeats of an action in a bout (exerted effort) was no different between animals with electrodes in the NAcc core or shell. I, Distribution of the exerted effort for bouts across trials plotted as the median with the error bars denoting the 25% and 75% quartiles. For head entry bouts, the lower quartile is identical to the median (one head entry per bout) and is not illustrated.
The dopamine response to cues and rewards in this PR task was no different between the NAcc core and shell (Wanat et al., 2010). However, it is unknown whether the dopamine response when initiating a bout of activity depends upon the type of behavior that is initiated, as well as where dopamine is released in the ventral striatum. To address these questions, we analyzed the dopamine response during trials when rats initiated a bout of activity. We excluded bouts occurring within 10 s of the trial-onset cue to prevent any contamination of the cue-evoked dopamine response in our analysis (Wanat et al., 2010). The dopamine response following the reward delivery during active lever bouts was not evaluated to prevent the confounding influence of reward-evoked dopamine release on the analysis. Significant changes in dopamine levels from null values were based upon a permutation test between the collected data and the mean centered null dataset (Fig. 2A–C). Dopamine release to an unexpected reward delivered outside of the behavioral task was no different between the NAcc core and NAcc shell (unpaired Student's t test, p > 0.05; Fig. 2D,E). While dopamine levels in the NAcc core across rats were unchanged when initiating a bout of active lever presses (permutation test, p < 0.05, n = 18 electrodes; Fig. 2F, middle), NAcc shell dopamine levels were elevated before the first active lever press in a bout (permutation test, p < 0.05, n = 10 electrodes; Fig. 2F, right). Following unreinforced head entries into the food tray, dopamine levels were transiently reduced in the NAcc core (permutation test, p < 0.05; Fig. 2G, middle), a response that was absent in the NAcc shell (Fig. 2G, right). Initiating an inactive lever-pressing bout was not associated with a change in dopamine levels in either the NAcc core or shell (permutation test, p > 0.05; Fig. 2H). A comparison of the dopamine response between transitions found that the transient reduction in NAcc core dopamine levels after initiating a head entry bout was not significantly different from the dopamine response after initiating active or inactive lever bouts (permutation test, p > 0.05). In contrast, the elevation in NAcc shell dopamine levels before initiating an active lever bout was significantly different from the dopamine response before initiating a head entry bout (permutation test, p < 0.05). Collectively, these results illustrate that the dopamine response when initiating an unreinforced behavioral bout is both region- and action-specific.
The dopamine response when initiating a behavior is both region- and action-specific. A–C, Illustration of an example of collected data, the mean centered null dataset, and results from the permutation test (example is from data presented in E). A, Collected data of dopamine levels in the NAcc core in response to the first head entry in a bout (n = 18 electrodes). B, Null datasets derived by mean centering the collected data at zero (n = 18). C, The mean of the collected data plotted alongside the 95% confidence interval of the permutation test performed on the collected and null datasets (1000 permuted datasets). D, E, No difference in NAcc core and shell dopamine release in response to the delivery of an unexpected food reward administered outside of the operant task. F, Initiating a bout of active lever presses was not associated with a change in dopamine levels in the NAcc core but was accompanied by a significant rise in NAcc shell dopamine release preceding the first active lever press in a bout. G, Unreinforced head entries into the food tray led to a transient drop in dopamine levels in the NAcc core, with no effect on NAcc shell dopamine levels. H, No change in dopamine levels when initiating a bout of inactive lever presses. *p < 0.05, differences from null values (permutation test).
The inherent behavioral variability in this self-paced task allowed us to determine whether the dopamine response when initiating a bout reflects the initiation vigor (time elapsed since the end of the previous bout) and the effort exerted in a bout (number of consecutive repeats of a particular action). The behavioral data from each rat were sorted into tertiles, which allowed for comparing the dopamine response across rats exhibiting different levels of activity (Wanat et al., 2010), and ensured equal weight was given to each electrode in the analysis (Fig. 3A,B). For each electrode, we calculated the average dopamine response corresponding to the tertile-sorted behavioral data (Fig. 3C). To quantify the relationship between the dopamine response and behavior, we plotted the tertile-sorted dopamine response as a function of the tertile-sorted behavior at each time point (Fig. 3D). The slope of this dopamine response versus behavior was calculated at each time point (Fig. 3E) and averaged across electrodes to identify if and when the dopamine signal reflected the initiation vigor and effort exerted.
Illustration of the method to quantify the relationship between the dopamine response and behavioral output. A, The behavioral response for each rat and for each type of bout was sorted into tertiles. Presented here is the behavior from a rat when initiating an active lever-pressing bout sorted by initiation vigor. B, The tertile-sorted behavior was log transformed to linearize the data. C, The average dopamine response corresponding to the tertile-sorted behavioral data. D, The slope of the dopamine response as a function of the behavioral output was calculated at all points in the time series. Example calculations for two points in the time series are presented. E, The inverse slope of the dopamine response as a function of behavior from a single electrode.
Sorting active lever pressing bouts according to initiation vigor or effort exerted resulted in no behavioral differences between rats with NAcc core electrodes and rats with NAcc shell electrodes (unpaired Student's t test, p > 0.05; Fig. 4A–C). However, this behavioral sorting revealed differences in the pattern of dopamine transmission between the NAcc core and shell (Fig. 4D–K). Dopamine levels in the NAcc core were elevated during high vigor bouts and reduced during low vigor bouts (permutation test, p < 0.05 for high and low vigor sort, n = 18 electrodes; Fig. 4D). Calculating the slope of the dopamine response as a function of behavior (for example analysis, see Fig. 3) illustrates that NAcc core dopamine reflected initiation vigor before, and immediately following, the active lever bout initiation (permutation test, p < 0.05; Fig. 4H). Dopamine in the NAcc core also reflected the exerted effort after initiating the bout, which was driven by a reduction in dopamine levels during bouts with a low number of active lever presses (permutation test, p < 0.05; Fig. 4F,J). The behavioral measures of initiation vigor and effort exerted were not related across rats during active lever bouts (linear regression, r2 = 0.003, p > 0.05), illustrating that NAcc core dopamine reflects initiation vigor and effort exerted through separate processes. In contrast, the pattern of dopamine transmission in the NAcc shell was unaffected by sorting according to the initiation vigor (permutation test, p > 0.05, n = 10 electrodes; Fig. 4I) or the effort exerted (permutation test, p > 0.05; Fig. 4K). Thus, dopamine reflects distinct aspects of motivated behavior in a NAcc subregion-dependent manner.
NAcc core dopamine release reflects initiation vigor and exerted effort when initiating a bout of active lever presses. A–C, Sorting the behavioral responding into tertiles according to the time elapsed since the end of the previous bout or the number of lever presses in a bout revealed no differences between rats with electrodes in the NAcc core and rats with electrodes in the NAcc shell. D–G, Average dopamine response across electrodes in the NAcc core (D, F) and NAcc shell (E, G) reflecting the behavioral sort based upon initiation vigor (D, E) and number of repeated actions in a bout of active lever pressing (F, G). H–K, Relationship between tertile-sorted behavioral responding and dopamine levels, where deviations from null values indicates when dopamine transmission signals the assessed behavioral measure. Inverse slope of the dopamine response in the NAcc core (H) and shell (I) as a function of the initiation vigor in a bout of active lever pressing. Slope of the dopamine response in the NAcc core (J) and shell (K) as a function of the number of repeated actions in a bout of active lever pressing. *p < 0.05, differences from null values (permutation test).
Sorting head entry bouts according to the initiation vigor or the number of head entries performed in a bout revealed no differences between rats with NAcc core electrodes and rats with NAcc shell electrodes (unpaired Student's t test, p > 0.05; Fig. 5A–C). The dopamine response when sorted as a function of initiation vigor illustrates that dopamine levels were reduced after initiating a low vigor head entry bout in the NAcc core (permutation test, p < 0.05 for low and mid vigor sort; Fig. 5D), and the NAcc shell (permutation test, p < 0.05 for low vigor sort; Fig. 5E). However, the pattern of dopamine transmission in the NAcc core and shell was not affected by this behavioral sorting, illustrating that NAcc dopamine levels do not reflect the initiation vigor during head entry bouts (permutation test, p > 0.05; Fig. 5H,I). Because >60% of the bouts consisted of a single head entry (Fig. 5C), the analysis of the dopamine response sorted by exerted effort was performed between bouts with a single head entry relative to bouts with multiple head entries. NAcc core dopamine reflected the effort exerted in head entry bouts, driven by the reduced dopamine levels both before and following a single head entry bout (permutation test, p < 0.05; Fig. 5F,J). Dopamine signaled the exerted effort in a region-specific manner, as NAcc shell dopamine levels were not altered when initiating a head entry bout (permutation test, p > 0.05; Fig. 5G,K). In contrast to bouts of head entries and active lever presses (Figs. 4, 5), dopamine transmission throughout the NAcc when beginning a bout of inactive lever presses did not reflect initiation vigor or exerted effort (Fig. 6). Collectively, our results illustrate that the dopamine response when initiating an action, as well as the information contained in the dopamine signal, depends both upon where dopamine is released and what action is performed (Table 1).
Dopamine in the NAcc core reflects exerted effort when initiating a bout of unreinforced head entries to the food tray. A–C, Sorting the behavioral responding according to the time elapsed since the end of the previous bout or the number of head entries in a bout revealed no differences between rats with electrodes in the NAcc core and rats with electrodes in the NAcc shell. D–G, Average dopamine response across electrodes in the NAcc core (D, F) and NAcc shell (E, G) reflecting the behavioral sort based upon initiation vigor (D, E) and number of repeated actions in a bout of head entries (F, G). H–K, Relationship between tertile-sorted behavioral responding and dopamine levels, where deviations from null values indicate when dopamine transmission signals the assessed behavioral measure. Inverse slope of the dopamine response in the NAcc core (H) and shell (I) as a function of the initiation vigor in a bout of head entries. Slope of the dopamine response in the NAcc core (J) and shell (K) as a function of the number of repeated actions in a bout of head entries. *p < 0.05, differences from null values (permutation test).
NAcc dopamine release does not reflect initiation vigor or exerted effort when initiating a bout of inactive lever presses. A–C, Sorting the behavioral responding according to the time elapsed since the end of the previous bout or the number of inactive lever presses in a bout revealed no differences between rats with electrodes in the NAcc core and rats with electrodes in the NAcc shell. D–G, Average dopamine response across electrodes in the NAcc core (D, F) and NAcc shell (E, G) reflecting the behavioral sort based upon initiation vigor (D, E) and number of repeated actions in a bout of inactive lever pressing (F, G). H–K, Relationship between tertile-sorted behavioral responding and dopamine levels. Inverse slope of the dopamine response in the NAcc core (H) and shell (I) as a function of the initiation vigor in a bout of inactive lever pressing. Slope of the dopamine response in the NAcc core (J) and shell (K) as a function of the number of repeated actions in a bout of inactive lever pressing.
Summary of results
Discussion
The mesolimbic dopamine system is involved with facilitating motivated reward-seeking behavior (Salamone and Correa, 2012). In operant tasks using PR reinforcement schedules, the breakpoint is elevated when dopamine transmission is enhanced, and attenuated when dopamine transmission is inhibited (Hamill et al., 1999; Zhang et al., 2003; Cagniard et al., 2006). Phasic electrical or optogenetic activation of midbrain dopamine neurons is sufficient for initiating conditioned approach behavior (Phillips et al., 2003; Ilango et al., 2014). However, using techniques that exogenously activate or inhibit the dopamine system prevents determining how endogenous dopamine transmission encodes aspects of motivated behavior. In the present study, we analyzed the dopamine response just before, and immediately following, the initiation of action-specific bouts of behavior in rats working for food rewards under a PR reinforcement schedule. Although cue- and reward-evoked dopamine release is uniform across the ventral striatum in this task (Wanat et al., 2010), we demonstrate that the pattern of dopamine transmission and the information conveyed in this dopamine signal depend upon the type of behavior initiated as well as where dopamine is released in the NAcc.
We found dopamine levels in the NAcc shell were elevated before initiating an active lever bout, similar to previous reports of the ventral striatal dopamine response during reinforced actions (Phillips et al., 2003; Roitman et al., 2004; Owesson-White et al., 2009; Wassum et al., 2012). However, dopamine transmission in the NAcc shell during active lever bouts did not reflect initiation vigor or exerted effort. In contrast to the NAcc shell, dopamine in the NAcc core reflected both the initiation vigor and the exerted effort in a bout, but this was masked when examining the average dopamine response across all active lever bouts. In particular, the rise in NAcc core dopamine levels was evident only in high initiation vigor bouts of active lever presses. Interestingly, low effort active lever bouts were associated with a drop in dopamine levels following the bout initiation. These results collectively suggest dopamine is serving different functions in the NAcc core and shell when engaging in a series of active lever presses. The NAcc core and shell can differentially govern behavioral outcomes (Saddoris et al., 2013). For example, inactivating the NAcc shell can increase nondiscriminate actions, whereas inactivating the NAcc core reduces cue-based responding (Floresco et al., 2008a). Thus, elevated dopamine in NAcc shell could promote the initiation of an active lever bout, whereas the dopamine response in the NAcc core may be involved with regulating the vigor and effort exerted in the bout.
In addition to regional differences, the NAcc dopamine response also depends upon the type of behavior that was initiated. Whereas dopamine release in the NAcc shell rose before an active lever press bout, there was no change in NAcc shell dopamine levels when engaging in a head entry bout. Interestingly, NAcc core dopamine levels were transiently reduced following a single unreinforced head entry to the food tray but were unchanged upon initiating a bout comprised of multiple head entries. The different pattern of dopamine transmission in the NAcc core between a single head entry and initiating a bout of multiple head entries could reflect the difference between a goal-directed action and generalized motor activity, respectively. In this task, engaging in multiple consecutive head entries has no possibility of being reinforced. Therefore, an explorative head entry to the food tray could represent a goal-directed reward-seeking action, whereas engaging in a series of repetitive head entries (or inactive lever presses) could reflect nonspecific motor activity. In this regard, the transient decrease in dopamine levels following a single goal-directed unreinforced head entry is consistent with a negative reward prediction error signal (Schultz et al., 1997). Alternatively, the difference in the dopamine response between a single head entry and multiple head entries may reflect the exerted effort, where a decrease in NAcc core dopamine levels functions to attenuate the number of repetitious actions in a bout.
In this study, we assessed the dopamine response when initiating unreinforced behavioral actions to avoid the confounding influence cue- and reward-evoked dopamine release in our analysis. The magnitude of the dopamine response when initiating an unreinforced behavior is predictably smaller than cue- and reward-evoked dopamine release (Wanat et al., 2010). Importantly, these dopamine signals during bouts of activity are related behavior and are likely contributing to the responses observed in previous studies where motivation-related dopamine signals occur in close proximity to the reward delivery or to reward-associated cues (Phillips et al., 2003; Roitman et al., 2004; Wassum et al., 2012). While we demonstrated that dopamine transmission is related to measures of motivated behavior, it is possible that the effort exerted and initiation vigor are also related to uncontrolled factors in this self-paced task, such as the number of previously unrewarded actions in a trial.
Depleting dopamine and antagonizing dopamine receptor function within the NAcc impairs performance in operant tasks with high fixed ratio requirements, as well as in tasks using PR reinforcement schedules (Aberman and Salamone, 1999; Hamill et al., 1999; Salamone et al., 2001; Nicola, 2010). Because these pharmacological and dopamine depletion manipulations affect both tonic and phasic dopamine transmission, the contribution of these distinct patterns of dopamine release in motivated behavior has not been fully elucidated. Our results demonstrate that endogenous phasic dopamine transmission can reflect initiation vigor and the effort exerted when initiating a new action. These bouts of activity occurred in the last few trials in PR sessions, where the operant requirement to earn a reward is high and performance is highly dopamine-dependent (Aberman and Salamone, 1999; Hamill et al., 1999; Salamone et al., 2001). While phasic dopamine release reflects behavioral actions, the net function of dopamine on motivated behavior is likely mediated by an interplay between both tonic dopamine levels and phasic dopamine release to discrete events. Collectively, our analysis of endogenous dopamine release during a self-paced operant task illustrates the heterogeneity in dopamine transmission throughout the ventral striatum, where encoding of motivated behavioral output occurs in an action-specific manner selectively in the NAcc core.
Footnotes
This work was supported by National Institutes of Health Grant DA033386 to M.J.W. We thank Paul E.M. Phillips for the dataset, Scott Ng-Evans for assistance with data analysis, and Nick Hollon for scientific discussions.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Matthew J. Wanat, Neurosciences Institute, Department of Biology, University of Texas at San Antonio, One UTSA Circle, San Antonio, TX 78249. Matthew.wanat{at}utsa.edu
