Abstract
The costs associated with obtaining illicit drugs can fluctuate depending upon the relative drug availability. As a consequence of the changing costs, the effort that one must exert to obtain drugs is dynamic. Considerable evidence illustrates a critical role for dopamine in the ventral medial striatum in mediating drug reinforcement. However, little is known regarding how dopamine release is affected by changes in the costs associated with earning drugs. We used fast-scan cyclic voltammetry to determine how changes in the operant requirement affected dopamine release to self-administered cocaine in male rats. Dopamine release to cocaine infusions increased across trials during self-administration sessions using a fixed-ratio reinforcement schedule with a low operant requirement. However, increasing the operant requirement abolished the within-session elevation in dopamine release to drug rewards. This effect was not due to underlying changes in preinfusion dopamine levels and was not explained by cocaine levels in the brain. This within-session increase in dopamine release to cocaine infusions reemerged when the operant requirement was lowered. Under a progressive ratio reinforcement schedule, there was no increase in dopamine release to drug rewards across trials, which contrasts with prior studies demonstrating an increase in dopamine release to food rewards. Collectively, these findings illustrate that the influence of operant costs on reward-evoked dopamine release depends upon type of reward that can be earned (e.g., food or drug).
SIGNIFICANCE STATEMENT The mesolimbic dopamine system is involved with mediating drug reinforcement. Although the costs associated with earning drugs are dynamic, no studies to date have examined how dopamine release to drug rewards is affected by changing costs. By performing fast-scan cyclic voltammetry recordings in rats self-administering cocaine, the present work demonstrates that changing the operant costs reversibly modulates the dopamine response to cocaine rewards. Furthermore, these findings highlight that the influence of costs on dopamine release to drug rewards differs from the established effect of costs on dopamine release to food rewards.
Introduction
Reward-seeking behaviors are influenced by the value that one ascribes to a reward. The mesolimbic dopamine system conveys value-related information in response to the delivery of food and juice rewards (Tobler et al., 2005; Nasrallah et al., 2011; Lak et al., 2014). These dopamine neuron responses toward natural rewards are also affected by extrinsic factors, including the costs associated with earning the reward (Fiorillo et al., 2008; Kobayashi and Schultz, 2008; Wanat et al., 2010). In addition to signaling the delivery of natural rewards, dopamine levels in the ventral medial striatum (VMS) rapidly rise in response to the delivery of drug infusions during self-administration sessions (Phillips et al., 2003; Owesson-White et al., 2009; Willuhn et al., 2012, 2014). However, it is not known whether the dopamine response to drug infusions is also affected by the costs associated with earning the drug.
Rodents with a stable pattern of cocaine intake exhibit a pronounced VMS dopamine response to drug infusions that is evident across weeks of training (Willuhn et al., 2012, 2014). However, rats that escalate cocaine intake exhibit a progressive reduction in dopamine release to cocaine infusions (Willuhn et al., 2014). These findings suggest that the level of operant responding during cocaine self-administration sessions could be inversely related to the dopamine response to drug rewards. In contrast, the costs associated with obtaining food rewards are positively related to the reward-evoked dopamine release (Wanat et al., 2010, 2013). Here, we performed voltammetry recordings in the VMS of male rats self-administering cocaine using different reinforcement schedules to resolve whether increasing operant costs decreases dopamine release to cocaine infusions, as is observed in rats that escalate cocaine intake (Willuhn et al., 2014), or increases dopamine release to cocaine rewards, as is observed in response to food rewards (Wanat et al., 2010, 2013).
Our results demonstrate that dopamine release to cocaine infusions increases across trials during self-administrations sessions using a fixed-ratio (FR) reinforcement schedule with a low operant requirement. However, this within-session elevation in dopamine release to drug rewards was abolished when the FR operant requirement increased and reemerged when the operant requirement was lowered. There was no change in dopamine release to cocaine infusions across trials when using a progressive ratio (PR) reinforcement schedule, which contrasts from the increase in reward-evoked dopamine release to food rewards during PR sessions (Wanat et al., 2010, 2013). Collectively, these findings demonstrate that operant costs affect reward-evoked dopamine release in a reinforcer-specific manner.
Materials and Methods
Subjects and surgery.
All procedures were approved by the Institutional Animal Care and Use Committee at the University of Texas at San Antonio. Male Sprague Dawley rats (Charles River Laboratories) were pair-housed upon arrival, given ad libitum access to water and chow, and maintained on a 12 h light/dark cycle. Rats were single-housed following surgery for the duration of the experiment. Voltammetry electrode implantation surgeries were performed under isoflurane anesthesia on rats weighing 300–350 g. During these surgeries, carbon fiber electrodes were implanted bilaterally in the VMS (relative to bregma: 1.3 mm anterior; ±1.3 mm lateral; 7.0 mm ventral), along with an Ag/AgCl reference electrode placed under the skull at a convenient location. Intravenous jugular catheter surgeries were performed 1–3 weeks following the voltammetry electrode implantation surgery. Rats were allowed to recover for at least 1 week before initiating cocaine self-administration training.
Cocaine self-administration.
Cocaine self-administration sessions were performed in operant boxes (Med Associates) with grid floors, a house light, two nose poke ports with lights, and a tone generator. Behavioral sessions (1 h) begin with illumination of the house light and were performed only once per day. After completing the required number of nose pokes into the active port, rats received a 0.3 mg/kg intravenous infusion of cocaine and the tone and active nose poke light turned on (5 s), which coincided with the initiation of a 20 s timeout period (house light off). Nose pokes during the timeout or into the inactive port had no consequences. To familiarize the animals with self-administration procedures and with changes in the operant requirement, cocaine self-administration sessions were first performed without voltammetry recordings for each reinforcement schedule. This training facilitated operant responding as some rats exhibited a reduction in behavioral responding when tethered during voltammetry recording sessions. Voltammetry recordings during FR reinforcement schedules were performed once rats had earned at least 10 infusions per session. Rats progressed to the next reinforcement schedule after obtaining at least one voltammetry recording session at a given reinforcement schedule. For the PR reinforcement schedule, the operant requirement escalated according to the following equation: operant requirement = 5 * e(infusion number * 0.2) − 5. PR sessions ended after 1 h.
Voltammetry recordings.
Chronically implanted carbon fiber microelectrodes were connected to a head-mounted voltammetric amplifier for dopamine detection in behaving rats using fast-scan cyclic voltammetry as described previously (Clark et al., 2010; Wanat et al., 2010; Fonzi et al., 2017). The potential applied to the carbon fiber was ramped in a triangle wave from −0.4 V (vs Ag/AgCl) to +1.3 V and back at a rate of 400 V/s during a voltammetric scan and held at −0.4 V between scans at a frequency of 10 Hz. Chemical verification of dopamine was achieved by obtaining high correlation of the cyclic voltammogram during a reward-related event to that of a dopamine standard (correlation coefficient r2 ≥ 0.75 by linear regression). The voltammetry data and corresponding behavioral data for a session were not analyzed if the detected voltammetry signal did not satisfy the chemical verification criteria, identical to the exclusion criteria used in prior studies (Wanat et al., 2010, 2013; Fonzi et al., 2017).
Data analysis.
Dopamine was isolated from the voltammetry signal using chemometric analysis (Heien et al., 2005) with a standard training set accounting for dopamine, pH, and background drift. The background for voltammetry recordings analyses was set to 0.5 s before the cocaine infusion. Trials in which the chemometric analysis failed to identify dopamine on >25% of the data points were excluded from analysis. Voltammetry data from sessions in which rats earned <5 cocaine infusions were also excluded from analysis. The dopamine concentration was estimated based on the average postimplantation sensitivity of electrodes (34 nA/μm) (Clark et al., 2010). Dopamine release to the cocaine infusion was quantified as the average response for the 12 s following the infusion relative to the 2 s preceding the infusion. The 12 s postinfusion temporal window was based upon when the average dopamine response returned to baseline levels during FR1 sessions.
The voltammetry data were also normalized to dopamine levels 10 s preceding cocaine delivery to examine changes in the dopamine signal occurring before the cocaine infusion. The preinfusion change in dopamine levels was quantified as the average dopamine signal 9–10 s preceding the cocaine infusion relative to the average signal during the 1 s preceding the cocaine infusion. We calculated the dopamine response to the cocaine infusion with the data aligned in this manner as the difference between the 12 s postinfusion temporal window relative to dopamine levels 8–10 s before the infusion. To quantify the dopamine response at the end of the timeout (when cocaine is available) we calculated difference between the average dopamine response 4 s after the end of the timeout relative to the last 2 s of the timeout. Analyses involving the time since the previous infusion omitted the first trial of the session. To examine potential differences in the decay of the dopamine signal, voltammetry data for each electrode for each reinforcement schedule were normalized and aligned to the peak dopamine response following the cocaine infusion. These data were then fit to a single-phase decay curve to calculate the tau for each electrode at each operant requirement.
Bouts of active and inactive nose poke responding were calculated as the repeated instances of a given action during PR sessions (Ko and Wanat, 2016). If >1 min elapsed without a response, then the subsequent action was identified as initiating a new bout of activity. Nose pokes during the timeout were not included in this analysis. The data were analyzed based upon a median split of the number of repeats in a bout and the latency to initiate a bout for each animal. Inactive nose poke bouts were predominately composed of a single nose poke and some subjects did not perform any inactive nose pokes during PR sessions. Therefore, inactive nose pokes were analyzed based upon whether the bout was composed of a single nose poke or multiple nose pokes. We calculated the dopamine response before (2 s) and following (2 s) the initiation of a new bout of activity. This dopamine signal was then related to the number of repeated actions in a bout and the latency since the last bout ended or the trial began.
Cocaine levels were modeled using the following equation C = d*A(e−α*t − e−β*t), where C is the cocaine concentration (micromolar), d is the cocaine dose (0.3 mg/kg), and t is the time in minutes since the injection occurred, with constants for A (9.637), α (0.642), and β (0.097) (Pan et al., 1991; Nicola and Deadwyler, 2000). The cocaine concentration across trials was fit to a Weibull function to estimate the asymptotic cocaine concentration achieved during self-administration sessions (Gallistel et al., 2004; Clark et al., 2013). This analysis was performed to identify the load and maintenance phases during cocaine self-administration sessions. Specifically, the beginning of the maintenance phase was based upon when the mean cocaine concentration exceeded the 95% confidence interval of the calculated asymptote according to the Weibull function fit. All preceding trials were identified as the load phase. The load and maintenance phases were standardized based upon the data fit during FR1 sessions (see Fig. 2) and during the first week of voltammetry recordings (see Fig. 3). This analysis was performed over the first 14 trials because this was the minimum number of cocaine infusions experienced under an FR1 reinforcement schedule in the analyzed data set. To examine the effect of cocaine on the dopamine response to drug infusions, we performed a median split based on the estimated cocaine concentration for each individual electrode per reinforcement schedule. Behavioral analyses were performed only on sessions corresponding to the voltammetry recordings included in these experiments. Significant effects were determined by Student's t tests, ANOVAs, and linear regression analyses. Data were analyzed using MATLAB and Prism software.
Histology.
Electrical lesions were made in anesthetized rats for the histological verification of the voltammetry electrode placement. Rats were subsequently intracardially perfused with 4% paraformaldehyde and brains were removed and postfixed in the paraformaldehyde solution for at least 24 h. Brains were subsequently placed in 15% and 30% sucrose solutions in PBS. Brains were then flash-frozen in dry ice, coronally sectioned, and stained with cresyl violet. The location of voltammetry electrodes is presented in Figure 1.
Voltammetry electrode placement. The histologically verified location of voltammetry electrodes are shown from rats that underwent changes in the operant requirement over training sessions (black circles) and rats that were maintained on an FR1 reinforcement schedule (blue circles).
Results
Voltammetry recordings were performed in the VMS of rats self-administering cocaine (0.3 mg/kg, i.v.) under different reinforcement schedules to ascertain how operant costs affect dopamine release to drug rewards. To familiarize the animals with self-administration procedures, rats were first trained to self-administer cocaine under an FR1 reinforcement schedule until at least 10 infusions were earned per session (mean 9.9 ± 1.7 training sessions, n = 9 rats; Fig. 2A). Voltammetry recordings were then performed during FR1 self-administration sessions. Rats progressed to a higher operant requirement after obtaining at least one viable voltammetry recording session at a given reinforcement schedule (mean 1.8 ± 0.2 voltammetry sessions per operant requirement). After increasing the operant requirement, animals were required to earn at least 10 infusions during behavior-only self-administration sessions before self-administration sessions were performed with voltammetry recordings. The average occurrence of voltammetry recordings sessions for each reinforcement schedule following FR1 training is presented in Figure 2A.
Increasing the operant requirement abolishes the within-session increase in dopamine release to the cocaine infusion. A, Top, Outline of training. Bottom, Average occurrence of voltammetry recording sessions at each reinforcement schedule as a function of time following FR1 training sessions. B, Average number of active nose pokes per session for each reinforcement schedule. C, Average number of inactive nose pokes per session for each reinforcement schedule. D, Average infusions per session for each reinforcement schedule. E, Modeling the estimated cocaine concentration in the brain across trials for each reinforcement schedule. F, G, Representative color plots of voltammetry recordings from a single electrode during load and maintenance trials in FR1 and FR3 sessions. H, Average dopamine response to the cocaine infusion during the load phase and the maintenance phase of cocaine self-administration sessions. I, Increase in dopamine release during the maintenance phase is absent under an FR3 reinforcement schedule. J, Dopamine release to the cocaine infusion aligned to the first (left) and last (right) trial in the session. *p < 0.05; **p < 0.01; ***p < 0.001.
Increasing the operant requirement to earn cocaine infusions led to a corresponding increase in active nose pokes (one-way repeated-measures ANOVA: F(2,16) = 19.4, p < 0.001; post hoc Tukey's test: FR1 vs FR2, q(16) = 5.437, p < 0.01; FR1 vs FR3 q(16) = 8.7, p < 0.001; Fig. 2B) and inactive nose pokes (one-way repeated-measures ANOVA: F(2,16) = 4.2, p = 0.03; post hoc Tukey's test: FR1 vs FR3, q(16) = 4.0, p < 0.05; Fig. 2C), but did not affect the number of infusions earned per session (one-way repeated-measures ANOVA: F(2,16) = 1.7, p = 0.21; n = 9 rats; Fig. 2D). An increase in the operant requirement also led to a reduction in the rate of active nose poke responding following the first response on a trial (FR2: 3.4 ± 0.5 nose poke/s; FR3: 1.2 ± 0.5 nose poke/s; paired t test, t(8) = 4.6, p = 0.002). Although the dopamine response during behavioral sessions is often averaged across all trials, this method of analysis can potentially obscure meaningful within-session differences in the dopamine transmission. In particular, behavioral responding during drug self-administration using a FR reinforcement schedule typically consist of an initial load phase of rapid responding, which is followed by a maintenance phase in which drug infusions are earned at a lower rate (Wise et al., 1995). To empirically identify the load and maintenance phases, we first estimated the cocaine concentration in the brain across trials throughout self-administration sessions (Fig. 2E) using established models (Pan et al., 1991; Nicola and Deadwyler, 2000). The cocaine concentration across trials was then fit to a Weibull function to identify the asymptotic cocaine concentration achieved during self-administration sessions (Gallistel et al., 2004; Clark et al., 2013). Using this analysis, we identified the load phase as trials 1–6 and the maintenance phase as all subsequent trials.
Voltammetry recordings of VMS dopamine release illustrate the dopamine response to cocaine infusions increased during the maintenance phase relative to the load phase during FR1 and FR2 sessions (Fig. 2F–I). This effect was quantified using a temporal window based upon when the average dopamine signal returned to baseline levels during FR1 sessions (12 s postinfusion relative to the 2 s preinfusion; Fig. 2I). However, there was no difference in the dopamine response to cocaine infusions between the load and maintenance phases during FR3 sessions (two-way repeated-measures ANOVA: effect of reinforcement schedule F(2,37) = 0.5, p = 0.64, effect of phase F(1,37) = 14.7, p < 0.001, interaction effect F(2,37) = 4.2, p = 0.022; post hoc paired t tests: FR1 t(12) = 2.7, p = 0.02, n = 12 electrodes; FR2 t(12) = 3.4, p = 0.005, n = 12 electrodes; FR3 t(13) = 0.1, p = 0.89, n = 13 electrodes; Fig. 2G–I). Examining the dopamine response aligned to the first trial in the session demonstrates an increase in dopamine release to cocaine infusions across trials (two-way ANOVA: effect of trial F(9,346) = 4.0, p < 0.001, effect of reinforcement schedule F(2,346) = 2.2, p = 0.11; Fig. 2J). However, aligning the voltammetry data to the last trial in the session highlights that dopamine levels are significantly blunted during FR3 sessions (two-way ANOVA: effect of reinforcement schedule F(2,360) = 13.7, p < 0.001, effect of trial F(9,360) = 1.7, p = 0.09; post hoc Tukey's test: FR3 vs FR1 q(360) = 7.1, p < 0.001; FR3 vs FR2 q(360) = 5.0, p = 0.0013; Fig. 2J). Therefore, the dopamine response to cocaine infusions is affected by the operant costs associated with earning the drug reward.
Increasing the delay before juice/food rewards are delivered results in a larger reward-evoked dopamine response (Fiorillo et al., 2008; Kobayashi and Schultz, 2008; Wanat et al., 2010). Therefore, the time since the previous cocaine infusion could contribute to the observed dopamine response during self-administration sessions. However, there was no relationship between the time since the previous infusion and dopamine release to the drug delivery (r2 = 0.00, p = 0.37, n = 1276 trials), indicating that the influence of elapsed time on reward-evoked dopamine release differs between natural and drug rewards.
Although our findings suggest that increasing the operant requirement prevents the elevation in dopamine release during the maintenance phase, an alternative possibility is that this phenomenon is independent of the reinforcement schedule, but rather develops as a function of self-administering cocaine over multiple weeks. To address this possibility, voltammetry recordings were performed in a separate cohort of rats that were maintained on an FR1 reinforcement schedule for 4 weeks (Fig. 3A), which encompasses the duration of training for the rats exposed to changes in the reinforcement schedule (Fig. 2A). There was no change in the number of active or inactive nose pokes across weeks (one-way repeated-measures ANOVA: active F(3,15) = 1.4, p = 0.29; inactive F(3,15) = 2.4, p = 0.11, n = 6 rats; Fig. 3B,C). The load phase consisted of trials 1–6 and the maintenance phase consisted of all subsequent trials based upon the estimated cocaine concentration across trials in this cohort (Fig. 3D). Dopamine release to cocaine infusions increased during the maintenance phase relative to the load phase across all 4 weeks of self-administration sessions using an FR1 reinforcement schedule (two-way repeated-measures ANOVA: effect of week F(3,34) = 0.2, p = 0.88, effect of phase F(1,34) = 60.1, p < 0.001, interaction effect F(3,34) = 0.3, p = 0.82; post hoc paired t tests: week 1 t(9) = 2.8, p = 0.02, n = 10 electrodes; week 2 t(9) = 3.9, p = 0.0037, n = 10 electrodes; week 3 t(8) = 6.6, p < 0.001, n = 9 electrodes; week 4 t(8) = 4.9, p = 0.0011, n = 9 electrodes; Fig. 3E–G). These data indicate that the absence of an elevated dopamine response during the maintenance phase of FR3 sessions is due to the increased operant requirement and is not a consequence of self-administering cocaine for multiple weeks.
Maintaining an FR1 reinforcement schedule over weeks does not affect the within-session increase in dopamine release to the cocaine infusion. A, Outline of training. B, Average number of active nose pokes per session across weeks. C, Average number of inactive nose pokes per session across weeks. D, Modeling the estimated cocaine concentration in the brain across trials across weeks. E, Representative color plots of voltammetry recordings from a single electrode during load and maintenance trials in weeks 1 and 3. F, Average dopamine response to the cocaine infusion during the load phase and the maintenance phase of cocaine self-administration sessions across weeks. G, Dopamine release to the cocaine infusion increases during the Maintenance phase across weeks in rats trained on an FR1 reinforcement schedule. *p < 0.05; **p < 0.01; ***p < 0.001.
Underlying changes in background dopamine levels can influence the dopamine response to reward-related stimuli under certain conditions (Hamid et al., 2016; Fonzi et al., 2017). In particular, the absence of the elevated dopamine response to cocaine rewards during the maintenance phase of FR3 sessions could arise from a “ceiling effect” in which an increase in preinfusion dopamine levels effectively attenuates the relative change in dopamine release following the drug delivery. To address this, we normalized the voltammetry data to 10 s before the cocaine delivery, which allows for assessing changes in the preinfusion dopamine response. Preinfusion dopamine levels differed between the load and maintenance phases only during FR3 sessions (two-way repeated-measures ANOVA: effect of reinforcement schedule F(2,37) = 1.7, p = 0.19, effect of phase F(1,37) = 7.9, p = 0.008, interaction effect F(2,37) = 2.3, p = 0.12; post hoc paired t tests: FR1 t(12) = 1.6, p = 0.13, n = 12 electrodes; FR2 t(12) = 0.4, p = 0.72, n = 12 electrodes; FR3 t(13) = 2.7, p = 0.02, n = 13 electrodes; Fig. 4C). However, this effect was not mediated by an increase in preinfusion dopamine release during the maintenance phase, but rather was due to a decrease in preinfusion dopamine levels during the load phase (Fig. 4A).
Within-session changes in the preinfusion dopamine signal emerges after multiple weeks of cocaine self-administration. A, Dopamine response normalized to 10 s before the drug reward in rats self-administering cocaine under increasing operant requirements. Yellow overlay denotes the preinfusion window. Arrows denote the median occurrence of the preceding nose poke. B, Dopamine response normalized to 10 s before the drug reward in rats self-administering cocaine under a stable FR1 operant requirement over weeks. C, D, Preinfusion change in dopamine levels in rats self-administering cocaine under increasing (C) or stable (D) operant requirements over weeks. E, F, Dopamine response following the cocaine infusion relative to dopamine levels 8–10 s preceding the drug reward in rats self-administering cocaine under increasing (E) or stable (F) operant requirements over weeks. *p < 0.05; **p < 0.01; ***p < 0.001.
In rats maintained on an FR1 reinforcement schedule, this within-session change in preinfusion dopamine levels was initially absent during weeks 1–2, but developed during weeks 3–4 of self-administering cocaine (two-way repeated-measures ANOVA: effect of week F(3,34) = 0.8, p = 0.50, effect of phase F(1,34) = 17.64, p < 0.001, interaction effect F(3,34) = 1.9, p = 0.15; post hoc paired t tests: week 1 t(9) = 0.2, p = 0.82, n = 10 electrodes; week 2 t(9) = 1.9, p = 0.10, n = 10 electrodes; week 3 t(8) = 5.0, p = 0.001, n = 9 electrodes; week 4 t(8) = 3.1, p = 0.01, n = 9 electrodes; Fig. 4B). Together, these data indicate that changes in preinfusion dopamine levels emerge after self-administering cocaine over multiple weeks (Fig. 4A–D). In contrast, dopamine release to the cocaine infusion is not affected by the prior experience self-administering cocaine, but rather is affected by the operant costs whether this measure is calculated as a relative difference to dopamine levels immediately before the drug delivery (Fig. 2) or to dopamine levels 10 s before the drug delivery (two-way repeated-measures ANOVA: effect of reinforcement schedule F(2,37) = 0.4, p = 0.70, effect of phase F(1,37) = 22.2, p < 0.001, interaction effect F(2,37) = 0.3, p = 0.71; post hoc paired t tests: FR1 t(12) = 3.3, p = 0.007; FR2 t(12) = 3.0, p = 0.011; FR3 t(13) = 2.1, p = 0.054; Fig. 4E). Furthermore, the within-session increase dopamine release to cocaine infusions is evident across all weeks of training in rats maintained on an FR1 reinforcement schedule when the reference window is set immediately before (Fig. 3) or 10 s before the drug infusion (two-way repeated-measures ANOVA: effect of week F(3,34) = 0.3, p = 0.81, effect of phase F(1,34) = 76.3, p < 0.001, interaction effect F(3,34) = 1.2, p = 0.33; post hoc paired t tests: week 1 t(9) = 2.3, p = 0.049; week 2 t(9) = 4.7, p = 0.0012; week 3 t(8) = 7.7, p < 0.001; week 4 t(8) = 6.2, p < 0.001; Fig. 4F). Collectively, these data illustrate that the within-session changes in preinfusion and postinfusion dopamine levels are influenced by the prior experience of self-administering cocaine and operant costs, respectively.
Dopamine clearance can be altered depending upon the pattern of cocaine intake during self-administration sessions (Calipari et al., 2013). Therefore, we posited that an impairment in dopamine clearance could explain the lack of a within-session increase of dopamine release to cocaine infusions during FR3 sessions. Given that cocaine enhances electrically evoked dopamine release using in vitro preparations (Ferris et al., 2012; Calipari et al., 2013), we anticipated that increasing cocaine levels should accompany a larger dopamine response during FR1 sessions and FR2 sessions, but not during FR3 sessions. To explore this possibility, we performed a median split of the trials based upon the cocaine concentration for each electrode. However, we found that cocaine levels did not affect the dopamine response to the cocaine infusion at any reinforcement schedule (two-way repeated-measures ANOVA: effect of reinforcement schedule F(2,37) = 1.0, p = 0.38; effect of cocaine F(1,37) = 0.09, p = 0.77; Fig. 5A,B). Although microdialysis experiments demonstrate that tonic dopamine levels increase while self-administering cocaine (Wise et al., 1995), our results illustrate that the rapid dopamine response to the drug reward is not primarily affected by underlying changes in the cocaine concentration. Additionally, if dopamine clearance were affected by an increase in the operant requirement, then we would anticipate that the decay of the dopamine signal would be slower during FR3 sessions. To address this, the voltammetry data were aligned to the peak dopamine signal following the cocaine infusion and fit to a single-phase decay curve. However, there was no difference in the tau across reinforcement schedules (one-way ANOVA F(2,33) = 1.2, p = 0.30, n = 11–13 electrodes; Fig. 5C,D), which illustrates that the operant requirement does not affect the decay in the dopamine response following the cocaine infusion.
Dynamics of the dopamine response to the cocaine infusion is not affected by cocaine concentration or the operant requirement. A, B, Dopamine response to the cocaine infusion is not affected by the cocaine concentration. C, Peak normalized dopamine response. D, Decay half-life of the normalized dopamine response.
Animals were subsequently trained on a PR reinforcement schedule following FR3 sessions to ascertain whether a within-session increase in the operant requirement affected dopamine release to cocaine infusions. Rats exhibited an increase in active nose pokes (unpaired t test, t(16) = 2.25, p = 0.04), no change in inactive nose pokes (unpaired t test, t(16) = 1.3, p = 0.22), and a decrease in the infusions earned per session (unpaired t test, t(16) = 2.8, p = 0.01; n = 9 per group; Fig. 6A). Although rats exerted greater effort to obtain fewer cocaine infusions, there was no difference in the dopamine response to drug rewards between FR3 and PR sessions (unpaired t test t(26) = 0.3, p = 0.80 n = 14 per group; Fig. 6B,C). Additionally, there was no systematic change in the dopamine response when aligning the data to the last completed trial in a session (two-way ANOVA: effect of reinforcement schedule F(1,216) = 0.4, p = 0.54; effect of trial F(8,216) = 0.4, p = 0.90; n = 14 per group; Fig. 6D). Cocaine levels were significantly lower during PR sessions relative to FR3 sessions (two-way ANOVA: effect of reinforcement schedule F(1,137) = 55.3, p < 0.001; effect of trial F(8,137) = 0.7, p = 0.67; n = 9 per group; Fig. 6E), which further demonstrates that dopamine release to drug infusions are not primarily driven by cocaine levels in the brain (Fig. 4). Dopamine release toward food rewards scales according to the time since the previous reward delivery (Wanat et al., 2010, 2013). In contrast, dopamine release toward cocaine infusions does not systematically change with the increasing time since the previous drug delivery during PR sessions (two-way ANOVA: effect of reinforcement schedule F(1,137) = 10.6, p < 0.001; effect of trial F(8,137) = 7.8, p < 0.001; interaction effect F(8,137) = 5.4, p < 0.001; n = 9 per group; Fig. 6F). Therefore, the influence of the operant requirement on reward-evoked dopamine release during PR reinforcement schedules depends upon the type of reward that can be earned.
Dopamine response to the cocaine infusion under a PR reinforcement schedule. A, Average number of active nose pokes (left), inactive nose pokes (middle), and infusions (right) under an FR3 and PR reinforcement schedule. B, Representative color plots of voltammetry recordings from a single electrode during the third to last and last trial of a PR session. C, Average dopamine response to the cocaine infusion during FR3 and PR sessions. D–F, Data aligned to the last trial in FR3 and PR sessions illustrating the average dopamine response to the cocaine infusion (D), the estimated cocaine concentration (E), and the time since the previous cocaine infusion (F). *p < 0.05.
The lack of a within-session increase of dopamine release during FR3 sessions could reflect a permanent change in the dynamics of the dopamine response across trials during FR reinforcement schedules or could be a unique property of the cocaine-evoked dopamine response when using higher operant requirements. To address these possibilities, a subset of rats was returned to an FR1 reinforcement schedule following PR sessions (n = 4; Fig. 7A). Dopamine release to the cocaine infusion was elevated during the maintenance phase relative to the load phase (paired t test, t(5) = 4.9, p = 0.0045, n = 6 electrodes; Fig. 7B,C), as was observed during earlier training sessions (Fig. 2). These data demonstrate that the absence of the within-session elevation in dopamine release during high operant requirement self-administration sessions is reversible when the operant requirement is lowered.
Within-session increase in dopamine release to the cocaine infusion reemerges when the operant requirement is lowered. A, Outline of training. B, C, Dopamine response to the cocaine infusion during the load phase and the maintenance phase of cocaine self-administration sessions in rats returned to an FR1 reinforcement schedule. **p < 0.01.
Finally, we investigated whether operant costs affect dopamine transmission to all task-related elements during self-administration sessions. The illumination of the house light that coincides with the end of the timeout denotes the availability of cocaine. In contrast to the dopamine response to the cocaine delivery, dopamine release at the end of the timeout was not affected by the operant requirement or the phase of the self-administration session (two-way ANOVA: effect of reinforcement schedule F(2,37) = 0.2, p = 0.82; effect of phase F(1,37) = 0.0, p = 0.99; Fig. 8A,B). Dopamine signals the amount of activity in a bout as well as the time since the previous action in rats lever pressing for food rewards under a PR reinforcement schedule (Ko and Wanat, 2016). Therefore, we also examined dopamine transmission upon initiating bouts of active or inactive nose poke responses during PR sessions. However, dopamine does not convey these vigor-related behavioral parameters either before or immediately after initiating bouts of active nose pokes (two-way ANOVA: effect of nose poke repetition F(1,48) = 0.4, p = 0.55, effect of pre vs post bout initiation F(1,48) = 0.1, p = 0.79; two-way ANOVA: effect of latency F(1,48) = 0.3, p = 0.62, effect of pre vs post bout initiation F(1,48) = 0.0, p = 0.88; n = 13 electrodes; Fig. 9A–G) or inactive nose pokes during cocaine self-administration sessions (two-way ANOVA: effect of nose poke number F(1,34) = 0.9, p = 0.35, effect of pre vs post bout initiation F(1,34) = 1.6, p = 0.22; two-way repeated-measures ANOVA: effect of latency F(1,32) = 0.4, p = 0.54, effect of pre vs post bout initiation F(1,32) = 0.0, p = 0.99; n = 10–11 electrodes; Fig. 9H–M). These data illustrate that operant costs do not globally affect dopamine transmission throughout cocaine self-administration sessions, but rather, increasing operant costs reversibly modulates the dopamine response to drug rewards.
Dopamine response at the end of the timeout when cocaine is available. A, Average dopamine response to the end of the timeout. B, No effect of reinforcement schedule or of phase of the cocaine self-administration session on the dopamine response at the end of the timeout.
Dopamine response when initiating bouts of activity during PR sessions. A, Example of behavioral responding (new bouts of activity are denoted by red triangles). Median split is based upon the number of active nose pokes performed in a bout for each electrode (B) and the corresponding dopamine response before and following the first active nose poke in the bout (C, D). Median split for active nose poke bouts is based upon the time elapsed since the end of the previous bout (E) and the corresponding dopamine response before and following the first active nose poke in the bout (F, G). H, Number of inactive nose pokes performed per bout. I, J, Dopamine response before and after the first inactive nose poke in single or multiple nose poke bouts. K, Median split for inactive nose poke bouts based upon time elapsed since the end of the previous bout for each electrode (K) and the corresponding dopamine response before and following the first inactive nose poke in the bout (L, M).
Discussion
A challenge with drug addiction research is that no single preclinical behavioral procedure can effectively model the range of behaviors that accompany substance use disorders. For example, escalated drug intake can be elicited though long-access and intermittent-access self-administration protocols (Ahmed and Koob, 1998; Zimmer et al., 2012). Escalated cocaine intake is accompanied by a reduction in phasic dopamine release both in vivo and ex vivo (Willuhn et al., 2014; Siciliano et al., 2018). Furthermore, pharmacological treatments that enhance dopamine signaling reverse the increase in drug intake (Willuhn et al., 2014; Siciliano et al., 2018), which illustrates that the mesolimbic dopamine system is a critical node in regulating drug taking. Substance use disorders are also characterized by spending greater time and effort to obtain the drug, which requires using protocols in which the costs associated with earning the drug are increased. To date, phasic dopamine transmission during cocaine self-administration has only been examined under static, low-cost operant requirements (Phillips et al., 2003; Stuber et al., 2005; Owesson-White et al., 2009; Willuhn et al., 2012, 2014). Here, we demonstrate that the within-session elevation in dopamine release to cocaine infusions is absent when the operant costs to earn a drug reward are increased. Therefore, a blunted dopamine response to drug rewards could be a general marker for multiple addiction-related behaviors.
The pharmacological effects of cocaine on rapid dopamine transmission have primarily been examined under conditions in which dopamine release is evoked using electrical stimulations (Wu et al., 2001; Stuber et al., 2005; Ferris et al., 2012; Calipari et al., 2013; Brodnik et al., 2017). The manner by which cocaine influences electrically evoked dopamine release depends upon the frequency of the stimulation as well as the dose of cocaine administered (Wu et al., 2001; Brodnik et al., 2017). However, recent work illustrates that dopamine transporter function is only minimally affected by reinforcing doses of cocaine used in self-administration studies (Brodnik et al., 2017). Consistent with these findings, our results demonstrate that the endogenous dopamine response to the cocaine infusion is not affected by cocaine levels achieved during self-administration sessions and instead is likely mediated by the afferent input to midbrain dopamine neurons (Oliva and Wanat, 2016).
Our analyses identified within-session changes in the dopamine response to preinfusion dopamine levels and the cocaine infusion. Whereas the within-session increase in dopamine release to drug rewards was affected by the operant requirement, the within-session change in the preinfusion dopamine response emerged after self-administering cocaine over multiple weeks. Interestingly, this effect was mediated by a decrease in preinfusion dopamine levels during the load phase. Dopamine levels decrease toward cues predicting delayed cocaine availability, which is thought to convey a negative affective state (Wheeler et al., 2011). Therefore, the decrease in preinfusion dopamine levels could signal the negative affective state before earning the cocaine infusion.
Our results illustrate that dopamine release to cocaine infusions is not monotonically affected by the operant costs or to cocaine levels in the brain during self-administration sessions. Rather, we demonstrate that the within-session elevation in dopamine release toward drug rewards is abolished as the operant requirement increases. Dopamine release to natural rewards is regulated by physiological state and conveys the relative subjective value of the reward (Lak et al., 2014; Cone et al., 2016; Papageorgiou et al., 2016). Therefore, we speculate that the within-session elevation in cocaine-evoked dopamine release could reflect an increase in the subjective value of the cocaine reward. This prediction is challenging to assess experimentally with the current dataset given that behavioral metrics assessing motivated output, such as the rate of responding, cannot be calculated for FR1 sessions. Additionally, a change in the subjective value of the drug reward is not necessarily linked to drug intake in the context of drug addiction. For example, the incentive sensitization theory of addiction posits the subjective liking of the drug diminishes with the development of addiction (Berridge and Robinson, 2016), which could be conveyed by the dopamine response to drug rewards. In support of this, rats that escalate cocaine intake, a hallmark feature of drug addiction, exhibit a progressive reduction in dopamine release to cocaine infusions (Willuhn et al., 2014). Therefore, a change in the dopamine response to cocaine rewards could reflect a corresponding change in the subjective value of the drug reward.
The current results coupled with prior work demonstrate that the effect of increasing operant costs on reward-evoked dopamine release is reinforcer specific. As the operant costs escalate across trials under a PR reinforcement schedule, dopamine release to food rewards increase (Wanat et al., 2010, 2013), whereas we identified no change in the dopamine response to cocaine infusions. Although the time elapsed since the previous reward delivery influences dopamine release to food rewards (Wanat et al., 2010), this is absent in the dopamine response to drug rewards. Although these findings illustrate discrepancies in the reward-evoked dopamine response between food and drug rewards, it is important to note experimental differences between the tasks used for earning food and drug rewards. As with prior studies, the delivery of cocaine is paired with audio and visual cues (Phillips et al., 2003; Stuber et al., 2005; Owesson-White et al., 2009; Willuhn et al., 2012, 2014). These cocaine delivery cues are presented concomitantly with the drug reward, which contrasts with cues that commonly precede the reward delivery in behavioral tasks involving food rewards (Fiorillo et al., 2008; Kobayashi and Schultz, 2008; Wanat et al., 2010, 2013; Fonzi et al., 2017). Although the presentation of drug delivery cues invigorates behavioral responding even though the drug cannot be earned during reinstatement tests, drug delivery cues are inextricably linked to the cocaine infusions during self-administration sessions. For clarity, and since the cocaine infusion (rather than the drug delivery cue) is the event that is analogous to the delivery of a food rewards, we discuss our voltammetry data as in response to the cocaine infusion. Despite the noted methodological differences between operant responding for food and drug rewards, our data highlight that the effect of operant costs on reward-evoked dopamine release depends upon the type of reward that can be earned.
Dopamine release in the VMS, in particular the nucleus accumbens core, signals the initiation vigor (time since the previous action) and the exerted effort when initiating a bout of lever presses to earn a food reward (Ko and Wanat, 2016). However, we find that dopamine transmission in the nucleus accumbens core does not reflect these vigor-related parameters when rats nose poke for cocaine infusions. Prior studies illustrate that a premotor dopamine response is not evident when animals nose poke to earn drug rewards (Willuhn et al., 2012, 2014) or perform head entries into a food port (Ko and Wanat, 2016). In contrast, dopamine levels increase before a lever press in rats working for either food or drug rewards (Phillips et al., 2003; Roitman et al., 2004; Wassum et al., 2012; Ko and Wanat, 2016). Together, these studies illustrate that the presence or absence of dopamine before an action depends upon the nature of the response that is made (e.g., lever press or nose poke).
A diverse array of self-administration protocols and training regimens are used to model specific aspects of addiction-related behaviors (Ahmed and Koob, 1998; Deroche-Gamonet et al., 2004; Zimmer et al., 2012; Marchant et al., 2018; Singer et al., 2018). Here, we demonstrate that the reinforcement schedule used during self-administration sessions influences the mesolimbic dopamine response to drug rewards. These findings underscore the importance of characterizing the differential engagement of neural systems across self-administration protocols to identify the circuits contributing to distinct addiction-related behavioral symptoms.
Footnotes
This work was supported by the National Institutes of Health (Grants DA033386 and DA042362 to M.J.W.). We thank Claire Stelly and Merridee Lefner for helpful comments throughout this project.
The authors declare no competing financial interests.
- Correspondence should be addressed to Matthew J. Wanat at Matthew.wanat{at}utsa.edu
