Efficient reward seeking is essential for survival and invariably requires overcoming costs, such as physical effort and delay, which are constantly changing in natural settings. Dopamine transmission has been implicated in decisions weighing the benefits and costs of obtaining a reward, but it is still unclear how dynamically changing effort and delay costs affect dopamine signaling to rewards and related stimuli. Using fast-scan cyclic voltammetry, we examined phasic dopamine release in the nucleus accumbens (NAcc) core and shell during reward-seeking behavior in rats. To manipulate the effort and time needed to earn a reward, we used instrumental tasks in which the response requirements (number of lever presses) were either fixed throughout a behavioral session [fixed ratio (FR)] or systematically increased from trial to trial [progressive ratio (PR)]. Dopamine release evoked by cues denoting reward availability was no different between these conditions, indicating insensitivity to escalating effort or delay costs. In contrast, dopamine release to reward delivery in both the NAcc core and shell increased in PR, but not in FR, sessions. This enhancement of reward-evoked dopamine signaling was also observed in sessions in which the response requirement was fixed but the delay to reward delivery increased, yoked to corresponding trials in PR sessions. These findings suggest that delay, and not effort, was principally responsible for the increased reward-evoked dopamine release in PR sessions. Together, these data demonstrate that NAcc dopamine release to rewards and their predictors are dissociable and differentially regulated by the delays conferred under escalating costs.
Our actions are dynamically molded by reward-related decisions that weigh perceived benefits against perceived costs and involve multiple neurobiological processes (McClure et al., 2004), including the mesolimbic dopamine system (Wise, 2004; Schultz, 2006). For example, phasic dopamine neurotransmission encodes the size of anticipated rewards (benefit) when they are preceded by predictive stimuli (Tobler et al., 2005; Gan et al., 2010). However, less is known on how dopamine neurotransmission is modulated by the costs involved with obtaining a reward, such as the response requirement in instrumental behaviors or the delay until the reward is delivered. Importantly, the cost of obtaining rewards in natural settings is often dynamic; for instance, the time to explore and the distance to travel for a foraging animal constantly changes as a function of food availability, and these temporal and energetic costs are often intimately linked (Stephens and Krebs, 1986; Stevens et al., 2005; Berger-Tal et al., 2009). Although dopamine neurotransmission, elicited by both cue presentation and reward delivery, is sensitive to discrete switches in both the delay and the effort associated with attaining rewards (Roesch et al., 2007; Gan et al., 2010), it is not known how dopamine release to rewards and their predictors are affected when these costs are continuously changing.
Dynamic changes in the effort and delay costs to obtain rewards can be modeled in animals using progressive ratio (PR) reinforcement schedules, in which the number of lever presses required to earn a reward is systematically increased from trial to trial. To fulfill the escalating response requirement, the animal must exert more effort and endure longer delays on each successive trial. Thus, the PR instrumental paradigm incorporates compound effort and delay costs that escalate throughout a behavioral session. Under this instrumental schedule, animals behave in a manner that is consistent with their ability to track these compound costs to earn a reward (Wanchisen et al., 1988). Additionally, with PR reinforcement schedules, one can assess the breakpoint, or rather the highest response requirement an individual will overcome before ceasing responding. Previous studies have demonstrated a link between dopamine in the nucleus accumbens (NAcc) and performance during PR paradigms. Specifically, enhancing dopamine transmission elevates breakpoints (Zhang et al., 2003; Cagniard et al., 2006), and inhibiting dopamine signaling in the NAcc is associated with attenuated breakpoints (Aberman et al., 1998; Hamill et al., 1999). However, there are mixed findings over the relative contribution of the core and shell subregions of the NAcc. For instance, dopamine-specific lesions implicate the NAcc core but not the NAcc shell in overcoming high-effort constraints (Sokolowski and Salamone, 1998), whereas dopamine receptor antagonism implicates both the NAcc core and shell (Nowend et al., 2001). Indeed, phasic dopamine release can be differentially regulated between these subregions (Aragona et al., 2008, 2009).
Thus, in the current study, we examined phasic dopamine release in the NAcc core and shell to rewards and predictors of their availability during instrumental tasks under different reinforcement schedules. Specifically, in separate sessions, rats lever pressed for food rewards in discrete trials under either fixed costs [fixed ratio (FR sessions)], escalating effort and time costs (PR sessions), or escalating time with fixed-effort costs (low-effort yoked sessions). Within these tasks, rewards were available in discrete trials, which permitted the analysis of dopamine release to both reward-related cues and reward delivery. Using fast-scan cyclic voltammetry at chronically implanted microelectrodes (Clark et al., 2010), we found that phasic dopamine release to reward-availability cues in the NAcc core and shell was unaffected by escalating effort and delay costs. In contrast, increasing response requirements augmented reward-evoked dopamine release in the NAcc core and shell, which was a function of the delay to reward delivery, independent of effort-related costs.
Materials and Methods
Subjects and surgery.
All procedures were approved by the University of Washington Institutional Animal Care and Use Committee. Thirty-nine male Sprague Dawley rats (Charles River Laboratories) were used for this study. Animals were group housed, given ad libitum access to water and lab chow, and maintained on a 12 h light/dark cycle (lights on at 7:00 A.M.). For voltammetry electrode implantation surgeries, rats (∼300 g) were anesthetized with isoflurane, and holes were drilled in the skull for anchor screws, a Ag/AgCl reference electrode, and recording electrodes. The reference electrode was implanted, and carbon fiber electrodes were lowered into the NAcc core (coordinates relative to bregma: anteroposterior, 1.3 mm; mediolateral, 1.3 mm; dorsoventral, −7.0 mm) and/or the NAcc shell (anteroposterior, 1.5 mm; mediolateral, 0.6 mm; dorsoventral, −7.5 mm) and were secured with cranioplastic cement. Animals were allowed to recover for at least 3 weeks before beginning behavioral training. Of the 39 animals entering the study, 11 provided data for the NAcc core, seven for the NAcc shell, and two for both subregions (from bilateral electrodes). The remaining 19 animals were excluded as a result of postsurgical complications or not satisfying the criteria for reliable dopamine detection (see below). Electrode recording locations are presented in Figure 1.
Rats were placed on mild food restriction until they reached ∼90% free-feeding weight (∼2–3 d). Rats were maintained on food restriction throughout the experiment and were given standard lab chow (∼15 g/d) in the home cage to allow for an increase in weight of 1.5% per week. Experimental 45 mg food pellets (F0021; BioServ) were placed in their home cages on the day before the first training session to familiarize the rats with the food pellets. Operant chambers (Med Associates) had sloped floors and a house light and contained a food tray and two cue lights above two retractable levers on a single wall. The cue lights and their corresponding levers were located on either side of the food tray. All behavioral sessions commenced with illumination of the house light. On the first training session (single-lever training), one lever extended with its corresponding cue light illuminated. The lever would remain extended throughout the session, and a single lever press would deliver a food pellet (maximum of 100 pellets earned within 90 min). In some instances, inaccessible food pellets were placed behind the lever to promote lever pressing. After successful completion of the single-lever training session for one lever, rats were then trained to lever press on the opposite lever using the same training conditions. In the following session (pseudorandom single-lever training), the side of the active lever was alternated between trials within a session in which a lever press on the active lever would cause the delivery of a food pellet, retraction of the lever, and the cue light to turn off for a 15 s intertrial interval (ITI). A pseudorandom pattern dictated which side (left or right) was the active lever for a given trial. The pseudorandom single-lever training ended after the successful completion of 80 trials within 2 h. Finally, rats were trained to lever press multiple times for a single food pellet in FR sessions consisting of 60 trials in which the active lever side remained constant throughout the session. FR sessions began with both levers (active and inactive) extending and illumination of the house light and the cue light over the active lever. Completion of the correct number of lever presses led to a pellet delivery, retraction of the levers, and the cue and house lights turning off for a 30 s ITI. Rats were trained under an FR4 reinforcement schedule (four lever presses to complete a trial), first with one lever active and then with the opposite lever active in separate sessions. In a similar manner, rats were then trained using an FR8 reinforcement schedule. To habituate rats to the recording procedure, they performed a single FR4 session with voltammetry recordings, although data from this session were not included in the analysis.
After successful completion of the training described above, rats then performed a single session per day according to the schedule in Figure 2A [i.e., a single FR4 session (session 1) would be performed on the first experimental day, and a single PR session (session 2) would be performed on a subsequent experimental day]. FR4 sessions were the same to those during training sessions with 60 pellets earned typically over 30–35 min. PR sessions were identical to FR4 sessions except that the operant requirement on each trial (T) is the integer (rounded down) of 1.4(T − 1) lever presses, starting at one lever press (i.e., 1, 1, 1, 2, 3, 5, 7, 10, 14, 20, 28, 40, 56, 79, 111, 155, 217, 304, 426). PR sessions ended after 15 min elapsed without the subject completing the response requirement in a trial. In a preliminary set of animals, we found that PR sessions lasted ∼30 min under this schedule, to which the duration of FR4 sessions was matched. To minimize inflexible behaviors, we interleaved FR4 sessions between PR sessions and alternated the side of the active lever (Fig. 2A). Voltammetry data were collected from PR sessions and the FR4 sessions immediately preceding PR sessions (see below). A subset of animals (n = 5) were tested on a low-effort yoked session that was used to discern the effects of effort and time on dopamine release to reward delivery in PR sessions. The low-effort yoked sessions were similar in structure to PR sessions, except that (1) the required effort to complete a trial remained constant throughout the session (FR1), (2) the time until pellet delivery in a trial was modeled after the average trial duration to complete that same number trial in the PR sessions, and (3) the session ended after 18 trials. These low-effort yoked sessions were assessed on days in which a PR session would normally be performed (Fig. 2A).
During experimental recording sessions, the chronically implanted carbon-fiber microelectrodes were connected to a head-mounted voltammetric amplifier for dopamine detection by fast-scan cyclic voltammetry as described in detail previously (Phillips et al., 2003a; Heien et al., 2005; Clark et al., 2010). In brief, the potential applied to the carbon fiber was ramped from −0.4 V (vs Ag/AgCl) to +1.3 V and back at a rate of 400 V/s during a voltammetric scan and held at −0.4 V between scans. Scans were repeated at a frequency of 10 Hz throughout the session. The application of this triangular waveform causes redox reactions in electrochemically active species at the carbon fiber (including dopamine: approximately +0.7 and −0.3 V peak oxidation and reduction potentials, respectively), which can be measured as changes in current. To ensure that electrodes were capable of detecting dopamine, unexpected food pellets were delivered before and after a recording session to elicit dopamine release (Clark et al., 2010). Chemical verification of dopamine was achieved by obtaining high correlation of the cyclic voltammogram (electrochemical signature) to that of a dopamine standard (correlation coefficient, r2 ≥ 0.75 by linear regression). The voltammetry data for a session were not analyzed if food pellet delivery did not elicit dopamine release that satisfied the chemical verification criteria.
To be included for data analysis, animals must have a minimum of two FR4 and two PR sessions with dopamine release satisfying the chemical verification criteria described above. Voltammetric data analysis were performed using software written in LabVIEW and low-pass filtered at 2000 Hz. Dopamine was isolated from the voltammetric signal using chemometric analysis (Heien et al., 2005) using a standard training set of stimulated dopamine release detected by chronically implanted electrodes. Dopamine concentration was estimated based on the average postimplantation sensitivity of electrodes (Stuber et al., 2008; Clark et al., 2010). Data were smoothed using a 0.5 s moving average. Analysis of extracellular dopamine concentration was restricted to a period of 3 s after cue onset or reward delivery. Because of the close temporal relationship between cue onset and reward delivery in FR sessions, dopamine concentration to reward delivery in FR sessions was calculated by determining the peak change in dopamine levels in the 3 s after pellet delivery relative to the 0.5 s before reward delivery. Statistical analyses were performed in Prism (GraphPad Software) using linear regression or one or two-way ANOVA with repeated measures as needed, followed by post hoc Bonferroni's t tests. Data are presented as mean ± SEM.
After completion of the experimental sessions, animals were anesthetized with ketamine/xylazine (100 mg/kg) and the recording site was marked by making a small electrolytic lesion at the electrode tip by passing a current (∼70 μA) through the carbon-fiber microelectrode for 20 s. Animals were subsequently perfused transcardially with physiological saline and then with 4% paraformaldehyde in PBS, before the brains were removed and postfixed in a paraformaldehyde solution. The brains were then placed in 30% sucrose solution in PBS for 48 h, flash frozen, and sectioned coronally (60 μm). All sections were mounted and stained with cresyl violet.
In this study, we explored how phasic dopamine release to rewards and related stimuli was affected by changing effort and delay costs during instrumental behavior. Specifically, we examined dopamine release to the cues presented at the start of each trial and to reward delivery in the NAcc core and shell either with fixed response requirements, under a FR reinforcement schedule, or when effort and delay costs escalated under a PR reinforcement schedule. Animals were subjected to a single behavioral session per day with FR and PR sessions interleaved and the side of the active lever alternated between sessions (Fig. 2A). This design was used to minimize concerns related to satiety and to maintain flexible goal-directed behaviors.
Behavioral performance was consistent across multiple days in FR sessions and PR sessions. As expected, rats completed all 60 trials in FR sessions (FR4, total of 60 pellets earned, n = 20 rats in 234 sessions). In PR sessions, the number of food pellets earned (F(3,72) = 0.7, p = 0.6, one-way ANOVA) (Fig. 2B) and the lever presses on the last, uncompleted (breakpoint) trial (F(3,72) = 0.4, p = 0.8, one-way ANOVA) (Fig. 2C) were unchanged over multiple PR sessions. On average, rats earned 14.4 ± 0.2 food pellets per PR session, corresponding to a breakpoint of 107.7 ± 8.2 lever presses for a single food pellet (n = 20 rats in 76 sessions) (Fig. 2B,C). Interestingly, on the breakpoint trial, rats completed 100.7 ± 3.3% of the operant requirement of the previous trial (F(7,67) = 1.7, p = 0.1, one-way ANOVA) (Fig. 2D), demonstrating that the behavior is stable even within a single PR session.
Dopamine release during fixed-ratio sessions
To control for aspects of dopamine transmission in instrumental tasks that do not relate to escalating response requirements, we first used an FR reinforcement schedule. Specifically, we examined phasic dopamine release in individual trials during FR sessions to the cues presented at the start of each trial (cue-light illumination and lever insertion) and to reward delivery. Throughout the session, the presentation of trial-onset cues consistently elicited phasic dopamine release (Fig. 3A). Although there was no difference in cue-evoked dopamine release between the NAcc core and shell, there was a notable trial-dependent decrease in dopamine release to the cue over the session (region effect: F(1,1180) = 1.4, p = 0.3; trial effect: F(59,1180) = 11.8, p < 0.0001, two-way repeated measures ANOVA; n = 13 core, n = 9 shell rats) (Fig. 3B). Cue-evoked dopamine release reached asymptotic levels by the third trial in FR sessions in both the NAcc core and shell (region effect: F(1,1180) = 1.7, p = 0.2; trial effect: F(57,1180) = 1.2, p = 0.2; for trials 3–60, two-way repeated measures ANOVA) (Fig. 3B). We also examined the change in dopamine release to the food pellet delivered at the completion of a trial and found that reward-evoked dopamine release was negligible in both the NAcc core and shell throughout FR sessions (region effect: F(1,1140) = 0.2, p = 0.6; trial effect: F(59,1140) = 1.3, p = 0.08, two-way repeated measures ANOVA) (Fig. 3A,C). Therefore, under fixed-effort and delay costs, reward-evoked dopamine release in the NAcc was minimal, whereas cue-evoked dopamine release rapidly attenuated and then remained stable throughout the behavioral session.
Dopamine release during progressive-ratio sessions
After determining how reward-related stimuli elicited dopamine release in FR sessions, we next assessed the effect of escalating effort and delay costs on dopamine release to trial-onset cues in PR sessions. Similar to FR sessions, there was a trial-dependent decrease in cue-evoked dopamine release present in both the NAcc core and shell during PR sessions (trial effect: F(13,260) = 21.0, p < 0.0001 for trials 1–14; region effect: F(1,260) = 0.03, p = 0.9; two-way repeated measures ANOVA; n = 13 core, n = 9 shell rats) (Fig. 4A,B). Additionally, dopamine release to trial-onset cues reached stable levels after the first few trials throughout the NAcc (trial effect: F(10,200) = 1.6, p = 0.1; region effect: F(1,200) = 0.3, p = 0.6; for trials 4–14, two-way repeated measures ANOVA) (Fig. 4B). To assess cue-evoked dopamine release in relation to the decision to cease responding, it is necessary to examine the pattern of striatal dopamine release when referenced to the breakpoint trial. Therefore, we analyzed cue-evoked dopamine release when aligned to the last trial of the PR session and again found no effect of trial or difference between the NAcc core and shell on phasic dopamine release (trial effect: F(9,180) = 1.3, p = 0.2; region effect: F(1,180) = 0.4, p = 0.5; last 10 trials, two-way repeated measures ANOVA) (Fig. 4C). Importantly, a subsequent analysis determined that the pattern of cue-evoked dopamine release across trials is identical between FR and PR sessions in both the NAcc core (trial effect: F(13,312) = 25.0, p < 0.0001; session type: F(1,312) = 0.9, p = 0.4; analyzed for trials 1–14 with two-way repeated measures ANOVA; n = 13) (Fig. 5A) and NAcc shell (trial effect: F(13,208) = 23.0, p < 0.0001; session type: F(1,208) = 0.2, p = 0.7; analyzed for trials 1–14 with two-way repeated measures ANOVA; n = 9) (Fig. 5B). Together, these results suggest that escalating effort and delay costs do not influence dopamine release to trial-onset cues.
In contrast to the cue-evoked response, there was a notable difference between FR and PR sessions in dopamine release to reward delivery. Specifically, reward-evoked dopamine release was negligible in FR sessions (Fig. 3C) but increased in magnitude in later trials during PR sessions (Fig. 6A). There was a significant effect of trial on reward-evoked dopamine release with no difference between the NAcc core and shell (trial effect: F(7,140) = 18.7, p < 0.0001; region effect: F(1,140) = 0.01, p = 0.9; last eight trials, two-way repeated measures ANOVA; n = 13 core, n = 9 shell rats) (Fig. 6B,C), suggesting that overcoming escalating effort and/or delay costs in PR sessions elicits more dopamine release to reward delivery.
Dissociating the contribution of effort and delay costs on reward-evoked dopamine release
The costs in PR sessions comprise two components, those relating to the exerted effort and those associated with the delay to reward delivery (i.e., time to complete a trial). The delay is highly correlated with the response requirement (p < 0.0001, r2 = 0.93) (Fig. 7A), making it impossible, from the PR-task data, to discern whether effort and/or delay costs are driving the increase in reward-evoked dopamine release. Therefore, data were compared between the PR task and a low-effort yoked control task in which a single lever press yielded a food pellet delivered with a progressive delay matched to the behavior in PR sessions. In agreement with our previous results (Fig. 5), there was a significant effect of trial on cue-evoked dopamine release, but, importantly, there was no difference between PR, FR, and low-effort yoked sessions (trial effect: F(13,598) = 21.8, p < 0.0001; session effect: F(2,598) = 0.7, p = 0.5; analyzed for trials 1–14 with two-way repeated measures ANOVA; n = 22 for FR and PR, n = 5 for low-effort yoked). When assessing reward-evoked dopamine release, there was a significant effect of trial duration, but not session type, in both PR and low-effort yoked sessions (trial duration: F(2,575) = 15.0, p < 0.0001; session type: F(1,575) = 1.8, p = 0.2, two-way ANOVA) (Fig. 7B). Importantly, there was not a significant interaction between session type and trial duration on reward-evoked dopamine release (F(2,575) = 0.1, p = 0.9, two-way ANOVA) (Fig. 7B), indicating that trial duration in PR sessions accounts for the increasing reward-evoked dopamine release. Subsequent analyses identified a main effect of trial duration in both PR and low-effort yoked sessions (PR: F(2,526) = 49.3, p < 0.0001, one-way ANOVA, n = 529 trials from 20 rats; Low-effort yoked: F(2,49) = 6.5, p < 0.01, one-way ANOVA, n = 52 trials from 5 rats) (Fig. 7B), with significant differences found between all trial duration comparisons in PR sessions (***p < 0.001, Bonferroni's multiple comparison test) and a significant difference between short- and long-duration trials in low-effort yoked sessions (**p < 0.01, Bonferroni's multiple comparison test). Together, these findings demonstrate that increasing reward-evoked dopamine release in PR sessions was a function of time costs (i.e., trial duration) independent of costs resulting from exerted effort.
Reward availability is variable in natural settings, which confers constantly changing costs that must be overcome to obtain rewards (Stephens and Krebs, 1986; Stevens et al., 2005; Berger-Tal et al., 2009). Thus, decisions to pursue rewards need to accommodate these dynamic conditions. There is increasing experimental evidence that dopamine encodes reward-related information in the execution of economic cost–benefit decisions (Fiorillo et al., 2003, 2008; Phillips et al., 2003b, 2007; Roitman et al., 2004; Tobler et al., 2005; Roesch et al., 2007; Kobayashi and Schultz, 2008; Stuber et al., 2008; Salamone et al., 2009; Zhang et al., 2009; Gan et al., 2010). Previous work demonstrated that phasic dopamine transmission to reward-related stimuli was insensitive to the effort and delay associated with obtaining rewards, except under conditions when these costs changed (Roesch et al., 2007; Gan et al., 2010). Therefore, in the current work, we examined phasic dopamine release to reward-related stimuli under reinforcement schedules that incorporate cost structures with different dynamics. We found that progressively increasing the response requirement did not affect dopamine release in the NAcc core or shell to cues that predict reward availability. However, this manipulation elicited increases in dopamine release to the reward delivery itself in both of these NAcc subregions. This elevation in reward-evoked dopamine release was a function of the delay to reward delivery, independent of the effort exerted to obtain the reward.
Cue-evoked dopamine release
In all of the tasks used in this study, we noted that dopamine release to trial-onset cues was greatest for the first trial, decreased over the next few trials, and remained stable thereafter. Satiety is unlikely to be a significant contributor to this pattern because the attenuation of cue-evoked dopamine release occurred early in the session, whereas animals continued to work for food for many more trials (e.g., 60 total in FR sessions). Larger dopamine release in the first trial could be an effect of novelty associated with the unexpected start of the session, as has been suggested by other studies observing a similar profile of dopamine transmission (Takikawa et al., 2004; Roesch et al., 2007; Gan et al., 2010), or could be explained if this trial acts as an incentive cue for the entire session (Berridge, 2007). Regardless of the interpretation, attenuation of dopamine release to cue presentation over the first few trials is not attributable to changes in effort or delay costs because this pattern was identical between reinforcement schedules that had different cost structures.
Electrophysiological and neurochemical studies demonstrate that the predicted reward value is encoded by phasic dopamine transmission to reward-associated cues (Tobler et al., 2005; Gan et al., 2010). Pharmacological and genetic studies demonstrate that dopamine in the NAcc is used to promote high-effort behaviors (Aberman et al., 1998; Hamill et al., 1999; Salamone and Correa, 2002; Zhang et al., 2003; Cagniard et al., 2006). These seemingly divergent functions have made it difficult to conceptualize the role of dopamine signaling in the NAcc. However, the assessment of reward value and the decision to overcome costs to obtain rewards are necessarily linked. For example, anticipation of larger rewards promotes greater effort to be exerted to earn these rewards and elicits high levels of dopamine in the NAcc that facilitates this effort. Accordingly, it is suggested that phasic dopamine sets a threshold for the maximum cost that should be overcome to obtain rewards (Phillips et al., 2007). In the current work, cue-evoked dopamine release was stable for all but the first few trials of a session, which implies that the threshold to work remains constant from trial to trial. Consistent with this notion, the effort that rats exerted in the last (uncompleted) trial of PR sessions was very similar to that on the previous (completed) trial. Thus, the steady cue-evoked dopamine release with escalating effort and delay costs is indicative of a stable valuation and willingness to work for a single food pellet in the PR session.
Reward-evoked dopamine release
In contrast to the cue-evoked response, we found that reward-evoked dopamine release scaled with the increasing response requirement (Fig. 8). Although dopamine release to reward delivery was minimal in FR sessions, it increased throughout PR sessions. The cost of obtaining rewards in PR sessions comprised two highly correlated components: effort exerted and delay to reward delivery. As such, we could not determine which component(s) mediated the effect on reward-evoked phasic dopamine release from the data obtained in the PR task. Therefore, we experimentally separated effort and time costs in low-effort yoked sessions in which the effort costs were fixed and the delay costs progressively increased throughout the session. The identical pattern of reward-evoked dopamine release between PR and low-effort yoked sessions (i.e., no main effect of session type, no difference between session type for any trial duration, and no interaction) indicates that increasing delays, and not exerted effort, mediated the elevation of dopamine release to food pellet delivery in PR sessions. Analogous to our results in instrumental tasks, dopamine neuron activity to reward delivery is elevated with longer delays during pavlovian paradigms in primates (Fiorillo et al., 2008; Kobayashi and Schultz, 2008).
Contemporary theories of dopamine function can provide a framework to interpret the increased dopamine release to delayed rewards in the context of cognition. The prediction error theory posits that dopamine neurons respond to rewards only when they are unexpected (Schultz et al., 1997; Fiorillo et al., 2003). Because timing becomes less accurate for longer intervals (Weber's law), it follows that dopamine release to rewards, which are delayed under escalating costs, could be a function of the temporal uncertainty of reward delivery. Indeed, dopamine neurons are phasically activated by rewards delivered at unpredictable times (Fiorillo et al., 2008). However, it is problematic to empirically test whether temporal uncertainty is the primary source of delayed-reward evoked dopamine release because there is no overt behavioral measure of uncertainty (Fiorillo et al., 2008). Nonetheless, others have concluded that enhanced phasic dopamine transmission associated with delayed reward delivery reflects uncertainty in the timing of the reward delivery (Fiorillo et al., 2008; Kobayashi and Schultz, 2008). Under an alternative framework, dopamine in the nucleus accumbens has been posited to represent the attribution of incentive value to stimuli (Berridge, 2007). Because dopamine release to reward delivery scaled with increasing delays, this raises the provocative question whether rewards are perceived as more valuable after longer responses have been endured to obtain them. Behavioral studies have examined this question by assessing the choice between distinct stimuli that were paired to the reward after completing the required response under different cost conditions. Interestingly, these studies support the prediction from the incentive hypothesis of dopamine function by demonstrating a preference for conditioned stimuli associated with reward delivery after high costs (Clement et al., 2000; Kacelnik and Marsh, 2002; Friedrich and Zentall, 2004; Navarro and Fantino, 2005; Alessandri et al., 2008). Therefore, both the reward prediction error and incentive salience theories of dopamine function provide competing but plausible rationales for the elevated reward-evoked dopamine release observed with the delay conferred by escalating costs.
Comparing dopamine release between the nucleus accumbens core and shell
In our assessment of anatomical specificity of reward- and cue-evoked dopamine release, we found no differences between the NAcc core and shell. Functionally, these subregions are thought to mediate different aspects of behaviors (Di Chiara, 2002). Recent studies have determined that phasic dopamine release can be heteregenous (Wightman et al., 2007) and differentially regulated throughout the NAcc core and shell in certain behaviors (Aragona et al., 2008, 2009). However, in the operant tasks used in the current study, reward-related stimuli elicited identical phasic dopamine responses throughout the NAcc core and shell, demonstrating that the degree of cooperability of dopamine signaling between these NAcc subregions is task specific.
Overall, the current findings demonstrate that, in both the NAcc core and shell, dopamine release to rewards and their predictors are separable and independently modulated during increasing instrumental-response requirements. Reward-evoked dopamine release in both regions is affected by escalating costs in a manner mediated by delays to reward delivery rather than increased work requirements, whereas cue-evoked dopamine release is unaffected by either temporal- or effort-related costs. Together, these results reveal the pattern of phasic dopamine release to reward-related stimuli in economic decisions under escalating costs.
This work was supported by National Institutes of Health Grants R01 MH079292 (P.E.M.P.), R21-AG030775 (P.E.M.P.), R01-DA016782 (P.E.M.P.), T32-AA009455 (M. E. Larimer), and F32-DA026273 (M.J.W.). We thank S. Ng-Evans for invaluable technical support, C. Akers for technical assistance, and J. Clark, S. Sandberg, J. Gan, and N. Daw for insightful comments.
- Correspondence should be addressed to Paul E. M. Phillips, University of Washington, 1959 NE Pacific Street, Health Sciences Building, Box 356560, Seattle, WA 98195-6560.