Abstract
Tonic dopamine (DA) signaling is widely regarded as playing a central role in effort-based decision making and in the motivational control of instrumental performance. The current study used microdialysis to monitor changes in extracellular DA levels across subregions of the nucleus accumbens and dorsal striatum of rats as they lever pressed for food reward on a probabilistic schedule of reinforcement, a procedure that ensured they would experience variation in the amount of effort needed to earn rewards across tests. Each rat was given three tests. Rats were hungry for the first and last test, but were sated on food before the middle test, allowing us to assess the effects of a downshift in motivational state on task performance and conditioning-induced DA efflux. During hungry tests, DA levels rose in both the shell and core of the accumbens and, to a lesser degree, in both the medial and lateral divisions of the dorsal striatum. Interestingly, changes in DA efflux across hungry tests in the accumbens core were negatively correlated with changes in the effort required to obtain rewards. We also found that—across regions—the DA response to instrumental conditioning was attenuated when rats were sated before testing. Furthermore, the effect of satiety on DA efflux in the accumbens shell was positively correlated with its effect on task performance. Together, the results indicate that tonic DA contributes to the control of instrumental performance by conveying information about the costs and benefits of responding to different striatal subregions.
Introduction
Striatal dopamine (DA) signaling is known to be involved in effort-based decision making and in regulating the vigor with which instrumental actions are performed (Walton et al., 2006; Niv et al., 2007; Salamone et al., 2007). Microdialysis studies have found that nucleus accumbens DA levels rise as rats lever press for food rewards, with a few studies finding a positive correlation between the magnitude of conditioning-induced DA efflux and response rate (McCullough et al., 1993; Sokolowski et al., 1998; Cousins et al., 1999), suggesting that DA may provide a response-activating function. However, in apparent contrast to these findings, studies using fast-scan cyclic voltammetry have found evidence that DA may encode the cost of responding, with high-effort actions being accompanied by equal or smaller elevations in DA than low-effort actions (Day et al., 2010; Gan et al., 2010).
In free operant performance, response rate is modulated by motivational state. Rats will, for example, press a lever that delivers food pellets more rapidly when they are hungry than when they are sated. Not surprisingly, striatal DA has been assigned an important role in mediating the effects of primary motivation on instrumental performance (Willner et al., 1988; Niv et al., 2007), with some researchers suggesting that its involvement may be limited to the motivational support provided by stimuli paired with reward (Dickinson et al., 2000). If so, then conditioning-induced DA efflux should be modulated by motivational state. Although there is some support for this claim (Wilson et al., 1995; Ahn and Phillips, 1999), existing studies have used passive feeding tasks, which may not engage the same motivational processes as instrumental conditioning and provide no information about the relationship between DA efflux and instrumental performance variables.
Although the dorsal striatum receives strong DAergic projections and has been implicated in the control of instrumental performance (Yin et al., 2008; Balleine et al., 2009), little is known about the role of dorsal striatal DA signaling in instrumental control. We used microdialysis to monitor changes in striatal DA levels in rats lever pressing for food rewards, targeting sites in both the nucleus accumbens (core or shell) and dorsal striatum (lateral or medial). Rats received three test sessions on separate days using an ABA design. The first and last tests were conducted when the rats were hungry, allowing us to characterize conditioning-induced DA efflux in these areas under a state of high food motivation. Before the second test, rats were fed to satiety so that we could determine whether the DA response to instrumental conditioning was modulated by motivational state. This design also allowed us to determine whether changes in DA efflux across tests varied with changes in either the rate of lever pressing or number of rewards earned per session. Furthermore, our use of a probabilistic (random ratio) reinforcement schedule ensured that the rats would experience variability in the number of responses (or effort) required to earn rewards across tests, allowing us to explore the relationship between conditioning-evoked DA efflux and response cost.
Materials and Methods
All procedures were approved by the Animal Research Committee of University of California, Los Angeles, and were performed in accordance with National Institutes of Health Guide for the Care and Use of Laboratory Animals.
Subjects and surgery.
Male Long–Evans rats (∼90 d of age at the beginning of the experiment; Harlan Laboratories) were housed singly in a temperature- and humidity-controlled vivarium. Experiments were conducted during the light phase of a 12 h light/dark schedule. Rats were deeply anesthetized with isoflurane inhalation and implanted unilaterally (left or right hemisphere, counterbalanced across placements) with a guide cannula (CMA 11) lowered into the core [anteroposterior (AP), +1.2; mediolateral (ML), ±1.8; dorsoventral (DV), −6.5] or shell (AP, +1.7; ML, ±0.8; DV, −6.5) of the nucleus accumbens, or the dorsolateral (DLS) (AP, +0.7; ML, ±3.6; DV, −4.0) or dorsomedial striatum (DMS) (AP, −0.4; ML, ±2.6; DV, −3.5) (all coordinates in millimeters relative to bregma). Dental cement was used to form a head cap, which sealed the wound and anchored the guide cannula and a tethering screw to three stainless-steel screws placed in the skull. Rats were given 5–7 d to recover before behavioral training. During the recovery period, they were placed on food restriction regimen (∼12 g of home chow per rat per day) to maintain them at ∼85% their free-feeding body weight throughout training and testing.
Instrumental training.
All behavioral training and testing sessions were conducted in identical MED Associates chambers. Rats were given a 180 min session of magazine training, during which grain-based food pellets (45 mg; Bio-Serv) were delivered on a 5 min random-time (RT) schedule into a recessed food magazine. A 2 s clicker stimulus (10 Hz; ∼75 dB) accompanied pellet deliveries during all training and testing sessions. They were then given 7 d of instrumental conditioning. Each session began with the insertion of a retractable lever located to the left of the food magazine. Rats were allowed to earn 50 pellets on a continuous reinforcement schedule in session 1. Reinforcement was shifted to a probabilistic, random-ratio (RR) schedule for the rest of training and testing, with the probability of reward given lever press performance decreasing over sessions: a RR-5 schedule [p(reward/press) = 0.20] was used in sessions 2–4, and a RR-10 schedule [p(reward/press) = 0.10] was used in sessions 5–7. Rats were allowed to earn up to 30 pellets in each of sessions 2–7. Each training session began after an initial context exposure period (60 min in session 1 and 90 min in sessions 2–7). The rats were tethered to a liquid swivel during sessions 6–7 to habituate them to this experience. The house light (3 W, 24 V) was illuminated as soon as the rats were placed in the chamber and was extinguished once all of the available outcomes had been earned.
Microdialysis procedures and behavioral testing.
After the last session of instrumental training, rats were implanted with a single CMA 11 microdialysis probe (length, 2 mm; outer diameter, 0.24; molecular cutoff, 6 kDa; CMA), attached to a dual-channel liquid swivel (Instech Laboratories) with FEP tubing (inner diameter, 0.12 mm; CMA). Connections were made using short lengths of PVC tubing. The rats were then individually housed in transparent Perspex cylindrical containers (height, 38 cm; diameter, 27 cm; subsequently referred to as home cages) in the vivarium for the remainder of the experiment when not undergoing dialysis testing. Rats were given their daily allotment of food after probe implantation. The microdialysis probe was continuously perfused with artificial CSF (125 mm NaCl, 2.5 mm KCl, 0.9 mm NaH2PO4, 5 mm Na2HPO4, 1.2 mm CaCl2, 1 mm MgCl2, pH 7.4) for 3 d. A flow rate of 2.0 μl/min was used during behavioral testing. The flow rate was reduced to 0.3 μl/min between tests. On the day after probe implantation, rats were given the first of three test (conditioning) sessions, which were conducted at the normal time of training over consecutive days (i.e., at intervals of ∼24 h). For each session, rats were placed in the operant chamber, with the liquid swivel mounted on a balance arm attached to the side wall. After a 2 h context habituation period, the lever was inserted into the chamber and rats were allowed to lever press for food pellets on a RR-10 schedule for 30 min. No limit was placed on the number of outcomes that could be earned. The lever was then retracted and the house light was extinguished. The rats were left in the chamber for an additional 30 min before returning them to the vivarium. The three test sessions were identical, with one important exception: rats were hungry for tests 1 and 3, but were sated on food for test 2. Specifically, rats were given their daily allotment of food in the home cage shortly after test 1 was complete. On the following day, rats were given 12 g of home chow 2 h before they placed in the chamber for test 2. Any uneaten food was removed from the home cage at this time. Since no additional food was provided after test 2, the rats entered test 3 having been deprived of food for ∼23 h.
Dialysate sampling and electrochemical detection.
Dialysate was collected into refrigerated (8°C) tubes containing 0.6 μl of 12.5 mm perchloric acid/250 μm EDTA at 3 min intervals (6 μl per sample) before (3 samples), during (10 samples), and after (10 samples) each test session. Collection times were adjusted to account for the estimated time lag in flow from the probe membrane to the FEP tubing outlet (∼9 min). HPLC with electrochemical detection (Antec Leyden) was used to determine the dialysate concentrations of DA, as previously described (Murphy and Maidment, 1999). The mobile phase (17% acetonitrile, 0.68 mm SDS, 10 μm EDTA, 35 mm phosphate buffer, pH 5.5) was pumped (Shimadzu model LC-10AD) at a rate of 0.2 ml/min through a 2 × 100 mm column (2 μm particle size; TSK-GEL Super ODS; Tosoh Biosciences). The working electrode was maintained at +450 mV. Standard concentrations of DA were analyzed with each set of test samples. For each test, dopamine levels were normalized as a percentage of the average of the three presession (baseline) samples.
Histology.
After the experiment, rats were given an overdose of sodium pentobarbital (100 mg/kg, i.p.). Their brains were removed, postfixed, and cryoprotected in a 30% sucrose–formalin solution, and cut into 50 μm coronal sections across the brain region being targeted. Sections were mounted on glass slides, stained with cresyl violet, and analyzed under a light microscope to determine probe placement.
Results
Probe placement
Figure 1 presents the individual microdialysis probe placements for all rats. In general, the probes were successfully implanted in the targeted brain area, either the accumbens core (n = 6), accumbens shell (n = 7), DLS (n = 7), or DMS (n = 8).
Schematic representation of microdialysis probe placements for rats in groups core (n = 6; black), shell (n = 7; gray), DLS (n = 7; white), and DMS (n = 8; horizontal stripes). The numbers represent distance (in millimeters) of coronal section from bregma.
Acquisition of lever pressing
As can be seen in Figure 2, rats from the four probe placement conditions acquired comparable rates of instrumental performance. An ANOVA using surgical group (core, shell, DMS, and DLS) and session (1–7) as factors found a significant main effect of session (F(6,144) = 99.72; p < 0.001), but found no effect of surgical group (F(3,24) = 1.16; p = 0.35) or session by group interaction (F(18,144) = 0.97; p = 0.50).
Acquisition of lever pressing. Shown are mean (±SEM) lever presses performed per minute for rats in groups core, shell, DLS, and DMS.
Baseline dialysate DA levels
Mean DA levels during pretesting baseline periods were 0.83 nm (SEM, ±0.15) for the core, 0.73 nm (SEM, ±0.13) for the shell, 1.46 nm (SEM, ±0.17) for the DLS, and 0.96 nm (SEM, ±0.14) for the DMS. Analysis of these data revealed a significant main effect of probe placement (F(3,26) = 4.72; p = 0.01). Bonferroni's post hoc tests found that basal DA levels were significantly higher in the DLS than in core (p = 0.05) or shell (p < 0.01). No other differences reached significance, indicating that DA levels in the DMS were intermediate between those observed in the DLS and nucleus accumbens.
Conditioning-induced DA efflux in the nucleus accumbens
Tests 1 and 3 allowed us to measure changes in extracellular DA concentrations as rats lever pressed for food reward in a state of high food motivation. Although task performance was comparable across these two tests, a mixed ANOVA using probe placement (core and shell) and test (1 and 3) as factors found that rats with accumbens probe placements responded at a marginally higher rate in test 3 than in test 1 (F(1,11) = 3.41; p = 0.092), which may have resulted from the additional conditioning given during the test sessions. No effect of placement (F(1,11) = 0.24; p = 0.634) or placement by test interaction (F(1,11) = 0.20; p = 0.667) was detected. Analysis of the number of rewards earned during these tests found no effect of test (F(1,11) = 0.24; p = 0.632) or placement (F(1,11) = 0.06; p = 0.808) and found no test by probe interaction (F(1,11) = 0.11; p = 0.742).
Instrumental conditioning resulted in an elevation of accumbal DA during the session, which returned to near baseline levels after the session ended (Fig. 3). These data were analyzed using a mixed ANOVA with probe placement, test, and time (23 × 3 min samples) serving as factors. The ANOVA detected a significant effect of time (F(22,242) = 11.37; p < 0.001), confirming that DA levels changed over the course of the session. There was no effect of test (F(1,11) = 0.94; p = 0.354) or probe placement (F(1,11) = 0.26; p = 0.619). Nor was there a significant test by group (F(1,11) = 0.01; p = 0.929), test by time (F(22,242) = 0.83; p = 0.682), or test by time by group interaction (F(22,242) = 0.75; p = 0.785).
DA efflux in the core (left panel) and shell (right panel) of the nucleus accumbens of hungry rats lever pressing for food pellets. The black lines represent mean (±SEM) DA concentrations [expressed as percentage of baseline (i.e., average for bins −3, −2, and −1)] for consecutive 3 min dialysate samples collected during tests 1 (solid line) and 3 (broken line). Bins 1–10 (gray rectangle) correspond to the 30 min test session. The gray lines represent mean (±SEM) lever presses per minute (alternate y-axis).
Although the magnitude of conditioning-induced DA efflux in the accumbens was similar for the two hungry tests when the data were averaged across animals, our multitest experimental design allowed us to explore the relationship between conditioning-induced DA efflux and important experimental variables using an approach that minimizes between-subjects variability (e.g., probe function). Specifically, we examined whether individual differences in the average DA response during instrumental conditioning (average of bins 1–10 for each subject during test sessions) between hungry tests were correlated with differences in either the number of lever presses performed or the number of pellets earned between the two sessions. Furthermore, since reinforcement was delivered in a probabilistic manner according to a RR-10 schedule, rats experienced variability in the number of responses needed to obtain reward (i.e., total number of presses performed per session/total number of rewards obtained per session) between the two tests, allowing us to investigate the relationship between DA and the effort (or response cost) required to gain each outcome. For each variable, test 1 values were subtracted from test 3 values to produce a difference score. Table 1 presents the mean (SEM) and range of difference scores for each of these variables. We found that changes in DA efflux in the nucleus accumbens (n = 13) between tests 1 and 3 were not significantly correlated (Pearson's, two-tailed) with changes in response rate (r = −0.049; p = 0.874) or changes in the number of rewards earned (r = 0.432; p = 0.140) between the two tests. Therefore, our data do not support the view that accumbal DA mediates a simple response-activating influence over instrumental performance, or else DA levels should have varied with response vigor. Nor does DA appear to simply track the number of actions performed or number of pellets earned. However, changes in DA were found to be negatively correlated with changes in the number of presses required to obtain reward (r = −0.580; p = 0.038) (Fig. 4), such that the DA response during conditioning dropped as the work requirement increased, and vice versa. This correlation was significant when the data from rats with accumbens core placements were separately analyzed (n = 6; r = −0.851; p = 0.032), indicating that the modulation of conditioning-induced DA efflux by response requirement in this structure is particularly strong. Although no correlation was found for rats with shell placements (n = 7; r = −0.567; p = 0.184), this lack of effect should be interpreted with caution, particularly given the significant correlation found when both probe placement groups were included in the analysis.
The mean difference in conditioning-induced DA efflux (average of bins 1–10), total lever presses performed, total rewards earned, and ratio of responses per reward (i.e., average response requirement) between the two hungry tests (test 3 − test 1)
Scatter plot showing the relationship between changes in conditioning-induced DA efflux and response requirement (i.e., the mean number of lever presses required to earn each reward) across hungry tests (test 3 − test 1) for rats in groups shell (black circles; solid trend line) and core (open circles; broken trend line).
To assess the effects of general satiety on tonic DA signaling and instrumental performance, we contrasted data collected during tests conducted when the rats were hungry (average of tests 1 and 3) with the data from test 2, which was conducted after the rats had been sated for 2 h on home chow. As shown in Figure 5, this satiety induction procedure was generally effective in reducing the rate of lever press performance (main effect of motivational state: F(1,11) = 25.79; p < 0.001). No effect of probe placement (F(1,11) = 0.65; p = 0.438) or state by placement interaction (F(1,11) = 0.93; p = 0.355) was detected. An ANOVA performed on the number of rewards earned during hungry and sated tests found that rats earned significantly fewer rewards when sated than when hungry (F(1,11) = 28.22; p < 0.001). No effect of placement (F(1,11) = 0.41; p = 0.536) or state by placement interaction (F(1,11) = 0.99; p = 0.340) was detected.
Effect of satiety on conditioning-induced DA efflux in the core (left panel) and shell (right panel) of the nucleus accumbens. The black lines represent mean (±SEM) DA concentrations [expressed as percentage of baseline (i.e., average for bins −3, −2, and −1)] for consecutive 3 min dialysate samples collected during the hungry (solid line; average of tests 1 and 3) and sated (broken line; test 2) tests. Bins 1–10 (gray rectangle) correspond to the 30 min test session. The gray lines represent mean (±SEM) lever presses per minute (alternate y-axis).
In general, the effects of instrumental conditioning on accumbal DA efflux also appeared to be sensitive to downshift in motivational state (Fig. 5). A mixed ANOVA performed on these data detected a marginally significant main effect of motivational state (F(1,11) = 4.24; p = 0.064), a significant effect of time (F(22,242) = 12.92; p < 0.001), and, more importantly, a significant state by time interaction (F(22,242) = 1.78; p = 0.020), indicating that the time course of conditioning-induced DA efflux was affected by the rats' motivational state at test, with rats exhibiting an attenuated DA response when they were sated. There was no main effect of probe placement (F(1,11) = 0.14; p = 0.718), nor was there any interaction between placement and state (F(1,11) = 0.04; p = 0.843), placement and time (F(22,242) = 0.62; p = 0.910), or between these three variables (F(22,242) = 0.98; p = 0.49).
The correlational analysis described above indicated that accumbal DA does not exert a direct response-activating effect on instrumental performance, or track the number of responses performed or rewards earned. However, if accumbal DA is involved in the motivational control of instrumental performance, one might expect the effects of satiety on DA efflux to vary with the behavioral effects of satiety. To assess this possibility, we examined whether individual changes in the average DA response during instrumental conditioning (average of bins 1–10 for each subject) between sated (test 2) and hungry tests (data averaged across tests 1 and 3) correlated with differences in the rate of responding or number of pellets earned between these conditions. For each variable, the mean value for the hungry tests was subtracted from the value for the sated test to generate a difference score; therefore, a negative difference score indicates that the sated value was lower than the hungry value. The mean difference scores (SEM) and the range scores are presented in Table 2. When the data were collapsed across sites, a marginally significant correlation was detected between changes in accumbal DA efflux and changes in response rate (r = 0.514; p = 0.072). Although a similar relationship was apparent between changes in DA and changes in the number of rewards earned, this correlation also failed reach the conventional threshold for significance (r = 0.455; p = 0.118). Additional analysis limited to accumbens shell placements, however, found significant positive correlations for both response rate (r = 0.827; p = 0.022) (Fig. 6A) and rewards earned (r = 0.778; p = 0.039) (Fig. 6B), which were highly correlated with one another (r = 0.973; p < 0. 001) since rewards were delivered according to a ratio schedule. Changes in core DA efflux were not correlated with either variable (response rate: r = −0.241; p = 0.646; rewards earned: r = −0.243; p = 0.643). Thus, it appears that, at least for the accumbens shell, the suppressive effects of satiety on conditioning-induced mesolimbic DA efflux vary with its behavioral effects.
The mean difference in conditioning-induced DA efflux (average of bins 1–10), total lever presses performed, and total rewards earned between the sated and hungry tests (test 2 − average of tests 1 and 3)
Scatter plot showing the relationship between the effect of satiety (sated test − hungry test) on conditioning-induced DA efflux and its effects on lever press performance (A) and on the number of rewards earned (B) for rats in groups shell (black circles; solid trend line) and core (open circles; broken trend line).
Conditioning-induced DA efflux in the dorsal striatum
A mixed ANOVA using probe placement (DLS, DMS) and test (1 and 3) as factors found that rats with dorsal striatal probe placements responded at similar rates across the two hungry tests (F(1,13) = 0.87; p = 0.368). No effect of placement (F(1,13) = 0.20; p = 0.662) or placement by test interaction was detected (F(1,13) = 0.26; p = 0.621). Analysis of the number of rewards earned in tests 1 and 3 found no effect of test (F(1,13) = 1.03; p = 0.328) or placement (F(1,13) = 0.16; p = 0.692), and found no interaction between these factors (F(1,13) = 1.16; p = 0.300). Although somewhat more modest than in the nucleus accumbens, dorsal striatal DA levels also increased as hungry rats lever pressed for food reward during tests 1 and 3 (Fig. 7), an effect that was observed in both lateral and medial regions. A mixed ANOVA using probe placement group, test, and time as factors detected a significant main effect of time (F(22,286) = 2.83; p < 0.001), confirming that DA levels increased during these test sessions. There was no effect of test (F(1,13) = 0.07; p = 0.791) or probe placement (F(1,13) = 0.48; p = 0.502), nor was there any significant interactions between group and test (F(1,13) = 0.02; p = 0.905), group and time (F(22,286) = 1.16; p = 0.286), or between the combination of these three variables (F(22,286) = 0.81; p = 0.720).
DA efflux in the DLS (left panel) and DMS (right panel) of hungry rats lever pressing for food pellets. The black lines represent mean (±SEM) DA concentrations [expressed as percentage of baseline (i.e., average for bins −3, −2, and −1)] for consecutive 3 min dialysate samples collected during tests 1 (solid line) and 3 (broken line). Bins 1–10 (gray rectangle) correspond to the 30 min test session. The gray lines represent mean (±SEM) lever presses per minute (alternate y-axis).
Although dorsal striatal DA levels significantly increased during tests conducted when rats were hungry, this effect did not appear to be associated with any of the behavioral variables being recorded. Changes in the DA response (n = 15, collapsed across placements) to instrumental conditioning between tests 1 and 3 were not correlated with changes in response rate (r = 0.077; p = 0.785), number of rewards earned (r = 0.112; p = 0.691), or response requirement (r = 0.220; p = 0.431).
Figure 8 shows the effects of satiety on lever pressing and conditioning-induced DA efflux in the dorsal striatum. Rats with dorsal striatal placements also responded at a significantly lower rate when sated than when hungry (F(1,13) = 8.43; p = 0. 012). There was no effect of probe placement (F(1,13) = 0.87; p = 0.369) and no state by placement interaction (F(1,13) = 0.73; p = 0.407). Analysis of the number of rewards earned in hungry and sated tests found a main effect of motivational state (F(1,13) = 10.02; p = 0.007). No effect of placement (F(1,13) = 0.26; p = 0.622) or interaction between state and placement (F(1,13) = 0.06; p = 0.816) was detected. Although DA levels in the dorsal striatum appeared to increase slightly at the beginning of the sated test session, these changes were much less persistent than those observed when the rats were tested hungry (average of tests 1 and 3). This interpretation was supported by the statistical analysis, which, although failing to detect a significant overall effect of motivational state (F(1,13) = 1.89; p = 0.192), did find a significant state by time interaction (F(22,286) = 1.62; p = 0.041). There was no effect of group (F(1,13) = 0.89; p = 0.362), nor was there any significant interactions between group and state (F(1,13) = 0.001; p = 0.975), group and time (F(22,286) = 1.25; p = 0.206), or between group, state, and time (F(22,286) = 0.51; p = 0.969). Although it appears that conditioning-related DA efflux in the dorsal striatum was modulated by motivational state, unlike in the accumbens, subject-specific changes in DA efflux between hungry and sated tests were not significantly correlated (n = 15; collapsed across placements) with changes in response rate (r = −0.108; p = 0.702) or rewards earned (r = 0.011; p = 0.969).
Effect of satiety on conditioning-induced DA efflux in the DLS (left panel) and DMS (right panel) of the nucleus accumbens. The black lines represent mean (±SEM) DA concentrations [expressed as percentage of baseline (i.e., average for bins −3, −2, and −1)] for consecutive 3 min dialysate samples collected during the hungry (solid line; average of tests 1 and 3) and sated (broken line; test 2) tests. Bins 1–10 (gray rectangle) correspond to the 30 min test session. The gray lines represent mean (±SEM) lever presses per minute (alternate y-axis).
Discussion
The current study replicates previous reports of instrumental conditioning-induced DA efflux in the nucleus accumbens and finds evidence of a similar, if more modest, effect in the dorsal striatum. Importantly, however, we found that task-related accumbal DA efflux varied with the work requirement rather than tracking reward receipt or behavioral output. Across the striatum, conditioning-induced DA efflux was attenuated when rats were sated on food before testing, an effect that correlated with task performance in rats with shell placements.
DA and instrumental conditioning
Striatal DA signaling appears to be involved in several aspects of instrumental control. DA plays a central role in incentive motivation (Beninger and Phillips, 1981; Robinson and Berridge, 1993; Dickinson et al., 2000). The response-invigorating effects that reward-paired cues have on instrumental performance are blocked by treatments that disrupt accumbal DA transmission (Murschall and Hauber, 2006; Corbit et al., 2007; Lex and Hauber, 2008). Furthermore, microdialysis studies have found cue-evoked DA efflux in the absence of any instrumental response requirement (Cheng et al., 2003), suggesting that DA efflux observed during instrumental tasks may be generated by pavlovian conditioning occurring in the background (cf. Cheng and Feenstra, 2006; Ahn and Phillips, 2007). However, DA signaling also plays a role in effort-based decision making (Walton et al., 2006; Salamone et al., 2007). Behavior maintained on high ratio schedules is more sensitive to disruption by DA depletion than behavior maintained on low ratio or interval schedules of reinforcement, which require less effort (Salamone et al., 2007). This contribution of DA signaling to instrumental control appears to be gated by primary motivation, as DA antagonists are more effective in suppressing performance in hungry than in sated rats (Dickinson et al., 2000) and appear to mimic the effects of satiety on instrumental performance (Willner et al., 1988).
A recent computational model of instrumental performance (Niv et al., 2007) proposes that tonic DA levels in the accumbens provide a running estimate of the net rate of reward, integrating the various costs and benefits of responding in the current environment. DA is assumed to play an indirect role in controlling response vigor by signaling what one should expect to lose by failing to act (i.e., the cost of sloth). According to this theory, the magnitude of DA efflux during instrumental conditioning should decrease if the effort needed to obtain reward increases, or if the value of reward decreases. An alternative approach to modeling the role of DA in decision making (Phillips et al., 2007) proposes that DA release signals the expected value of an available reward, allowing this information to be contrasted with an independent estimate of the effort needed to obtain reward when deciding whether or not an action should be selected. Note that this account predicts that DA levels should be unaffected by manipulations of response cost.
In the current experiment, rewards were delivered according to a probabilistic schedule, such that average number of lever presses (or effort) required to obtain reward varied randomly across tests. We found that changes in conditioning-evoked accumbal DA efflux between tests were negatively correlated with changes in the amount of effort needed to obtain reward, a relationship that was particularly striking for the accumbens core. This finding indicates that information about response cost is conveyed to the accumbens via tonic DA signaling, consistent with predictions of Niv et al. (2007). Interestingly, this result mirrors recent findings from fast-scan cyclic voltammetry studies showing that phasic accumbal DA release to discriminative cues encodes the effort needed to obtain reward (Day et al., 2010; Gan et al., 2010). However, it should be noted that the single-action, free-operant procedure used here differs considerably from the two-action, discrete-trial procedures used in these other studies. The similarity between these findings should also be considered in light of the existing literature indicating that tonic and phasic DA release represent distinct channels of neurotransmission and that phasic DA signaling does not significantly contribute to the changes in DA concentrations that are measured with microdialysis (Floresco et al., 2003). It therefore seems likely that tonic and phasic DA independently convey information about response costs (and benefits), perhaps mediating fundamentally different neural and psychological functions.
Our failure to find a significant correlation between changes in response rate and accumbal DA efflux across hungry tests suggests that tonic DA does not simply track the number of rewards received or responses performed. Nor do these data support the view that DA mediates a simple response-activating function. There is, however, some support for this latter view. There have been multiple reports that conditioning-induced DA efflux in the ventral striatum positively correlates with response rate (McCullough et al., 1993; Sokolowski et al., 1998; Cousins et al., 1999). Our finding of a positive correlation between changes in shell DA levels and response rate across hungry and sated tests could also be taken as support for this view. However, other studies have failed to find evidence of a relationship between response rate and DA efflux (Cousins and Salamone, 1996; Ahn and Phillips, 2007). Interestingly, when higher ratio schedules are used, the correlation between DA and lever pressing can become hyperbolic, with rats responding at intermediate and high rates exhibiting equivalent levels of DA efflux (Salamone et al., 1994). Such mixed results suggest that the relationship between tonic DA signaling and lever press performance is complex and is likely to depend on the nature of the task and/or the schedule of reinforcement being used.
Dorsal striatum
The finding that DA levels in the dorsal striatum increased during instrumental conditioning is perhaps not surprising given the massive DAergic projections to this region and given recent findings implicating it in instrumental action selection (Yin et al., 2008; Balleine et al., 2009). As with the nucleus accumbens, treatments that disrupt dorsal striatal DA transmission tend to suppress instrumental performance (Amalric and Koob, 1987; Salamone et al., 1993). It has also been reported that rats lever pressing for food exhibit increased DA metabolism in dorsal striatum (Church et al., 1986). It is therefore noteworthy that a recent study monitoring DA levels in the dorsal striatum during instrumental conditioning found no evidence of conditioning-induced DA efflux (Ahn and Phillips, 2007). However, the area being targeted in that study was considerably more anterior and ventral than either of the two dorsal striatal areas targeted in the current experiment, which were selected because of their established roles in goal-directed [DMS (Yin et al., 2005)] and habitual [DLS (Yin et al., 2004)] performance.
Conditioning-induced DA efflux in the dorsal striatum was of a substantially smaller magnitude than that observed in the nucleus accumbens. Consistent with this finding, it has recently been reported that food-evoked increases in DA are smaller in the dorsal striatum than in the nucleus accumbens (Zhang et al., 2009). It is also noteworthy that, unlike in the accumbens, dorsal striatal DA efflux was not significantly correlated with any of our behavioral variables. Such findings indicate that there is functional heterogeneity in striatal DA signaling, with accumbal—but not dorsal striatal—DA playing a central role in the motivational control of instrumental performance (Yin et al., 2007). However, dorsoventral differences in DA transporter density (Marshall et al., 1990) and DA reuptake (Stamford et al., 1988; Garris et al., 1994) suggest that the relatively weak dorsal striatal DA response that we observed may reflect the rapid clearance of extracellular DA in this area rather than a regional difference in DA release.
Downshift in motivational state
Our results indicate that conditioning-related tonic DA signaling is modulated by primary motivation. Specifically, sated rats showed an attenuated DA response to instrumental conditioning, an effect that was observed in both the nucleus accumbens and dorsal striatum. This finding is generally consistent with previous reports that satiety treatments tend to decrease the accumbal DA efflux observed during and in anticipation of feeding (Wilson et al., 1995; Ahn and Phillips, 1999). However, these previous studies used passive feeding procedures and therefore could not address questions about the relationship between DA and instrumental performance. In the current study, we found that the effect of satiety on conditioning-evoked DA efflux in the accumbens shell was correlated with its effects on lever pressing and on the number of rewards earned. Although our use of a ratio schedule of reinforcement resulted in a strong positive correlation between response rate and reward delivery, thus preventing us from determining whether one of these variables was preferentially associated with DA signaling, the current findings indicate that tonic DA signaling in the shell may play a particularly important role in the motivational control of instrumental performance.
Conclusion
The current study found broad but heteregeneous instrumental conditioning-related tonic DA signaling across subregions of the striatum. Although we found little evidence that tonic DA directly controlled behavioral output, it did appear to track shifts in primary motivation and response cost, variables that influence the vigor with which instrumental actions are performed. These results may inform future research investigating the role of tonic DA signaling in addiction and other disorders of behavioral control.
Footnotes
-
This work was supported by National Institute on Drug Abuse Grants DA09359 and DA05010 (N.T.M.), National Institute on Alcohol Abuse and Alcoholism Grant AA18014 (B.W.B.), and National Institutes of Health Training Fellowship T32 MH17140 (S.B.O.).
- Correspondence should be addressed to Sean B. Ostlund, University of California, Los Angeles, Semel Institute for Neuroscience and Human Behavior, 760 Westwood Plaza, Los Angeles, CA 90024. sostlund{at}ucla.edu
