Abstract
A wide range of behaviors is impaired after disruption of dopamine (DA) transmission, yet behaviors that are reflexive, automatic, or elicited by salient cues often remain intact. Responses triggered by strong external cues appear to be DA independent. Here, we examined the possibility that a single behavior may become DA independent as a result of extended training. Rats were trained to execute a head-entry response to a cue signaling food delivery. Vulnerability of the response to D1 or D2 receptor blockade was assessed on day 3, 7, or 17 of 28-trial-per-day training. During the early stages of training, the D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH 23390) increased response latencies; however, the same behavior was unaffected by SCH 23390 in animals tested during the later stages of training. Other aspects of behavior such as locomotion and head-entry responses during the uncued intertrial interval remained vulnerable to SCH 23390 throughout the experiment. This D1-mediated response was unaffected by the D2 antagonist raclopride, even at a dose that strongly suppressed locomotion. The results provide strong evidence that a D1-dependent behavior becomes less dependent on DA with extended training. A number of fundamental neurobiological changes occur as behaviors become learned habits; at least for some responses, this change involves a shift from D1-mediated to D1-independent responding.
Introduction
Impairments in locomotion and in the initiation of goal-directed movements have been observed after dopamine (DA) receptor blockade (Fowler and Liou, 1998; Pitts and Horvitz, 2000), experimental lesions of DA neurons (Carli et al., 1985; Salamone and Correa, 2002), and parkinsonian DA neuron degeneration (Marsden, 1984; Stern et al., 2005). Under certain conditions, however, patients with Parkinson's disease show “paradoxical kinesia,” a surprising ability to generate normal movements in otherwise motor-impaired patients. For example, patients with Parkinson's disease have been reported to walk quickly or even run out of a hospital room in response to a fire alarm or step normally over salient lines drawn on the ground (Martin, 1967; Jahanshahi and Frith, 1998). Parkinsonian motor deficits are most pronounced when the movement requires internal guidance and are greatly reduced when the movement is cued by a salient external stimulus in tasks that include reach-to-grasp movement (Schettino et al., 2004) and finger tapping (Frischer, 1989). Such observations have given rise to the view that under conditions of compromised DA transmission, the subject suffers a disruption not in the motor apparatus per se, but in the ability to voluntarily generate motor acts, i.e., to initiate movements in the absence of strong external elicitors (Jahanshahi and Frith, 1998).
Consistent with these data from human subjects, DA antagonist-induced motor impairments in rats are greatest when the behavioral response is elicited by relatively weak environmental stimuli. For instance, DA antagonists strongly disrupt the ability of rats to lick a metal spout when the lick is generated as an operant response for subsequent reward delivery but not when it is a reflexive response to water on the spout (Gramling and Fowler, 1985). Similarly, DA receptor blockade disrupts avoidance responses to a conditioned stimulus (CS) that signals subsequent shock but produces virtually no deficit in the animal's escape from shock itself (Beninger et al., 1980). DA-independent responding therefore occurs in response to external stimuli that are “strong” by virtue of innate response-eliciting properties; however, there is evidence to suggest that conditioned stimuli also become capable of eliciting DA-independent responses after extended habit training (Beninger and Hahn, 1983; Horvitz and Ettenberg, 1991). To directly test the hypothesis that a conditioned response becomes DA independent as a result of extended habit training, we examined the ability of both D1 and D2 antagonists to disrupt a simple behavior after different amounts of training. We report here that D1, but not D2, family receptor blockade disrupts the expression of a simple cued head entry during early stages of learning but has little or no effect on the response after extended training. Even after the well acquired, cued head entry has become invulnerable to D1 receptor blockade, noncued generation of the response remains suppressed by the D1 antagonist.
Materials and Methods
Subjects. A total of 173 male albino Sprague Dawley rats (275-350 g), obtained from Charles River Laboratories (Kinsington, NY), were housed in pairs within Plexiglas cages (22 cm high × 22 cm wide × 46 cm deep) mounted on a rack within an animal colony, with food and water available ad libitum. The colony was maintained at ∼23°C, with a 12 h light/dark cycle (lights on at 8:00 A.M.). Rats were handled gently during their first week of arrival and placed on a 23 h food-deprivation schedule for the remainder of the experiment.
Apparatus. Behavioral sessions were conducted in chambers (29 cm high × 29 cm wide × 25 cm deep; Coulbourn Instruments, Allentown, PA) housed individually within sound- and light-attenuated enclosures. Two opposite walls of the chamber were Plexiglas; the other two walls were metal. A house light was located at the top center of one metal wall, 2 cm below the ceiling. Recessed within the bottom center of this wall, 2 cm above the floor, was a food compartment (4.0 cm high × 3.0 cm wide × 2.5 cm deep), into which food pellets (45 mg; Bioserve F0021, Frenchtown, NJ) were delivered. Activation of the food magazine produced a 400 ms, 78 dB sound that served as the CS. The pellet settled at the bottom of the feeder trough ∼600 ms after feeder activation. An infrared photo-emitter detector was located on the sides of the food compartment, and interruption of the photobeam signaled to the computer (Dell Pentium; Dell, Round Rock, TX) the presence of the animal's head within the food compartment. A Coulbourn Instruments H24-61 activity monitor detected movement of infrared body heat across small compartments of a ceiling-mounted lens. The computer (running Coulbourn L2T2 software) recorded the time of head entries and withdrawals, pellet deliveries, and movement counts with 0.05 s resolution.
Drugs. The selective D1 antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1 H-3-benzazepine hydrochloride (SCH 23390) (Iorio et al., 1983) and the selective D2 receptor antagonist raclopride (Protais et al., 1994) (Sigma, St. Louis, MO) were dissolved in isotonic saline. Drugs were injected intraperitoneally in a volume of 1 ml/kg body weight 30 min before testing.
Procedure. A house light was illuminated several seconds after the rat was placed in the chamber and remained illuminated throughout the session. During each daily session, 28 pellets were delivered individually into the food compartment on a variable-time 70 s schedule (with a minimum 30 s interpellet interval). Time stamps for each motor count and each head entry into and removal out of the food compartment were recorded. At the end of the sessions, animals were returned to their home cages and provided food ad libitum for 1 h.
Rats received 0, 0.04, 0.08, or 0.16 mg/kg of the D1 antagonist SCH 23390 30 min before the CS-food session on day 3 (n = 8 per dose), day 7 (n = 9 per dose), or day 17 (n = 9, 9, 8, and 8 per dose, respectively) of training. Rats in the “3-7” control group received 2 d of cued head-entry training, rested in their home cages for 4 d, and were tested under the influence of 0 mg/kg (n = 7) or 0.16 mg/kg (n =8) SCH 23390 on day 7. During the home-cage days, animals continued to receive food ad libitum for 1 h daily at the scheduled time. The 3-7 group was therefore matched to the day 3 group in terms of the number of CS-food trials but was matched to the day 7 group in terms of the passage of time since initial training. Additional groups of rats received 0, 0.2, or 0.4 mg/kg of the D2 antagonist raclopride on day 3 (n = 11, 7, and 14 per dose, respectively). Pilot data showed that 0.4 mg/kg raclopride produces locomotor suppression greater than or equal to that produced by the high (0.16 mg/kg) dose of SCH 23390 (Choi and Horvitz, 2003). Because cued head-entry latencies were not disrupted by raclopride on day 3, only the high (0.4 mg/kg) dose and vehicle were tested on subsequent test days 7 and 17 (n = 6 per dose). Each rat in the experiment received a single injection in this between-subjects design.
Latencies of >10 s were assigned a 10 s score and designated “missed trials.” The frequency of head entries during a baseline period 10 s before CS presentation was examined to assess drug effects on uncued head entries occurring during the intertrial interval (ITI).
Results
An SCH 23390 dose (0, 0.04, 0.08, or 0.16 mg/kg) × training day (3, 7, or 17) two-way ANOVA on latency to enter the food compartment on cue presentation revealed the following: a main effect of dose (F(3,90) = 13.67; p < 0.00001), indicating that SCH 23390 increased response latencies; a main effect of training day (F(2,90) = 64.48; p < 0.00001); and a significant dose × training day interaction (F(6,90) = 5.47; p < 0.0001), indicating that the SCH 23390-induced increase in latency varied as a function of training days before SCH 23390 treatment.
Top, Mean ± SEM proportion of the 28 test trials that rats missed (i.e., showed a response latency of >10 s) as a function of the dose of the D1 antagonist SCH 23390 (SCH) on test days 3, 7, and 17 (left) and for the 3-7 control group (right). The D1 antagonist increased missed trials when administered during the third conditioning session (i.e., the day 3 group and the 3-7 control group). In contrast, rats that received the same doses of the D1 antagonist on either day 7 or day 17 of training showed no increases in missed trials. This decrease in missed trials over the course of training mirrored and accounted for the reduced overall latency scores on days 7 and 17 (see Results). Bottom, The D1 antagonist SCH 23390 produced locomotor suppression on all test days.
To better understand the nature of the SCH 23390-induced response deficit, a two-way ANOVA was conducted with missed trials excluded from the latency scores. Under this analysis, a dose × training day interaction was not observed (F(6,90) = 0.426; p = NS). In contrast, ANOVA conducted solely on the missed trials (mean proportion of trials missed) revealed a significant dose × day interaction (F(6,90) = 9.39; p < 0.00001). As seen in Figure 1 (top), and as confirmed in tests of simple main effects, the D1 antagonist increased the proportion of trials missed on day 3 (F(3,28) = 9.42; p < 0.0005) but not on day 7 (F(3,32) = 1.88; p = NS) or day 17 (F(3,30) = 1.12; p = NS). Because a Kolmogorov-Smirnov test indicated violations of normality for this measure, these data were also subjected to the nonparametric Kruskal-Wallis test. This analysis confirmed that SCH 23390 increased misses on day 3 (p < 0.001) but not on day 7 (p = NS) or day 17 (p = NS). Tolerance to the drug cannot account for these results, because each animal received only one injection.
It was of interest to determine whether the reduced vulnerability of the cued response to the D1 antagonist challenge in the day 7 group compared with the day 3 group was the result of the increased number of training trials or the passage of time between days 3 and 7, independent of the number of trials. A 3-7 control group (Fig. 1, top right) was matched to the day 3 group in terms of the number of CS-food trials but was matched to the day 7 group in terms of the passage of time since initial training. A dose (vehicle or 0.16 mg/kg SCH 23390) × condition two-way ANOVA comparing misses in the day 3 and the 3-7 control group (matched for amount of training but differing according to time passage before drug challenge) revealed a main effect of dose (F(1,27) = 21.27; p < 0.0001) but no dose × condition interaction (F(1,27) = 2.71; p = NS). Thus, with the number of training trials held constant, the passage of time itself did not produce resistance to the D1 antagonist challenge. In contrast, a two-way ANOVA conducted on data for the 3-7 control group versus day 7 conditions (matched for time passage before drug challenge and differing according to the number of training trials) revealed a main effect of dose (F(1,29) = 16.34; p < 0.001) and a significant dose × condition interaction (F(1,29) = 11.41; p < 0.01), indicating that vulnerability to the drug varied as a function of the number of training sessions. Kruskal-Wallis nonparametric analyses showed that SCH 23390 significantly increased missed trials in the 3-7 control group (p < 0.05) and in the day 3 group (reported above). Together, these data suggest that resistance to the D1 antagonist challenge from day 3 to day 7 of training cannot be accounted for by the passage of time itself, but can be accounted for by the increased number of CS-food trials over the course of the 7 training days.
Head entries into the food compartment (horizontal bars) were recorded during the period from 16 s before to 10 s after food delivery. Consecutive trials (1-28) are represented as successive rows on the y-axis (bottom to top). The vertical line at time 0 indicates the presentation of the auditory cue and food delivery 600 ms later. Additional time between trials (ITI time beyond this 26 s period) is not shown. On test day 3, SCH 23390 (SCH) disrupted head entries both during the ITI (before sec 0) and in response to the food cue (compare Saline with SCH on day 3; top left vs bottom left). By test day 17, the cue-elicited response had become invulnerable to D1 antagonist challenge (bottom right), whereas execution of the same response during the ITI remained vulnerable to the D1 antagonist (compare top right with bottom right). Although only representative rasters from days 3 and 17 are shown, performance on day 7 was similar to day 17 in that the cue-elicited response had become invulnerable to D1 receptor blockade, whereas ITI responses remained D1 dependent (see Results).
Locomotor scores for day 3, 7, and 17 are shown in Figure 1 (bottom). An SCH 23390 dose × day two-way ANOVA revealed a main effect of dose (F(3,90) = 10.27; p < 0.00001) and of day (F(2,90) = 6.35; p < 0.05) but no interaction (F(6,90) = 0.42; p = NS). SCH 23390 suppressed locomotion on day 3 (F(3,28) = 3.90; p < 0.05), day 7 (F(3,32) = 6.76; p < 0.01), and day 17 (F(3,30) = 2.9; p = 0.05). Although the conditioned head entry became DA independent with extended training, locomotion remained D1 dependent throughout the stages of cued head-entry training.
Figure 2 shows raster plots of head entries (horizontal bars) from 16 s before (-16 to 0 s) to 10 s after (0-10 s) pellet delivery. The rasters show the behavior of representative rats receiving either vehicle or the high dose of SCH 23390 on day 3 (Fig. 2, left) or day 17 (Fig. 2, right). As can be seen, the D1 antagonist continued to suppress the uncued expression of the head entry during the ITI even after the cued expression of the response had become invulnerable to the drug (Fig. 2, bottom right). A day × dose two-way ANOVA conducted on the frequency of head entries during the 16 s baseline period before presentation of the auditory cue revealed the following: a main effect of SCH 23390 dose (F(3,90) = 17.20; p < 0.00001), reflecting the suppressive effect of SCH 23390 on baseline head entries; no main effect of day (F(2,90) = 1.66; p = NS), indicating that baseline head entries did not vary over training days; and no dose × day interaction (F(6,90) = 0.79; p = NS), indicating that the effect of SCH 23390 on baseline head entries did not vary over the course of training. Therefore, although the conditioned head entry became D1 independent with extended training, the identical (or nearly identical) response during the ITI remained dependent on D1 receptor activity.
Finally, it was of interest to determine whether this D1 antagonist-induced suppression of cued head entries was similarly produced by the selective D2 antagonist raclopride and, if so, whether this effect was reduced with extended training. A raclopride dose × training day two-way ANOVA conducted on latency scores indicated the following: no effect of dose (F(2,49) = 1.59; p = NS); a main effect of day (F(2,49) = 3.50; p < 0.05); and no dose × day interaction (F(2,49) = 0.27; p = NS). A two-way ANOVA conducted on misses revealed no effect of dose (F(2,49) = 2.74; p = NS) or of training day (F(2,49) = 1.94; p = NS) and no dose × day interaction (F(2,49) = 1.12; p = NS). (Note that because the 0.2 mg/kg raclopride dose was tested on day 3 and not on days 7 and 17, data for this dose are excluded from the interaction analyses.) The Kolmogorov-Smirnov test showed that misses again violated assumptions of normality and therefore were subjected to the nonparametric Kruskal-Wallis tests. The tests showed that raclopride did not increase misses on day 3, 7, or 17 (p = NS for each day); however, raclopride strongly suppressed locomotion. A two-way ANOVA on locomotor scores showed a main effect of drug (F(1,49) = 15.54; p < 0.005), no effect of day (F(2,49) = 0.92; p = NS), and no interaction (F(2,49) = 1.23; p = NS). Raclopride, which did not affect expression of the cued head-entry response, produced locomotor suppression that was comparable with (and tended to be greater than) that produced by 0.16 mg/kg SCH 23390 (71 vs 52% suppression). Together, these data suggest that initiation of the conditioned head entry during early stages of training is dependent on activity at D1 and not D2 family receptors and that the response becomes D1 independent with extended training.
Discussion
During early stages of training, D1 receptor blockade disrupts both the expression of a head entry response to a CS and the spontaneous expression of the response during ITIs. After extended training, expression of the CS-elicited response becomes resistant to D1 receptor antagonist challenge. Reductions in the vulnerability of the conditioned response to D1 antagonist challenge cannot be attributed to drug tolerance, because each animal received only one drug injection. Furthermore, the extended training sessions produced a reduced vulnerability to SCH 23390, specifically in the expression of the conditioned response; other aspects of behavior, e.g., locomotion and head entry responses during the noncued ITI, remained suppressed by the D1 antagonist throughout stages of cued head-entry training. The results of the 3-7 control group showed that the decreased DA dependence of the conditioned response was the result of increased training trials and could not be accounted for simply on the basis of the passage of time. The D2 receptor family antagonist raclopride, at a dose that strongly suppressed locomotion, did not affect expression of the conditioned head-entry response during any stage of training. These data suggest that activity at the D1 receptor family is critical for the expression of cued and noncued head entries during early stages of training but that expression of the cued head entry becomes DA independent over the course of training.
Disruption of the cued response during early stages of training was not the result of a general slowing of responses to the CS but rather an increased probability that the animal would fail to execute a response during a given trial, i.e., an increase in the proportion of missed trials. In animals receiving extended training before SCH 23390 challenge, no increase in missed trials was observed (and virtually no trials were missed). These data suggest that during early stages of training, D1 receptor blockade decreases the likelihood that the eliciting stimuli in the environment will engage response systems rather than reduce the speed of response systems once activated. It is possible that the reduction in DA-mediated responding is related to the enhanced ability of a discrete external cue to elicit a response: under conditions of noncued operant responding, rats remain highly vulnerable to the response-suppressing effects of DA depletions even when the depletions are administered after extended training (Correa et al., 2002).
Some neural changes must occur over the course of extended training to render the cued response invulnerable to D1 receptor blockade. At least two general kinds of change may be possible. First, it is possible that DA transmission mediates the conditioned appetitive response during early but not late phases of learning because the response is originally represented in DA target regions; however, with extended training, the response shifts to mediation by non-DA target areas. Consistent with this possibility, neurons in DA target regions such as the lateral striatum show single-unit responses that are time locked to an operant lever press during early phases of lever-press training; however, with extended training, these cells no longer show activity time locked to the operant response (Carelli et al., 1997). This suggests that the appetitive response, originally represented in the lateral striatum, is not represented there after overtraining. Similarly, frontal (Jueptner et al., 1997a) and striatal (Jueptner et al., 1997b) regions become less active with extended training of a motor sequence in humans. It is therefore possible that a behavior, mediated by activity in DA target regions during early stages of training, shifts to non-DA regions after extended training and becomes invulnerable to the otherwise disruptive effects of DA receptor blockade. It has been suggested that over the course of habit learning, response representation may shift from the basal ganglia to direct corticocortical mediation (Carelli et al., 1997).
Alternatively, it is possible that as the conditioned response becomes well acquired, the response continues to be mediated by the same neurons that originally mediated response expression, but DA plays a declining role in response modulation. For example, cortical glutamate input to striatal cells may depend on striatal D1 transmission to amplify the strength of task-relevant input signals (Horvitz, 2002; O'Donnell, 2003) during early phases of learning. During later stages of learning, these glutamate synapses may become so efficient that DA facilitation of glutamate transmission is no longer necessary for normal response expression. If strengthened corticostriatal synapses represent enhanced salience of the CS, then with extended training, CS salience might become sufficient to overcome a raised sensorimotor threshold produced by D1 receptor blockade. From this point of view, striatal neurons may depend on DA facilitation during early stages of habit acquisition and performance (Packard and McGaugh, 1996; Jog et al., 1999), but with extended training, effective throughput of corticostriatal input signals occurs even in the absence of DA transmission.
One might argue that with extended training the DA neuronal response to the CS becomes so strong that a given DA antagonist dose is no longer sufficient to block the increasing amount of DA released into the synapse at the time of CS presentation. This explanation appears unlikely given single-unit and dialysis data showing that although DA neurons respond to appetitive conditioned stimuli (Mark et al., 1994; Schultz, 2001), extended training reduces rather than increases the magnitude of CS-elicited DA release (Ljungberg et al., 1992; Phillips et al., 2003; Ikegami and Duvauchelle, 2004). Indeed, these data are consistent with the two previously described hypotheses that assume a reduced DA dependence over the course of habit training.
In summary, DA plays an important role in the process by which incoming stimuli gain access to appropriate motor outputs (Redgrave et al., 1999; Horvitz, 2002; Salamone and Correa, 2002). Our data suggest that the role of dopamine in the performance of a simple appetitive response to an auditory cue diminishes with extended habit training.
Footnotes
This work was supported by National Institutes of Health-National Institute on Drug Abuse Grants R29 DA11653 (J.C.H.) and R21 DA015210 (P.D.B.).
Correspondence should be addressed to Jon C. Horvitz, Department of Psychology, 301 McGuinn Hall, Boston College, Chestnut Hill, MA 02467. E-mail: jon.horvitz{at}bc.edu.
Copyright © 2005 Society for Neuroscience 0270-6474/05/256729-05$15.00/0
