Abstract
Central to the production of adaptive behavior is the ability to learn the temporal order of behavioral elements (e.g., A, B, C). Yet little is known about neural substrates of serial order in self-initiated behavioral sequences. The present study assessed the contributions of specific dorsal striatal regions to the acquisition of serial order in mice, using a two-action sequence task without instructive cues. Excitotoxic lesions of the sensorimotor (dorsolateral) striatum dramatically impaired the acquisition of a simple sequence; in contrast, lesions of the associative (dorsomedial) striatum had no significant effect. Neither lesion caused gross motor impairments or affected the learning of nonsequential actions. These results demonstrate for the first time a critical role of the sensorimotor striatum in the learning of serial order.
Introduction
Serial order is a fundamental property of behavior. As I type this sentence, each word is a unique sequence of letters, and the letters themselves are part of an alphabetical sequence that I learned long ago, starting with A, B, C. Although genetic programs can organize certain fixed action patterns with predetermined serial order independently of experience, the order of many behaviors is learned.
The basal ganglia have long been considered important for the organization of sequential behavior (Benecke et al., 1987; Graybiel, 1998). In particular, the striatum is anatomically well situated to integrate inputs from cortex and thalamus and sends projections that ultimately influence the thalamocortical network as well as various brainstem nuclei. Previous work suggested a role for the striatum in the sequential organization of natural behaviors in monkeys (Van den Bercken and Cools, 1982) and rats (Berridge and Whishaw, 1992; Pellis et al., 1993). For example, Berridge and colleagues found that the dorsal striatum is critical for serial order in grooming sequences in rats (Berridge and Whishaw, 1992; Aldridge and Berridge, 1998). Because grooming sequences are relatively independent of learning, however, it remains unclear whether, in addition to a role in the expression of innate action sequences, the dorsal striatum is also needed for the learning of any arbitrary serial order.
This study examined the role of the dorsal striatum in the acquisition of serial order using a discrete-trial, self-initiated sequence task (Balleine et al., 1995; Yin, 2009). Previous work has demonstrated considerable functional heterogeneity in the dorsal striatum [e.g., sensorimotor striatum or dorsolateral striatum (DLS) and associative or dorsomedial striatum (DMS) can be functionally dissociated] (Sabol et al., 1985; Devan et al., 1999; Yin et al., 2004, 2005a). An important question, therefore, is whether these striatal regions also play distinct roles in the acquisition of serial order. I therefore compared the effects of DLS and DMS lesions on the acquisition of a simple sequence using a discrete-trial operant task. Mice were required to press the two levers in a specific order (i.e., left lever, right lever) to earn a food reward.
Materials and Methods
Animals.
All experiments were conducted in accordance with the Duke University Institutional Animal Care and Use Committee guidelines. Male C57BL/6J mice (∼3 months, Jackson Laboratory) were used in the experiments. They were placed on a food deprivation schedule to reduce their weight to ∼85% of their normal weight. All mice were fed 1.5–2 g of home chow each day after testing and training. Water was available at all times in the home cages.
Instrumental training.
Training took place in 8 Med Associates (St. Albans, VT) operant chambers (21.6 cm long × 17.8 cm wide × 12.7 cm high) housed within light-resistant and sound-attenuating walls. Each chamber was equipped with a food magazine that received Bio-Serv 14 mg Dustless Precision Pellets from a pellet dispenser, with two retractable levers on either side of the magazine and a 3 W, 24 V house light mounted on the wall opposite the levers and magazine. Computers with the Med-PC-IV program were used to control the chambers and record behavior.
Behavioral training started ∼1 week after surgery. Initial lever-press training consisted of three consecutive days of continuous reinforcement (CRF), during which the animals received a pellet for each lever press. At the beginning of each session, the house light was illuminated and the left lever was inserted. At the end of each session, the house light turned off and the lever retracted. Sessions ended at 90 min or 30 rewards.
The discrete trial serial order task was first used in a previous study (Yin, 2009). Briefly, two levers were inserted at the beginning of each trial, which ended after two presses. A specific sequence of two presses (left to right) was immediately followed by a food pellet. There was no limit on trial length, but the intertrial interval was always 8 s. Each session ended after 50 rewards or 90 min.
To measure sensitivity to partial reinforcement and ability to press a single lever at higher rates, 18 of the mice (7 controls, 5 DLS, and 6 DMS) were exposed to a cyclic ratio schedule on the left lever following the completion of sequence training ∼4 weeks after surgery (Ettinger and Staddon, 1983). The cyclic ratio schedule consisted of five values: 2, 4, 8, 16, and 32. These values were presented in an ascending sequence followed by a descending sequence, with a complete cycle of 10 ratios. After the completion of each ratio, a pellet was delivered. Each session ended after six complete cycles.
Surgery and histology.
Mice were anesthetized with isoflurane (induction at 3%, maintained during surgery at 1%) and placed on a stereotaxic frame (Kopf). The scalp was cut to expose the skull surface and small holes were drilled into the skull just above the target sites. Bilateral lesions were made by manually infusing quinolinic acid (10 mg/ml) into either the DMS or DLS at a rate of 0.1 μl/min for 5 min (total volume: 0.5 μl per site) using a 1 μl Hamilton syringe. For DMS lesions, the coordinates (in millimeters relative to bregma) were: anteroposterior +0.3, mediolateral ±2, dorsoventral (relative to dura) −3. For DLS lesions, the same volume was injected at the following coordinates: anteroposterior +0.5, mediolateral ±2.5, and dorsoventral −3. The needle was left in place for 5 min to allow the drug to diffuse. Sham lesions were made using the same procedures except no drug was infused.
After the completion of the experiments, all mice were anesthetized with isoflurane and transcardially perfused with 0.9% saline followed by 10% buffered formalin solution. The brains were sliced into 100 μm coronal sections with a Vibratome 1000 Plus, stained with thionin, and examined under a light microscope to verify the placement and extent of the lesions.
Results
Histological analysis of striatal lesions
The brains of all mice were examined. Two mice with inaccurate lesions were excluded from the final analysis. The extent of striatal lesions is shown in Figure 1, with dotted lines indicating the largest lesions and filled areas indicating the smallest lesions. In all cases included in the final data analysis, the lesions were bilateral and characterized by gliosis and loss of neurons. The DMS lesions were often associated with enlarged ventricles but did not extend to lateral regions. The DLS lesions spared tissue in the medial striatum as well as in the ventrolateral striatum. Very little damage to the overlying cortex was observed. Final group sizes were: n = 12 for Sham, n = 6 for DMS, and n = 8 for DLS.
Illustration of the largest (dotted lines) and smallest (black) lesions and initial acquisition. The coronal view diagrams are taken from a standard mouse brain atlas (Paxinos and Franklin, 2003). Numbers indicate distance (anterior-posterior) in millimeters from bregma.
Single lever training
Figure 2A illustrates the initial acquisition of lever pressing under CRF, in which each press is reinforced with a food pellet. All mice learned to press the lever after three sessions. Striatal lesions had no effect on the rate of pressing or head entries, as confirmed by a two-way mixed ANOVA conducted on the acquisition data, with lesion and time as factors. For the rate of lever presses (Fig. 2A), there was a main effect of time (F(2,48) = 36.0, p < 0.05), no main effect of lesion (F(2,48) = 1.5, p > 0.05), and no interaction between lesion and time (F < 1). For the rate of head entries (Fig. 2B), there was no main effect of time, no main effect of lesion, and no interaction between lesion and time (F < 1). Thus, selective lesions of the dorsal striatum did not cause any gross motor deficits, nor did such lesions create any significant deficit in instrumental learning per se, because the lesioned mice learned to normally press the lever under CRF.
Striatal lesions did not affect lever pressing on a single lever. A, The rate of lever presses on the left lever and head entries into the food magazine during the first 3 d of lever press training under a CRF schedule. B, Mean rate of lever pressing on five different fixed ratios in a cyclic ratio schedule. Striatal lesions had no effect on the rate of lever pressing under either continuous or partial reinforcement schedules on a single lever when the reward was not contingent upon the learning of serial order.
To assess lever pressing under leaner reinforcement schedules, 18 mice (7 controls, 5 DLS, and 6 DMS) were tested under a cyclic ratio schedule, which mixed five different ratio requirements in a single session. Figure 2C shows the rate of lever pressing under different fixed ratio requirements. A two-way ANOVA (lesion by ratio) showed no main effect of lesion (F(2,60) = 1.0, p > 0.05), a main effect of ratio (F(4,60) = 18.6, p < 0.05), and no interaction between these factors (F < 1.0). Thus, all three groups showed higher rates of lever pressing under higher ratio requirements, but there was no significant effect of striatal lesions on rate of lever pressing.
Acquisition of serial order
As shown in Figure 3D, DLS lesions impaired the acquisition of the LR sequence. This observation is confirmed by two-way mixed ANOVA conducted on the proportion of correct (rewarded) sequences, with lesion and time (2 d blocks) as factors. There was a main effect of lesion (F(2,138) = 8.0, p < 0.05), a main effect of time (F(6,138) = 41.5, p < 0.05), and a significant interaction between these factors (F(12,138) = 5.1, p < 0.05). Post hoc analysis revealed that the DLS group was significantly worse than the control group and the DMS group after 6 training sessions (p < 0.05 for days 6, 8, 10, 12, and 14).
Acquisition of serial order. A, Rate of reward. B, Trials per session. C, Rate of lever pressing on the two levers. D, Proportion of the four possible sequences: LL, RR, LR, RL. Only LR was followed by a food reward.
For the discrete-trial sequence task, four sequences [left-left (LL), left-right (LR), right-left (RL), and right-right (RR)] were possible. The proportion of incorrect sequences was also analyzed. There was no group difference in RL and LL, but whereas the proportion of RR sequences decreased in the control and DMS groups, it remained high in the DLS group. A two-way ANOVA revealed a main effect of lesion (F(2,138) = 7.8, p < 0.05), a main effect of time (F(6,138) = 3.5, p < 0.05), and no interaction between these two factors (F(12,138) = 1.7, p > 0.05). Post hoc analysis showed a significant difference between DLS and control for the last three 2 d blocks (p < 0.05).
Figure 4 displays two additional measures of the sequence performance: the latency to start pressing after trial onset (insertion of both levers) and the interpress interval between two lever presses in a correct sequence. During training, latency was reduced in all groups, but it was higher in the DLS group than in the control group. There was a main effect of time (F(6,138) = 10.0, p < 0.05), a main effect of lesion (F(2,138) = 13.6, p > 0.05), and a significant interaction (F(12,138) = 3.3, p < 0.05). Post hoc analysis revealed that the DLS group showed higher latency than the other two groups for the first two 2 d blocks (p < 0.05). The interpress interval also decreased significantly during training but was much higher in the DLS group (Fig. 4). There was a main effect of time (F(6,138) = 12.0, p < 0.05), a main effect of lesion (F(2,138) = 12.1, p < 0.05), and a significant interaction (F(12,138) = 2.3, p < 0.05). Again, the interpress interval was higher in the DLS group for the first four 2 d blocks (p < 0.05).
Latency to initiate sequence, interpress interval, and duration of lever presses in the LR sequence. A, B, Lesions of the DLS increased the latency to initiate any sequence (A) and the interpress interval (B). C, Whereas the control and DMS groups showed a longer duration on the first press of the sequence, the DLS group did not.
Finally, the duration of presses in the correct sequence was analyzed. A recent study using the discrete-trial sequence task demonstrated longer duration of the first press in a sequence (Yin, 2009). Here, this result was replicated. Figure 4 shows the mean press durations during the course of training. For both the control and DMS groups, the press duration was higher on the first (left) press, but no clear difference between the first and second presses was observed in the DLS group. To analyze the difference in press duration between the first and second presses in the correct sequence, a mixed two-way ANOVA was conducted for each group. For the control group, there was a main effect of time (F(6,132) = 17.3, p < 0.05) and a main effect of lever (F(1,132) = 10.7, p < 0.05) but no significant interaction between these factors (F(6,132) = 1.6, p > 0.05). For the DMS group, there was a main effect of time (F(6,60) = 3.96, p < 0.05) and a main effect of lever (F(1,60) = 10.8, p < 0.05) but no significant interaction between these factors (F < 1). For the DLS group, there was no main effect of time (F < 1), no main effect of lever (F < 1), and no significant interaction between these factors (F(6,84) = 1.5, p > 0.05).
Discussion
This study examined the effects of selective lesions of DLS and DMS on the acquisition of a simple sequence. Neither lesion produced gross motor deficits, nor did they impair the acquisition of lever pressing per se (Fig. 2). However, a significant and selective deficit in the acquisition of serial order was observed after lesions of the sensorimotor striatum. Lesions of the associative striatum, in contrast, produced no significant effect. These data reveal a critical striatal substrate for the acquisition of serial order in mice.
Clinical observations have long suggested the importance of the frontal lobes for serial order (Petrides and Milner, 1982; Milner et al., 1985; Fuster, 2001). In monkeys, sequence-specific neurons have also been found in the supplementary motor cortex, lateral prefrontal cortex, and supplementary eye field cortex (Mushiake et al., 1990; Aizawa et al., 1991; Shima and Tanji, 1998; Lu et al., 2002). But most studies did not examine the learning of serial order, only its expression in well-trained monkeys, and few data are available on the effects of lesions or inactivation on serial order of actions (Miyachi et al., 1997; Lu and Ashe, 2005; Ashe et al., 2006).
Recent work in rodents using self-initiated behavioral tasks has begun to elucidate the neural substrates of serial order (Ostlund et al., 2009; Yin, 2009). In mice, the primary motor cortex does not seem to be critical for the acquisition of serial order, though a significant deficit was observed after lesions of the secondary motor cortex (Yin, 2009). Given the convergence of inputs from primary somatosensory and motor cortices to the sensorimotor striatum, perhaps larger lesions extending to the somatosensory cortices are needed to affect serial order (McGeorge and Faull, 1987, 1989; Ragsdale and Graybiel, 1990; Flaherty and Graybiel, 1991; Gerfen and Wilson, 1996). In addition, because damage to the parietal association cortices in humans can produce a variety of apraxic symptoms, including deficits in serial order (Luria, 1966), additional cortical regions may be involved in the expression of serial order (McGeorge and Faull, 1989). Because of the striatum's unique role in the cerebrum in integrating diverse cortical inputs, sensorimotor striatal lesions may result in greater deficits in serial order compared to comparable damage to any particular cortical region whence the corticostriatal projections arise.
In addition to the proportion of correct sequences, latency to initiate the sequence once the trial begins and the interpress interval were also measured to characterize serial order acquisition. Damage to the DLS made the mice slower in initiating the sequence and in producing the second press in the correct sequence (Fig. 4). There was therefore impaired grouping of the two actions in a sequence. The press duration data were also analyzed for the correct sequence. Previous work has found that the duration of the first lever press in a sequence is on average longer and more variable than that of the second lever press (Yin, 2009). As shown in Figure 4, whereas the control and DMS groups showed longer duration on the left lever press (first action), the DLS group did not.
Further insight into the deficits caused by DLS lesions can be gathered from an analysis of the errors made. The sequence studied here consisted of two actions, one distal and the other proximal to the reward. Two types of errors are possible: repeats (LL, RR) or alternates (RL). The most common error made by the DLS group is the production of repeats, in particular the persistence of RR, the repeat of the proximal action (Fig. 3). The alternative sequence of RL, in contrast, is almost never produced. Note also that the LL sequence was frequent early in acquisition, which is not surprising, because the animal had learned to press the left lever during the preliminary lever press training (Fig. 2). Yet over time, the LL sequence was extinguished. In contrast, the relative frequency of RR increased in all mice along with an increase in LR (Fig. 3). Thus, in controls and DMS lesioned mice, this increase in RR was transient, whereas it persisted in the DLS lesioned mice even though it was never rewarded. The effect of reward following LR, then, was initially a nonspecific increase in frequency of all sequences ending with R (LR and RR). Gradually, mice were able to distinguish between these two and perform the correct LR sequence more frequently, unless they had DLS lesions. In the DLS group, both LR and RR increased in frequency, though the incorrect RR remained the most common pattern produced.
The pattern of errors suggests three conclusions. First, in sequence acquisition, the proximal action is initially favored; the reinforcement selects the closest operant in the recent past that leads to reward. If LR is rewarded, then the animal will initially repeat R, the action immediately preceding the reward. Only after repeated non-reinforcement of the RR does the animal stop performing the incorrect sequence. Second, DLS lesions did not produce a general inability to discriminate between different sequences; the lesioned mice did not produce RL and LL sequences any more frequently than did the control and DMS groups. Finally, the persistence of the incorrect RR sequence in the DLS group cannot be explained by increased perseveration, because the LL sequence was rapidly reduced in all groups, or by a general tendency to repeat an action, as only the sequence with the same proximal action as the correct sequence was repeated. Rather, the overall pattern of errors made by the DLS-lesioned mice suggests a selective deficit in the binding of different actions to form a performance unit.
Comparison with cued tasks
The discrete-trial, two-action sequence task used here promises to be a useful tool for studying the neural substrates of serial order. Although serial order of actions has long interested theorists (Lashley, 1951; Bernstein, 1967), the investigation of its neural basis has been hampered by the lack of analytical behavioral assays. Most studies used cued tasks in which each element of a sequence is instructed by some stimulus, but successful performance of such tasks does not require the internal representation of serial order. Consider the difference between typing your password and having each letter lit up in the right order before you touch each key. In the latter scenario, you do not have to know the password at all, yet it is more similar to traditional cued tasks used to study sequence learning.
For example, in the popular serial reaction time task, visual cues instruct the animal which response to make (e.g., nose poke when a light is turned on above a hole) (Robbins, 2002; Bailey and Mair, 2006). Even if a series of cues is presented in a specific order, the ability to represent serial order is confounded by the ability to follow cues. To address this problem, often performance on a new sequence is compared with that on a learned sequence on probe tests. This comparison, however, is made only in reaction time, which is not a direct measure of serial order learning. Improvements in reaction time with training on repeated sequences of cues can be explained by a number of factors (e.g., faster movements, improvements in visual attention, learning the optimal position from which to detect a stimulus). Of course, actually learning the serial order would also result in shorter reaction time on familiar sequences, but this possibility cannot be experimentally demonstrated using the serial reaction time task.
Functional heterogeneity in the striatum
Many studies have now shown considerable functional heterogeneity in the striatum (Sabol et al., 1985; Devan et al., 1999; Yin and Knowlton, 2004; Yin et al., 2004, 2005b), but caution is needed in comparing current results with those from previous studies in rats. Clearly, DMS lesions did not affect serial order acquisition whereas DLS lesions did. But without assessing the effects of the DMS lesions on other behavioral tasks, it is difficult to claim that, for example, the same sensorimotor striatal region is necessary for habitual lever pressing on a single lever and for the “chunking” of actions. Further work is needed to functionally map striatal regions in the mouse.
In conclusion, although mice with DLS lesions were not impaired at acquisition of a single lever press (Fig. 2), their deficits were revealed when two actions must be performed in a particular order to earn the reward. The present results raise several questions. How does neural activity in the DLS change during sequence acquisition? What are the contributions of brain regions that are strongly connected with the DLS? Although it remains for future investigations to answer these questions, the use of simple and quantitative behavioral analysis in a genetically tractable organism, as shown here, should facilitate the analysis of the neural mechanisms of serial order in behavior.
Footnotes
-
This work was supported by National Institute on Alcohol Abuse and Alcoholism Grants 018018 and 016991 to H.H.Y. I thank Oksana Shelest, Yedema Hayrapetyan, Alberto Lopez, David Fan, and Jay Gupta for their help with the experiments.
- Correspondence should be addressed to Henry H. Yin, Department of Psychology and Neuroscience, Department of Neurobiology, Center for Cognitive Neuroscience, Duke University, Durham, NC 27708. hy43{at}duke.edu
