Abstract
The ability to change strategies in different contexts is a form of behavioral flexibility that is crucial for adaptive behavior. The striatum has been shown to contribute to certain forms of behavioral flexibility such as reversal learning. Here we report on the contribution of striatal cholinergic interneurons—a key element in the striatal neuronal circuit—to strategy set-shifting in which an attentional shift from one stimulus dimension to another is required. We made lesions of rat cholinergic interneurons in dorsomedial or ventral striatum using a specific immunotoxin and investigated the effects on set-shifting paradigms and on reversal learning. In shifting to a set that required attention to a previously irrelevant cue, lesions of dorsomedial striatum significantly increased the number of perseverative errors. In this condition, the number of never-reinforced errors was significantly decreased in both types of lesions. When shifting to a set that required attention to a novel cue, rats with ventral striatum lesions made more perseverative errors. Neither lesion impaired learning of the initial response strategy nor a subsequent switch to a new strategy when response choice was indicated by a previously relevant cue. Reversal learning was not affected. These results suggest that in set-shifting the striatal cholinergic interneurons play a fundamental role, which is dissociable between dorsomedial and ventral striatum depending on behavioral context. We propose a common mechanism in which cholinergic interneurons inhibit neurons representing the old strategy and enhance plasticity underlying exploration of a new rule.
Introduction
Behavioral flexibility—the ability to adapt to changes in contingencies, rules, or strategies—is crucial for survival. The prefrontal cortex is considered to be a key brain region in behavioral flexibility, representing the rules governing ongoing tasks. However, to express these rules requires activation of interconnected structures involved in motor outputs (Miller, 2000). The striatum is one such structure and plays an important role in expression of behavioral flexibility. In particular, the dorsomedial striatum is associated with reversal learning (Pisa and Cyr, 1990; Castañé et al., 2010) and set-shifting (Ragozzino et al., 2002b; Nicolle and Baxter, 2003) while the ventral striatum plays a role in learning involving changes in behavioral strategies, or shifting attention to previously irrelevant stimuli (Floresco et al., 2006a). The cholinergic interneurons are a key element in the neuronal circuits of all striatal subregions. Recent studies that have targeted cholinergic interneurons (Brown et al., 2010; Bradfield et al., 2013b; Okada et al., 2014) suggest that they play an important role in the striatal mechanisms of behavioral flexibility.
Tonically active striatal neurons that are presumed to be cholinergic interneurons acquire responses to sensory stimuli after they have been repeatedly paired with reward (Kimura et al., 1984; Aosaki et al., 1994, 1995; Graybiel et al., 1994). Acquisition of these responses is dopamine dependent (Aosaki et al., 1994), but the neurons do not simply mirror the activity of dopamine neurons (Morris et al., 2004). More recent studies (for review, see Apicella, 2007) suggest that cholinergic interneurons also respond to spatial attributes such as the location of a stimulus or movement direction (Ravel et al., 2006) and the behavioral context (Lee et al., 2006). These findings suggest that cholinergic interneurons are involved in flexible switching of responses to directional cues in different contexts.
Consistent with electrophysiological findings, cholinergic interneurons and their thalamic afferents have been implicated in behavioral flexibility. Brown et al. (2010) showed that inactivation of the thalamic afferents to cholinergic interneurons did not affect place learning in a cross maze, but impaired learning when the rewarded side was switched. The impairment was failure to maintain new correct choice patterns (regressive errors). Similarly, Bradfield et al. (2013b) found that loss of afferents from the parafascicular nucleus, which reduced cholinergic cell activity, impaired goal-directed learning but only after changes in the action-outcome contingency. In contrast, Okada et al. (2014) found that specific lesions of cholinergic interneurons caused enhanced reversal of a spatial discrimination task.
We investigated the hypothesis that cholinergic interneurons are involved in flexible switching of responses to directional cues using a strategy set-shifting task (Floresco et al., 2008) involving different types of attention, and reversals. We compared intact rats with rats in which immunotoxin-induced specific lesions of cholinergic interneurons were made in dorsomedial or ventral striatum. In support of the hypothesis, we found dissociable effects of cholinergic lesions of dorsomedial or ventral striatum on set-shifting.
Materials and Methods
Ethical approval
All procedures involving animals were approved by the Committee for Care and Use of Animals at Okinawa Institute of Science and Technology.
Animals and groups
Male Long–Evans rats weighing 250–300 g on arrival were used in a series of experiments. All animals were provided with food and water ad libitum and housed under standard conditions (12 h light/dark cycle, at 23°C) until 5 d before behavioral experiments. Thereafter animals were food restricted to ∼85% of their average weight. Animals were tested in three experimental conditions detailed below (Fig. 1). Three groups of animals were compared: control, dorsomedial striatum lesion (DMS), and ventral striatum lesion (VS).
Surgical procedures
Stereotaxic surgery was conducted under isoflurane anesthesia (initial, 3.5%; maintenance, 2.5–3%). In lesion groups, we injected anti-choline acetyltransferase saporin (Advanced Targeting Systems) to induce specific lesions of cholinergic interneurons while leaving other types of cells intact (Laplante et al., 2011). The agent was pressure injected bilaterally in either DMS or VS. Control groups were injected with saline in either DMS or VS in a counterbalanced manner. Coordinates of injection sites were as follows: from bregma or dural surface: DMS: AP +1.0 mm, ML 2.0 mm, and depth 3.7 mm; VS: AP +1.6 mm, ML 1.9 mm, and depth 6.6 mm. The injection volume for lesions and controls was 0.5 μl for DMS and 0.45 μl for VS in each hemisphere. After recovery, behavioral experiments were commenced.
Behavioral procedure and analysis
Habituation and conditioning.
After recovery from surgery, animals were habituated to an operant chamber (Med Associates). After magazine training, animals completed a continuous reinforcement schedule to learn to press a lever for a food reward (a sucrose pellet, 45 mg; TestDiet), until 60 pellets had been obtained (60 lever presses) or 40 min had passed. Next, lever-press training was performed for five to eight sessions under the same trial schedule as testing procedures mentioned below. In lever-press training, the left or right lever was inserted randomly and animals were required to press the lever within 10 s of lever insertion. Finally, side bias was tested to see the animal's preference for left or right lever (Floresco et al., 2008).
Trials.
A trial was commenced with a 3 s tone. Two seconds after the tone ceased, two levers were presented and animals were allowed to choose either the left or right lever within 10 s. During lever-press training, only one lever was presented. If there was no response within 10 s, both levers were retracted and the trial was counted as an omission. In some conditions there was a light stimulus above a lever, which was turned on immediately after the termination of tone and turned off when animals pressed a lever or in 10 s after lever insertion when omissions.
Testing procedures.
A daily session was composed of 80 trials with intertrial intervals of 20–30 s. Figure 1 shows the testing procedure. Phase 1 comprised four consecutive sessions. In all conditions a response strategy was required in which animals learned to press a lever based on lever location. The correct side was opposite to their preference based on the side-bias test. Phase 2 comprised 10 sessions. In all conditions a visual cue strategy was required in which the correct lever was indicated by an illuminated light. Three different conditions of attentional shifts were compared. In condition 1 (Fig. 1A) no light was shown in phase 1, but in phase 2 a light indicated the correct lever. Animals in condition 1 thus had to attend to a novel visual cue. In condition 2 (Fig. 1B) a light stimulus was shown above the correct lever in phase 1, and again in phase 2. In this condition the visual cue had been relevant, but not necessarily used in making the choice in phase 1. In condition 3 (Fig. 1C) the light stimulus was randomly presented above either the left or right lever in phase 1; in phase 2 animals had to pay attention to the light stimulus that was previously irrelevant. After completing this set-shifting paradigm (phases 1 and 2), animals entered phase 3 in which they were retrained in the initial response strategy for 3 sessions and then tested in reversal learning as phase 4, where they had to respond to the previously incorrect side (Fig. 1).
Data analyses.
Percentage of correct responses in each daily session was measured in all conditions, excluding omission trials. During learning of a visual cue strategy (Fig. 1, phase 2), accumulated errors over 10 sessions were counted and classified as perseverative, regressive, or never-reinforced errors (Floresco et al., 2008). Perseverative errors were defined as incorrect responses made on the previously correct lever while the level of performance was still significantly below chance (Jones and Mishkin, 1972; Dias et al., 1996, 1997; Hunt and Aggleton, 1998; Dias and Aggleton, 2000). Various, but conceptually similar, criteria for the perseverative error have been used in the past (Ragozzino et al., 2002b; Floresco et al., 2006a, b, 2008). Using a principled approach, we defined the criterion for perseveration as the point at which the animal began scoring fewer than 8 of 10 incorrect responses (probability of scoring 8/10 errors or more = 0.054, cumulative binomial distribution) in a moving window of 10 trials. To determine this point, we started the 10 trial window from first trial and then advanced it by one trial at a time until fewer than eight errors out of 10 trials were counted. Subsequent errors made after the same point were counted as regressive. We performed this analysis across all trials where a light cue was on the previously incorrect side during visual cue learning. Never-reinforced errors were counted when animals pressed the previously incorrect lever above which the light was not illuminated. Those errors were subdivided into an early or late component; errors that occurred in the first half of visual cue learning (sessions 1–5) were regarded as early and those in the second half (sessions 6–10) were considered late.
For error analysis of reversal learning, we used the same criterion as in the set-shifting. Again, we counted number of errors in a moving window of 10 trials and advanced the window by one trial until fewer than eight errors out of 10 trials were scored. Errors before this point were regarded as perseverative errors. Once the score became <8/10 incorrect responses, subsequent errors were counted as regressive errors.
Histology
After the completion of behavioral experiments, the animals were perfused with 4% paraformaldehyde. Brains were extracted and postfixed in the same fixative. We prepared coronal sections (60 μm) using a vibratome (VT1000S; Leica) and divided sections into four vials. Choline acetyltransferase (ChAT) and nuclei (NeuN) immunohistochemistry were performed in one or two of four vials. For ChAT staining, briefly, sections were incubated overnight in a polyclonal anti-ChAT antibody raised in goat (AB144P; Millipore) and later incubated in rabbit anti-goat IgG-conjugated biotin secondary (Sigma). NeuN staining was performed by Fox3 primary antibody (AB104224; Abcam) and a secondary anti-mouse IgG-conjugated biotin (Invitrogen). Both ChAT and NeuN signals were enhanced by an avidin–biotin complex method (ABC Elite; Vector Laboratories) and visualized using a metal-enhanced DAB Substrate Kit (#34065; Thermo Scientific). Lesioned extent in each animal was examined by ChAT staining under a light microscope (CX22; Olympus). Images of representative sections (Fig. 2) were photographed using a digital microscope (BZ-9000; Keyence). To indicate the range of lesion extent the smallest and largest lesions in each treatment group were drawn for all the experimental conditions (Fig. 2; Paxinos and Watson, 2004).
Statistical analysis
All analyses were conducted with SPSS Statistics version 21. Percentage of correct responses in a series of behavioral tasks was analyzed using a two-way ANOVA for repeated measures with treatment as a between-subjects factor. Analysis of errors was performed using a one-way ANOVA with treatment as the between-subjects factor followed by Dunnett's test for multiple comparisons. Before ANOVAs, when a Levene's test for homogeneity of variance indicated unequal variance, a more conservative Welch's ANOVA followed by Dunnett's T3 test was applied. For the number of perseverative errors in set-shifting (Phase 2) in conditions 1 and 2 where a violation of the normality assumption of ANOVA was obvious, analysis was performed using the equivalent nonparametric Kruskal–Wallis test followed by Dunn's test with the Bonferroni correction for pairwise comparisons. A difference of p < 0.05 was considered significant.
Results
Immunotoxin injections caused specific lesions of the cholinergic interneurons (Fig. 2A), consistent with a previous study using the same immunotoxin (Laplante et al., 2011). Small nonspecific lesions were seen in some sections close to the injection site (Fig. 2B). Since the nonspecific lesions were much smaller than the extent of the associated cholinergic lesion, we included them into later analyses. The extent of the lesions was similar between conditions, as illustrated by comparisons of the smallest and largest lesion cases in each group (Fig. 2C,D). Note that DMS lesions included mostly anterior DMS and a part of posterior DMS (Fig. 2C; compared with Bradfield et al. (2013b), and that VS lesions affected both core and shell subdivisions (Fig. 2D).
Across all the three experimental conditions, initial learning of the response strategy was not affected by either DMS or VS lesion (Fig. 3A–C; in all conditions: F < 2.4, p > 0.05), suggesting that lesions of cholinergic interneurons did not affect acquisition of the initial strategy.
When switching a strategy in condition 1, where a novel stimulus was introduced as a new directional cue, a percentage of correct responses in rats with lesions of cholinergic interneurons in VS seemed lower but not significant (Fig. 3A; treatment effect: F(2,46) = 0.120, p = 0.887; time-by-treatment interaction: F(18,414) = 1.327, p = 0.234). However, analysis of each class of error indicated that the number of perseverative errors was significantly different between groups (Fig. 3A; Kruskal–Wallis test, χ2(2) = 6.292, p = 0.043). Multiple comparisons showed a significant increase of perseverative errors in VS lesions (Dunn's test, VS vs control: p = 0.037; DMS vs control: p > 0.05). In contrast, there were no differences in the numbers of regressive errors, never-reinforced errors, nor their subdivided early and late components (Fig. 3A; one-way ANOVAs, F(2,46) < 1, p > 0.05).
During strategy set-shifting in condition 2 where animals were required to shift their attention to a previously relevant cue, neither lesion had an effect on the learning performance (Fig. 3B; treatment effect: F(2,42) = 0.092, p = 0.912; time-by-treatment interaction: F(18,378) = 1.209, p = 0.291). Separate error analyses showed no effects on the number of each error type (Fig. 3B; Kruskal–Wallis test for the perseverative error, χ2(2) < 1.4, p > 0.05; one-way ANOVAs for the other types of errors, F(2,42) < 2.6, p > 0.05).
When attention to a previously irrelevant stimulus was required to switch a behavioral strategy in condition 3 (Fig. 3C), the percentage of correct responses showed slightly slow increment in DMS lesions in the middle of learning, but it was not significant (Fig. 3C; treatment effect: F(2,51) = 0.530, p = 0.592; time-by-treatment interaction: F(18,459) = 1.395, p = 0.222). Importantly, however, error types were clearly changed after cholinergic interneuronal lesions in that the number of perseverative errors was significantly different (Fig. 3C; Welch's ANOVA, F(2,28.128) = 6.315, p = 0.005). Multiple comparisons with the control indicated a significant increase of the perseverative errors in DMS lesions (Dunnett's T3, p = 0.007) but not in VS lesions (p > 0.05). In contrast, there was no effect on regressive errors (Fig. 3C; one-way ANOVA, F(2,51) < 1, p > 0.05). Analyzing the number of never-reinforced errors showed a significant difference between groups (Fig. 3C; one-way ANOVA, F(2,51) = 5.224, p = 0.009). Compared with the control, a significant decrease was found in both types of cholinergic lesions in DMS (Dunnett's, p = 0.007) and VS (Dunnett's, p = 0.045), particularly in the early phase of learning (Fig. 3C; one-way ANOVA, F(2,51) = 8.930, p < 0.001; Dunnett's, DMS vs control: p < 0.001; VS vs control: p = 0.019) but not in the late phase of learning (one-way ANOVA, F(2,51) < 2, p > 0.05).
These findings demonstrate that lesions of cholinergic interneurons in DMS affected strategy set-shifting, particularly when it required attending to the previously irrelevant stimulus (Fig. 3C, condition 3), in that an abnormal distribution of error types was evident. On the other hand, cholinergic ablation in VS had a specific influence on the set-shifting when a novel cue was introduced (Fig. 3A, condition 1), resulting in more perseverative errors during a shift.
Last, Figure 4 shows that the performance in retraining of the response strategy was not different between groups (Fig. 4; in all conditions: F < 1.2, p > 0.05). Also, neither lesion had an effect on performance in reversal learning (Fig. 4; in all conditions: F < 1.6, p > 0.05). Comparisons of the number of perseverative and regressive errors during reversal learning revealed no statistically significant differences between groups (Fig. 4; in all conditions, F < 2.5, p > 0.05). These results indicated that the lesions of the cholinergic interneurons did not affect retraining of a previously learned strategy and its reversal.
Discussion
We investigated the effect of a selective lesion of striatal cholinergic interneurons on set-shifting and reversal learning. We found that lesions of cholinergic interneurons in the DMS caused a significant impairment of learning after a shift that required attending to a previously irrelevant stimulus. This impairment was evident in an increase of the perseverative errors after the shift when animals attempted to find a new behavioral rule. There was an associated decrease in the number of never-reinforced errors, which was particularly evident at an early stage of learning, suggesting that the perseverative errors were made at the expense of exploration of alternative strategies (Floresco et al., 2008). In set-shifting requiring attention to a novel stimulus, lesions of the VS resulted in a significant increase of perseverative errors, implying that animals were unable to inhibit the use of a now invalid strategy and to facilitate exploration toward the novel cue. Neither of the lesions affected set-shift requiring attention to a previously relevant stimulus, retraining of the original response strategy, nor reversal of the original response. To our knowledge, this is the first study to demonstrate a specific role of striatal cholinergic interneurons in set-shifting, namely suppressing the use of an old strategy and promoting exploration of a new rule. Furthermore, this role is possibly dissociable between dorsomedial and ventral striatum depending on behavioral context.
We observed no effect of lesions of the cholinergic interneurons on reversal learning. However, in previous studies locally injected cholinergic antagonists have been found to impair reversal learning (Ragozzino et al., 2002a, 2003; McCool et al., 2008). In contrast, specific cholinergic interneuronal ablation has been reported to enhance learning of a spatial reversal task (Okada et al., 2014). It is possible that impaired reversal learning caused by muscarinic receptor antagonists (Ragozzino et al., 2002a; McCool et al., 2008) results from a lack of extrinsic cholinergic modulation by brainstem nuclei (Dautan et al., 2014), as opposed to the intrinsic cholinergic neurons (Okada et al., 2014). In agreement with the present findings, a previous study showed that loss of DMS cholinergic transmission did not affect reversal learning (Bradfield et al., 2013b). However, the same study showed that animals were unable to use the knowledge of reversed contingency when tested by a devaluation test. This more specific role of cholinergic interneurons in reversal learning was not assessed in the present study.
The present study focuses specifically on cholinergic function localized to the striatum, mediated by the cholinergic interneurons. The contribution of other cholinergic systems to attentional set-shifting has been considered in previous studies. Generalized blockade of muscarinic cholinergic receptors induced by systemic injections of scopolamine disrupted extradimensional shifts but not initial learning (Chen et al., 2004). However, localized immunotoxin lesions of basal forebrain cholinergic fibers projecting to medial frontal cortex did not impair set-shifting (McGaughy et al., 2008). These findings are consistent with the proposal that cholinergic receptors outside of neocortex play an important role in regulating attentional shifts (Chen et al., 2004). The present findings extend these results by showing specifically that the intrinsic cholinergic neurons of the striatum play a crucial role in extradimensional shifts (condition 3).
Previous work has shown that the prefrontal cortex plays a role in inhibiting the use of a previously learned strategy (Ragozzino et al., 1999; Floresco et al., 2006b), consistent with the strong connectivity of the prefrontal cortex with the DMS and VS. The perseverative deficit we observed due to the specific cholinergic lesions was different from the regressive deficit caused by injection of local anesthetic in the DMS or VS (Ragozzino et al., 2002b; Floresco et al., 2006a), a procedure that inactivates the principal neurons as well as the cholinergic interneurons. These findings are consistent with a model in which cholinergic interneurons are necessary to inhibit the use of a previously learned strategy and promote exploration, rather than maintain responding after switching to a new strategy. One possible physiological function of the cholinergic interneurons is inhibition of striatal projection neuron activity (Ding et al., 2010). Another is cholinergic modulation of dopamine (DA) release in the striatum. Recent studies suggest that DA release is triggered by synchronous activities of cholinergic interneurons in both dorsal (Threlfell et al., 2012) and ventral striatum (Cachope et al., 2012) and such DA release is related to exploration (Rebec et al., 1996a, b).
If, as we propose, in set-shifting cholinergic interneurons suppress a now invalid strategy and facilitate exploration of a new strategy, then how can we explain the dissociation of VS and DMS findings? One possibility is that both findings result from a similar mechanism, but one that operates on different inputs and outputs. Unlike the DMS, the VS receives dense direct projections from hippocampus (Kelley and Domesick, 1982; Naber and Witter, 1998; Groenewegen et al., 1999; Voorn et al., 2004; Floresco, 2015), which plays a role in novelty detection (Knight, 1996; Lisman and Otmakhova, 2001; Ranganath and Rainer, 2003; VanElzakker et al., 2008; Mannella et al., 2013). For example, Mannella et al. (2013) proposed that the VS plays a role in biasing action selection based on the novelty of stimuli as detected by the hippocampus. Thus the VS is suitably situated to facilitate responses (potentially exploration) toward a novel stimulus away from the old rule. Conversely, we also showed that DMS cholinergic interneurons play a role when contingency of a stimulus changes. Bradfield et al. (2013a) suggested that cholinergic transmission in posterior DMS is important for learning about response-outcome contingency, whereas the anterior DMS is involved in a stimulus-outcome contingency. Our DMS lesions were more anterior than the posterior DMS lesions made by Bradfield et al. (2013b). Thus, our results are consistent with the suggested involvement of anterior DMS in set-shifting based on change in contingency of a stimulus.
Since novelty itself has motivational value (Mannella et al., 2013), responses to novel stimuli may entail Pavlovian approach behavior controlled by the VS (Di Ciano et al., 2001; Floresco, 2015). Yet, importantly, the fact that VS cholinergic lesions had no effect on initial learning in any condition suggests that this role cannot be generalized; rather, it is specialized to an attentional/directional shift to the novel stimulus by which a switch of strategies is accomplished. In contrast, the absence of effect of DMS lesions in condition 1 could be due to the intact ventral cholinergic system. As noted above, condition 1 might involve VS-dependent response to novelty (Wittmann et al., 2008; Mannella et al., 2013), such that the intact VS overrides the effect of the DMS cholinergic loss.
Lack of significant effect of VS lesions on perseveration in condition 3 might also be due to nonselectivity of the lesions for core and shell subdivisions, which play dissociable roles (Weiner, 2003; Floresco et al., 2006a; Schiller et al., 2006; Dalton et al., 2014). Thus, the effect of cholinergic lesions could be canceled out. Note, however, that both DMS and VS lesions led to a decrease of never-reinforced errors in condition 3, suggesting that not only the dorsomedial but also the ventral striatal cholinergic system helps facilitate exploration over perseveration in this context.
Consistent with the present findings, other evidence suggests that there is a cholinergic mechanism for inhibition of a previously learned response in the striatum. Lee et al. (2006) recorded putative cholinergic interneurons in monkeys performing a go/no-go task that involved action initiation and inhibition. Cholinergic interneurons showed burst firing activity when a monkey correctly withheld his movement in no-go trials. Lee et al. proposed that cholinergic interneurons suppress planned movements by inhibiting striatal projection neurons. It has also been shown that burst firing of cholinergic interneurons caused by thalamic inputs transiently suppresses cortical drive to striatal spiny projection neurons (Ding et al., 2010), which Ding et al. proposed as a possible mechanism for shift of attention and redirection of behavior. By such mechanisms, burst firing of cholinergic interneurons may suppress a previously correct but now incorrect strategy, and enable a search for an alternative strategy. Loss of this mechanism may explain the increase of perseverative errors after the shift when animals attempted to find a new behavioral rule, and the associated impairment of exploratory behavior in the present study.
Footnotes
This study was supported by Human Frontier Science Program and the Sasakawa Scientific Research Grant from the Japan Science Society.
The authors declare no competing financial interests.
- Correspondence should be addressed to either Sho Aoki or Jeffery R. Wickens, Neurobiology Research Unit, Okinawa Institute of Science and Technology, 1919-1 Tancha, Onna, Kunigami, Okinawa 904-0495, Japan. sho.aoki{at}oist.jp or wickens{at}oist.jp