Abstract
Theories of corticocerebellar function propose roles for the cerebellum in automating motor control, a process thought to depend on plasticity in cerebellar circuits that exchange information with the motor cortex. Little is known, however, about automating behaviors beyond the motor domain. The present study tested the hypothesis that cerebellar plasticity also subserves the development of automaticity in behavior based on low-order rules. Human subjects were required to learn two sets of first-order rules in which visual stimuli of different shapes each arbitrarily instructed a particular finger movement. We used event-related functional magnetic resonance imaging to scan subjects while these response rules became increasingly automatic with practice, as assessed with a dual-task procedure. We found that the amplitude of the blood oxygenation level-dependent signal gradually decreased as a function of practice, as responses became increasingly automatic, and that this effect was greater for a set of rules that became automatic rapidly compared with a second set, which became automatic more slowly. These trial-by-trial activity changes occurred in Crus I of cerebellar cortical lobule HVIIA, in which neurons exchange information with the prefrontal cortex rather than the motor cortex. Activity in Crus I was time locked specifically to the processing of these rules, rather than to subsequent actions. The results support the hypothesis that decreases in cerebellar cortical activity underlie the automation of behavior, whether related to motor control and motor cortex or to response rules and prefrontal cortex.
Introduction
A number of theories posit that plastic changes in cerebellar circuits support behavioral adaptation seen in motor learning tasks (Brindley, 1964; Marr, 1969; Albus, 1971; Gilbert, 1974; Ito, 2000). These have been supported by empirical demonstrations of motor learning-related cerebellar plasticity using electrophysiology in nonhuman primates (Gilbert and Thach, 1977; Ojakangas and Ebner, 1992; Greger and Norris, 2005; Soetedjo and Fuchs, 2006; Medina and Lisberger, 2009) and numerous functional neuroimaging studies in humans (Friston et al., 1992; Imamizu et al., 2000, 2003; Ramnani et al., 2000; Ramnani and Passingham, 2001; Ungerleider et al., 2002; van Mier et al., 2004; Miall and Jenkinson, 2005; Penhune and Doyon, 2005; Puttemans et al., 2005). Motor learning is likely to depend on information flow between cortical motor areas and cerebellar cortex via the pontine nuclei (Allen and Tsukahara, 1974; Wolpert and Kawato, 1998), and changes in the strength of synapses onto Purkinje cells (PCs; the principal computational units and output neurons of the cerebellar cortex).
Axonal fiber-tracing studies have suggested the cerebellum exchanges information with the cerebral cortex within independent sets of closed corticocerebellar loops (Middleton and Strick, 2000; Kelly and Strick, 2003). These consist of neocortical projections to the cerebellar cortex via the pontine nuclei, and return projections back to the neocortex via the cerebellar nuclei and thalamus. Such loops involve functionally diverse cortical regions, which interact with different cerebellar lobules. Kelly and Strick (2003) showed that the primary motor cortex, for example, interacts specifically with cerebellar lobules V, VI, VIIb, and VIIIa. These parts of the cerebellum are implicated in motor learning by specific electrophysiological signatures of plasticity, as revealed by trial-by-trial changes in the firing properties of PCs (Gilbert and Thach, 1977; Medina and Lisberger, 2008).
Although prominent, motor areas of the cerebral cortex are not the only ones that send information to the cerebellar cortex. Other areas, such as prefrontal area 46, do so as well, although they target different cerebellar cortical areas (specifically, Crus I and Crus II) (Kelly and Strick, 2003). Although it is well known that the prefrontal cortex (PFC) plays important roles in the processing of rule-related representations (Miller and Cohen, 2001; Wallis et al., 2001), the role of prefrontal-related regions of the cerebellar cortex is unclear.
Ideas about cerebellar function in motor learning can provide testable hypotheses for rule learning by analogy. During motor learning, the cerebellum may acquire representations of movement that can mimic operations initially established in motor cortex and be used to execute these actions automatically (Marr, 1969; Thach, 1998). An important characteristic of automaticity is the ability to perform a primary task “with little or no interference by a demanding secondary task” (Poldrack et al., 2005) because of decreased dependency on attention. Cerebellar patients perform poorly in motor tasks if they are performed with distractors (Lang and Bastian, 2002). Here, we tested the hypothesis that Crus I and Crus II operate similarly to acquire representations of prefrontal information processing that contribute to the automation of first-order rules (Ramnani, 2006).
Materials and Methods
During functional magnetic resonance imaging (fMRI) scanning, subjects practiced the execution of two sets of rules that they had learned in a training phase beforehand. Performance of each set became increasingly automatic (less prone to the distracting effects of a concurrently performed task relative to the other). Furthermore, since prescan training used a differential reinforcement schedule, subsequent automatization during scanning progressed at different rates in the two sets of rules. Changes in automaticity were determined by investigating performance improvements under dual-task conditions. In line with neurophysiological investigations of motor learning-related cerebellar plasticity, we hypothesized gradual trial-by-trial changes in the amplitude of the blood oxygenation level-dependent (BOLD) signal from Crus I and/or Crus II, reflecting underlying changes in neuronal excitability. This activity would be time locked specifically to events in which subjects processed rules. Since our manipulation involved the development of automaticity at different rates, we looked for interactions in the time courses of event-related BOLD activity, where the rate of change differed between the two sets of rules. In line with our hypothesis, in cerebellar Crus I the amplitude of BOLD activity time locked to symbolic instruction cues decreased more quickly for cues that progressed to automaticity more rapidly.
Subjects
Nineteen right-handed participants were used for this study (age range, 18–30 years; 12 females, 7 males). Participants gave written informed consent, and the study was approved by the Royal Holloway University of London Psychology Department Ethics Committee. In line with the requirements of the Combined Universities Brain Imaging Centre (CUBIC; http://www.pc.rhul.ac.uk/sites/cubic/), we obtained additional consent for participation in MRI data collection from these subjects. The study also conformed to regulations set out in the CUBIC MRI Rules of Operations.
Apparatus
Subjects lay supine in a 3 T Siemens Trio MRI scanner and were wearing MRI-compatible headphones. The fingers of their right hand were positioned on a four-button, MRI-compatible response box. Stimuli were back-projected onto a screen behind the subject and viewed in a mirror positioned above the subjects' eyes. A dedicated stimulus personal computer with Presentation software (Neurobehavioral Systems) was used for stimulus presentation and response collection. The Presentation personal computer received transistor–transistor logic (TTL) pulse inputs from the response keypad (via a custom-built parallel port interface). The personal computer also received TTL pulses from the MRI scanner at the onset of each volume acquisition, allowing events in the experiment to be precisely synchronized with the onset of each scan. A behavioral dual-task paradigm was applied at three stages of the experiment (see Dual task trial structure), in which a personal computer was used to deliver auditory cues to the subject. The scanner was not running during these trials. The timings of all events in the experiment were sampled continuously and simultaneously (independent of Presentation) at a frequency of 1 kHz using an A/D 1401 unit (Cambridge Electronic Design). This allowed us to calculate all event timings with an accuracy of ∼1 ms. Spike2 software running on a separate personal computer was used to create a temporal record of these events. Reaction times (RTs) were calculated off-line, and event timings were prepared for subsequent general linear model (GLM) analysis of fMRI data (see First-level single-subject analysis, below).
Experimental design
The aim of the experimental design was to compare the adaptation of BOLD activity (the hemodynamic consequences of excitability changes) time locked to two groups of symbolic instruction cues. The only difference between these was the extent to which associations between symbolic cues and actions were learned in a previous training session (Fig. 1). In brief, we sandwich the fMRI session, during which we hoped to see differential automation and adaptation, between two dual-task sessions. The before and after dual-task sessions allowed us to verify differential learning in terms of reaction times and error rates.
a, Experimental design: schematic figure of the experimental sessions. S 1, Dual-task training; S 2, prescanning behavioral training; S 3, dual task before scanning; S 4, training to automaticity during scanning; S 5, dual task after scanning and greater automaticity; orange, unrepeated training stimuli; red, low-learning condition; blue, high-learning condition (see Materials and Methods for specific details about each session). Note that the RT bar plots are schematic only; see Figure 2 for the data. b, Conditional learning trial structure. Each trial was divided into two 6 s periods (instructions occurred in the first period, and the trigger, response, and outcome (modeled as a single event) occurred in the second). Onsets of instructions and trigger-related events occurred pseudorandomly between 0.5 and 6 s within these periods, allowing us to statistically partition activity time locked to instructions from activity time locked to subsequent events (see Materials and Methods).
After extensive instrumental training, behavior that is initially flexible and goal oriented becomes increasingly habitual and automatic (Dickinson, 1985). The hallmarks of behaviors that reach this state include their relative insensitivity to the value of reinforcers, and their relative immunity to the distracting effects of secondary tasks afforded by a decreased dependency on attentional mechanisms (Ashby et al., 2010). Adams and Dickinson (1981a,b) have demonstrated that the rate of learning in the initial phases of instrumental learning enhances later resistance to reinforcer devaluation. Our initial pilot studies confirmed that the reinforcement schedule in earlier phases was related to levels of automaticity in later learning. Our experimental design was therefore based on this manipulation. Differential levels of learning were systematically manipulated using differential reinforcement schedules. Dual-task sessions were incorporated into the experimental design to establish the effects of differential schedules of reinforcement on automatic performance of the primary task. The experiment consisted of five sessions, which are described below in detail (Fig. 1, in which the order is described). All behavioral data were collected in the scanner to maintain comparable psychophysical conditions.
An important part of our analytical approach was to ensure that our behavioral data were drawn from the same trials that were used in the analysis of the fMRI data, the aim of which was to compare trial-to-trial learning-related changes in excitability in two conditions. During learning, trials in which errors are present are followed by trials in which there is an increased probability of a correct response, and theories of cerebellar learning have suggested that in these later trials there should be excitability changes compared with earlier trials. We aimed to investigate trial-to-trial changes in excitability that relate to learning, holding other confounding factors constant. It was not possible, for example, to validly compare trials with and without errors, since activity differences between trials would otherwise be explained by the presence of error-related processing rather than long-term plastic changes. However, it is appropriate to compare across early and late trials that are both free of errors. Our event-related design allowed us to model correctly executed trials separately from trials in which errors were present, and our analysis of reaction times similarly excluded these trials. Hence, fMRI results were comparable to reaction time results because exactly the same trials contributed to both. Furthermore, neither the BOLD signal nor the behavioral RT data were contaminated by the presence of errors.
Task
Conditional task trial structure.
Subjects were required to learn the arbitrary relationships between visual instruction cues (green shapes) and finger responses by trial and error. This task is commonly known as a conditional learning task. For each of two sets of stimuli, five different shapes mapped onto the four finger responses (for all stimulus-response mappings, see supplemental Fig. 1, available at www.jneurosci.org as supplemental material). We chose to have five shapes per condition to avoid subjects using higher-order learning strategies. For example, if there were only four shapes per condition, then participants may have calculated stimulus-response contingencies through a process of elimination (i.e., shape 1 corresponds to button 1, thus shape 2 must correspond to a different button). The use of five shapes per condition ensured that participants could not rely on more complex strategies and were forced to learn through trial and error. Participants were told there were more shapes than finger responses before entering the scanner. Each trial began with the presentation of an instruction cue (500 ms). After a variable delay period (see Experimental timing), subjects saw a “Go!” signal immediately followed by the four adjacent underscores (trigger cue), prompting subjects to execute a response. After the trigger cue, the subject would receive either relevant feedback (a green or red dot indicating a correct or incorrect response respectively) or ambiguous feedback (a yellow dot). This ambiguous feedback contained no information that could inform the participant about whether or not they correctly selected their response on that trial. If participants failed to execute a response within a 1000 ms time window, the word “Missed” was displayed instead of these feedback cues (this was rare, occurring on average in 1.73% of trials; SD, 1.69%). The identities and timings of motor responses were logged by Presentation and Spike2, respectively.
Dual-task trial structure.
Dual-task paradigms have been consistently used as rigorous tests of automaticity (Passingham, 1996; Oliveira et al., 1998; Poldrack et al., 2005; Grol et al., 2006). We assessed the level of automaticity attained in both conditions to determine the robustness of performance to distractors. Subjects performed the task described above (the variable delay between instruction cues and trigger cues was removed). While performing this task, subjects were required to simultaneously perform a verb-generation task. Subjects heard a word through the MRI compatible headphones every 3 s (e.g., “beer”). Immediately after this, the subject responded by saying an appropriate verb (e.g., “drink”). The onset of the conditional task was randomly jittered across 3 s. Previous versions of this dual task have simultaneously presented the visual instruction with the auditory stimulus. However, this leads to a “bottlenecking” effect such that interference occurs with only one part of the primary task. In order to properly assess the automaticity of all components of the primary task, it is necessary to jitter the secondary task with respect to it.
Our primary task involved the use of a manual response. Our secondary task therefore needed to be one that avoided the use of the hand because subjects sometimes needed to execute responses in primary and secondary tasks simultaneously. This would not have been physically possible, so the secondary task therefore required output to be delivered by a different effector. We were also concerned that the if visual instructions were presented simultaneously for both the primary and secondary tasks, the visual display for the primary task would not be the same as that used in the training sessions (see Session 2: initial behavioral training, and Session 4: training during scanning). Furthermore, it could be argued that task failure on the primary task might be related not to the demands of processing two decisions, but instead to the failure to see the instructions of the primary task because they would have to be located in a different part of the visual field to those of the secondary task. We therefore opted to use a task that required instructions to be delivered in a different (auditory) sensory modality (as opposed to visual input, as used in the primary task), verbal output (as opposed to manual output, as used in the primary task), and a rule that was independent of the primary task (generation of a verb following the presentation of a noun).
This dual-task paradigm was performed before and after scanning to determine whether changes in automaticity occurred during scanning. It should also be noted that to avoid the effects of novelty of the dual-task situation on performance, subjects were familiarized with the dual task using a set of novel visual cues before the experiment.
Experimental sessions
All experimental sessions took place inside the MRI scanner so that the learning and training environments were exactly the same.
Session 1: dual-task training.
Subjects were familiarized with the requirements of performing conditional trials concurrently with verb generation (8 min; 10 instruction cues, 6 repetitions per instruction cue). On a few occasions at the early stages of this session, subjects sometimes failed to generate a manual response at all in the primary task, or generated associated nouns rather than verbs (e.g., “car”–“bus” instead of “car”–”drive”). The purpose of the session was to eliminate these procedural errors through familiarization, rather than to increase automaticity or to reduce errors in response selection in the primary task. By the end of the session, these procedural errors were eliminated (all subjects were able to execute at least 15 trials consecutively without such errors). It was important that this session should contribute only to the learning of the context of concurrent task performance, and not to the learning of the actual associations between instructions and actions. The instruction cues that were used in session 1 were therefore not carried forward into subsequent sessions (a new set of associations was used from session 2 onward). Any learning of associations in session 1 could therefore not contribute to subsequent sessions.
Session 2: initial behavioral training.
After the dual-task training, subjects started to learn the associations between instructions and responses (10 instruction cues × 8 repetitions—pseudorandomly intermixed). Subjects were unaware that the 10 different instruction cues fell into two categories; high learning (HL) and low learning (LL). These were pseudorandomly intermixed. In the HL condition, subjects always received relevant feedback about their responses (green or red dots). In the LL condition, subjects only received relevant feedback for 50% of the trials (50% yellow dots—ambiguous feedback pseudorandomly intermixed with relevant feedback). This design enabled us to apply different reinforcement schedules to each condition, thereby slowing learning in the LL condition systematically while providing subjects with identical exposure to visual cues in these otherwise identical conditions. Subjects were unaware that there were differential reinforcement schedules. This is the only stage in the experiment where trial parameters differed between conditions. Subjects were unaware that the 10 instructions fell into two categories, HL and LL.
Session 3: dual task before scanning.
Subjects performed conditional trials with the same instruction cues with the same associations as those used in session 2 (5 shapes × 2 conditions × 6 repetitions), concurrently with verb generation. Trials in both HL and LL conditions were given 50% relevant feedback so as to minimize the levels of learning within this session while maintaining comparability with other sessions. Instruction cues were randomly ordered as in session 2.
Session 4: training during scanning.
During this session, trial parameters and reinforcement schedules were exactly matched across the two conditions (2 conditions × 5 instruction cues × 7 repetitions; 86% relevant feedback). As with session 2, HL and LL trials were pseudorandomly intermixed. Differential performance between conditions could therefore be attributed to the training received in session 2.
Session 5: dual task after scanning.
This was an exact replication of session 3.
Experimental timing
An important feature of our study was the ability to time lock activity specifically to instruction cues. A variable delay was introduced between the instruction cue and the “Go!” signal. This allowed us to isolate BOLD activity time locked to the instruction cue without the contaminating effects of subsequent trial events (“Go!” signal, trigger cue, motor response, and error feedback), as in previous studies (Ramnani and Miall, 2003, 2004; Balsters and Ramnani, 2008).
Events in each trial took place across four repetition times (TRs; 0–12 s; TR = 3 s) (Fig. 1b). To optimally sample evoked hemodynamic responses (EHRs), we randomly varied the interval between scan onset and instruction cue onset from trial to trial over the range of the first two TRs (jittering). This achieved an effective temporal sampling resolution much finer than one TR. These intervals were uniformly and randomly distributed, ensuring that EHRs time locked to the instruction cue were sampled evenly across the time period following instruction cues. The “Go!” signal (along with motor responses and feedback) occurred in the period occupied by the third and fourth TR, and the “Go!” cue's onset was varied in the same manner (in the range 6–10.5 s after the onset of the first TR). The range of the variable delay between the onset of instruction cues and the onset of the “Go!” signal varied from 1549 to 9080 ms.
Functional imaging and analysis
Data acquisition.
Four hundred ten echo-planar imaging (EPI) scans were acquired from each participant using a 3 T Siemens Trio scanner (Royal Holloway University of London, UK). The field of view covered the whole brain: 192 × 192 mm (64 × 64 voxels); 36 axial slices were acquired (25% slice gap, 0.8 mm) with a voxel size of 3 × 3 × 3 mm [TR = 3 s; echo time (TE) = 32; flip angle = 90°]. The functional sequence lasted 20 min. Immediately after the functional sequence, phase and magnitude maps were collected using a standard gradient recalled echo (GRE) field map sequence (default echo times were used: TE1 = 5.19 ms; TE2 = 7.65 ms). High-resolution T1-weighted structural images were also acquired at a resolution of 1 × 1 × 1 mm using an MPRAGE sequence.
Image preprocessing.
Scans were preprocessed using SPM5 (www.fil.ion.ucl.ac.uk/spm) on a Dual Core AMD Athlon 64 MHz personal computer with 2 gigabytes of RAM, running Windows XP and Matlab 2007a (MathWorks Inc). Before analysis, all images were realigned and unwarped using field maps to correct for motion artifacts, susceptibility artifacts, and motion-by-susceptibility interactions (Andersson et al., 2001; Hutton et al., 2002). Images were subsequently normalized to the ICBM EPI template using both linear affine transformations and nonlinear transformations (Friston et al., 1995a). Last, a Gaussian kernel of 8 mm was applied to spatially smooth the images to conform to the Gaussian assumptions of a GLM as implemented in SPM5 (Friston et al., 1995b,c).
Statistical analysis
First-level single-subject analysis.
Four event types were modeled at the first level. A GLM was constructed from regressors formed by the convolution of event delta functions with the canonical hemodynamic response function (HRF). The learning-related conditions (conditions 1 and 2) were also parametrically weighted, and these weighted regressors were included as separate regressors (see Time × condition interaction). Trials in which responses were incorrect, too early (before the trigger cue), or too late (RT > 1000 ms) were modeled separately as a fourth event type and were differentiated from events of experimental interest. This fourth event type included both the onsets of the instruction cue and “Go!” signal in error trials. Thus, activity time locked to incorrect trials were excluded from regressors explaining instruction-related activity: (1) HL instruction cues (correct only) [parametrically weighted HL instruction cues (HLT)]; (2) LL instruction cues (correct only) [parametrically weighted LL instruction cues (LLT)]; (3) trigger cues; and (4) error (both instruction cue and trigger cue onsets).
The residual effects of head motion were modeled in the GLM by including the six head motion parameters acquired from the realignment stage of the preprocessing as covariates of no interest.
Time × condition interaction.
Our modeling strategy accounted for the individual rates of learning for each instruction cue in HL and LL.
Instruction cues for correctly executed trials in the two learning conditions (HL and LL) were parametrically weighted to produce two further regressors (HLT and LLT). These regressors modeled the expected linear changes in hemodynamic response amplitudes to instruction cues, where changes were linear increases from one correctly executed trial to the next for each of the five given instruction cues (any intermediate trials containing errors were excluded and were modeled separately). The resulting model approximated the expected time course of excitability changes in our learning conditions.
Before the study, a set of planned experimental timings was carefully checked so that the timings resulted in an estimable GLM in which the statistical independence of the event types was preserved.
Second-level random-effects group analysis
In this article, we focus on the differential adaptation between the high and low learned tasks as tested for statistically at the between-subject level. This was done using one-sample t test to produce random-effects statistical parametric maps (SPMs) in the usual way. These t tests were applied to contrast images (on a per subject basis) comparing the effects due to HLT vs LLT.
Given our anatomically specific hypothesis, a small volume correction was used (bilateral Crus I and Crus II mask) to correct for multiple comparisons. This mask was generated using the atlas of Diedrichsen et al. (2009). SPMs were thresholded at p < 0.001 for display purposes, and all significant results reported survived a correction for multiple comparisons over the appropriate search volume.
Localization
Anatomical details of significant signal changes were obtained by superimposing the SPMs on the T1 canonical single-subject image from the Montreal Neurological Institute series. Results were checked against normalized T1 images of each subject. The atlas of Duvernoy and Bourgouin (1999) was used as a general neuroanatomical reference. The atlases of Schmahman et al. (2000) and Diedrichsen et al. (2009) were used as a specific neuroanatomical reference for cerebellar activations. We used the nomenclature of Larsell and Jansen (1972) to label cerebellar lobules. The SPM anatomy toolbox (Eickhoff et al., 2005) was used to establish cytoarchitectonic probabilities where applicable.
Results
Behavioral results: dual task
To assess the levels of automaticity attained by subjects, it was important to determine the robustness of the task to distractors. For this reason, we used a dual-task paradigm before and after the main scanning session. A two-way repeated-measures ANOVA was used to investigate differences in both RTs and error rates before and after scanning (sessions 3 and 5) for the two conditions (Fig. 2). When RT was the dependent variable, only trials with correct responses were used. There was a main effect of the sessions: RTs were significantly faster in session 5 than in session 3 (F(11,198) = 10.97; p < 0.001). There was also a significant main effect of condition (F(1,18) = 18.75; p < 0.001): RTs in HL were significantly faster than in LL. This main effect indicates that differential reinforcement schedules applied in the prior training session (session 2) were effective in manipulating levels of automaticity attained during the training that occurred in session 4. There was also a significant condition-by-session interaction effect (F(11,198) = 2.38; p < 0.01): the RT difference between conditions was greater after the session 4 scan compared with the difference before the scan.
Reaction time behavior: reaction times in each session. HL, Black; LL, gray. Subjects' reaction times were the same in HL and LL conditions in both initial training (S2) before scanning and during scanning itself (S4). However, HL trials became more automatic than LL trials (the RT differences between HL and LL were larger in S5 than in S3). It is important to note that the errors themselves did not contribute to the trial-by-condition interaction reported in the cerebellum since activity was time locked to instruction cues, not motor responses or errors, and only correctly executed trials were included in the regressors related to this contrast. The decline in error rate is therefore unlikely to explain our observation that cerebellar activity decreased.
Similar results were obtained when error rates were the dependent variable. Our subject's error rates showed significant main effects of session (F(1,18) = 13.182; p < 0.005) and condition (F(11,198) = 8.214; p < 0.05). There was also a significant interaction (F(11,198) = 3.955; p < 0.05). Error rates were lower in the HL condition than in the LL condition, lower in session 5 than in session 3, and the difference between conditions was greater in session 5 than in session 3. These results, particularly the significant interaction terms, demonstrate that there were significant improvements in the level of automaticity during the fMRI session (session 4) that intervened between the two dual-task sessions (sessions 3 and 5). Taking the RT and error data together, we conclude that automaticity was greater in HL trials than in LL trials, and that this effect was greater in session 5 than in session 3.
Behavioral results: learning during scanning (session 4)
RTs and error rates were analyzed using two-way repeated-measures ANOVAs. Only correctly executed trials were included in this analysis. RTs during the training and scanning session (session 4) showed no significant effects of trial (F(1,17) = 0; p = 0.99), condition (F(6,102) = 0.616; p = 0.72), or trial-by-condition interactions (F(6,102) = 1.44; p = 0.21. On these error-free trials, training in the previous initial training session (session 2) was therefore effective in ensuring that overt performance reached a plateau before session 4. The time-by-condition interactions observed in our imaging results are better explained by changes in automaticity (see Behavioral results: dual task, above) than by overt improvement in RT performance.
Subjects showed a significant difference in error rates between conditions (F(1,18) = 19.13; p < 0.001): the LL condition was associated with more response errors. There was also a significant main effect of trial (F(5,90) = 9.77; p < 0.001). We did not find a significant trial-by-condition interaction (F(5,90) = 2.1; p = 0.073). However, the processing of errors cannot explain our neuroimaging results because trials in which there were errors were excluded from the fMRI analysis.
Functional imaging
Our differential reinforcement schedule resulted in differential levels of automaticity between HL and LL conditions. Our hypothesis predicted that differences between HL and LL conditions that were modulated by practice (a time × condition interaction) would be found in cerebellar cortical areas connected with the PFC.
As hypothesized, excitability changes time locked to the increasingly automatic processing of instruction cues were found in prefrontal-projecting cerebellar territories (Crus I). The significant effect lay between the superior posterior fissure and the horizontal fissure of the right hemisphere (Fig. 3). Figure 3 also shows the temporal dynamics of the BOLD signal that exhibited the time-by-condition interaction in session 4. Activity related to the increasingly automatic processing of HL instruction cues decreased during scanning, while BOLD activity remained stable in the LL condition. These effects are not explained by the erroneous processing of instruction cues, or by the error feedback itself, since the reductions in BOLD activity were time locked to cues in which trials were correctly performed (other trials were excluded), and not to any other part of the trial, such as those in which error feedback was delivered.
a, b, Cerebellar activations during training to automaticity: section from Schmahmann et al. (2000) cerebellum atlas (a) comparable to that in b; excitability changes in event-related BOLD activity in Crus I associated with the condition in which there are greater changes in automaticity (b); data points represent the means and SEs of estimated peak hemodynamic response amplitudes for each trial in HLT and LLT (c). These are derived from the subject-specific fitted responses in SPM. Lines of best fit were calculated using linear least squares. HL, Blue; LL, red.
Speculatively, the stable responses in the LL condition are consistent with the relatively late engagement of cerebellar plasticity. LL trials may represent a state through which the HL trials have already passed. It is possible that cerebellar plasticity is only engaged after a certain level of learning has been achieved in other systems. This possibility may be tested empirically in future work by continuing to train LL trials until they achieve the same levels of automaticity as those of the HL trials in this experiment. One could then test whether the same areas of the cerebellar cortex exhibit learning-related changes in excitability.
Discussion
First-order rules can be those in which a given response is arbitrarily paired with a specific stimulus property. In contrast, second-order rules are more abstract because they specify which rule set to apply. Some suggest that habit formation involves the automation of first-order rules through plastic mechanisms in the basal ganglia (Yin and Knowlton, 2006; Ashby et al., 2010). However, this view is not universally accepted (Wise, 1996; Balleine et al., 2009; Desmurget and Turner, 2010). Networks important for automation of first-order rules likely include cerebellar circuitry (Bracke-Tolkmitt et al., 1989; Canavan et al., 1994; Balsters and Ramnani, 2008). Here, we provide evidence that cerebellar plasticity may underlie such learning (Ramnani, 2006).
We tested the hypothesis that areas of the cerebellar cortex interconnected with the PFC exhibited signatures of plasticity as the execution of first-order rules became increasingly automatic (Ramnani, 2006). Under dual-task conditions, performance on the application of rules to symbolic cues improved through repeated practice during scanning. Performance under dual-task conditions was better following training compared with performance before training, and furthermore, the improvement for cues that were reinforced more before scanning (HL) was greater than that for cues that were reinforced less (LL) (Fig. 2). This suggests that the processing of these cues became automatic as a result of practice, and the rate at which it became automatic depended on the level of reinforcement before practice.
We identified activity time locked to symbolic instruction cues that changed from trial to trial, and the rate depended on whether performance became automatic at slow (LL cues) or fast (HL cues) rates. We have not just used time-by-condition interactions to study cerebellar adaptation (Friston et al., 1992) but have demonstrated differential adaptation by manipulating the automation of information processing. This constitutes a three-way interaction among cue, time, and the degree of automation. This difference in cerebellar adaptation can only be explained by a difference in the automation of rule-based learning and is orthogonal to cerebellar adaptation to execution per se. In line with our hypothesis, activity in Crus I was time locked to the processing of HL cues, which declined rapidly compared with LL cues (Fig. 3). The statistical effects found in Crus I were greater than for any other region (see supplemental Table 3, available at www.jneurosci.org as supplemental material). Whereas previous studies have demonstrated such plasticity in the human cerebellum for motor learning (Imamizu et al., 2000, 2003; Ramnani et al., 2000; Ramnani and Passingham, 2001; Ungerleider et al., 2002; van Mier et al., 2004; Miall and Jenkinson, 2005; Penhune and Doyon, 2005; Puttemans et al., 2005), for the first time our study demonstrates plastic changes in cerebellar cortical parts of the “prefrontal loop” that occur as a direct consequence of manipulating the automaticity of rule-based information processing. Such activity reflects the acquisition of internal models of prefrontal information processing that contribute to the automatic execution of cognitive operations (Balsters and Ramnani, 2008).
Cerebellar cortical physiology and synaptic plasticity
Was the directionality of the changes observed in our study physiologically plausible? Among the many models that attempt to explain cerebellar plasticity at the cellular level (Hansel et al., 2001; Kim and Linden, 2007), long-term depression (LTD) remains one of the most influential (Ogasawara et al., 2008) partly because it is consistent with classic models that explain cerebellar learning in terms of changes in the excitability of PC-parallel fiber (PF) inputs. Memory formation is explained in terms of concurrent PF and climbing fiber inputs that depress the sensitivity of PC-PF inputs, leading eventually to a reduction in simple spike frequency in in vitro cerebellar preparations. While the progressive depression in simple spike activity observed in LTD is consistent with decreasing activity in our study, such observations are more compelling if made in vivo (Ito and Kano, 1982; Ito et al., 1982; Ito, 1984; Hartell, 2002).
Gilbert and Thach (1977), recording from PCs in cerebellar lobules III, IV, and V of rhesus monkeys, found a high frequency of complex and simple spikes at the start of learning, which declined to background levels as learning progressed. Classical eyeblink conditioning, a simple form of cerebellar-dependent motor learning (De Zeeuw and Yeo, 2005; Lepora et al., 2009), is also accompanied by a learning-dependent decline in the frequency of simple spikes following the onset of predictive conditioned stimuli that generate conditioned responses (Jirenhed et al., 2007). Similarly, Medina and Lisberger (2008) investigated excitability changes in PCs in the monkey cerebellum during smooth pursuit eye movement adaptation, and also reported a progressive depression in simple spike activity and decreases in complex spike probability in PCs. Furthermore, the complex spikes on a given trial corresponded to a depression in simple spikes on the following trial. These studies provide support for the hypothesis that changes in PC-PF transmission encodes aspects of memory traces associated with cerebellar motor learning.
The directionality of change observed in our study is the same as that observed in these experiments. Could declining BOLD signal in our study have been caused by learning-related declines in complex and simple spikes? The cerebellar cortex is an important model for establishing in vivo relationships among neural activity, blood flow, and tissue oxygenation (the latter two being important contributors to the BOLD signal). Cerebellar blood flow increases monotonically with summed field potentials caused by climbing and PF stimulation (Mathiesen et al., 1998), suggesting that decreasing BOLD signal in our study is consistent with decreasing discharge rates in simple and/or complex spikes. However, synaptic inputs to PCs contribute to cerebellar blood flow more reliably than postsynaptic effects such as simple and complex spikes (Gold and Lauritzen, 2002; Thomsen et al., 2004). There may be a more reliable relationship between oxygen consumption (component of the BOLD effect) and postsynaptic neural activity in the cerebellar cortex. Oxygen consumption increases linearly with local field potentials caused by both climbing fiber (Offenhauser et al., 2005) and PF stimulation (Thomsen et al., 2009), and appears to be dependent upon postsynaptic processes, most likely in PCs.
It has been suggested that the uniformity of cellular organization across the cerebellar cortex implies uniformity in the nature of the information processing. The processes that took place in Crus I as the application of rules became increasingly skilled and automatic were probably comparable to those discussed above in which motor skills are acquired. We suggest that learning-related decreases in BOLD activity observed in our study may have been caused by plastic processes that involve learning-related decreases in simple and complex spikes or the synaptic events correlated with spiking activity.
Dual systems: automatic versus controlled processing
Shiffrin and Schneider (Schneider and Shiffrin, 1977; Shiffrin and Schneider, 1977) proposed a “dual-process” account of information processing that distinguished between two particular modes. In one, information processing is effortful and “controlled,” and learning enables a transition to another in which it is “automatic” and robust to distractors. Friston et al. (1992) systematically investigated the effects of motor learning in the cerebellum by introducing the use of time-by-condition interactions (see Materials and Methods). These test for changes in signal over time that are driven specifically by learning rather than other time-varying confounds unrelated to learning [e.g., physiological or scanner drift (Bandettini et al., 1993; Smith et al., 1999)]. It has been previously suggested that areas in the lateral parts of the PFC are important for earlier phases where attention is required for performance. Prefrontal activity sustained over working memory delays is suppressed by distractors that cause forgetting of rehearsed material (Sakai et al., 2002), elevated when subjects start to learn a novel sequence, and much less active during the execution of prelearned sequences executed automatically (Jueptner et al., 1997a,b). Attending to prelearned sequences even though they can be executed automatically, reactivates the PFC (Jueptner et al., 1997a). Hence, prefrontal circuits are engaged when there is a requirement to attend to actions, as in the early phases of learning. Jueptner et al. (1997b) demonstrated bilateral activity in the cerebellar cortex that was greater during the performance of new sequences than prelearned sequences. These authors have not localized activity to particular cerebellar structures, but we find their coordinates to be localized to Crus II—a cerebellar target of the PFC in monkeys (Kelly and Strick, 2003). In these studies, it is not possible to discriminate the experimental variance ascribed to motor learning and rule learning as it is in the present experimental design, but, considering the rule-related requirements of their experiment, the profile of activity is consistent with ours. The progress to automaticity in such studies may be supported by interactions between lateral parts of the PFC and interconnected areas in the cerebellar cortex.
We report that increasingly automatic execution of first-order rules is accompanied by changes in excitability in a cerebellar target of the PFC. This parallels findings that report excitability changes in cerebellar targets of the motor system during motor learning. It remains for future work to test whether such plasticity shares common cellular mechanisms to support both forms of learning. Future work from our laboratory will investigate whether higher-order rules engage the cerebellum, and whether neocortical–cerebellar interactions change during the automation of abstract rules.
Footnotes
- Received August 18, 2010.
- Revision received November 26, 2010.
- Accepted December 14, 2010.
This study was supported by grants to N.R. from the United Kingdom Biotechnology and Biological Sciences Research Council (Grant BB/H012990/1) and the Royal Society. We thank Steven P. Wise (National Institute of Mental Health) for helpful comments on an earlier version of our manuscript.
- Correspondence should be addressed to Dr. Narender Ramnani, Department of Psychology, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK. n.ramnani{at}rhul.ac.uk
- Copyright © 2011 the authors 0270-6474/11/312305-08$15.00/0
This article is freely available online through the J Neurosci Open Choice option.