Abstract
Visuomotor adaptation has been thought to be an implicit process that results when a sensory-prediction error signal is used to update a forward model. A striking feature of human competence is the ability to receive verbal instructions and employ strategies to solve tasks; such explicit processes could be used during visuomotor adaptation. Here, we used a novel task design that allowed us to obtain continuous verbal reports of aiming direction while participants learned a visuomotor rotation. We had two main hypotheses: the contribution of explicit learning would be modulated by instruction and the contribution of implicit learning would be modulated by the form of error feedback. By directly assaying aiming direction, we could identify the time course of the explicit component and, via subtraction, isolate the implicit component of learning. There were marked differences in the time courses of explicit and implicit contributions to learning. Explicit learning, driven by target error, was achieved by initially large then smaller explorations of aiming direction biased toward the correct solution. In contrast, implicit learning, driven by a sensory-prediction error, was slow and monotonic. Continuous error feedback reduced the amplitude of explicit learning and increased the contribution of implicit learning. The presence of instruction slightly increased the rate of initial learning and only had a subtle effect on implicit learning. We conclude that visuomotor adaptation, even in the absence of instruction, results from the interplay between explicit learning driven by target error and implicit learning of a forward model driven by prediction error.
Introduction
Adaptation to a visuomotor rotation has served as a paradigmatic task for studying sensorimotor learning (Cunningham, 1989; Imamizu et al., 1995; Pine et al., 1996; Krakauer, 2009), with the learning in such tasks attributed solely to the updating of a forward model (Mazzoni and Krakauer, 2006; Tseng et al., 2007; Synofzik et al., 2008). Recent studies, however, have shown that use-dependent and reinforcement learning also operate during visuomotor adaptation tasks (Huang et al., 2011; Izawa and Shadmehr, 2011; Shmuelof et al., 2012). Furthermore, instructed explicit strategies can be used to counter an imposed rotation resulting in lower target error initially during training (Mazzoni and Krakauer, 2006; Benson et al., 2011). However, the role of explicit learning in motor tasks has mainly been addressed using sequence-learning paradigms, such as the serial reaction time task, but these tasks focus on the ordering of discrete action elements and not on the action elements themselves (Nissen and Bullemer, 1987; Curran and Keele, 1993). The potential role for instruction and explicit learning processes in motor adaptation tasks is interesting given that, since the seminal studies of motor learning in amnesic patients such as H.M., such tasks have been taken as emblematic of a kind of learning that does not require declarative memory (Scoville and Milner, 1957; Corkin, 1968).
When participants are instructed to offset a visuomotor rotation by reaching to an instructed location that is in the equal and opposite direction of the rotation, they show immediate compensation with the cursor landing in or close to the target (Mazzoni and Krakauer, 2006; Benson et al., 2011). If a visual landmark is provided to help participants employ the instructed strategy, then over the course of training performance deteriorates with the cursor drifting in the opposite direction of the rotation (Mazzoni and Krakauer, 2006). This puzzling increase in error with training is consistent with the idea that implicit learning is driven by a sensory-prediction error signal (difference between the aiming and feedback location) rather than target error (difference between the target and feedback location). If the training is extended, the learning curve shows an intriguing non-monotonic form: at ∼100 trials, performance reverses direction and eventually reaches a stable state in which the cursor again hits the target (Taylor and Ivry, 2011). We hypothesized that this non-monotonicity reflects the parallel operation of two processes, an implicit process driven by sensory-prediction error and an explicit process driven by target error (Taylor and Ivry, 2011). Indeed, the saliency of the sensory-prediction error is critical with respect to implicit learning, such that when the landmark is absent participants are still able to employ the strategy to reduce target error but at the cost of reduced implicit learning (Benson et al., 2011; Taylor and Ivry, 2011). However, evidence for this two-process account has only been indirect, with the degree of implicit learning inferred by the size of the aftereffect and the degree of explicit learning inferred through catch trials (Benson et al., 2011) or modeling (Taylor and Ivry, 2011). Moreover, previous studies were unable to measure how explicit learning evolves directly throughout training.
In the current study, we sought to directly assess explicit learning through a novel task design that allowed us to obtain continuous verbal reports of aiming direction. Participants were provided with a continuous array of visual landmarks surrounding the target and were required to report their aiming direction before each movement. They were not informed about the rotation nor given a strategy to counter it. For this reason, we are wary of calling this verbally reported aiming direction a strategy per se in recognition of the fact that the participants may not have discovered that the perturbation is a rotation. Nonetheless, the aiming reports provide a direct assay of how participants are explicitly attempting to reduce target error. Moreover, assuming that explicit and implicit processes are additive, this method provides a novel way to measure the time course of implicit learning.
We also investigated whether the degree of explicit learning depends on the kind of feedback provided. Recently, it has been shown that reduced error feedback leads to what appears to be nonforward model learning (Izawa and Shadmehr, 2011). We predicted that providing the motor system with on-line error feedback will favor forward model-based learning whereas endpoint feedback will weight learning toward explicit learning.
Materials and Methods
Participants and experimental apparatus.
Sixty young adults (35 females/25 males, aged 18–30) were recruited in exchange for class credit from the research participation pools of the Department of Psychology at Princeton University and the Department of Psychology at the University of California, Berkeley. All participants were right-handed, verified with the Edinburgh handedness inventory (Oldfield, 1971). The experimental protocol was approved by the Institutional Review Boards at Princeton University and the University of California, Berkeley.
Participants made center-out, horizontal reaching movements to visually displayed targets arranged in a circle, sliding their right hand across a digitizing tablet while holding onto a digitizing pen (Intuous 3; Wacom). Movement trajectories were sampled at 100 Hz. The stimuli and feedback cursor were displayed on a 15 inch, 1280 × 1024 pixel resolution LCD computer monitor (Dell), horizontally mounted 25.4 cm above the tablet. Since the monitor occluded vision of the hand, visual feedback was provided in the form of a small circular cursor (3.5 mm).
Procedure.
Participants were assigned to one of four experimental groups according to a 2-by-2 factorial design, with 15 participants per group. The design involved two levels of verbal instruction (instruction and no instruction) and two levels of feedback (endpoint and on-line feedback).
Each trial started with the participant moving his or her hand such that the cursor was positioned within a 5 mm starting circle located at the center of the screen (Fig. 1). After maintaining this position for 1 s, a green target circle (7 mm diameter) was presented. The target could appear at one of eight locations that were separated by 45° along an invisible ring with a radius of 7 cm (0, 45, 90, 135, 180, −135, −90, and −45°). The sequence of target locations were pseudorandomly presented, such that each target location was experienced before a particular target location was repeated, and each participant received a different randomized sequence of target locations. The participants were instructed to make a fast reaching movement to the target, “slicing” through the target.
Experimental task. Top, Participants learned to overcome a 45° counterclockwise rotation while reaching to eight different target locations, separated by 45°. In the Instruction conditions, the workspace included numbered landmarks that flanked the target location. Before each movement, the participants verbally report where they planned to aim to make the cursor land on the target. In the No Instruction conditions, only the target location was presented. Vision of the hand in all conditions was occluded by a monitor that was mounted horizontally above the arm. For participants in the Endpoint Feedback conditions, the cursor disappeared at movement onset and reappeared as soon as the hand crossed a virtual ring, 7 cm from the start position. For participants in the Online Feedback conditions, the cursor remained visible throughout the reach. Bottom, In the baseline block, feedback was veridical (no rotation). In the second-baseline block, feedback was veridical and participants in the Instruction conditions reported their aiming location. In the rotation block, the cursor was rotated 45°. In the no-feedback block, the visual landmarks and cursor feedback were removed, and participants were instructed to aim directly to the target. In the washout block, veridical cursor feedback was restored.
For groups receiving endpoint feedback, equivalent to knowledge of results, the cursor disappeared when the participant's hand exited the starting circle (7 mm). Feedback, in the form of a red circle, was provided when the movement amplitude exceeded 7 cm. On no-rotation trials, the endpoint feedback was presented at the crossing position on the virtual ring of targets. For groups receiving on-line feedback, the cursor was visible during the first 7 cm of the movement, changing from white to red and becoming static when the hand intersected the ring.
The difference between the position of the green target and red cursor provided participants with feedback of their angular error. If the cursor overlapped any part of the target, the participant received 1 point. The points were not displayed to the participant on each trial; rather they were accumulated over each block and displayed as a summary at the end of the block (total number of points accumulated within a block). Participants were also provided auditory feedback on movement speed. If the 7 cm radial distance was traversed in <500 ms, a pleasant “ding” sound was played; otherwise an unpleasant “buzz” sound was played. After the feedback display had been maintained for 1 s, the target and cursor feedback (and landmarks, when present–see below) were erased from the screen, and replaced by a white ring that indicated the distance from the current hand position to the starting position. This ring was used to guide the participant to the starting position without providing information about the rotation. By moving toward the starting position, the ring became progressively smaller. When the hand was within 1 cm of the center of the starting position, the ring was transformed into the white feedback cursor, allowing the participant to precisely position the hand within the starting circle.
The two feedback conditions were crossed factorially with two types of instruction: Instruction and No Instruction. Participants in all groups were instructed that the task goal was to make their cursor land on the target. Participants in all groups were reminded of this task goal every 40 trials automatically by the computer software controlling the game. For participants in the Instruction conditions, the target was surrounded by a ring of 63 numbered visual landmarks spaced 5.625° apart (Fig. 1). Since the target could appear at multiple locations, the numbers rotated with the target such that they increased and decreased in the clockwise and counterclockwise directions from the target, respectively. Before each reach, the participants were instructed to verbally report the landmark number they planned to reach toward to make the cursor hit the target location. Importantly, they were not informed of the correct aiming direction to offset the rotation. Furthermore, their reported aiming direction may not be the same as their actual reach direction. Their actual reach location could be affected be movement variability or implicit learning, while their aiming direction simply reflects their planned direction of movement. The location of the aiming directions was recorded by an experimenter. Participants moved before reporting the aiming location on <1% of the report trials and these trials were excluded from analysis. In the No Instruction conditions, the numbered landmarks were absent (and thus no instruction was required), a design similar to that used in standard tests of visuomotor rotation.
The experiment was divided into five blocks (Fig. 1). The first block of 48 trials had veridical feedback (baseline block), allowing participants to become familiar with the reaching task. The second block of eight trials was identical to the first block, except that participants in the Instruction conditions were now required to report the aiming landmark before each movement (second-baseline block). A visuomotor rotation was introduced in the third block (rotation block), with feedback of the cursor displaced by 45° in the counterclockwise direction from the actual hand position. This rotation was present for 320 trials. Participants in the Instruction conditions were required to report their aiming direction throughout this block. In the fourth block of 40 trials (no-feedback block), the visual feedback of the cursor was removed and the rotation was turned off. In addition, all of the participants were instructed to aim directly to the target and, for the Instruction groups, the numbered landmarks were removed (and verbal reports were no longer required). In the final block of 40 trials (washout block), veridical cursor feedback was restored, in the same format for each group as they had experienced during the initial three blocks (on-line or endpoint).
Movement analysis.
Kinematic and statistical analyses were performed with MATLAB (MathWorks). To assess task performance, we focused on the initial heading angle of the hand. Each movement trajectory, regardless of the actual target location, was rotated to a common reference axis with the target location set at 0°. The average heading angle was computed by drawing a straight line between referent points positioned at 1 and 3 cm along the trajectory and computing the angle of this line. Positive angles indicate a clockwise deviation from the target and negative angles indicate a counterclockwise deviation from the target. These heading angles are reported in hand space. To determine the heading angle early in rotation training, we subtracted the average of the heading angle for the last eight trials in the second-baseline block from the average of the first eight trials in the rotation block for each participant. Similarly, to determine the size of the aftereffect for each group, we averaged the heading angle for the first eight trials in the no-feedback block and subtracted the average of the last eight trials in the second-baseline block for each participant. Note that because the target locations were chosen in a pseudorandom fashion, the average includes one reach to each of the eight target locations. In addition, since the sequence of target locations was randomized across participants, the averaging procedure removes any variability associated with specific target locations.
Learning in many adaptation studies is estimated by fitting an exponential function to the time series. This estimation procedure assumes learning is monotonic, with deviations from monotonicity attributed to noise. The aiming report data made evident that the learning functions can be non-monotonic, especially during the early phase of learning (see below). As such, we opted to use measures that did not require assumptions regarding the shape of the learning function.
To assess other kinematic features of the movements, we calculated the peak movement speed and movement curvature, defined as the total absolute curvature in Cartesian coordinates (Taylor et al., 2013). Velocity was computed with a fourth-order Savitzky–Golay filter, which introduces less noise than basic difference differentiation (Savitzky and Golay, 1964; Smith et al., 2000). We also measured reaction time, defined as the time between target onset and when the participant's hand position was 1 cm from the starting circle, and movement time, defined as the time required to traverse from the 1 cm position to the 7 cm position.
For all dependent measures, we report the mean and the 95% confidence interval of the mean for all dependent variables subjected to statistical evaluation.
Power analysis.
Our two primary dependent measures of interest were the heading angle during rotation training and the size of the aftereffect. We computed minimum sample sizes on assumed effect sizes for these dependent measures. To this end, we used a dataset from Taylor and Ivry (2011) and an unreported dataset in which participants learned a 45° rotation without any instructions or an explicit strategy over the course of an identical number of trials. We estimated the power for an independent samples t test using a two-tailed α of 0.05 and power of 0.95. For the heading angle early in training, the effect size is d = 6.48 (based on group means and SDs of μStrategy = 45.3°, σStrategy = 4.39° and μNoStrategy = 6.05°, σNoStrategy = 7.32°), requiring a minimum sample size of three participants. For the size of the aftereffect, the effect size is d = 1.91 (μStrategy = 5.61, σStrategy = 2.72 and μNoStrategy = 21.7, σNoStrategy = 11.6) requiring a minimum sample size of nine participants. Thus, to have sufficient power to detect significant effects, even if the effects are slightly smaller than in the previous datasets, we recruited 15 participants for each of the four groups (Button et al., 2013).
Results
Participants in all groups practiced bringing the cursor to the target during the initial baseline block with veridical feedback. During the second-baseline block, participants in the Instruction groups were asked to verbally report the number of the landmark they planned to reach toward to make the cursor hit the target location. In the Instruction-Endpoint group, nine participants reported always aiming toward the target (“0°” landmark). The other six participants reported aiming to other landmarks on some trials; however, these were generally at a landmark neighboring the target location. The mean aiming location showed a slightly clockwise bias relative to the target direction (1.76 ± 8.65°). The participants in the Instruction-Online group always reported aiming to the target location (0°). Compared with the No Instruction groups, the instruction to report the aiming landmark on each trial did not have an appreciable effect on any kinematic features of the subsequent movements, including heading angle, speed, or curvature (Table 1). The reaction time was considerably longer for the Instruction groups since these participants needed to verbally report the aiming direction before each movement (Table 1). For the instructed groups, reaction times were, on average, 25 ms longer for the Endpoint group compared with the Online group, perhaps because some participants in the former reported aiming to landmarks other than the target in the second-baseline block. However, the pattern was inconsistent and the reaction time difference was not statistically reliable (t(28) = 1.3, p = 0.2).
Movement parameters during the second-baseline block
Following the short second-baseline block, a 45° counterclockwise rotation was introduced and remained present for 320 trials. The time course of target errors for all four groups showed a stereotypical learning curve (Fig. 2A,B). To determine whether there were any group differences in heading angles early in rotation training, we performed a two-way ANOVA (Instruction, Feedback) on the average hand heading angle computed over the first eight trials relative to the average heading angle in the second-baseline block (Fig. 2D). There was a main effect of Instruction (F(1,28) = 5.21, p = 0.03), but no effect of Feedback (F(1,28) = 0.03, p = 0.87) nor a significant interaction (F(1,14) = 0.42, p = 0.52). The Instruction groups showed a greater change in heading angle early in training (Instruction-Endpoint: 12.1 ± 5.45°; Instruction-Online: 10.9 ± 7.19°) compared with No Instruction groups (No Instruction-Endpoint: 4.65 ± 2.54°; No Instruction-Online: 6.72 ± 3.59°). To determine whether there were any group differences in heading angles at learning asymptote during the rotation block, we performed a two-way ANOVA (Instruction, Feedback) on the average hand heading angle computed over the last eight trials relative to the average heading angle in the second-baseline block (Fig. 2D). There was a no effect of Instruction (F(1,28) = 1.88, p = 0.17), Feedback (F(1,28) = 1.72, p = 0.19), or interaction (F(1,14) = 0.1, p = 0.75).
A, Target error (hand heading angle plus the 45° counterclockwise rotation) for Instruction-Endpoint (blue) and No Instruction-Endpoint groups (magenta). B, Target error for Instruction-Online (red) and No Instruction-Online groups (cyan). The rotation was present between 56 and 376 (dashed vertical lines). C, Angle of aiming location (landmark number multiplied by a spacing constant of 5.625°) for Instruction-Endpoint (blue) and Instruction-Online groups (red). Participants made verbal reports of aiming locations between movements 48 and 376. D, Left, Difference in the hand heading angle between the first eight trials of the rotation block and the last eight trials of the second-baseline block. Center, Difference in hand heading angle for the last eight trials of the rotation block and the last eight trials of the second-baseline block. Right, Difference in hand heading angle for the first eight trials of the no-feedback block and the last eight trials of the second-baseline block. Bar graphs represent the mean, while the circles are the individual participants.
During the rotation block, the reaction times for the Instruction groups were considerably longer (1.24 ± 0.18 s) than for the No Instruction groups (0.59 ± 0.09 s). Reaction times were also longer for the Endpoint groups (1.02 ± 0.10 s) compared with the Online groups (0.81 ± 0.12 s). Both of these effects were reliable in a two-way ANOVA (Instruction: F(1,28) = 51.8, p < 0.001; Feedback: F(1,28) = 5.55 p = 0.022), with no interaction (F(1,14) = 0.26, p = 0.61). The increase in reaction time for the Instruction groups is expected given that these individuals have to identify and report their aiming landmark before the reach. The reason for the decrease in reaction time for the Online groups is unclear; one possibility is that participants in the Online group were less cautious since they could rely on feedback control during the movement.
To determine whether there were any changes in feedback control as a result of Feedback or Instruction, we computed the movement curvature during the course of rotation training. We submitted the average movement curvature during the rotation block to a two-way ANOVA (Instruction and Feedback). There was no significant effect of Instruction (F(1,28) = 0.78, p = 0.38), Feedback (F(1,28) = 1.15, p = 0.29), or Interaction (F(1,28) = 1.78, p = 0.19). As such, it appears that the availability of on-line feedback did not influence the trajectories, presumably because we required ballistic reaches.
Explicit aiming direction
Compensation for the rotation may occur through the combination of both explicit and implicit learning processes. Unlike previous studies of visuomotor rotation learning in which the contribution of multiple processes was inferred, our experimental design allowed for the direct assessment of an aiming direction. Participants in both Instruction groups readily chose to aim to locations other than the goal target, and the selected locations changed in a systematic way over the course of the rotation block. Moreover, participants in the Instruction-Endpoint group chose to aim to locations that were farther away from the target location than participants in the Instruction-Online group (Fig. 2C). This difference was confirmed by binning the aiming directions over the 320 trials of the rotation block into bins of eight trials each (Taylor and Ivry, 2011), and submitting the mean of these values to a two-way ANOVA with the factors Feedback and Trial Number. There was a main effect of Feedback (F(1,28) = 98.32, p < 0.01) and a main effect of Trial Number (F(1,39) = 2.05, p < 0.01); there was no significant interaction (F(1,14) = 0.27, p = 1.00). Together, these results suggest that the type of feedback modulates the magnitude of the explicit contribution to performance during visuomotor adaptation, but does not influence the time course of this component over training.
As can be seen in Figure 2C, the aiming locations, even in group averaged data, is highly nonstationary over the course of training exhibiting high variance early in training and low variance at the end of training. To analyze this change in variance over training for each participant, we binned the aiming directions over the 320 trials of the rotation block into bins of eight trials each and computed the variance with each bin. When the variance of each bin was submitted to a two-way ANOVA with the factors of Feedback and Trial Number, we found both a main effect of Feedback (F(1,28) = 4.93, p = 0.03) and Trial Number (F(1,39) = 4.57, p < 0.001), with no interaction between these factors (F(1,14) = 1.07, p = 0.35). Thus, suggesting that the statistics of the aiming directions are nonstationary.
The nonstationarity in the aiming direction data motivated a finer grained analysis. We calculated the probability that the participants changed aiming direction over successive trials during the rotation block (Fig. 3A). As might be expected, the probability of aim change was very high during the first trials after the introduction of the rotation. More interesting, the probability of change gently decreased over the 320-trial rotation block, dropping from an initial value of ∼60% to a final value of ∼40% during the final 100 rotation trials even when performance was quite stable (Fig. 2A,B). To statistically evaluate these data, we conducted a two-way ANOVA (Feedback and Trial Number) with the probability of aim change as the dependent variable. There was a main effect of Trial Number (F(1,39) = 2.13, p < 0.001), but no effect of Feedback (F(1,28) = 1.27, p = 0.26) nor a significant interaction (F(1,14) = 0.51, p = 0.99). Thus, participants were more likely to change their aim in the beginning than at the end of training.
A, Probability of aim change during the report phase for the Instruction groups. B, Magnitude of aim change, the average change from trial (n) and trial (n − 1) across participants. C, Probability of aim change following a trial in which the cursor hit or missed the target. Instruction-Endpoint, blue; Instruction-Online, red.
While the probability of a change in aiming direction decreased in a gradual manner over the rotation block, the size of the aim change rapidly decreased (Fig. 3B). Initially, the changes were quite large as the participants explored the task space, but near the end of the rotation block, the changes were consistently smaller. This was confirmed by a two-way ANOVA with factors for Feedback and Trial Number. Again, we found a main effect of Trial Number (F(1,31) = 8.28, p < 0.001), but no effect of Feedback (F(1,28) = 2.13, p = 0.14) and no interaction (F(1,14) = 0.66, p = 0.94). Thus, while participants in the Endpoint Feedback condition tended to show larger overall shifts from the target in their aiming directions, late adjustments in the aiming locations were similar for the two feedback groups.
We also sought to identify whether there was an underlying pattern of how aiming changed during training. One such pattern may be win-stay, lose-shift where changes in aim would be more likely following a trial in which the cursor missed the target. We conducted a two-way ANOVA with the factors Feedback and Hit, using the probability of a change in aim direction as the dependent variable. There was a main effect of Hit (F(1,28) = 9.18, p = 0.004), but no effect of Feedback (F(1,28) = 0.02, p = 0.90) nor an interaction (F(1,14) = 0.27, p = 0.61). Thus, participants were more likely to change their aiming direction following a miss compared with a hit (Fig. 3C). It is interesting, that even after hits, participants changed their aiming location on >20% of the trials. While this could reflect efforts to further refine performance (e.g., hit the center of the target), it is possible that participants were learning specific aiming directions for different target locations. However, the current experiment lacks sufficient power to track trial-by-trial changes in aiming direction for the different targets, a question that can be addressed in future studies.
Aftereffects
During the no-feedback block, visual feedback was removed along with the rotation and the participants in all four groups were instructed to aim directly to the goal target. Note that in most studies in which aftereffects have been measured, participants are not informed of the removal of the rotation. However, we wanted specifically to remove the contribution of explicit learning, with the idea that residual directional error is attributable to implicit adaptation of a forward model. The aiming landmarks were also removed for the two Instruction groups. Importantly, none of the groups were provided with cursor feedback, a method to probe for the degree of motor adaptation in the absence of learning (Galea et al., 2011; Kitago et al., 2013; Taylor and Ivry, 2013; Taylor et al., 2013).
All of the groups showed a large aftereffect: the heading angle at the start of the no-feedback block was in the opposite direction of the rotation (Fig. 2A,B; each group compared to zero: p < 0.01). Note that the aftereffects are much lower than 45° and their magnitude decreases gently over trials without feedback, consistent with previous reports that adaptation decays over time (Galea et al., 2011; Kitago et al., 2013; Taylor and Ivry, 2013; Taylor et al., 2013).
To compare the amount of adaptation between groups, the average of the last eight trials of the second-baseline block were subtracted from the average of the first eight trials of the no-feedback block. There was a significant effect of Feedback (F(1,28) = 10.8, p = 0.002). On average, the aftereffect was larger for the Online groups (Fig. 2A,B), averaging 31.5 ± 3.76° for the No Instruction-Online group and 25.8 ± 4.92° for the Instruction-Online group. The comparable values for the Endpoint groups were 22 ± 5.62° (No Instruction) and 21.5 ± 4.46° (Instruction). While the Instruction groups had smaller aftereffects, this difference was not reliable (F(1,28) = 2.89, p = 0.09), nor was the interaction term (F(1,14) = 1.16, p = 0.29). While our power analysis suggested that we should be sufficiently powered to detect a large effect (d > 1), we may not have sufficient sensitivity to detect a smaller effect size (d < 1).
After the 40 trials of the no-feedback block, visual feedback was reintroduced in the washout block. For all groups, target error was rapidly attenuated, although it remained different from zero at the end of the washout block for all groups (p < 0.05).
Uncovering forward model adaptation
Since we independently measured target errors and the aiming directions, we used subtractive logic to extract the state of forward model adaptation during rotation training for the two Instruction groups (Fig. 4A). In contrast to previous studies where adaptation functions have been described by a fast exponential function, our results suggest a relatively slow process of implicit learning once explicit learning contributions are subtracted out (Fig. 4B). To determine whether there were any group differences in implicit learning early in rotation training, we compared the average “implicit learning” heading angle over the first eight trials relative to zero for both Instruction groups. We find that there is an advantage in implicit learning for the Instruction-Online group (10.5 ± 5.33°) compared with the Instruction-Endpoint group (2.32 ± 3.69°; t(28) = 2.49, p = 0.02). This difference appeared to persist throughout training and was still evident by the end of rotation training. To compute the final state of implicit learning, we computed the average implicit learning heading angle over the last eight trials and found that there was a significant difference in implicit learning (t(28) = 2.8, p = 0.01) between the Instruction-Online (33.9 ± 4.83°) and the Instruction-Endpoint groups (26.2 ± 2.39°).
A, Target error (performance) is the sum of the explicit aiming direction and implicit learning of a forward model minus the perturbation. B, Implicit learning can be estimated by subtraction of the aiming direction (see Fig. 2C) from the target error. Instruction-Endpoint, blue; Instruction-Online groups; red.
While the aiming directions exhibited nonstationarity and non-monotonicity during the course of rotation training, the inferred implicit learning function appears much more stationary and monotonic. To quantify this, we binned the implicit learning over the 320 trials of the rotation block into bins of eight trials each and computed the variance within each bin (identical to the analysis performed on the aiming direction data). When the variance of each bin was submitted to a two-way ANOVA with the factors of Feedback and Trial Number, we found a main effect of Trial Number (F(1,39) = 4.42, p < 0.001), but no effect of Feedback (F(1,28) = 0.96, p = 0.33) or interaction between these factors (F(1,14) = 0.56, p = 0.99). Surprisingly, while implicit learning appears much more monotonic compared with explicit learning, this analysis revealed that the variance of implicit learning changes during training suggesting that it was also nonstationary. This may be the result of explicit learning's variability being injected into implicit learning. Alternatively, it may be an unintended consequence of our experiment design, such that participants may have reached to a direction other than their reported aiming direction early in training.
The current design provides two measures of the extent of implicit learning: the final estimate of the forward model and the initial size of the aftereffect. Theoretically, we would expect these two values to be identical. However, as seen between Figures 2 and 4, the size of the aftereffect is smaller than the estimated state of the internal model. Over the last eight trials of the rotation block, the estimated change in the forward model is 25.9 ± 2.3° and 31.5 ± 6.6° for the Endpoint and Online feedback groups, respectively. Over the first eight trials of the no-feedback block, the corresponding values of the aftereffect are 21.6 ± 4.1° and 25.9 ± 4.9°. A two-way ANOVA with the factors Feedback (Endpoint, Online) and Block (rotation, no-feedback) showed main effects for both Feedback (F(1,28) = 4.13, p = 0.047) and Block (F(1,28) = 4.23, p = 0.045), and no interaction of these factors (F(1,14) = 0.07, p = 0.80). Combined over the two feedback conditions, the estimate of the forward model is ∼4.9° greater than the aftereffect. We also performed a regression analysis on these two measures of implicit learning. When the participants in the two groups are combined, we observed a modest correlation (r = 0.36, p = 0.049). The slope and intercept values deviated from the predicted values if the relationship was perfect (slope = 0.32 vs 1, intercept = 14.4° vs 0°). Thus, at an individual level, there is only a weak relationship between the two measures and, as evidenced by the positive intercept, the estimate of the forward model is greater than the estimate of the aftereffect.
We believe that this difference between the final estimate of the forward model and the aftereffect is largely due to two factors, all of which have the effect of attenuating the size of the observed aftereffect. First, the aftereffect was computed by averaging over the first eight trials of the no-feedback block. Previous studies have shown substantial decay of an adapted forward model when reaches are made without feedback (Galea et al., 2011; Kitago et al., 2013; Taylor and Ivry, 2013; Taylor et al., 2013). Indeed, if we restrict our analysis to only the last trial of the rotation block and the first trial of the no-feedback block, the difference between the two estimates of implicit learning is no longer reliable (F(1,28) = 0.7, p = 0.41). Second, before the start of the no-feedback block, a short break (∼40 s) was required to instruct the participants that they should aim directly at the green target on the forthcoming trials. Force field and visuomotor adaptation studies have shown that adaptation decays with time even in the absence of movement (Miall et al., 2004; Krakauer et al., 2005), an effect that can be as large as 20–25% in just 15–20 s (Hadjiosif and Smith, 2013). Both of these factors would converge to lower the magnitude of the measured aftereffect.
Discussion
Summary
The current study was designed to examine the contribution of explicit and implicit processes to the learning of a visuomotor rotation. To assay for an explicit component, participants were instructed to report their aiming direction before each movement. Importantly, we found that explicit aiming persists throughout learning regardless of whether cursor feedback was provided continuously or restricted to the movement endpoint. The current protocol also offers a novel method to measure the time course of implicit learning, estimated as the residual function after subtracting the aiming component from the target error. Implicit learning was driven by sensory-prediction error, was relatively slow, and was incomplete in both feedback conditions. Moreover, the magnitude of the aftereffect, a measure of the degree of implicit learning, was similar for groups with and without instruction. Together, these results provide compelling evidence for both implicit and explicit contributions to learning in a visuomotor adaptation task.
The interaction of explicit and implicit learning
How do explicit and implicit learning combine during a visuomotor adaptation task? Providing participants with aiming targets led to slightly faster performance gains without significantly affecting the contribution of implicit learning, as measured by the size of the aftereffects. We interpret the results as follows: initially subjects explicitly explore aiming directions in an attempt to reduce target error but are concomitantly adapting implicitly to prediction error. If an appropriate aiming direction is selected, the combination of the two processes can achieve zero target error. However, even though performance is accurate, implicit learning continues because sensory-prediction error is not yet zero. Continued operation of the implicit process would lead to a systematic increase in target error (Mazzoni and Krakauer, 2006), this drift in target error is countered by adjustments of aiming direction in the opposite direction; indeed, adjustment of the aiming direction appears to be driven by the size of the target error (Taylor and Ivry, 2011). The influence of error size can explain why exploration is of greater amplitude in the case of endpoint feedback: implicit learning is less effective with this kind of feedback, resulting in persistently larger target errors. This dynamic suggests that the gain on exploration is determined by the size of target error. We hypothesize that, in the current study, the two feedback conditions led to similar performance because of differential weighting on the two processes. With endpoint feedback, higher gain exploration was coupled with slower implicit learning; with on-line feedback, lower gain exploration was coupled with faster implicit learning.
The benefit of being able to counter target error with an explicit process is made readily apparent given the evident limitations of the implicit learning process, revealed here by subtraction of the reported aiming directions: it was both slow and incomplete by the end of rotation block. Notably, implicit learning was also incomplete for both No Instruction groups as their aftereffect was <45°. The degree of implicit learning was greater, however, with on-line feedback compared with endpoint feedback. This difference can be attributed to the idea that on-line feedback provides a richer source of signed sensory-prediction error, the signal used to update a forward model (Izawa and Shadmehr, 2011).
We have recently suggested that, whereas implicit error-based learning dominates during the acquisition phase of adaptation tasks, reinforcement, especially at asymptote, is responsible for long-term retention (Huang et al., 2011; Shmuelof et al., 2012). We also suggested that, at least for rotation adaptation, these two processes may map onto the slow and fast distinction made by Smith et al. (2006) for force-field adaptation. The current results, however, point to limitations with this hypothesis: the multiple processes that contribute to rotation adaptation do not map simply onto a binary opposition between fast and slow processes. As shown here, a fast explicit process speeds up acquisition when combined with slower implicit adaptation of a forward model. It may well be that the combined effects of explicit and implicit processes that operate during acquisition can converge on actions that are then reinforced to facilitate long-term retention (Huang et al., 2011; Shmuelof et al., 2012).
In the current study, implicit learning, measured by the size of the aftereffect, was incomplete following 320 trials of rotation training, which seems at odds with previous studies showing aftereffects during washout that are close to 100% of the rotation (Zarahn et al., 2008; Fernandez-Ruiz et al., 2011; Kitago et al., 2013). The apparent discrepancy between the size of the aftereffects can be explained by the fact that in previous studies the washout phase is initiated without a change in instruction or display; the rotation in these studies was just turned off unexpectedly. Under such conditions, participants would be expected to continue to reach in the same manner as on the last trials of the rotation phase, making it trivially inevitable that the initial errors will be ∼100% of the rotation and likely that the standard washout procedure thereby masks the contribution of an explicit component. By instructing our participants to aim directly for the green target and withholding visual feedback, we obtained an uncontaminated estimate of the state of the internal model, one that revealed an aftereffect that was smaller than the size of the rotation even for the No Instruction groups. These smaller than expected aftereffects suggest that an explicit component is present even in adaptation tasks in which there are no landmarks or instructions. Exploration of potential aiming directions may be a fundamental process for motor learning and, in fact, may also occur in nonhuman primates (Jarosiewicz et al., 2008; Legenstein et al., 2010).
The nature of explicit learning
The contribution of explicit knowledge to motor learning has been surprisingly understudied despite the pervasive use of coaching in sports. One reason for this might be that sensorimotor adaptation tasks have been treated as paradigmatic examples of implicit learning. This perspective has been advanced given the seminal findings with amnestic patients such as H.M. (Corkin, 1968), as well as by studies showing that adaptation is highly dependent on a cerebellar-dependent process in which a forward model is updated by sensory-prediction errors (Taylor et al., 2010; Izawa et al., 2012). Indeed, in our previous work, we showed that an aiming strategy was quickly overridden by implicit learning, even at the cost of task performance (Mazzoni and Krakauer, 2006).
This raises the question: What is the role of explicit learning in visuomotor adaptation tasks? There have been several studies suggesting a role for attention, awareness, strategies, and declarative memory in visuomotor adaptation tasks (Redding and Wallace, 1996; Hwang et al., 2006; Michel et al., 2007; Taylor and Thoroughman, 2007, 2008; Hegele and Heuer, 2010, 2013; Benson et al., 2011). However, in these studies the evidence has largely been indirect, measured through changes in learning rate, size of aftereffects, or post-experiment tests of knowledge of the perturbation. For example, the degree of implicit learning has been found to be negatively related to awareness of the perturbation (Michel et al., 2007; Benson et al., 2011). However, these indirect approaches do not tell us what the overt strategy was, nor do they probe how a strategy was used throughout the entire time course of learning. Our protocol provides, for the first time, a method to capture trial-by-trial fluctuations in the explicit component during learning.
We find that good task performance (i.e., zero target error) can result from the combined contributions of explicit and implicit learning processes. Behavior in adaptation paradigms has been characterized within the framework of state-space models, in which a gradient descent process reduces error in a continuous and monotonic manner (Thoroughman and Shadmehr, 2000; Zarahn et al., 2008). We have previously applied a similar modeling framework to characterize the interaction of implicit and explicit processes (Taylor and Ivry, 2011). While this approach provided an excellent fit for group-averaged data, the performance of some individuals suggested more abrupt, discontinuous changes in explicit aiming. The aiming data in the current study definitively showed that, early in learning, the explicit process is non-monotonic, consisting of large fluctuations in aiming amplitude. Exploratory behavior of this form is often described as reflecting model-free reinforcement learning (Sutton and Barto, 1998). However, the exploration observed here was not random, but biased toward the correct solution, suggesting some influence of vector error. This influence may have been based on a simple heuristic of the kind “when the error is to the left, go right and vice-versa.” Broadly consistent with a simple heuristic, we observed a win-stay/lose-shift pattern in the aiming direction time course data, such that the aim was less likely to change on the next trial if the previous trial was successful (Worthy et al., 2013). Alternatively, biased exploration could reflect a more sophisticated model or strategy based on inferring the precise nature of the perturbation. Further experiments would be required to dissociate between these possibilities.
Having argued that there is an explicit component to motor learning, one might ask why people do not rely on this process, especially since it is capable of fostering fast learning (e.g., one trial learning). Indeed, why can't explicit learning processes substitute entirely for implicit learning in visuomotor adaptation tasks? A clue comes from the performance of patients with cerebellar disease on sensorimotor adaptation tasks. These patients are unable to compensate for large perturbations (Martin et al., 1996; Rabe et al., 2009; Criscimagna-Hemminger et al., 2010). It would appear that, for these individuals, self-generated exploration is insufficient for solving such tasks (Vaca-Palomares et al., 2013); they must be given the explicit strategy (Taylor et al., 2010). Thus, it is possible that the cerebellum is necessary to provide some form of an error signal that is used by explicit learning to reduce the search space of aiming direction; however, this remains an open question. Finally, while we have emphasized the parallel operation of implicit and explicit processes, it is also possible that the systems operate in a push–pull manner: recent reports suggest that error-based learning may suppress reward-based learning and retention (Izawa and Shadmehr, 2011; Shmuelof et al., 2012).
In summary, our novel task design allowed us to identify the time courses of explicit and implicit processes during visuomotor adaptation. We found that explicit learning is driven by task success (target errors) and exhibits large exploratory fluctuations early in training before settling into smaller adjustments late in training. This time course occurs because the participants attempted to explicitly counter errors caused by the rotation and then had to compensate for the slow drift resulting from concomitant implicit learning. Interestingly, providing instructions for aiming did not dramatically change the overall time course of learning (measured as target error) and only slightly affected the degree of implicit learning, consistent with other studies (Benson et al., 2011). These results support the hypothesis that visuomotor adaptation, even in conventional paradigms, entails both fast learning of an explicit aiming direction and slower implicit learning of a forward model, with overall task performance reflecting the joint operation of both processes.
Footnotes
J.A.T. was supported by R01NS084948 from the National Institute of Neurological Disorders and Stroke (NINDS). J.W.K. was supported by Grant 220020220 from the James S. McDonnell Foundation and R01NS052804 from the NINDS. R.B.I. was supported by R01HD060306 from the National Institute of Child Health and Human Development and R01NS074917 from the NINDS. We thank Melissa Burney, Nawsheen Shoaib, and Samuel Brudner for help with data collection and Samuel McDougle for help with manuscript preparation. We also thank Adrian Haith for helpful discussions.
- Correspondence should be addressed to Jordan A. Taylor, Department of Psychology, Princeton University, 429 Peretsman-Scully Hall, Princeton, NJ 08544. jordanat{at}princeton.edu
