Abstract
Can memories be unlearned, or is unlearning a form of acquiring a new memory that competes with the old, effectively masking it? We considered motor memories that were acquired when people learned to use a novel tool. We trained people to reach with tool A and quantified recall in error-clamp trials, i.e., trials in which the memory was reactivated but error-dependent learning was minimized. We measured both the magnitude of the memory and its resistance to change. With passage of time between acquisition and reactivation (up to 24 h), memory of A slowly declined, but with reactivation remained resistant to change. After learning of tool A, brief exposure to tool B brought performance back to baseline, i.e., apparent extinction. Yet, for up to a few minutes after A+B training, output in error-clamp trials increased from baseline to match those who had trained only in A. This spontaneous recovery and convergence demonstrated that B did not produce any unlearning of A. Rather, it masked A with a new memory that was very fragile. We tracked the memory of B as a function of time and found that within minutes it was transformed from a fragile to a more stable state. Therefore, a sudden performance error in a well-learned motor task does not produce unlearning, but rather installs a competing but fragile memory that with passage of time acquires stability. Learning not only engages processes that adapt at multiple timescales, but once practice ends, the fast states are partially transformed into slower states.
Introduction
One of the defining characteristics of primates is the ability to use tools. Learning to use a tool requires practice, and the result of this practice is a motor memory that allows the user to display skilled performance the next time the tool becomes available. Classic texts on memory hypothesized that motor memory differed from cognitive or declarative memories because it did not appear to include a short-term phase in which the memory was fragile (Squire, 1987). This picture changed when it became clear that motor memories exhibited a phenomenon termed spontaneous recovery (Kojima et al., 2004; Smith et al., 2006; Ethier et al., 2008). For example, after people learned to use a novel tool (a robotic arm) that exhibited dynamics A, they could quickly “unlearn” that behavior if the dynamics of the tool suddenly changed to B. However, in the subsequent “error-clamp” trials in which motor output could be assayed while minimizing error-dependent learning, behavior reverted back toward A (Smith et al., 2006). This suggested that during motor learning, changes in behavior were attributable to two processes: a fast learning process that was highly sensitive to error but had poor retention, and a slow learning process that had poor sensitivity to error but had robust retention. Learning of B installed a fast memory that decayed rapidly after acquisition. Extensions of this simple idea appeared to account for a range of behaviors in motor learning (Kording et al., 2007; Chen-Harris et al., 2008).
If motor memory is supported by a fast and a slow process, what are their characteristics? For example, do these processes show different sensitivities to passage of time? Are the processes independent of each other, or does passage of time transform one process into another? Finally, can motor memories be unlearned, or is “unlearning” merely learning of a new memory that competes with the old? These questions are of fundamental importance because they deal with consolidation patterns of motor memory, a topic that despite years of research in various laboratories has produced a large body of apparently conflicting results. Whereas some have found that passage of time increases resistance of the memories to unlearning (Shadmehr and Brashers-Krug, 1997; Krakauer et al., 2005; Overduin et al., 2006), others have seen no evidence for a temporal gradient (Caithness et al., 2004; Mattar and Ostry, 2007).
Here, we assayed the effects of time on motor memory through error-clamp trials, in which the memory was reactivated but error-dependent learning was minimized (Scheidt et al., 2000). The crucial advantage of this approach was that it allowed us to simultaneously measure both the magnitude of the reactivated memory and its resistance to change. We found that the large errors subjects experienced during deadaptation did not produce unlearning of tool A, but rather installed a competing memory of tool B. However, in contrast to predictions of the fast/slow model, with passage of time after A+B training, memory of B did not rapidly fade away. Rather, when subjects stopped performing the task, the initially fragile memory gained stability. These results together suggest a new memory model in which learning not only engages processes that adapt at multiple timescales, but that once practice ends, the fast states can be partially transformed into slower states. We suggest that this framework can account for much of the apparently conflicting results in motor memory consolidation research.
Materials and Methods
One hundred and seven neurologically intact right-hand-dominant participants were involved in this study. Fifty-six volunteers (24 male, 32 female) were recruited for experiment 1 (average age, 26.0 years; SD, 4.3 years). Fifty-one different volunteers (26 male, 25 female) were recruited for experiment 2 (average age, 28.5 years; SD, 9.1 years). All volunteers were naive to the purpose of the experiment. Experimental procedures were approved by the Johns Hopkins University School of Medicine Institutional Review Board and all participants signed a consent form.
The volunteers were trained in the standard force field reach adaptation paradigm (Shadmehr and Mussa-Ivaldi, 1994). They held a two-joint robotic manipulandum with their right hand and made point-to-point reaching movements from a center starting position to a single 1 cm square target positioned at 10 cm directly along the body midline. Once at the target, the robot brought the subject's hand back to the center. Subjects were rewarded with an “explosion” for completing the movement within a 50 ms window centered at 0.5 s after movement start time. The subject's hand was covered by a horizontal screen onto which a small cursor (5 × 5 mm) representing hand position was projected at all times. We recorded force at the handle, hand position, and hand velocity at a rate of 100Hz.
The experiment (see Fig. 1A) began with training in a null field (no forces, 192 trials) followed by training in a curl field in which forces were dependent on hand velocity f = Aẋ, in which A = [0 13; −13 0] N · s/m. This adaptation phase consisted of 384 trials. One subject was excluded from the analysis because of failure to adapt to the force field during the training session.
Error-clamp trials.
We placed error-clamp trials randomly in the baseline and adaptation phases with one-eighth probability (no error-clamp trials were present during the exposure to field B). During the error-clamp trials, the motion of the hand was constrained to a straight line to the target by a stiff one-dimensional spring (spring coefficient = 2500 N/m; damping coefficient = 25 N · s/m) that counteracted any forces perpendicular to the target direction. Error-clamp trials, however, were no different from regular trials in the type of feedback that the subject received: they were rewarded with an explosion for completing the movement within a 50 ms window centered at 0.5 s after movement start time. Because the curl field perturbed the hand perpendicular to the direction of motion, the forces that the hand produced against the “channel” wall in error-clamp trials served as a proxy of adaptation, i.e., the change in the motor output. Previous work has shown that after training, forces in error-clamp trials faithfully represent the ideal force trace that one should produce to cancel the robot forces, whether in velocity-dependent (Scheidt et al., 2000; Hwang et al., 2006b; Smith et al., 2006) or acceleration-dependent fields (Hwang et al., 2006a).
Experiment 1: group A.
This experiment assayed the sensitivity of the motor memory to passage of time. After the completion of adaptation, subjects were reexamined once at 0, 2, or 10 min, or 1, 6, or 24 h. The reexamination consisted of 30 error-clamp trials only.
Experiment 2: group A+B.
In this experiment, volunteers who had completed their training in field A were immediately exposed to 20 trials of field B (viscosity matrix [0 −13; 13 0] N · s/m). This small number of trials was sufficient to bring the motor output back to baseline. However, according to our theory, the trials set up a competition between a “fast” memory acquired in response to training in B and a “slow” memory acquired in response to training in A. As a result, we expected to see spontaneous recovery of A immediately after completion of the 20 trials in B. We were interested in quantifying the sensitivity of this spontaneous recovery to passage of time after completion of training in B. Therefore, after completion of training in B, subjects were reexamined once at 0, 2, or 10 min, or 1, 6, or 24 h. The reexamination consisted of 30 error-clamp trials only.
We accounted for the time that passed between end of training and start of the test session in all subjects. In the 0, 2, and 10 min groups, all subjects remained seated in front of the robot. In the 1, 6, and 24 h groups, all subjects left the chair during the break and were instructed to proceed throughout their day normally. Subjects in the 24 h group were required to sleep a minimum of 6.5 h between day one and day two.
Data analysis.
Performance was measured via the force that subjects produced against the channel wall of the error-clamp trials. Our performance measure, termed force output as a percentage of perturbation, was simply the ratio between this force as measured at the maximum velocity and the ideal force in field A. That is, regardless of whether the block of trials was in field A or null, the ideal force was described as f = Aẋ, where ẋ is the hand velocity on that trial. In this way, in the null field we assayed motor output with the same yard-stick as in subsequent training trials in field A. Note that there were no error-clamp trials during exposure to field B. Repeated-measures ANOVA and post hoc Tukey's tests were used to quantify effects of time passage and differences between groups. All analyses were done using Matlab and SPSS.
When subjects were using the robot, the memory was “active” and it changed as a function of time/trial. When subjects were not using the robot, the memory was “inactive” but it still continued to change as a function of time. To assay the sensitivity of the active memory to time/trial, we focused on the rate of decay of the force output during the error-clamp trials of the test period. This rate was estimated by fitting a single exponential of the form f (n) = aexp(−bn) to the data set during the test period for each subject. In this equation, f is force and n is trial number. This continuous-domain equation can be well approximated in the discrete domain: f(n + 1) = (1 − b) f(n), in which (1 − b) is an estimate of sensitivity of the memory to trial. Therefore, b is fraction of the force that is lost from one trial to the next. This measure quantifies the fragility of the memory as assayed at a particular time after its acquisition.
We performed a boot-strapping procedure to estimate strength of memory for B at each time point after A+B training. To do so, we had two subject groups to choose from: one group of subjects who had learned A and was then tested at time point t for 30 trials, and another group of subjects who had learned A+B and was then tested at the same time point t for 30 trials. We picked one subject at random (with replacement) from group A, and another subject at random (with replacement) from group A+B, and then subtracted performance of subject in group A from subject in group A+B, i.e., B̂ = (A + B) − A. We then repeated this procedure 50 times to arrive at a distribution for the motor output at each trial during the test period at time t. This produced the data for B̂, shown in Figure 4.
When the subjects were not reaching with the robot, the memory was inactive. The sensitivity of the inactive memory to passage of time was estimated using a boot-strapping procedure. We selected at random one subject (with replacement) from each group and calculated the average of the first two error-clamp trials of the test session. We then took the six data points (one for each time delay) and fitted them to a single exponential of the form f (t) = aexp(−t/τ). We repeated this procedure 50 times.
Active–inactive state model of motor memory.
Previous models of motor memory assumed that during learning, the observed state of the environment, was assigned to one of two general states: a fast state and a slow state. Learning was a procedure in which one tried to estimate the state of the environment (Kording et al., 2007). With passage of time, the fast state decayed rapidly whereas the slow state decayed gradually. Here, one of our main observations was that effect of time was not merely a decay in the states, but rather a transformation of the fast state into a slow state. In particular, when the subject was not performing the task (the inactive state of the memory), passage of time produced a partial transformation of the fast state into a slow state. To represent this idea, we considered a generative model in which the environment was sometimes observable and sometimes unobservable. In the observable condition, the generative model was identical to our previous model (Kording et al., 2007). However, in the inactive state, the state transitions allowed some transformation of the fast states into slow states.
The graphical representation of the model is shown in Figure 5A. The measured variables are filled circles, whereas the estimated variables are unfilled circles. When the environment is observable (active memory), measurements are simply the sum of the fast and slow states, with equal weight, i.e., y(t) = cTx(t) + εy where cT= [1, 1] and xT = [xs xf]. Time affects the states in both the active and inactive conditions. When the environment was observable, we assumed x(t + Δ) = Aax(t) + εa, where Aa was diagonal, representing the state transition for the active memory. When the environment was unobservable, we assumed x(t + Δ) = Aix(t) + εi, where Ai was not diagonal, representing the state transition for the inactive memory. To fit the model to the data, the only relevant variables are Aa (two parameters) and Ai (three parameters) because the noises play no role in the decay rates in error-clamp trials (as long as we assume that there are no errors in error-clamp trials, and the noises are Gaussian). We assume that by end of training in A, 83% of the memory was attributable to the slow state and 17% was attributable to the fast state. Training in B introduced a competing memory that was 83% fast and 17% slow. These values were found to be the optimal initial condition proportions for the fast and slow states for the memory of A and the memory of B. We estimated the five parameters of the model by fitting it to the average data using nonlinear optimization.
Results
We imagined that practice resulted in a motor memory that had at least two functional states: a fast state and a slow state. To assay the sensitivity of the hypothetical slow memory to passage of time, we trained subjects for a long period of time (384 trials) (Fig. 1A) on a reach adaptation protocol (field A), and then divided them into six groups and tested each group at a single time point after completion of practice (experiment 1). To assay the sensitivity of the hypothetical fast memory to passage of time, we performed an adaptation/deadaptation experiment in which the same training in A was followed by a brief period of training in B (20 trials) until performance reached naive levels (experiment 2). Although A+B training produced apparent extinction, we hypothesized that the training in fact produced a fast memory of B that competed with the slow memory of A. We divided the subjects into 6 groups and tested each group at a single time point after completion of A+B. By comparing A with A+B at various time points, we quantified how passage of time affected the hypothetical slow memory of A and fast memory of B.
Study protocol and performances during learning of the task. A, Volunteers were divided into two groups (group A and group A+B). They held the handle of a light weight robotic arm and reached to a target. In the first 192 trials, the robot produced a null field (no forces). In the subsequent 384 trials, a curl force field was introduced (field A), perturbing the hand perpendicular to its direction of motion. In group A+B, the force field was reversed in sign for 20 additional trials. The gray bars schematically represent “error-clamp” trials, trials in which the robot produced a stiff channel that allowed us to measure the subject's motor output perpendicular to the direction of motion. After a break of variable length, subjects returned and were asked to hold the robotic tool and perform the task. We quantified the strength of the reactivated memory through 30 error-clamp trials. B, Force output (mean for each group for each trial) during the error-clamp trials in the baseline and learning periods (field A only). Learning of A was similar among the groups, exhibiting the classic double exponential pattern: rapid initial learning followed by slow, gradual learning. Bin size is one trial.
Performance was measured via the force that subjects produced against the channel wall of the error-clamp trials. Adaptation in A produced performance measures that had the familiar initial fast rise followed by a period of gradual increase (Fig. 1B). We found no significant differences in the performance measures of the 12 groups during adaptation in A (repeated-measures ANOVA, F(11,95) = 0.683, p = 0.752). A one-way ANOVA on the average of the last five trials showed no differences between groups (F(11,95) = 0.174, p = 0.999).
Memory of A decayed as a function of time, but when reactivated remained resistant to trial
After completion of 384 trials in A, subjects were assigned to one of six groups and waited 0, 2, or 10 min, or 1, 6, or 24 h. In the 0, 2, and 10 min groups, the subjects remained seated in front of the robot. In the 2 and 10 min groups, subjects released the handle of the robot during the delay period. The subjects in other groups left the room and returned at their scheduled time. After return, they were asked to hold the handle and reach to the target. In these movements, the robot always produced a channel. Therefore, the test period consisted of only error-clamp trials, allowing us to assay the memory without contamination from additional relearning.
The performance of each group during the test phase is shown in Figure 2A. During the time between end of training and start of testing, the memory decayed as a function of time, as reflected in the force that subjects produced in the first two trials (Fig. 2B) (F(5,50) = 8.469, p < 0.05). A Tukey's post hoc test revealed differences between the 24 h group and the 0, 2, and 10 min and 1 h groups, and the 6 h group and the 0 and 10 min groups (p < 0.05 in all cases). A repeated-measures ANOVA suggested that groups that waited a longer period of time produced smaller forces during the course of the entire test session than those that waited a brief period (main effect of wait period, F(5,50) = 3.773, p < 0.01).
Performance during test of recall in group A. A, Force output (mean ± SEM) for each subgroup after completion of training in A. The dashed line represents the force output at end of training (average of last 5 error-clamp trials across all subgroups). The output started higher for subjects that acquired the task recently, and decayed slowly during test of recall in all subgroups. Bin size is one trial. B, Magnitude of the memory as a function of time since acquisition. Each bar plot represents the initial force output (bin size is two trials) averaged across subjects in each subgroup (error bars are SEM). C, Fragility of the memory. Force output as a function of trial for each subject was fitted to a single exponential. The decay rates, shown here as mean ± SEM, did not change with passage of time.
Suppose that we consider the memory active when the subject is using the tool, and inactive during the period that the subject is not holding the handle of the tool (i.e., time between training and testing). The effect of passage of time is a measure of decay of the inactive memory. To estimate the sensitivity of the inactive memory to passage of time, we used a boot-strapping procedure to select at random one subject from each group in Figure 2B and fitted their data to a single exponential. The result suggested that the strength of the inactive memory decayed with a time constant of 18.5 ± 3.89 h (SEM).
Once the test trials began, the activated memory decayed as a function of time/trial. Intriguingly, this resistance did not appear to change with the passage of time. Figure 2C illustrates the distribution of time constants when a single exponential was fitted to the force output of each subject during the error-clamp trials in the test period. A one-way ANOVA found no significant effect of passage of time on sensitivity to trial (F(5,50) = 0.562, p = 0.729). A Tukey's post hoc analysis revealed no significant differences between individual groups.
When the memory was reactivated (Fig. 2C), trial-dependent loss was distributed between 1 and 3% per trial, which translates into a time constant of 200–600 s. In contrast, when the memory was inactive, the time constant was ∼18.5 h (Fig. 2B). Therefore, decay of the activated memory was about two orders of magnitude faster than the inactive memory.
In summary, after a long period of training in A, the resulting inactive memory decayed when the subject was not in the context of the task with a time constant of 18.5 h. Once the memory was reactivated in the test session, it further decayed as a function of time/trial with a loss of between 1 and 3% per trial (time constant of 200–600 s). This sensitivity of the reactivated memory did not change, despite the fact that time caused a decay in the inactive memory.
A+B training did not alter the memory of A, but produced a competing memory of B
After 384 trials in A, subjects were exposed to 20 trials of B (experiment 2). The small number of trials in B was sufficient to drop the performance to naive levels (Fig. 3A), producing an apparent extinction. Subjects then waited 0, 2, or 10 min, or 1, 6, or 24 h. In all but the 0 min group, subjects released the handle. In the 0, 2, and 10 min groups, the subjects remained seated in front of the robot. The subjects in other groups left the room and returned at their scheduled time. After return, they were asked to hold the handle and reach to the target. The testing was always in error-clamp trials.
The patterns of spontaneous recovery. A, In the A+B group, the 20 B trials were sufficient to bring the motor output to baseline, as assayed at 0 or 2 min. Yet, in the subsequent error-clamp trials, output spontaneously rose and precisely converged to the output of group A. B, At 10 min after A+B acquisition and for all subsequent time intervals, motor output no longer started at zero and no longer rose during the error-clamp trials. All data points are mean ± SEM. Bin size is one trial.
By comparing performance of subjects who learned A with subjects who learned A+B we can answer a fundamental question: does learning of B produce any destructive effects in the memory of A? At 0 or 2 min after A+B training, the force output for the first error-clamp trial was at zero (Fig. 3A). Trial by trial, the force output increased and converged to the output observed in subjects who had only learned A (Fig. 3A). This rise is a form of spontaneous recovery and has been reported previously (Smith et al., 2006). However, the crucial new result is that the rise is so strong as to precisely converge to the falling memory of A. A repeated-measures ANOVA on the last four trials of the test session for the 0 min A and A+B groups revealed no significant differences (F(1,16) = 0.186, p = 0.672). Similarly, there were no significant differences in the last four trials for the 2 min A and A+B groups (F(1,13) = 0.052, p = 0.824). If learning of B produced any destructive effects on A, this rise would have fallen short of A. Therefore, the data for the 0 and 2 min groups suggested that the long-term training in A produced a memory that was essentially unaffected by the brief training in B, despite the fact that this brief training brought performance down to baseline.
A repeated-measures ANOVA on the last four trials of the test session revealed no significant differences between the A and A+B groups for the 6 h (F(1,18) = 0.686, p = 0.418) and 24 h groups, (F(1,16) = 1.074, p = 0.315). The A and A+B group were different for the 10 min (F(1,14) = 5.37, p = 0.036) and 1 h (F(1,18) = 5.69, p = 0.028) wait periods.
The 20 trials in B produced a memory that competed with A, effectively learning a force that was equal in magnitude but opposite to A. However, at 0 or 2 min after acquisition, the memory of B washed out to zero within 30 error-clamp trials, whereas the same 30 trials after A did not wash out A (Fig. 2A). Therefore, whereas A decayed slowly as a function of time/trial, at 0 or 2 min after acquisition, B decayed nearly completely within 30 trials. The memory of B was extremely fragile at 0 and 2 min after it was acquired.
B starts out as a fast motor memory, but with time becomes slow-like
If at 0 or 2 min after acquisition, B starts out with a magnitude equal to A but decays away within 30 trials (these trials took a total of 1.85 ± 0.02 min), does passage of time alter the characteristics of this memory? That is, at 10 min and beyond, what is the strength of the B memory and what is its sensitivity to trial? We found that as we waited a longer period of time after A+B training, the motor output changed considerably. Whereas at 0 and 2 min after A+B training the motor output rose from zero to meet A, at 10 min and beyond the force output started out significantly higher than zero (Fig. 3B): first trial was not significantly different from zero at 0 min (t test, p = 0.595) and at 2 min (t test, p = 0.451). However, force output was significantly higher than zero for all other test periods (t test, p < 0.005). Whereas at 0 and 2 min after acquisition A+B converged onto A, at 10 min and 1 h, performances no longer converged: a repeated-measures ANOVA on the last four trials of the test session revealed a significant difference between A and A+B groups at 10 min (F(1,14) = 5.37, p = 0.036) and 1 h (F(1,18) = 5.69, p = 0.028). Therefore, the patterns of “spontaneous recovery” changed drastically within minutes after A+B training.
The change in the pattern of spontaneous recovery could not have been attributable to the effects of time on memory of A, because data in experiment 1 suggested that fragility of A did not change significantly with passage of time. It is likely that the changes were attributable to effects of time on memory of B. To estimate how the passage of time affected the magnitude and sensitivity of B, we used a bootstrapping procedure to estimate B at each time point and each trial after A+B training. We selected a subject at random from each of the A and A+B groups and subtracted their force output for each trial, i.e., B̂ = (A + B) − A (see Materials and Methods).
The results, plotted in Figure 4A, illustrate two points. First, when subjects were away from the task (what we termed inactive memory), passage of time caused forgetting. However, the rate of this forgetting was much faster for memory of B than memory of A (Fig. 4B). For example, the two memories appeared to be equal in magnitude at 0 and 2 min, yet B was significantly smaller at 10 min, 1 h, and 6 h (0 min: F(1,57) = 0.009, p = 0.924; 2 min: F(1,55) = 0.176, p = 0.676; 10 min: F(1,56) = 30.692, p < 0.05; 1 h: F(1,60) = 58.244, p < 0.05; 6 h: F(1,59) = 11.401, p < 0.05). Second, when subjects were brought back to the task (what we termed reactivated memory), immediately after acquisition the reactivated memory of B was far more fragile than A, but with passage of time it gained stability. For example, at 0 and 2 min after acquisition, the reactivated memory of B displayed a decay (loss per trial) that was five times the rate of decay of memory of A (Fig. 4C). However, at 10 min, the memory of B had a decay rate that was no different from A. This increased resistance was maintained for as long as the memory of B could be assayed (6 h). Therefore, memory of B started out fragile (large loss per trial), but then appeared to be transformed with passage of time to a more stable memory (small loss per trial).
The effect of reactivation of memory as a function of time since acquisition. A, Force output as a function of trial after acquisition of A versus after acquisition of B. To estimate the memory of B, we used a boot-strapping procedure to subtract performance of the A group from the A+B group. The memory of B is a result of only 20 training trials. This memory at 0 and 2 min after acquisition is fragile in the sense that when activated in error-clamp trials it decays rapidly to baseline. However, at 10 min and beyond it acquires an increased resistance to trial. By 24 h, the memory of B is no longer measurable. B, Magnitude of memories of A and B as a function of time since acquisition. Each bar plot represents the initial force output (bin size is two trials) averaged across subjects in each subgroup. Error bars are SEM. With passage of time, the memory of A declines gradually, whereas memory of B declines rapidly. C, Fragility of the memory. Force output as a function of trial for each subject was fitted to a single exponential (shown as mean ± SEM). Within minutes after acquisition, the memory of B became more resistant to trial.
Together, these two observations suggest that at 0 and 2 min after acquisition, the reactivated memory of B was fragile, because it decayed rapidly as a function of time/trial. However, if that memory was not activated, it lost much of its content within 10 min, but the remaining content gained stability.
A model of motor memory consolidation
We observed that (1) after long-term training in A, brief exposure to B produced apparent extinction, but at 0 and 2 min error-clamp trials produced spontaneous recovery; (2) the rate of decay of an active memory (i.e., reaching with the robot) was two orders of magnitude greater than that of an inactive memory (i.e., simple passage without reaching with the robot); and (3) in the inactive state, over time, the initially fast memory of B was partially transformed into a more stable memory. The first observation is a fundamental prediction of a multistate memory model (Smith et al., 2006; Kording et al., 2007). This model is illustrated by the graphical representation in Figure 5A: the learner assumes that the environment has multiple states (here represented by a fast and a slow state), and that his/her observations are a sum of these states. However, to account for the second and third observations, we need to make a distinction between the condition in which the learner can observe the environment (learning from the environment through observations, active memory) and the condition in which the learner is outside the context of the environment (passage of time, inactive memory).
Active/inactive multistate model of motor learning. A, A generative model, describing the learner's hypothesis about the task. As in previous generative models (Kording et al., 2007), the learner assumes that the environment is composed of states with multiple timescales (two timescales are considered here, labeled fast and slow), and the problem of learning is state estimation. In the active condition (left), the learner can sample the environment (i.e., subject is using the tool) and learns from observation. In the inactive condition (right), the learner is not in the context of the task and therefore cannot sample the environment. Passage of time affects the two conditions differently, as specified by the state transition matrices Aa and Ai, referring to the active and inactive conditions, respectively. We assumed that Aa is diagonal but Ai allows for transformation of a portion of the fast states into the slow states. In total, there were five parameters in this model (components of the two matrices). B, Output of the model compared with the measured data. Aa = [0.984, 0; 0, 0.855] and AI = [0.9999, 0.0043; 0, 0.983]. Error bars indicate SEM.
The simplest generative model that best accounted for our data had a state transition matrix in the active state that was diagonal, Aa = [0.984, 0; 0, 0.855], and a nondiagonal transition matrix in the inactive state, Ai = [0.9999, 0.0043; 0, 0.983], where x(t + Δ) = Ax(t) + ε and Δ = 6 s (typical intertrial interval in the test period). The nondiagonal transition matrix in the inactive condition is essential to reproduce the result that, with passage of time, the initially fast memory of B gains stability. However, the change in the model from the active to inactive condition is a bit unsettling because it raises the possibility that, even in the activated state, the fast memory states may be transformed into slow states. Our experiment had no power to detect this possible transformation during the active state of the memories, and we therefore stayed with the simplest approach and kept the transition matrix in the active state diagonal. However, we noted the possibility of the transformation of the fast to slow states in the active condition with a dotted arrow in Figure 5A.
In Figure 5B, the data are replotted with the resulting curves from the model (we estimated the goodness of fit of the model to all of the data with an omnibus R2 = 0.83, p < 0.001). The model provided a reasonable fit to the data at 0, 2, and 10 min and 1 and 6 h groups. However, forgetting rates for the inactive state were smaller than expected for the 24 h group. This raises the possibility that sleep may play a role in further stabilizing the motor memories, something that would be consistent with some (Huber et al., 2004) but not all previous results in reach adaptation (Donchin et al., 2002).
Discussion
We trained people to reach with tool A until performance approached an asymptote (∼400 trials in a single direction). We then quantified properties of this memory as it was reactivated in error-clamp trials, i.e., trials in which error-dependent learning was minimized. We made three observations. (1) With passage of time (24 h), memory of A gradually declined, but when reactivated it remained resistant to trial. This resistance did not change with passage of time. (2) When dynamics of the tool were suddenly changed to B for 20 additional trials, the large performance errors brought the motor output back to baseline, demonstrating an apparent unlearning or extinction. When we assayed performance at 0 or 2 min after A+B training, we found that within 30 error-clamp trials (∼2 min) the motor output increased from baseline until it converged to that of subjects who had trained in A only. The magnitude of the spontaneous recovery suggested that the large performance errors introduced by B could not have produced any unlearning of A. Rather, it installed a memory that competed with A. This competing memory, termed “memory of B,” was initially fragile in the sense that when reactivated, it decayed to zero within 30 trials. (3) Within 10 min after A+B training, reactivation produced a changed pattern of spontaneous recovery. The nature of these changes suggested that with the passage of time, the memory of B gained stability.
To encapsulate these results, we suggested a new model of motor memory (Fig. 5A). Acquisition of a motor memory not only depends on processes that have multiple timescales, as has been proposed previously (Smith et al., 2006; Kording et al., 2007), but that during the inactive state, i.e., when time passes outside the context of the task, the fast states partially transform to slower states.
Our conclusions depend crucially on the assumption that memory of B can be assayed through mathematical subtraction of A from A+B. This assumption was tested at 0 and 2 min after A+B training, in which we observed that performance of A+B group rose from zero and converged to A. The convergence is a strong hint that there is superposition of the two memories. The convergence could not occur unless learning of B left memory of A virtually untouched.
If we now hypothesize that performance errors can produce unlearning in only the fast memory and not the slow memory, the resulting theory may explain a large set of apparently conflicting results. We had initially reported that if task A was followed by task B, subjects exhibited naive performance when retested on task A some days later (Shadmehr and Brashers-Krug, 1997). However, inserting a few hours between tasks A and B allowed for recall of A. Caithness et al. (2004) and Mattar and Ostry (2007) performed similar experiments but observed that, despite 24 h between A and B, there was no recall of A. Overduin et al. (2006) reproduced both results and explained that the key difference was the presence of “catch trials,” i.e., trials in which the dynamics were unexpectedly returned to null, causing occasional large errors, especially near the end of training. The important questions are why should learning of A with catch trials make it more vulnerable to B at 0 min than 6 or 24 h? Why should removal of catch trials fundamentally alter this time dependency?
Our theory explains that learning with catch trials produces a memory of A that contains a significant amount of the fast component (because of the large errors that catch trials produce). Learning of B probably destroys the fast memory at 0 min, but with passage of time, the fast memory of A is transformed to slow, becoming less vulnerable to B. This explains results of Brashers-Krug et al. (1996) and Shadmehr and Brashers-Krug (1997), because both studies used catch trials. However, if learning of A is without catch trials, then extended training in A produces a memory that is slow, making it resistant to B at all times. In that scenario, learning of B always installs a competing memory. No matter when the task is assayed, both memories are present and will compete, explaining the results of Caithness et al. (2004), Mattar and Ostry (2007), and Overduin et al. (2006). Indeed, Krakauer et al. (2005) showed that when training is without catch trials, there is robust recall of A if subjects are provided with washout trials before testing of recall. As they noted, naive performance is not because A is gone, but because both A and B are present and competing.
Our theory also explains why a relatively small amount of training in A produces a memory that is vulnerable to B at 0 min but less vulnerable at 24 h, yet a longer amount of practice produces a memory that is invulnerable both at 5 min and at 24 h (Krakauer et al., 2005). A small amount of training installs a fast memory, making it vulnerable to B at 5 min, whereas longer training installs a slow memory, making it invulnerable to B at all times.
The fundamental prediction of our theory is that once training installs a slow motor memory, it may not be possible to unlearn it through performance errors. Our current understanding of the biology of memory supports this idea. Retention of a motor skill (Bracha et al., 1998; Luft et al., 2004a,b) requires synthesis of new proteins. If unlearning is the formation of a new memory trace and not erasure of the original learning, then it should also require de novo protein synthesis for its long-term retention. A number of paradigms have examined this question. In inhibitory avoidance training (Vianna et al., 2001), conditioned taste aversion (Berman and Dudai 2001, Burgos-Robles et al., 2007), and classical conditioning of eyelid response (Inda et al., 2005), unlearning is termed “extinction.” In all cases, retention of the extinction memory (what we termed memory of B here) requires new protein synthesis, suggesting that unlearning is spawning of a new, competing memory, not erasure of the existing memory.
Our finding that memory of B is a fast memory that over time is transformed into a slow, stable memory closely parallels the conclusions of studies that have attempted to disrupt the extinction memory. Berman and Dudai (2001) found that whereas injection of anisomycin at 10 min after extinction training disrupted later recall of the extinction memory, at 30 min the drug had no effect on consolidation of the extinction memory. From our point of view, the extinction memory is analogous to the memory of B, which starts as a fast, vulnerable memory. Berman and Dudai (2001) found that within minutes after acquisition, this extinction memory gains resistance. In our data, we see that within 10 min after exposure to B, the resistance of memory of B to trial has more than doubled.
Because all of our conclusions are based on measurements in error-clamp trials, it is important to consider how this method of assaying memory differs from traditional approaches in which testing is either via aftereffects, or in the same type of trials as the original learning. In error-clamp trials, we can measure both the magnitude of the reactivated memory via motor output and its resistance to change via the derivative of the motor output with respect to trials. Whereas in a catch trial we would be assaying an aftereffect, in an error-clamp trial we are measuring the aftereffect but minimizing the error-dependent change that is caused by the aftereffect. In the traditional approach, retention is assayed via “savings” in which trials provide error feedback, resulting in faster relearning. Therefore, the motor output in error-clamp trials can be viewed as the bias from which the relearning curve would start in the traditional savings experiment.
Consolidation of motor memory has now been investigated in the reaching paradigm for >10 years, with sometimes conflicting results. Our results here provide a way to account for much of that data. Motor memory can exist in two functional states, a fast and a slow state. With passage of time away from the task, some of the fast state may be transformed into a slow state. Furthermore, once a slow motor memory has been established, large performance errors may not be able to change it.
Is the neural circuit for the fast and slow motor memories the same? We recently reported that brief disruption of M1 during adaptation did not affect rates of adaptation, but produced a memory that decayed more rapidly than normal (Hadipour-Niktarash et al., 2007). It is possible that M1 has a particularly important role for the slow human motor memory (Richardson et al., 2006), although the question of whether the neural basis of the fast memory is distinct has yet to be determined.
It appears that a typical long period of training (without catch trials) is sufficient to produce a slow motor memory, and subsequent performance errors do not erase his memory, but install a competing memory. The general implication of our work is that once a motor skill has been well learned, all additional learning may be instantiation of competing memories. This suggests that in biology, the cost of unlearning may be much higher than learning.
Footnotes
-
This work was supported by National Institutes of Health Grant NS37422. We are grateful for stimulating and insightful comments from Dr. John Krakauer.
- Correspondence should be addressed to Sarah E. Criscimagna-Hemminger, Johns Hopkins University School of Medicine, 720 Rutland Avenue, 416 Traylor Building, Baltimore, MD 21205. sarah.hemminger{at}jhu.edu
