Flexible Control of Safety Margins for Action Based on Environmental Variability

Effects of increased environmental variability on GFs and MFs

While previous work has focused on the adaptive responses of both GFs and MFs to changes in the expected value of environmental dynamics (Flanagan and Wing, 1993, 1997; Shadmehr and Mussa-Ivaldi, 1994; Wolpert and Kawato, 1998; Wolpert et al., 1998; Kawato, 1999; Krakauer et al., 1999; Smith et al., 2006; Danion and Sarlegna, 2007; Sing et al., 2009; Yokoi et al., 2011; Yousif and Diedrichsen, 2012), here we hypothesize that safety margins, which apply to GF but not MF control, are specified based on the variability rather than the expected value of environmental dynamics, leading to fundamental differences between the control of MFs and GFs. In particular, this would predict the control of GFs, but not MFs, to be strongly modulated by environmental variability.

To hold steady against a load, the motor system must apply an MF equal and opposite to the LF: a weaker MF would result in acceleration in the direction of the LF, whereas a stronger MF would accelerate the object against the load. Accordingly, a strategy that yields efficient MF control, minimizing the risk of accelerating toward either direction, would be to exert an MF that continually approximates the expected value of the LF during the course of each movement. This idea is in line with previous work showing that under variable dynamic environments, MFs adapt to the expected value of the perturbation (Scheidt et al., 2001; Takahashi et al., 2001) and that under variable visuomotor transformations, movements adapt to the expected value of the transformation (Burge et al., 2008; Saijo and Gomi, 2012), independent of the amount of variability in both cases.

To apply this strategy to GF control, however, would mean that the planned GF would often be insufficient to maintain grasp because producing a GF equal to the expected value of the required GF (i.e., the expected value of the orthogonal LF divided by the coefficient of friction, η) would result in compromised grasp whenever the required GF exceeds its expected value. This exposes a key dichotomy between the control of GFs and MFs: whereas undercompensating and overcompensating MFs both result in motion errors, undercompensating GFs result in grasp errors while overcompensating GFs do not (Fig. 1). A strategy for improved GF control would thus reduce the risk of slip by programming GFs to be greater than the expected value of the required GF (Johansson and Westling, 1984, 1988; Cole and Johansson, 1993). The specification of the size of this GF safety margin can be viewed as a tradeoff between the increased effort associated a higher margin and the reduced risk of slip that it affords. Here we hypothesize that when GFs are not limited by strength or fatigue, the safety margin for GF control is specified based on a fixed level of statistical confidence. Figure 1 illustrates that this hypothesis predicts higher safety margins as uncertainty increases, because the risk of slip at a fixed GF level would increase with higher uncertainty (Fig. 1E,F).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

GFs and MFs in variable environments. A, Diagram of the grasping device (left) and top view of the experiment setup. B, Top view of the grasped object illustrating the forces exerted on and by the subject during movement. Rightward MFs can compensate leftward LFs and compressive GFs maintain grasp. C, D, MF compensation in a low-variability (C) or a high-variability (D) environment, both with the same expected value μ_LF. The optimal MF level is equal to the expected value of the perturbation, which minimizes the amount of overcompensation and undercompensation in both cases. E, F, GF control in the same environments. The GF required to avoid slip has a mean value of the mean load divided by the coefficient of friction (μ_LF/η) in both cases. However, note that, in 50% of the cases, using this expected value (vertical dashed line) would result in slip whenever the required GF exceeded this value. In a low-variability environment (E), using a GF (GF*) that incorporates a small safety margin above this expected value ensures that the chance of slippage is low (dark green area under the bell curve). However, in a high-variability environment (F), a larger safety margin would be required to maintain a low slip probability. This leads to the hypothesis that GF control includes a safety margin that scales with environmental variability.

To investigate the fixed statistical confidence hypothesis, we tested the prediction that the safety margin for GF control would be modulated by environmental variability. We designed a task in which subjects were exposed to environments with different amounts of variability. Subjects performed rapid point-to-point reaching movements while grasping a small object with physical dynamics that changed randomly from one trial to the next, with the amplitude of these random changes systematically varying from one block to the next, as illustrated in Figure 2A–C. In this task, the direction of the LF was orthogonal to the grasp axis so that GF levels, rather than finger configuration, determined whether the object slipped (Fig. 1A,B). In addition to specifying the orientation of grasp relative to the LF, we restricted the possible grasp locations, as freedom to vary the finger configuration can influence the applied GF (Fu et al., 2010). Thus, subjects could modulate GF to prevent slip and independently modulate MF to control the motion of the object. After a practice period of 150 trials in which participants were familiarized with the basic task, they were exposed to 16 50-trial blocks with different levels of variability. During each block, the strength (b) of a viscous curl FF environment (Eq. 1) was varied from one trial to the next by drawing b from a Gaussian distribution with mean zero and SD σ. σ was fixed within each block at one of four possible values: 0, 1.8, 3.6, and 5.4 Ns/m, and four blocks were presented for each σ value during the 16-block session, in a randomized order. To give subjects time to adapt to each variability level, we focused our analysis on the last 70% of the data from each block (see Materials and Methods). It should be noted, however, that the adaptation was fairly rapid and so other choices for the window size yield similar results.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

GFs, but not MFs, scale with the level of environmental variability. A, Design of Experiment 1a. Following a baseline period (data not shown) participants were exposed to environments that randomly changed in variability from one 50-trial block to the next, as illustrated. Specifically, the viscosity (b) of a velocity-dependent FF perturbation was drawn from a normal distribution with mean zero and SD (σ) that assumed one of four values (see Materials and Methods). Note that the zero-σ variability blocks contain some nonzero values for b; these occur on EC trials, on which no predetermined FF is applied. Instead, the manipulandum applies the forces required to maintain a straight handpath. Using these forces, we estimated the effective b that the robot applied in these trials (see Materials and Methods). B, SD (σ) of the values of b for each block, illustrating the four distinct variability levels. C, Example FFs (blue arrows) presented to subjects on individual movements (black arrows) for different values of b. Note that a force offset F₀ prevents even a strongly negative b from reversing the direction of the net load force. D, Mean GF profiles for each of the four variability levels, showing increasingly stronger forces as σ increases. E, Changes in GF for the three nonzero variability levels compared with the zero-σ variability level. Error bars indicate SEM across subjects. The right panel shows the average amplitude of these changes for a short time window (±50 ms) around the peak speed point. F, Baseline-referenced adaptation levels (mean ± SEM) for the GF profiles in E, showing significant increases in GF levels with variability. *p < 0.05, **p < 0.01, ***p < 0.001. White asterisks indicate comparisons with the baseline level. Note how the separation between different variability groups is even more clear than in the right panel of E, despite the fact that here adaptation levels are based on the transient changes in GF during movement, and thus these comparisons do not benefit from the separation between premovement levels suggested in E. G–I, Same as D–F but for MF. Note that, similar to E, H shows error bars indicating SEM; however, they are small enough that they are largely obscured by line width and the fluctuations of the MF on the time axis.

Critically, we applied a leftward static force offset of 4 N (Eq. 1, F₀) throughout the experiment on top of the time-varying FF perturbation to ensure that the net direction of the force would always remain leftward even for negative values of b as shown in Figure 2C. This ensured that negative values of b required smaller GFs than baseline to prevent slip, whereas positive values of b required higher GFs. Without this offset, negative values of b would result in rightward forces, whereas positive values of b would generate leftward forces, with increased GF required to maintain grasp in both cases. Moreover, since load-increasing and load-decreasing values of b were equally likely, any symmetric positive versus negative trial-to-trial learning (Scheidt et al., 2001; Takahashi et al., 2001; Braun et al., 2009) would balance out. As we estimated variability-driven responses based on the average responses during each variability block, the net effect of trial-to-trial learning predicted by current models of motor adaptation (Thoroughman and Shadmehr, 2000; Donchin et al., 2003; Smith et al., 2006; Joiner and Smith, 2008; Lee and Schweighofer, 2009; Sing et al., 2009; Sing and Smith, 2010) should be near zero.

Increased environmental variability leads to increases in GF control but not MF control

We found that the average GF profiles displayed in each block systematically increased with variability. This is evident in both the raw GF profiles (Fig. 2D) and baseline-subtracted GF profiles, which show the variability-induced changes from the zero-variability baseline blocks (Fig. 2E). The pattern of variability-induced GF change was similar in shape for all nonzero variability levels, but its amplitude grew systematically as σ increased. Changes in GF began at movement onset and peaked near the peak velocity point, when the effect of the LF from the viscous curl FF perturbation would be the strongest (due to the velocity-dependent FF perturbation). The amplitude of these GF changes scaled with the variability level, peaking at 1.5 N for the high-variability blocks (Fig. 2E).

To quantify the magnitude of variability-induced changes in both GF and MF, we computed adaptation coefficients for each movement for both GF and MF. These adaptation coefficients were based on linear regressions of the corresponding GF or MF force profile onto the LF profile (Scheidt et al., 2000; Smith et al., 2006; Joiner and Smith, 2008; Gonzalez Castro et al., 2011, 2014; Joiner et al., 2011; Yousif and Diedrichsen, 2012; Sing et al., 2013; Wu et al., 2014; see Materials and Methods). We found that subjects exhibited significantly greater GF adaptation in the low-σ, medium-σ, and high-σ blocks compared with the zero-σ blocks (p < 0.001 in all three cases, one-tailed paired t tests; Fig. 2F). Moreover, the GF adaptation was greater in both the medium-σ and high-σ blocks than the low-σ blocks (p < 0.01 in both cases), and greater in the high-σ blocks than the medium-σ blocks (p < 0.05, one-tailed paired t tests).

In contrast, we found no consistent changes in MF control with variability. The subject-averaged MF profiles were essentially identical for different amounts of variability (Fig. 2G), and the variability-related changes in MF shown in Figure 2H were small. Correspondingly, the MF adaptation coefficients were not significantly different from zero (Fig. 2I). In particular, note that the high-σ environments display MF levels that are essentially the same as the zero-σ environment (p > 0.48, two-tailed paired t test; Fig. 2G,H). These findings indicate that when the mean dynamics are held constant, GF but not MF control systematically depends on the variability of environmental dynamics.

GF control is even more sensitive to changes in the LF variability than the mean LF

Since previous work maintained that GF is controlled based on the expected LF, we wanted to know how the strength of the dependence of GF on LF variability observed in Experiment 1a compares to the dependence of GF on the expected value of the LF. To that end, we estimated the sensitivity (i.e., the slope of the relationship) that GF control displays with respect to both the SD (σ_LF) and the mean (μ_LF) of the LF. Across the four different variability levels (zero, low, medium, and high) in Experiment 1a, we found the relationship between subject-averaged GF levels and σ_LF to be approximately linear, with a strong positive correlation between GF levels and σ_LF (R² > 0.99; slope = 1.02; Fig. 3B, squares and solid line). When we estimated the GF/σ_LF sensitivity for all 11 individual participants, we found positive slopes for all 11, with an average value of 1.02 ± 0.21 (mean ± SEM across subjects, p < 0.001, one-tailed paired t test; Fig. 3B,D). Note that these slopes are all dimensionless because the levels of GF adaptation and σ_LF are both quantified in the units of FF strength (newton-seconds per meter).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

GFs are threefold more sensitive to changes in the variability than changes in the expected value of environmental dynamics. A, Comparison between the design of Experiments 1a, 1b, 2a, 2b, and 1c/2c. Examples of training schedules are shown. The duration of variability blocks in Experiment 1b is double that of Experiment 1a. Experiments 2a and 2b present a step increase or decrease of the FF, respectively. Experiment 1c/2c begins with a step increase in the FF, which persists for 240 trials (Exp. 2c) and is then followed by the addition of a variable FF component (Exp. 1c) in an analogous fashion to Experiment 1a. Note that, for clarity given the small size of the plots in this panel, the effective FF strength b during EC trials is not shown, in contrast to Figure 2A. B, Baseline-referenced adaptation levels (mean ± SEM) for GF for each of the four types of variability blocks and each experiment (1a, 1b, and 1c) plotted against the corresponding increases in the SD of b with respect to the baseline. The black lines indicate linear fits to the data (thin dashed lines, in order of increasing density: Exps. 1a, 1b, and 1c; thick solid line: overall data from all 3 experiments). The three groups display similar slopes for GF versus σ and indicate that GF is sensitive to environmental variability. C, Baseline-referenced adaptation levels (mean ± SEM) for GF for Experiments 2a, 2b, and 2c, plotted against the corresponding changes in the mean of b with respect to the baseline. The colored lines indicate linear fits to the data for each experiment (the black line indicates the linear fit to the overall data from all 3 experiments). Note that these fits lie on top of each other, indicating that the sensitivity of GF to the μ is linear over the range of changes in μ that we examined. The aspect ratio is the same as in B, illustrating that the sensitivity of GF to μ (i.e., the slope of the fits) is substantially smaller than the sensitivity of GF to σ (the slope of the fits in B). D, GFs show significantly greater sensitivity to the SD, σ, compared with the mean, μ, of environmental dynamics. In the left panel, we calculate sensitivities to σ based on the slopes in B. In the right panel, we calculate sensitivities based on the high-variability blocks from Experiments 1 and 1c, where the magnitudes of the baseline-referenced changes in σ were similar to magnitude of the baseline-referenced changes in μ from Experiment 2c (3.75 Ns/m). **p < 0.01, ***p < 0.001. E–G, Manipulatory forces are not sensitive to changes in the SD, but only to the mean of environmental dynamics. The format of E–G parallels the format of B–D.

To put the amplitude of the GF/σ_LF sensitivity into perspective, we estimated the GF/μ_LF sensitivity by measuring GF responses to systematic changes in the LF environment. To that end, we presented subjects with a deterministic step change in the environmental dynamics, changing the viscosity coefficient (b) of a viscous curl FF from zero to a constant value of either +7.5 Ns/m (Exp. 2a, leftward, n = 18) or −7.5 Ns/m (Exp. 2b, rightward, n = 18) for 240 trials following a 160-trial baseline period with b = 0 (Fig. 3A). Note that, like in Experiment 1a, a leftward static offset force was present throughout to ensure that LFs were reduced in magnitude rather than reversed in direction during rightward −FF trials (Fig. 2C). We found highly significant and nearly symmetric changes in GF during asymptotic adaptation (last 60 trials) in Experiments 2a and 2b (2.46 ± 0.51 Ns/m, p = 0.0002 and −2.36 ± 0.74 Ns/m, p = 0.0055, respectively; two-tailed paired t tests; Fig. 3C). These changes in GF correspond to sensitivity estimates (slopes) of 0.33 ± 0.07 and 0.31 ± 0.10 for individuals in Experiments 2a and 2b, respectively, and 0.32 ± 0.06 for the combined data (Fig. 3C,D). Remarkably, these values are threefold lower than the sensitivity of GF to changes in variability observed in Experiment 1a (1.01 ± 0.19; Fig. 3, compare B, C), indicating LF variability to be a stronger determinant of GF control than mean LF.

We conducted three additional experiments to determine the robustness of the finding that GF control is more sensitive to σ_LF than to μ_LF. We first checked whether the stability of GF adaptation in the 50-trial blocks from Experiment 1a was sufficient for accurate GF/σ_LF sensitivity estimates. In Experiment 1b (Fig. 3A), variability was modulated from one block to the next like in Experiment 1a, but with longer (100 vs 50 trials) and correspondingly fewer (8 vs 16) blocks. Experiment 1b yielded sensitivity estimates of 1.15 ± 0.17 across individuals (p < 10⁻⁶, one-tailed paired t test), similar to the ones obtained from Experiment 1a (Fig. 3B,D), suggesting GF/σ_LF sensitivity estimates to be stable for blocks of 50 trials or greater.

In a second additional experiment (Exp. 2c), we checked whether the GF/μ_LF sensitivity estimates from Experiments 2a and 2b would be consistent with the sensitivity observed from a smaller change in μ_LF. Because Experiments 2a and 2b were based on changes in μ_LF with a magnitude (7.5 Ns/m) that was greater than the σ_LF changes examined in Experiments 1a or 1b (≤4.1 Ns/m), examining a smaller change in μ_LF allowed us to compare GF/σ_LF and GF/μ_LF sensitivity estimates using similar displacements for σ_LF and μ_LF. This also allowed us to examine the linearity of the relationship between GF and μ_LF. Experiment 2c mirrored 2a, except that the FF amplitude (b) was halved (+3.75 vs +7.5 Ns/m). The results showed an asymptotic adaptation level in the last 60 trials of the 240-trial FF exposure that was almost exactly half of that observed in Experiment 2a (1.16 ± 0.42 vs 2.46 ± 0.51 Ns/m). This corresponds to nearly identical GF/μ_LF sensitivity estimates for Experiments 2a, 2b, and 2c (0.33 ± 0.07, 0.31 ± 0.10, and 0.31 ± 0.11 Ns/m, respectively), suggesting a fixed level of GF/μ_LF sensitivity up to at least 7.5 Ns/m, in line with a linear relationship between GF and μ_LF. Comparison of the GF/σ_LF sensitivity estimates from the high-σ blocks in Experiment 1a and the GF/μ_LF sensitivity estimate from 2c revealed the former to be significantly greater than the latter (1.04 ± 0.21 vs 0.31 ± 0.11, p = 0.0043, one-tailed unpaired t test; Fig. 3D, right). This is in line with the idea that GF is substantially more sensitive to the variability than the expected value of the LF environment, even when displacements for σ_LF and μ_LF are similar (4.1 and 3.75 Ns/m, respectively).

In a third additional experiment (Exp. 1c), we checked whether the GF/σ_LF sensitivity estimates from Experiments 1a and 1b would be consistent with the sensitivity observed when GF variability was centered around a nonzero mean. We performed this experiment immediately after Experiment 2c in the same session with the same participants, which also enabled us to compare GF/σ_LF and GF/μ_LF sensitivity estimates in the same participants. Note that we used a value of +3.75 Ns/m for μ_LF in 1c to maintain continuity with the FF training in 2c. Again, we found GF to be more sensitive to σ_LF than μ_LF (0.78 ± 0.11 vs 0.31 ± 0.11, p = 0.0013, one-tailed paired t test; Fig. 3D, right), consistent with our previous findings. Also consistent with our previous findings, we found the GF/σ_LF sensitivity in Experiment 1c to be similar to that from Experiments 1a and 1b in that an ANOVA revealed no significant differences in GF/σ_LF sensitivity across experiments (F_(2,48) = 1.46, p = 0.24).

Analogous analysis of the MF data from Experiments 2a, 2b, and 2c revealed highly significant asymptotic adaptation of MF levels to changes in μ_LF [Exp. 2a (b = +7.5 Ns/m): 6.66 ± 0.27 Ns/m; Exp. 2b (b = −7.5 Ns/m): −7.14 ± 0.15 Ns/m; Exp. 2c (b = +3.75 Ns/m): 3.15 ± 0.25 Ns/m, p < 10⁻⁶ in all three cases, two-tailed paired t tests; Fig. 3F]. Note the nearly symmetric increases and decreases in the adaptive responses for Experiments 2a and 2b. These MF levels correspond to MF/μ_LF sensitivity estimates of 0.89 ± 0.04, 0.95 ± 0.02, and 0.84 ± 0.07, for Experiments 2a, 2b, and 2c, respectively (Fig. 3G). The similarity between these estimates corresponds to the nearly linear relationship between MF and μ_LF observed in Figure 3F. In contrast, the overall MF/σ_LF sensitivity we observed in Experiments 1a, 1b, and 1c was much smaller and not different from zero (0.002 ± 0.020, p > 0.9, two-tailed paired t test; Fig. 3E,G) in line with the lack of MF modulation observed in Figure 2G–I when variability was increased.

GF control maintains a safety margin based on statistical confidence

Remarkably, our findings show not only that GF control is sensitive to the SD of environmental dynamics, but that this sensitivity is higher (p < 10⁻⁶, two-tailed unpaired t test between Exps. 1a, 1b, and 1c vs Exps. 2a, 2b, and 2c) than the sensitivity to the environment's mean dynamics as illustrated in Figure 3 (2.82 ± 0.27-fold higher, estimated using bootstrap when combining the data from Exps. 1a, 1b, 1c, 2a, 2b, and 2c). Since the GF/σ_LF sensitivity acts to scale the GF safety margin against slip by the LF variability, the ratio between the GF/μ_LF sensitivity and the GF/σ_LF sensitivity can be used to determine the statistical confidence associated with the observed safety margin (see Materials and Methods). In particular, the ratio of 2.82 ± 0.27 that we observed corresponds to GF control in a variable environment that is 2.82 SDs above that required for the expected value of dynamics. For normally distributed environmental perturbations, like the ones administered in Experiments 1a, 1b, and 1c, this corresponds to a z-score of 2.82 ± 0.27, which amounts to >99% confidence interval against slip. Unfortunately, the precise frequency of slippage incidents is difficult to measure experimentally as these incidents usually result in microslips, which subjects rarely notice and often correct almost instantaneously (Johansson and Westling, 1984, 1987). In fact, we were not able to detect any instances of loss of grasp from slip during movement in our experiments. The low likelihood of overt slippage we observed is consistent with the high statistical confidence against slip that we report here.

A computational model for understanding how GF control responds to both the mean and variability of LF dynamics

We next investigated how the adaptation of GF levels to environmental variability might manifest following a sudden but enduring change in the environment as in Experiment 2. We hypothesized that the motor system might estimate the current levels of the mean and the variability of the LF environment based on a window of recent experience. If this is the case, a recent sudden change in the environment should increase the motor system's estimate of variability because some trials in the window would be at the previous perturbation level and others would be at the new perturbation level. This increase should wane after prolonged exposure when most or all of the trials in the recent memory window would be associated with the new environment. The ACEVU model, illustrated in Figure 4, posits that the overall adaptive response simply reflects the sum of a mean-driven and a variability-driven component, based on windowed estimates of the expected value and the variance of the environment.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Predictions of a model for the adaptation of GFs and MFs based on distinct mean-driven and variability-driven components. A, B, Running estimates of mean and variability during a step change in environmental dynamics. These estimates weigh recent memories more strongly than older ones using an exponentially shaped memory window. The mean estimate (A) will tend monotonically toward the post-step value, whereas the SD estimate (B) will transiently increase, rising rapidly following the step change and then receding as memories that include the prestep environment fade away. C, Mean-driven and variability-driven components of the adaptive GF response following a positive (+FF) or negative (−FF) change in environmental dynamics. Critically, whereas the mean-driven component will follow positive or negative changes in environmental dynamics, the variability-driven component will be positive for changes in either direction. Thus our ACEVU model, which incorporates these two components, predicts asymmetrical, nonmonotonic GF adaptation curves. A positive FF would elicit overshooting adaptation (blue) due to complementary mean-driven and variability-driven components, whereas a negative FF would elicit bimodal adaptation (red) due to opposing mean-drive and variability-driven components. D, Same as C but for MFs. As MFs are insensitive to the variability of environmental dynamics, the variability-driven component will be zero throughout adaptation, leading to monotonic adaptation curves resulting in predictions of symmetrical, monotonic adaptation for the ACEVU model for both positive and negative changes in environmental dynamics.

While the exact shape of the memory window is not crucial for the model, we used an exponentially decaying memory window to estimate both the mean and the variability of the environment because it describes an estimation process that weighs past experiences less and less as they become more distant. This window shape is also compatible with linear state-space models of trial-by-trial adaptation with finite retention (retention coefficients <1), and it captures the approximately exponential learning curves that have been observed for motor adaptation. Moreover, a recent study found evidence that subjects track the variance of a visuomotor perturbation using an exponential memory window (Landy et al., 2012).

On-line estimates of the mean and variability of the environment for the ACEVU model in response to a step change in the environment are illustrated in Figure 4. Following the step, the mean estimate changes monotonically, asymptotically approaching a level appropriate for the new environment (Fig. 4A) in a symmetric manner that is specific for positive and negative steps. In contrast, the variability estimate increases transiently as discussed above, but in an identically positive manner after either step change (Fig. 4B). Thus a critical feature of the ACEVU model is that the mean-driven component of GF adaptation produces opposite responses for opposite changes in the environment (i.e., positive vs negative FFs), whereas the variability-driven component produces identical responses, which cannot be specific to the direction of the step change. This is the case because the variance of a signal is equal to the variance of its opposite. Figure 4C,D shows how estimates of the expected value and variability would combine in contributing to the adaptation of GF and MF throughout the time course of this adaptation. In these predictions, MFs and GFs both respond to changes in the mean of the environment as previous studies have shown (Flanagan and Wing, 1993, 1997; Scheidt et al., 2001; Takahashi et al., 2001; Mawase and Karniel, 2012); however, only GFs respond to changes in the variability, based on the results of Experiment 1.

Uncertainty versus variability

Our instantiation of the ACEVU model (Eq. 2) uses a running estimate of variability to estimate the amount of uncertainty about environmental dynamics. Variability and uncertainty are closely related: for example, the systematic block-to-block manipulation of variability in Experiments 1a–1c results in corresponding changes in the uncertainty about the LFs likely to be encountered in the environment. It should be noted, however, that variability and uncertainty, though related, are distinct qualities. Variability is easy to rigorously and specifically define as the statistical variance around the mean of the data. However uncertainty is trickier: it is related to the unpredictable part of the variability. Thus, to estimate uncertainty, the properties of the predictor must be understood. This would allow uncertainty to be defined as the statistical variance of the difference between actual and predicted values of the environment rather than the variance of actual values alone. The running estimate of the mean experienced load in the ACEVU model can be seen as a LF prediction relative to which environmental variability is computed. This bases the ACEVU model on an estimate of uncertainty rather than the variability around a fixed mean. In fact, neither the width of the LF prediction window nor the form of the prediction itself (i.e., using a predictor not based on an exponential time window) need be analogous to that for variability estimation as in the instantiation of the model in Equation 2, highlighting the dichotomy between LF prediction and the estimation of uncertainty around that prediction.

The ACEVU model predicts nonmonotonic adaptation

The superposition of a specific μ-driven and a nonspecific σ-driven component of the adaptive response in the ACEVU model results in a series of counterintuitive predictions for GF adaptation. In combining separate contributions from μ-driven and σ-driven adaptation, our model predicts that positive perturbations (increases in LF) will yield adaptation curves that rise especially rapidly but overshoot their asymptotic levels before settling (Fig. 4C). According to the model, the rapid rise will occur because during initial adaptation, the task-specific μ-driven and task-nonspecific σ-driven components will both contribute to an increase in GF levels. However, the σ-driven contribution will fall off as training proceeds, and an overshoot will result if this fall off outpaces the continued rise of the task-specific μ-driven component.

Perhaps even more counterintuitive is a second prediction also illustrated in Figure 4C: that decreases in LF will initially lead to increases in GF. This will occur if the increase in GF due to the nonspecific σ-driven component of adaptation to a negative perturbation outweighs the reduction in GF due to the task-specific μ-driven component. Prolonged training with a decreased LF will then eventually lead to a reduction of GF levels toward those appropriate for the environment as the nonspecific σ-driven component subsides. Thus the ACEVU model remarkably predicts an initial adaptive response in the “wrong” direction followed by a reversal resulting in appropriately decreased GF levels when decreased LFs are encountered.

Unlike GF control, nonspecific σ-driven adaptation should not contribute to MF control according to the results of Experiment 1, which show MF to be insensitive to σ_LF. Consequently our model predicts that MF adaptation curves will be fully determined by task-specific μ-driven adaptation and thus be monotonically directed toward an asymptotic level appropriate for each new environment be it positive or negative as shown in Figure 4D.

GFs adapt to an additive combination of uncertainty and the expected value

To examine how GFs and MFs actually evolve following a sudden change in physical dynamics, we analyzed the full time course of the adaptation data from Experiment 2. For the leftward +FF group, we found that the GF learning curve displayed a clear overshoot. It reached a maximum value at ∼10 trials after exposure before settling at a significantly lower asymptotic adaptation level as shown in the blue GF profiles in Figure 5A,B and the blue learning curve in Figure 5C (adaptation levels of 7.90 ± 1.15 Ns/m vs 2.46 ± 0.51 Ns/m for the first 20 trials of adaptation vs the last 60, p < 0.001, two-tailed paired t test; Fig. 5D). Individually, 16 of the 18 participants in the +FF group displayed higher average GFs in the first 20 trials than in the last 60 of the FF training epoch.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

GFs, but not MFs, exhibit asymmetric, nonmonotonic adaptation curves following opposite perturbations. A, B, Baseline-referenced GF profiles early and late (mean of first 20 vs last 60 trials) following the onset of positive (A, leftward) and negative (B, rightward) FF perturbations. Note that in B, the early response is opposite of what is appropriate for the applied FF, whereas the late response is appropriately directed. The shaded areas represent SEM across subjects. The gray vertical line represents the 95% confidence interval for movement onset. C, Corresponding GF adaptation curves based on the adaptation coefficient (see Materials and Methods). Each datapoint in the adaptation curve represents a bin of four trials in each of the two movement directions (8 movements in total). GFs initially increase in both FFs, but eventually reach asymptote at levels that are approximately equal and opposite (2.46 ± 0.51 Ns/m for the +FF and −2.36 ± 0.74 Ns/m for the −FF). This corresponds to overshooting adaptation for the +FF group and bimodal adaptation for the −FF group as predicted by the model in Fig 4C. The colored horizontal lines in the bottom of the graph indicate the learning periods shown in A and B: early training (first 20 trials, blue for +FF and red for −FF) and late training (last 60 trials, cyan for +FF and orange for −FF). D, E, GF levels for the early and late (mean of first 20 vs last 60 trials) stages of adaptation. F–J, Same as A–E but for MF, illustrating symmetric, monotonic adaptation patterns as predicted by the model in Fig. 4D.

For the rightward −FF group, the initial adaptation resulted in seemingly inappropriate increases in GF above baseline, which persisted for ∼40 trials before the adaptive responses appropriately decreased below baseline levels, where they slowly settled to asymptote as shown in Figure 5C. Correspondingly, the adaptation level during the first 20 trials was significantly greater than baseline, whereas the adaptation level during the last 60 was significantly lower than baseline as shown in the red GF profiles in Figure 5A,B and the red learning curve in Figure 5C (adaptation levels of +3.80 ± 0.75 Ns/m vs −2.36 ± 0.74 Ns/m, p < 0.0001 and p < 0.01, respectively, two-tailed paired t tests; Fig. 5E). Individually, the GF level was higher than baseline in 16 of 18 subjects during the first 20 trials, and lower than baseline in 15 of 18 subjects during the last 60 trials. Both the direction reversal observed in the −FF data (red) and the pattern of overshoot and subsequent fall off observed in the +FF data (blue) were remarkably well predicted by the ACEVU model displayed in Figure 4, which combines nonspecific σ-driven adaptation and FF-specific μ-driven adaptation.

It should be noted that EC trials were interspersed with FF trials during training, perhaps complicating the findings. These trials are less disruptive to training than catch trials on which FF perturbations are withheld (Thoroughman and Shadmehr, 2000; Smith and Shadmehr, 2005) because motor errors that drive learning are held near zero for EC trials in contrast to the large errors often associated with catch trials. However to hold lateral errors near zero, an external force must be applied by the robot arm on EC trials, which effectively cancels the lateral force output of the learner. Because learning is generally partial, this external force pattern can be viewed as a certain fraction of the force level applied during FF trials, introducing some variability in the effectively applied FF between FF and EC trials, and thus perhaps increasing variability-driven adaptation in line with the ACEVU model. Thus, the increased variability-driven response early during training in Experiment 2 might be driven by variability that stemmed from the presence of EC trials rather than variability stemming from the abruptness of FF onset as modeled in Figure 4. However, when we examined our data to determine whether this was the case, we found evidence for both key predictions of the ACEVU model even before the first EC trial could affect the variability estimate: a GF overshoot in the +FF group (5.6 ± 2.3 Ns/m overshoot, p = 0.012) and adaptation toward the wrong direction in the −FF group (+5.1 ± 1.3 Ns/m, p = 0.0006). This indicates that the overshooting and wrong direction learning we observe do not require the presence of EC trials. However, we cannot rule out the possibility that the amplitude of these effects is exaggerated by it.

MFs adapt chiefly to the expected value of environmental dynamics

In contrast to GF adaptation, we found MF adaptation to be characterized by highly symmetric, monotonic responses to increases and decreases in LF, as evidenced by the MF learning curves displayed in Figure 5H and the force profiles displayed in Figure 5F,G. Accordingly, the amplitude of the average MF adaptation level during the first 20 trials was significantly lower in magnitude than the average adaptation level during trials 181–240 (leftward FF: 4.56 ± 0.24 Ns/m vs 6.66 ± 0.27 Ns/m, p < 10⁻⁴; rightward FF: −3.92 ± 0.21 Ns/m vs −7.14 ± 0.15 Ns/m, p < 10⁻⁶, two-tailed paired t-tests; Fig. 5I,J). The highly symmetric learning curves indicate that there is little nonspecific adaptation, with MF adaptation highly specific to the experienced environment. This is consistent with the idea that the nonspecific component of the adaptive response arises from σ-driven adaptation, which we found to be minimal for MF in Experiment 1.

Robustness of results using different metrics

To ensure that our observations were not specifically related to the adaptation coefficient metric we used to quantify FF adaptation, we repeated the analysis using an even simpler metric: the GF observed when the effect of b on the LF produced by the imposed environment was maximal (Sing and Smith, 2010; Pekny et al., 2011; Joiner et al., 2013). We found that the learning curves derived from this measure were essentially identical in shape to those displayed in Figure 5C,H (R² > 0.97 for the +FF and −FF GF learning curves and for the +FF and −FF MF learning curves).

Decomposition of GFs into perturbation-specific and perturbation-nonspecific components

Based on the ACEVU model, task-specific adaptation should differentially affect the adaptive responses to two opposite perturbations, whereas the task-nonspecific adaptation will be common to both. We used this fact to decompose the experimentally observed adaptive responses to the +FF and −FF environments into specific (differential) and nonspecific (common) components (see Materials and Methods). Specifically, we applied this differential versus common component analysis to the GF profiles [GF(t)] in Experiment 2 to determine the specific and nonspecific components of these profiles (Fig. 6A,C). This analysis reveals that over the course of adaptation, the amplitude of the nonspecific common-mode GF profile rises from baseline during initial adaption and wanes as adaptation proceeds. Despite the changes in amplitude, the shape of the common-mode GF profile remains remarkably constant as training progresses (Fig. 6A,B, colored lines). The extent of this shape consistency is apparent when amplitude-normalized force profiles are compared with each other in Figure 6B. Critically, this shape is strikingly similar to the shape of the GF profiles observed in Experiment 1a, when the effects of environmental variability were measured (R² values between 0.92 and 0.99 for the six common-mode profiles shown normalized in Fig. 6B, and R² = 0.98 for the mean of these). Note that the GF profile from Experiment 1a is shown as the dark gray dashed line in Figure 6B for reference. The close match between the shapes of the force profiles for the transiently occurring common-mode component of the step response from Experiment 2 and the variability-driven adaptive response directly measured in Experiment 1a is consistent with the idea that the transient nonspecific component the GF learning curve reflects the motor system's estimate of environmental variability as predicted by the ACEVU model.

• Download figure
• Open in new tab
• Download powerpoint

Figure 6.

Distinct shapes of the specific and nonspecific components of GF profiles during learning reflect the shape of mean-driven and variability-driven responses, respectively. GF profiles were averaged within bins of 20 trials and across subjects. The FF-specific (non-FF-specific) component was obtained by taking, for each such bin, the differential (common) mode of GF profiles from the positive and negative FF experiments. A, Nonspecific components of GFs (warmer colors indicate later parts of training). B, Normalized profiles for the nonspecific components for the first six bins after perturbation onset. The gray dashed curve shows the normalized shape of the average variability-driven change in GF shown in Figure 2E. Its similarity to the average shape of the nonspecific GF components (R² = 98%) is in line with the idea that nonspecific GF adaptation is driven by environmental variability. C, FF-specific components of GFs (color scheme the same in all panels). D, Normalized profiles of the specific components for the last eight bins. The gray dashed curve shows the shape of the ideal compensatory force pattern from Experiment 2. Its similarity to the average shape of the specific GF components (R² = 93%) is in line with the idea that specific GF adaptation is driven by the expected value of the environment.

On the other hand, the task-specific differential mode component of the GF profile assumes a shape very close to the shape of the viscous LF profile (Fig. 6C,D). This similarity increases with training, with R² values increasing to 0.91 by the end of the training period (R² = 0.93 for the average profile). Thus the shapes of the force profiles associated with the specific and nonspecific components of the GF adaptation data from Experiment 2 were more closely matched to the mean LF profiles observed in Experiments 2a and 2b and the variability-driven changes in GF observed in Experiment 1a, respectively, than to each other (R² values of 0.93 and 0.98, respectively, vs an R² of 0.50). This finding provides direct experimental support for the theoretical links posited in the ACEVU model between task-specific adaptation and expected value estimation on one hand and between task-nonspecific adaptation and variability estimation on the other.

Learning curves for the specific and nonspecific components of GF and MF adaptation

We then leveraged our knowledge about the shapes of the force profiles associated with the nonspecific (Fig. 6A,B) and task-specific (Fig. 6C,D) components of adaptation to estimate the learning curves associated with each component. Instead of the standard practice of singly regressing the adaptive responses onto the shape of the viscous load force profile to estimate an adaptation coefficient, as in Figure 5, we simultaneously regressed the adaptive responses we observed onto the shapes of both the common-mode and differential-mode components (see Materials and Methods), to obtain learning curves for nonspecific (green) and task-specific (purple) adaptation, respectively, for both GF (Fig. 7A) and MF (Fig. 7B). This analysis reveals that the task-specific differential-mode component of adaptation (purple) displays a monotonic increase toward an asymptote, following a similar time course for both GF and MF (Fig. 7C for a direct comparison). In contrast, the task-nonspecific common-mode component (green) for the GF is transient (it displays a rapid increase toward a sharp peak and then a gradual decay toward baseline) and present only during GF adaptation. This is consistent with the mirror-opposite MF learning curves observed for the leftward and rightward FF groups in Figure 5I, which suggest fully-specific MF learning, and with the finding in Experiments 1a and 1b that MFs are unrelated to environmental variability. Remarkably, we find that the shapes of these task-specific and task-nonspecific components of the GF learning curves (Fig. 7A,B) closely match the predictions of the ACEVU model (Fig. 4) for distinct mean-driven and variability-driven responses based on exponentially windowed running estimates of these quantities.

• Download figure
• Open in new tab
• Download powerpoint

Figure 7.

Adaptation curves for specific and nonspecific components. A, B, Estimates of the specific (magenta) and nonspecific (green) components of the learning curves for GF (A) and MF (B). The shapes of these components match the predictions in Figure 4C,D. C, Comparison of the learning curves for the specific components of GF and MF, showing that they display similar time courses of adaptation. Adaptation curves were normalized by the overall asymptotic adaptation level (last 60 trials). Shading indicates SEM across subjects. The inset shows the time constants of adaptation for these two specific components (mean ± SEM estimated using bootstrap). D, Same as C but for the nonspecific component. E, Comparison of the adaptation levels for the specific components of GF (light) and MF (darker) in five-trial windows centered around trials 5, 10, 20, and 40. None of the differences are significant (p > 0.5 in all cases). F, Same as E but for the nonspecific component. All differences are highly significant (p < 10⁻⁶ in all cases).

The decomposition of the learning curves into task-specific and task-nonspecific components provides the opportunity to compare the rate of task-specific GF and MF adaptation. We observed a strong similarity between the learning curves for the task-specific (purple) adaptation for GF and MF (r = 0.84), suggesting similar adaptation rates (Fig. 7C, which compares asymptote-normalized task-specific learning curves). Correspondingly, fitting single exponentials to these learning curves revealed no difference between the time constants for task-specific GF versus MF adaptation (35.1 ± 19.6 vs 26.6 ± 2.6 trials, p = 0.334, estimated using bootstrap; Fig. 7C, inset). Moreover, we found no differences between normalized adaptation levels for GF and MF at trials 5, 10, 20, and 40, estimated using five-trial bins (p > 0.5 in all cases, two-tailed paired t test; Fig. 7E). Together these findings indicate that the FF-specific components of GF and MF adapt at essentially the same rate.