Fast and accurate motor behavior requires combining noisy and delayed sensory information with knowledge of self-generated body motion; much evidence indicates that humans do this in a near-optimal manner during arm movements. However, it is unclear whether this principle applies to eye movements. We measured the relative contributions of visual sensory feedback and the motor efference copy (and/or proprioceptive feedback) when humans perform two saccades in rapid succession, the first saccade to a visual target and the second to a memorized target. Unbeknownst to the subject, we introduced an artificial motor error by randomly “jumping” the visual target during the first saccade. The correction of the memory-guided saccade allowed us to measure the relative contributions of visual feedback and efferent copy (and/or proprioceptive feedback) to motor-plan updating. In a control experiment, we extinguished the target during the saccade rather than changing its location to measure the relative contribution of motor noise and target localization error to saccade variability without any visual feedback. The motor noise contribution increased with saccade amplitude, but remained <30% of the total variability. Subjects adjusted the gain of their visual feedback for different saccade amplitudes as a function of its reliability. Even during trials where subjects performed a corrective saccade to compensate for the target-jump, the correction by the visual feedback, while stronger, remained far below 100%. In all conditions, an optimal controller predicted the visual feedback gain well, suggesting that humans combine optimally their efferent copy and sensory feedback when performing eye movements.
Limb movements are neither purely driven by sensory feedback, nor “ballistic” i.e., unfolding as a preprogrammed sequence of muscle contraction (Goodale et al., 1986; Desmurget and Grafton, 2000). Instead, a growing consensus is that the motor system uses a forward model of the limb dynamics to compute an internal estimate of the current sensorimotor state (e.g., position and velocity) and an inverse model to compute the motor command given the internal estimate (Wolpert et al., 1995; Wolpert and Kawato, 1998; Todorov, 2004).
This internal estimate is the result of a combination of two unreliable cues: the efferent copy of the motor command, and the sensory feedback. The forward model predicts the current sensorimotor state from the past state and the efferent copy, but is necessarily imperfect because movements are variable (Schmidt et al., 1979; van Beers et al., 2004). The sensory feedback (e.g., visual, proprioceptive) is noisy and delayed. The combination of the forward prediction and the feedback estimate is optimal, i.e., the internal model is most precise, when the two sources of information are weighted according to their reliability. The optimal relative contributions of the sensory feedback and the forward prediction are described by “Kalman gain” (Kalman and Bucy, 1961) computed recursively from the sensory and motor variance (Denève et al., 2007). The Kalman gain increases as a function of the motor noise and decreases as a function of the sensory noise (see Materials and Methods).
While the relative contribution of forward model and sensory feedback is still debated, they are both involved in the control of the upper limb (Prablanc and Martin, 1992; Connolly and Goodale, 1999; van Beers et al., 1999; Sabes, 2000; Ariff et al., 2002; Baddeley et al., 2003). Some experiments suggest that humans use a Kalman filter when performing arm movements (Wolpert and Ghahramani, 2000; Saunders and Knill, 2004). However, it is still unclear whether the same principles apply to saccadic eye movements (Pierrot-Deseilligny et al., 2003).
Saccadic eye movements are fast (50–80 ms) (Robinson, 1964) compared with sensory delays. They are more reproducible and less variable than arm movements (Bahill et al., 1975; Becker, 1989). Moreover, “saccadic suppression” prevents the visual feedback from influencing the trajectory of the eye (Bridgeman et al., 1975; Thiele et al., 2002). Robinson (1975) proposed that saccade generations were controlled by internal loops comparing the desired eye position with a prediction based on the efferent copy of the motor command, i.e., a forward model of eye position. Such internal loops have since been reported in subcortical structures (Gnadt et al., 2001).
However, while sensory feedback plays a minor role in single saccades, it could contribute significantly to the control of sequences of saccades i.e., successive eye movements separated by brief periods of fixation. Consistent with this proposal, sequences of memorized saccades shows a correction of fixation errors from one saccade to the next, whereas a forward controller would accumulate errors after each saccade (Karn et al., 1997; Ditterich et al., 1998). This correction, however, is not complete and might be due to proprioceptive feedback, internal corrective loops, or combination of both (Karn et al., 1997; Ditterich et al., 1998). Proprioception is not required to perform sequences of memorized saccades (Lewis et al., 2001), suggesting that this online correction relies mainly on internal loops.
While the importance of proprioceptive feedback is debated, visual cues anchored in egocentric space could also provide a reliable feedback. The goal of this study was to investigate whether visual cues and efferent copy are combined optimally in sequences of eye movements. We asked subjects to perform two eye movements in quick succession, and introduced an artificial motor error by randomly moving the target of the first saccade during the movement. The extent to which the second saccade was corrected by this visual feedback, allowed us to directly measure the Kalman gain (see Materials and Methods). We varied the size of the first saccade and found that this gain was well predicted by an optimal controller.
Materials and Methods
We performed two separate sets of experiment. Eleven healthy human subjects (9 males, 2 females; 22–32 years old) participated to the first study (experiment 1) while eight healthy human subjects (6 males, 2 females; 24–32 years old) participated to the second study (experiment 2). Subjects performed 4 blocks of 72 trials in experiment 1, and 6 blocks of 48 trials in experiment 2. All subjects had normal or corrected to normal vision. Four subjects participated to the two experiments.
Subjects were placed 35 cm in front of a 17 inch, 800*600 pixels, CRT screen at 100 Hz refresh rate, in a dimly lit room. Their head was maintained in a fixed position by a chin piece and by requiring the subjects to bite on their own dental print. Eye movements were recorded using an optometric system: EyelinkI system (SR Research). Eye positions were continually recorded at 250 Hz.
The task was adapted from the double-step paradigm, and is schematically represented in Figure 1. The subjects were instructed (1) to gaze at a white cross, 0.80° wide, the fixation point (FP), positioned in the center of the screen. (2) After a brief fixation period (300 ms), a green target (diameter 0.80°) was turned on at the right or left of the FP. (3) A red target was flashed for 200 ms, 490 ms later. This flash corresponded to the “go signal.” Subjects had to perform two saccades in rapid successions, first (4) to the green target (target 1, T1), and then (5) to the memorized location of the red target (target 2, T2).
(6) In experiment 1, T1 “jumped” unpredictably on two-thirds of the trials from its initial position in the direction of the first saccade (positive/centrifugal jump) or in the opposite direction (negative/centripetal jump) as soon as the distance between the average eye position during the last 6 records at 250 Hz and the new eye position reached 0.75°. The amplitude of this jump was 20% of the distance between FP and T1 (i.e., ∼20% of the amplitude of the first saccade). In agreement with previous studies using intrasaccadic target displacements (Prablanc and Martin, 1992) we found that subjects never detected the target jump, i.e., they never reported seeing a change in position of T1 during the first saccade. We thus postulate that the target jump introduced an artificial motor error on the first saccade endpoint. To limit the occurrence of short corrective saccades, T1 was extinguished 90 ms after completion of the first saccade. This always occurred before the start of the memorized saccade to T2.
In experiment 2, T1 was extinguished during the saccade on half of the trials, using the same criteria as for target jumps in experiment 1. In the other half of the trials, T1 was extinguished 90 ms after completion of the first saccade, as in the “no jump” condition in experiment 1.
T1 appeared at one of three eccentricities (6° for small saccades, 12° for medium saccades, and 18° for large saccades), on the right or left of the fixation point, for a total of 6 possible T1 locations. T2 appeared in one of 4 possible locations relative to T1 (9° above or below, and 9° left or right of T2). Finally, as stated above, T1 either stayed at the same location (one-third of the trials in experiment 1, half of the trials in experiment 2), jumped to the right by 20% of the distance between FP and T1 (one-third of the trials in experiment 1), jumped to the left by 20% of the distance between FP and T1 (one-third of the trials in experiment 1), or was extinguished (one-half of the trials in experiment 2). The order of presentation for the different conditions was randomized separately for each subject. If subject moved their eyes before the “go signal” or if the two saccades were not completed within 1710 ms, the trial was aborted. A saccade was considered to be completed when the distance between the average eye position during the last 6 records and the new eye position was <0.6°. Note that we allowed for the existence of a single small “corrective saccade” after the first saccade to T1 and before the saccade to T2.
We separated the trials into two main categories: Trials where the saccades to T1 and to T2 occurred in immediate succession without corrective saccades (trials without corrective saccades), and those where a single corrective saccade occurred between T1 and T2 (trials with corrective saccade). Other types of trials (those that did not contain a specified sequence of saccades as described below) were eliminated from the analyses. The first saccade to T1 had to bring the eye from a rectangular window of 2.5 × 2.5° around the fixation point to a rectangular window around the location of T1 (before the jump) representing 40% of the first saccade amplitude in length and 3° in height. A corrective saccade was detected when the second saccade brought the eye within a rectangular window of 5.7 × 2.3° around the new position of T1 after the jump (if any). A memorized saccade to T2 brought the eye within a 7.3 × 4.6° window around T2. A correct trial had to contain a saccade to T1 followed by a saccade to T2 (trial without corrective saccade), or a saccade to T1 followed by a single corrective saccade and a saccade to T2 (trial with corrective saccade).
Our main goal in experiment 1 was to measure the influence of the target jump of T1 on the memorized saccade to T2. This corresponds to the discrepancy between the dashed and dotted arrows in Figure 1D–I. We defined the “final percentage of correction” as the deviation introduced by the target jump in the final eye position around T2, divided by the size of the target jump. The final eye position around T2 is the eye position at the end of the main saccade to T2 or at the end of subsequent corrective saccade, when executed.
Thus, for positive target jump this percentage of correction is defined as follows: where e is the horizontal position of the final eye position around T2, 〈 〉jump-pos represents an average over all trials with positive jumps, 〈 〉no-jump represents an average over trials with no target jump, and T1 is short-cut notation for the horizontal position of target 1 before the jump minus the position of the fixation point. e.g., a small rightward saccade corresponds to T1 = 6°, a medium leftward saccade T1 = −12°. The target jump size is 0.2T1.
Similarly, for negative jumps (i.e., T1 is displaced in the opposite direction to the first saccade) the final percentage of correction for small saccades is defined as follows: where 〈 〉jump-neg is the average over trials with negative jumps. The global percentage of correction regardless of jump direction is defined as follows: We only report the results in terms of horizontal positions since the target jump occurred on the horizontal axis. Similar analyses performed on vertical positions did not show any significant effects of the target jump.
In addition, we also computed the percentage of correction after the first saccade to T1, the percentage of correction after the corrective saccade (if there is one), and the percentage of correction of the eye displacement (“saccade vector”) during the saccade to T2. This is done by replacing “e” in the previous equations by the corresponding positions or displacements. For example, the percentage of correction after the corrective saccade is given by the following: where 〈e(2) 〉jump-neg, corr-sacc is the position of the eye following the corrective saccade, averaged over all trials with corrective saccade and negative jumps. Similarly, the percentage of correction of the saccade vector (eye displacement) to T2, for trials without corrective saccade is given by the following: where e(3) − e(1) is the horizontal eye displacement for the memorized saccade to T2, and 〈 〉jump-pos, corr-sacc indicates an average over trials with positive jump and without corrective saccade.
The goal of experiment 2 was to measure the true contribution of “motor errors” to the variability of the first saccade end-point (i.e., how the actual movement differs from its prediction by the efferent copy). In this case, we restricted our analysis to trials with target extinction and without corrective saccades. We measured to what extent fixation errors on the first saccade were transmitted to the memorized saccade end-points, defining the “transferred variability” as follows: where the suffix “ext” refers to trials with target extinction and no corrective saccade in experiment 2. This “transferred variability” was used to predict the visual Kalman gain, and thus the percentage of correction in experiment 1 (see below, The Lakman filter model).
The Kalman filter model.
If the saccadic system completely trusted its visual feedback, i.e., if it completely assigned the discrepancy between T1 and the eye position after the first saccade to a motor error, it should adjust the final eye position around T2 in the same direction and with approximately the same amplitude than the target jump (Fig. 1D,E). This would correspond to a percentage of correction of 100%. If, on the other hand, the saccadic system ignored the visual feedback and relied entirely on the efferent copy to update the saccadic movement vector to T2, the jump should have no influence on the final eye position around T2 (Fig. 1H,I). This would correspond to a percentage of correction of 0%. If the subject combined both signals, the correction of the second saccade to T2 should fall between these two extremes, i.e., the percentage of correction should be intermediate between 0 and 100% (Fig. 1F,G). The optimal percentage of correction, i.e., the percentage of correction minimizing the final eye position error, reflects the reliability of the visual sensory feedback relative to that of the forward prediction. We illustrate this prediction by a simplified model of the double saccade paradigm.
Let us consider a one-dimensional model of the eye plant, where the eye position e(t) is “instantaneously” updated by the saccadic vector Δ(t), so that Δ(t) corresponds to the “motor command” sent to the eye plant. εm(t) represents the “motor noise,” or “execution noise,” which corresponds to noise in the eye plant. It is assumed to be Gaussian distributed with zero mean and variance Vm (Table 1).
For an eye movement planned toward target T, the motor command sent to the eye plant is Δ(t) = T + εl(t), where εl(t) is assumed to be a Gaussian “target localization” noise with zero mean and variance Vl. εl(t) corresponds to the localization error of target T, as a result of sensory noise and/or an imperfect sensory-motor transform of the sensory input into a motor command. Errors in localization of the target largely contribute to the variability of saccadic eye movements and exceed the motor noise (van Beers, 2007). However, contrary to motor noise, these errors can be predicted from the efferent motor commands. In our notations, the total variability of the saccade movement, Vn = Vl + Vm, is the result of the combination of target localization errors and motor noise.
If the target is still present after the end of the saccade, a visual sensory feedback is provided by the retinal position of the target after the saccade, s(t + 1) = T' − e(t + 1), where T' is the target location after the saccade; In normal situations, T' = T, but in cases when we introduced jumps in target location, T' = T + jump.
The motor system could use the efferent copy of the motor command Δ(t) and its previous eye position internal estimate to compute a forward prediction for the new eye position. To simplify the equations, we define the initial eye position estimate as the central position ê(t) = 0. The forward estimate is then given by the efferent copy êf (t + 1) = Δ(t). Unfortunately, while this estimate takes into account the localization errors εl(t), it does not take into account the motor noise εm(t) and thus will accumulate errors after each saccade. Alternatively, the eye controller could rely on its visual sensory feedbacks. The feedback estimate measures the eye displacement as the difference between the target location before the saccade, T + εl(t), and its location after the saccade, s(t + 1). The optimal way of combining these two unreliable signals is a weighed sum, with each cue weighted by the inverse of its variance (Ernst and Banks, 2002). The internal estimate is corrupted by the motor noise, εm(t), with variance Vm, while the sensory feedback is corrupted by visual localization noise εl(t) with variance Vl. Thus, the combined estimate of eye position is given by the following: ks, the Kalman gain, represents the relative contribution of the forward model (or “efferent copy”) and the visual feedback. It is given by the following: ks is zero when the localization noise of the visual target dominates the motor noise (Vl ≫ Vm), and 1 when the motor noise dominates (Vl ≪ Vm).
For our particular task, e(0) is the position of the eye at fixation, e(1) is the position of the eye after the first saccade, e(2) is the position after the corrective saccade, and e(3) is the position of the eye after the saccade to T2. Δ(1), Δ(2), and Δ(3) are the motor commands corresponding to the first saccade, corrective saccade and memorized saccade to T2 (see Table 1). To simplify notations, we assumed that for trials without corrective saccades, e(1) = e(2) and thus, Δ(2) = εl(2) = εm(2) = 0. The visual feedback corresponds to the retinal position of T1 after the jump, i.e., s(2) = T1 + jump − e(1). The first saccade vector, Δ(1) = T1 + εl(1), brings the eye toward the first target. As a consequence, the first eye position is given by the following: From there we need to separate different kinds of trials.
Target extinction, no corrective saccade (experiment 2).
In the absence of visual feedback (target extinction) the internal eye position estimate after the first saccade is equal to the forward estimate: ê(1) = Δ(1) = T1 + εl(1). The motor command for the next saccade is computed as the difference between the memorized location of the second target (corrupted by target localization noise), and the current estimate of eye position is as follows: The localization error on the memorized saccade, εl(3), corresponds to the error is the memorized visual location of the second target T2. As a consequence, the final eye position is given by the following: We use these trials to predict the Kalman gain for trials with visual feedback (i.e., with target jump). The Kalman gain can be predicted directly from the covariation between the eye position after the first saccade and the final eye position. If fixation errors on the first saccade were entirely due to localization noise, they would be completely corrected in the second saccade. As a result, e(3) would not be correlated to e(1) and the Kalman gain would be equal to zero. If, on the other hand, errors in the first saccade were entirely due to motor noise, they could not be corrected and would result in a strong positive correlation between e(1) and e(3), and a Kalman gain equal to one. In general, the Kalman gain can be predicted by the covariance between e(1) (Eq. 11) and e(3) (Eq. 13), divided by the variance of e(1): The measured motor variance Vm might in fact under-estimate the noise in the motor plant: Our experimental protocol does not allow us to separate the contribution of the efferent copy and proprioceptive sensory feedback. The motor noise reflects variability in the eye position estimate after taking into account the efferent copy and the proprioceptive feedback.
Target jump, no corrective saccade (experiment 1).
In the presence of visual feedback, the estimate for the eye position after the first saccade is given by the forward estimate corrected by the sensory feedback. Using Equation 9 with êf(1) = Δ(1) = T1 + εl(1) and ês(1) = e(1) − jump + εl(1) (Eq. 8), and replacing e(1) by its expression in Equation 11, we get the following: where “jump” is the amplitude of the jump of target T1. Thus “jump” is 0.2 × T1 for positive jumps, −0.2 × T1 for negative jumps, and 0 when there is no jump.
When no corrective saccade occurs, (Δ(2) = 0) the motor command for the second saccade (i.e., the memorized saccade) is given by the following: Using Equation 7 and replacing Δ(3) by its expression in Equation 16, we get the following: In conclusion, the predicted percentage of correction for trials with target jump and without corrective saccade is 100ks.
Target jump, with corrective saccade (experiment 1).
When a corrective saccade occurs, it is a visually guided saccade aimed at target T1 + jump. Thus, its motor command is as follows: As a consequence, the true eye position after the corrective saccade is as follows: The target is extinguished by the end of the corrective saccade, preventing subjects from receiving any additional visual feedback. However, since this corrective saccade is presumably directed toward T1 and subjects are unaware of a target jump, the initial target location estimate T1 + εl(1) can still provide information about eye position after the corrective saccade. For example, if the corrective saccade was perfectly precise in foveating the target, T1 + εl(1) would be the best estimate of eye position after the corrective saccade, without requiring a forward model. In our framework, relying completely on the initial target location T1 + εl(1) as an estimate of eye position after a corrective saccade corresponds to planning the memorized saccade in egocentric coordinates as the difference between the second and first target position, Δ(3) = T2 − T1 + εl(3) − εl(1), regardless of visual feedback and preceding motor commands.
The forward estimate and initial target location can be combined as a function of their respective reliability. Using Equation 9 with êf(2) = ê(1) + Δ(2) and ês(2) = T1 + εl(1), we obtain the following: The new Kalman gain ks′ depends on the variance of the forward eye position estimate êf (2) and of the feedback estimate ês(2). Here we assume that the small corrective saccade does not introduce additional motor noise, i.e., we neglect εm(2). This is reasonable since we found that the motor noise increased with saccade amplitude, and was already negligible for the small first saccade amplitude of 6 degrees (see Fig. 2C). The variance of the forward estimate is then (1 − ks)2 Vm, while the variance of the memorized target location is Vl. The new Kalman gain is given by the following: Using Equation 20 and replacing ê(1) by its expression in Equation 15, the resulting estimate for the eye position after the corrective saccade is as follows: Thus, the last saccade vector and final eye position are given by the following: In trials with target jump and with corrective saccade, the predicted percentage of correction is as follows:
To report the results we use the conventions in Table 1. All positions are projected on the horizontal axis, and position zero corresponds to the fixation point. Positions to the right of the fixation point are positive, and positions to the left are negative. Mean values for the timing characteristics of saccades during the task are reported in Table 2.
Correction of the memorized saccade vector by fixation errors
In agreement with previous studies (Bock et al., 1995; Ditterich et al., 1998) we found that fixation errors around T1 influence the saccade vector to the memorized location of T2. In other words, fixation errors are taken into account to correct the next saccade. Figure 2 shows an anti-correlation between the fixation error, defined as the position of the eye around T1 before the start of the saccade (i.e., e(2) − T1), and the saccade vector to T2, defined as the eye displacement from the beginning to the end of the saccade (i.e., e(3) − e(2)). Figure 2A plots all trials without target jumps in experiment 1. The strong anti-correlation (p < 0.001; r = −0.40; Spearman rank correlation) indicates that the two saccades do not unfold as a preprogrammed sequence of motor commands; the memorized saccade is corrected by the previous fixation error around T1. We found similar results when we restricted the analyses to small saccades (p < 0.001, r = −0.39), medium saccades (p < 0.001, r = −0.32) or large saccades (p < 0.001, r = −0.43). A strong anti-correlation was present for all conditions in both studies. In particular, Figure 2B shows trials with target extinction, i.e., cases where no visual feedback was provided after the start of the first saccade (experiment 2). The strong anticorrelation (p < 0.001, r = −0.41; Spearman rank correlation) suggests that a large part of the variability in the first saccade endpoint can be corrected using the efferent copy (and/or proprioceptive feedback) to update the memorized saccade. If the correction of fixation errors by the efferent copy was complete, the eye position after the first saccade and the eye position after the memorized saccade should be independent. e(1) and e(3) are not significantly correlated for trials with small (p = 0.93, r = 0.006; correlation analysis) and medium (p = 0.40, r = 0.056). However, this correlation is significant for large saccades (p = 0.01, r = 0.171). These results indicate that for large saccades, a part of the fixation errors cannot be predicted from the efferent copy and corrected in the next saccade. We used the “transmitted variance” to estimate the contribution of the motor noise to the total saccade variance, and thus predict the visual Kalman gain (Eq. 14). The transmitted variance (Eq. 14) measured from trials with target extinction in experiment 2 were respectively 0.01, 0.07, and 0.24 for small, medium and large saccades. The corresponding contribution of motor noise Vm and target localization error Vl to the total saccade variability Vn = Vm + Vl is plotted in Figure 2C. Our result suggest that most of variability in saccadic eye movements corresponds to errors in computing the motor command, that is, in transforming the position of the visual target into a motor plan. The contribution of motor variance, i.e., the fluctuations in movement properties due to noise in the motor plant, increases with saccade size. Localization errors also increase from small to medium and large saccades.
The fixation error signal used by the motor system to correct future saccade vectors could be the visual feedback from target T1, the proprioceptive feedback, or an internal prediction obtained by integrating the motor command sent to the eye plant (i.e., the forward model estimate). The goal of this study was to measure the contribution of the visual feedback to this correction, compared with the forward model and/or proprioceptive feedback. For this, we introduced unpredictable perturbations in the retinal location of the visual target during the first saccade. By varying the amplitude of this saccade, we also varied the motor noise, which we predicted would influence the Kalman gain.
Effects of target jumps on the final eye position (experiment 1)
The final eye position around T2, e(3), is significantly influenced by the direction of the target jump. As there were no significant differences in saccade accuracy between the four T2 endpoints, saccade error and correction data were averaged for each subject across all T2 locations for a given T1 location. Figure 3 plots the global percentage of correction of the final eye position for small, medium and large saccades (Fig. 3, black columns). Data are represented as percentage correction of the target jump and show significant correction for medium and large saccades (p < 0.001); one-sample t test) but not for small saccades. One-way ANOVA analysis showed a significant effect of saccade size [F (2,1639) = 7.27031; p < 0.001] with significantly larger amount of correction for medium (16.6%) and large (23.2%) saccades than for small (0.4%) saccades (p = 0.02 and p < 0.001 respectively; test post hoc honestly significant difference Tukey's). Note that the target jump is itself a fixed percentage of the first saccade size. Thus, the same percentage of correction corresponds to a larger absolute correction of the final eye position following a larger first saccade.
When separating trials without and with corrective saccades (Fig. 3, gray and white columns) we found that the percentage correction was significantly larger for trials with corrective saccades for both medium (25.4% vs. 10.5%, p = 0.038; independent samples t test) and large saccades (29.3% vs 19.3%, p = 0.046).
An optimal controller should adjust the contribution of the visual feedback as a function of its reliability compared with that of the efferent copy. As explained previously, the percentage of correction is a direct measure of this contribution (see methods). As a result, an optimal controller model predicts the following: where Vl is the variance of the visual feedback, and Vm is the variance of the forward prediction computed from the efferent copy. Since the motor noise Vm and the total saccade variance Vn both covary with the amplitude of the saccade (Fig. 2C), we expect the percentage of correction to covary with the variance of the first saccade.
In agreement with Abrams et al. (1989) we found that the total variance of the saccade end-points increased with the eye movement amplitude, from small to medium to large saccades. The percentage of correction as a function of the variance of the first saccade end-point is plotted in Figure 4A. The corresponding 6 point correlation has to be taken with caution, but is significant when we use a nonparametric test (r = 0.81; p < 0.05; Spearman rank correlation coefficient).
The variability of saccade end-point does not directly predict the Kalman gain, since it is a combination of motor noise and target localization error (see previous section). Thus, we used the “transferred variability” measured in experiment 2 to separate the two variances and estimate the true reliability of the efferent copy (Fig. 2C). We then compared the predicted and observed percentage of correction (Fig. 4B). We found a good quantitative agreement between the model prediction using transferred variance in experiment 2, and the percentage of correction by visual feedback observed in experiment 1 for the 3 saccade sizes, with and without corrective saccade (r = 0.94; p < 0.01; Spearman rank correlation coefficient). This suggests that human subjects combine optimally the visual feedback with the efferent copy (and/or proprioceptive feedback).
Origin of the final eye position correction (experiment 1)
The final eye position correction could originate from an on-line adjustment of the first saccade amplitude (i.e., e(1)), from a corrective saccade (i.e., e(2) − e(1)), or from the memorized saccade (i.e., e(3) − e(2)). To differentiate these 3 contributions, we computed the percentage of correction for each of these saccades.
Contribution of the first saccade
We found no sign of an on-line correction of the first saccade by the target jump during the saccade. As shown in Figure 5A, the target jump influenced neither the saccade end-point nor its velocity profile, suggesting that the first saccade is entirely ballistic. Thus, the correction of the final eye position due to the target jump is implemented either by the corrective saccade, or by the memorized saccade, or a combination of both.
Contribution of the corrective saccade
Relative to no-jump trials, corrective saccades were less frequent when the target was jumped back (negative jump), and more frequent when it was jumped forward (positive jump). Corrective saccades were also more frequent for larger saccades (Table 3). In the absence of target jumps, corrective saccades were generated more often after a hypo-metric first saccade (data not shown), in accord with previous reports (Prablanc et al., 1978; Viviani and Swensson, 1982; Becker, 1989, 1991).
We measured the percentage of correction of e(2) for trials with corrective saccade.
The results are reported in Figure 5B (hatched bars). The amount of correction introduced by the corrective saccade was between 70% and 80% for all saccade amplitudes, and thus compensated directly for most of the visual fixation error introduced by the target jump.
This result seemingly contradicts the results reported earlier on the final eye position correction. We found that even for trials with corrective saccades, the final eye position compensates for less than one-third of the target jump as if the eye position following the memorized saccade e(3) “loses” part of the compensatory effect of corrective saccades (compare Figs. 5B, hatched bars, 3, white columns). Even more strikingly, for small saccade, the corrective saccade compensate for 80% of target jump while the final correction is null. This is due to the fact that the effect of the corrective saccade is antagonized by the memorized saccade (see next section).
Contribution of the memorized saccade
For trials without corrective saccades, the memorized saccade vector is completely responsible for the final eye position correction. For trials with corrective saccades, on the contrary, the correction introduced by the memorized saccade is negative, i.e., the saccadic vector (e(3) − e(2)) is adjusted in the direction opposite to the target jump (Fig. 5B, gray columns). This is simply a question of geometry, as illustrated on Figure 1G: The corrective saccade almost completely compensates for the target jump (Fig. 5B, hatched columns) while the final eye position is only corrected partially by the target jump (Fig. 3, white columns). Thus, the saccadic vector is necessarily corrected in the direction opposite to the final eye position, and in fact annihilates most of the effect of the corrective saccade (Fig. 5B, gray columns). Since the correction introduced by the corrective saccades does not significantly depend on the first saccade amplitude (Fig. 5B, hatched columns), while the correction of the final eye position (e(3)) increases with it (Fig. 3, white columns), it follows that the negative correction decreases with the size of the first saccade, as observed in Figure 5B.
This effect is predicted by the model: If the corrective saccade vector endpoint completely compensate for the target jump, i.e., if Δ2 = jump + noise (see Materials and Methods), the third saccade that will bring the eye to the second target, Δ3 = e(3) − e(2) should be on average: Since (ks − 1) is negative, the average memorized saccade vector is corrected in the direction opposite to the jump, in contrast to the final eye position.
To measure the relative contribution of the visual feedback and the efferent copy of the motor command in the control of sequences of eye movements, we introduced artificial visual fixation errors by moving the target during saccadic eye movements and measuring its effect on a subsequent memorized saccade. We found that human subjects rely neither completely on the sensory feedback nor on the efferent copy. Rather, their behavior is compatible with that of a controller keeping an internal estimate of eye position, updating this estimate using the efferent copies after each eye movement, and correcting it with the noisy sensory feedback.
In addition, we found that the relative contribution of the visual feedback and the efferent copy (and/or proprioceptive feedback) is not fixed but varies with the amplitude of the eye movement, in a way that is quantitatively predicted by a Kalman filter model. As the amplitude of the eye movement increases, the motor noise also increases, both in absolute value and in its relative contribution to the variability of the saccade. As the efferent copy becomes less reliable compared with the visual feedback, the confidence given to the visual feedback, and thus the influence of a target jump on the eye position estimate increases.
Also in agreement with the rather consensual claim that saccades are ballistic movements, we found no evidence of an on-line correction of the saccadic eye movement due to the target jump. Other studies found that target displacement could introduce small perturbations, but they appear only for very large saccades (≥30°) and strong perturbations of target position (Becker, 1991; Gaveau et al., 2003). In the presence of a corrective saccade, we observed that the visual error is almost entirely compensated for, while part of this correction is annihilated by the next memorized saccade. Thus, we found an interesting dissociation between the partial update of the internal efferent copy and a more “automatic” corrective saccade system, directly driven (at least in part) by the visual fixation errors.
By measuring the amount of fixation errors transmitted to the next saccade, we were able to estimate the reliability of the efferent copy in saccadic eye movements. We found that this reliability is very high, i.e., only a small portion of the movement variance corresponds to unpredictable motor noise in the eye plant. As a result, most of fixation errors can be corrected by internal loops from one saccade to the next. This is in agreement with previous studies showing that saccade vectors are partially or totally corrected based on previous fixation errors during sequences of saccades in complete darkness (Bock et al., 1995; Ditterich et al., 1998).
Another study based on the covariances between saccade end-points and kinematic properties of the movement found that a large portion of end-point variance in saccades is indeed due to target localization errors, not motor noise (van Beers, 2007). This last study found, however, a stronger contribution of motor noise compared with the result reported here. This mismatch could be due to the fact that proprioceptive feedback contributes to decrease the variance of the “efferent copy” in our study. Alternatively, it could be due to the very different methods used for measuring these variances. Van Beers used the covariance between saccade end-point and the cinematic properties of the saccade to estimate the contribution of target localization error and motor noise. Part of our own measure of “target localization errors” could in fact correspond to errors in planning the motor command that could be in a motor frame of reference but still be predicted by the efferent copy.
Numerous cortical areas are implied in the cortical network of saccadic movement such as frontal eye field (FEF), supplementary eye field, dorsolateral prefrontal cortex, parietal eye field, cerebellum and different subcortical regions as the superior colliculus and the brainstem reticular formation (Gaymard et al., 1998; Quaia et al., 1999; Munoz and Fecteau, 2002; Dorris et al., 2007). Previous studies suggest that the forward and inverse models used for sensory-motor control involve the cerebellum and parietal cortex (Zee et al., 1980; Wolpert et al., 1998; Imamizu et al., 2004; Bursztyn et al., 2006). Meanwhile, the anterior intraparietal sulcus is critical for dynamic error detection during goal-dependent reach-to-grasp arm movements (Tunik et al., 2005). A compelling signature of an internal model for eye position is the “visual remapping” observed in the lateral intraparietal area (LIP), whereby cells starts responding to memorized saccadic targets or salient stimuli far outside of their receptive field before an eye movement that would bring this position into their receptive field (Duhamel et al., 1992; Colby et al., 1995). So, an “internal representation” of future targets is updated by an efferent copy of the motor command sent to the eye plant. Corollary discharges (i.e., efferent motor commands) used in sequences of eye movements involve FEF (Sommer and Wurtz, 2006; White and Snyder, 2007), an area strongly interconnected with LIP (Stanton et al., 1995). Thus the parietal cortex contains an internal model of target position, the question being whether this remapped target positions is also corrected by sensory feedback and modulated by the reliability of sensory and motor signals, or is an “open loop” system corresponding to a forward prediction. Recording neural activities in parietal, premotor areas and cerebellum in animals performing double step saccades with target jumps could answer this question and untie different signals that were previously confounded, such as the internal estimate of eye position, the forward prediction, motor errors and planned corrective saccades.
S.D. was supported by the IST European consortium project BACS FP6-IST-027140, The Marie Curie Team of Excellence Grant BIND MECT-CT-20095-024831, and ANR Jeunes Chercheurs Grant JC05-54686. J.-R.D. was supported by National Science Foundation Grant BCS0346785 and by Agence Nationale de la Recherche Grant ANR-05-NEUR-024-01. J.M. was supported by the Fondation pour la Recherche Médicale FDT20070910790.
- Correspondence should be addressed to Sophie Deneve, Group for Neural Theory (GNT), 3, rue d'Ulm, 75005 Paris, France.
- Abrams et al., 1989.↵
- Ariff et al., 2002.↵
- Baddeley et al., 2003.↵
- Bahill et al., 1975.↵
- Becker, 1989.↵
- Becker, 1991.↵
- Bock et al., 1995.↵
- Bridgeman et al., 1975.↵
- Bursztyn et al., 2006.↵
- Colby et al., 1995.↵
- Connolly and Goodale, 1999.↵
- Denève et al., 2007.↵
- Desmurget and Grafton, 2000.↵
- Ditterich et al., 1998.↵
- Dorris et al., 2007.↵
- Duhamel et al., 1992.↵
- Ernst and Banks, 2002.↵
- Gaveau et al., 2003.↵
- Gaymard et al., 1998.↵
- Gnadt et al., 2001.↵
- Goodale et al., 1986.↵
- Imamizu et al., 2004.↵
- Kalman and Bucy, 1961.↵
- Karn et al., 1997.↵
- Lewis et al., 2001.↵
- Munoz and Fecteau, 2002.↵
- Pierrot-Deseilligny et al., 2003.↵
- Prablanc and Martin, 1992.↵
- Prablanc et al., 1978.↵
- Quaia et al., 1999.↵
- Robinson, 1964.↵
- Robinson, 1975.↵
- Sabes, 2000.↵
- Saunders and Knill, 2004.↵
- Schmidt et al., 1979.↵
- Sommer and Wurtz, 2006.↵
- Stanton et al., 1995.↵
- Thiele et al., 2002.↵
- Todorov, 2004.↵
- Tunik et al., 2005.↵
- van Beers, 2007.↵
- van Beers et al., 1999.↵
- van Beers et al., 2004.↵
- Viviani and Swensson, 1982.↵
- White and Snyder, 2007.↵
- Wolpert and Kawato, 1998.↵
- Wolpert and Ghahramani, 2000.↵
- Wolpert et al., 1995.↵
- Wolpert et al., 1998.↵
- Zee et al., 1980.↵