Abstract
The brain makes use of noisy sensory inputs to produce eye, head, or arm motion. In most instances, the brain combines this sensory information with predictions about future events. Here, we propose that Kalman filtering can account for the dynamics of both visually guided and predictive motor behaviors within one simple unifying mechanism. Our model relies on two Kalman filters: (1) one processing visual information about retinal input; and (2) one maintaining a dynamic internal memory of target motion. The outputs of both Kalman filters are then combined in a statistically optimal manner, i.e., weighted with respect to their reliability. The model was tested on data from several smooth pursuit experiments and reproduced all major characteristics of visually guided and predictive smooth pursuit. This contrasts with the common belief that anticipatory pursuit, pursuit maintenance during target blanking, and zero-lag pursuit of sinusoidally moving targets all result from different control systems. This is the first instance of a model integrating all aspects of pursuit dynamics within one coherent and simple model and without switching between different parallel mechanisms. Our model suggests that the brain circuitry generating a pursuit command might be simpler than previously believed and only implement the functional equivalents of two Kalman filters whose outputs are optimally combined. It provides a general framework of how the brain can combine continuous sensory information with a dynamic internal memory and transform it into motor commands.
Introduction
In less than a quarter of a second, the motion of a tennis ball hit by an opponent elicits smooth pursuit eye movements with appropriate speed and direction. This is despite very noisy motion information provided to the pursuit system by motion-sensitive areas (Osborne et al., 2005, 2007). To overcome this noise, the brain integrates stimulus information over time across large populations of neurons (Treue et al., 2000), which improves the reliability of sensory information (Snowden and Braddick, 1991; Perrett et al., 1998). How exactly this integration of information occurs and how it contributes to the movement dynamics remain unknown. To fill this gap, we propose a new model for the sensory estimation of visual motion signals through Kalman filtering (Kalman, 1960) and show as a proof-of-concept that such a simple mechanism can reproduce many seemingly unrelated findings regarding the control of smooth pursuit eye movements.
The smooth pursuit system also relies on prediction of target motion to overcome sensory processing delays and to accurately follow the target. Predictive smooth pursuit can be observed during sinusoidal pursuit in which the phase lag between eye and target velocity is almost zero (Dallos and Jones, 1963). They are present before target motion onset in which the eyes start to move smoothly before the start of expected target motion (Barnes and Asselman, 1991). They are found during the transient disappearance of a moving target in which eye velocity is maintained at non-zero values and is predictively increased before target reappearance (Bennett and Barnes, 2003; Orban de Xivry et al., 2006; Coppe et al., 2012). These different aspects of predictive smooth pursuit have been viewed as different predictive systems (i.e., short-term vs long-term prediction; Barnes, 2008) that are believed to be mechanistically very different from visually guided smooth pursuit. How the visual and predictive signals are combined in the brain remains essentially unknown. Previous models combine the visually guided and predictive smooth pursuit responses by means of an artificial switch mechanism (Barnes, 2008). In addition, those models were purely deterministic while the brain has to deal with noisy sensory and internal signals.
Here, we specifically designed a model describing sensory and memorized motion processing of noisy sensory inputs and tested its principles for smooth pursuit eye movements. Our model is based on two Kalman filters: (1) one estimating retinal motion for visually guided movements; and (2) one computing a dynamic internal representation of target motion (Orban de Xivry et al., 2008) for predictive movements. The measure of uncertainty associated with sensory and predictive signals is used to combine those signals in a statistically optimal way (Ernst and Banks, 2002; Vaziri et al., 2006; Ronsse et al., 2009). The emerging properties of our stochastic model account for several aspects of pursuit dynamics, but similar mechanisms could be at play in controlling predictive and sensory-driven movements of other effectors, such as the arm or head.
Materials and Methods
Global structure
The model aims at estimating the current retinal slip (RS) through two different mechanisms: (1) Kalman filtering of noisy (delayed) sensory inputs; and (2) prediction of future RS. In Kalman filtering, noisy sensory inputs are combined with the previous estimate of RS to refine this estimation. Noisy sensory inputs are delayed by 80 ms (Krauzlis and Lisberger, 1994). The predicted RS is obtained from an efference copy of eye velocity and an internal representation of target motion (obtained from memory if available). Estimated and predicted values of the RS are then combined in a statistically optimal manner (Bayesian integration) and processed similarly as in previous models. We will develop the details of the model below.
More specifically, the general structure of the model is shown in Figure 1. The model takes target velocity as its input and produces eye velocity as its output. The model consists of three main parts. (1) Visual processing computes an estimation of the sensory RS (
Global structure of the model. Sensory pathway (red): visual processing uses the noisy target velocity and eye velocity to compute an estimation of the sensory RS (
Sens) and its uncertainty (ΣSens). Predictive pathway (blue): the internal representation block constructs a prediction of the estimated RS (
Mem) and an estimate of its uncertainty (ΣPred) from past sensory information and the current estimation of the eye velocity (coming from efference copy and an internal model of the eye; Ghasia et al., 2008). The motion integration step generates the motor command, using both sensory and internal RSs. This command is sent to the premotor system and then to the eye plant.
The processing of the motor command is a simplified version of classical pursuit models (Krauzlis and Lisberger, 1994). It is transmitted to the premotor system, which includes a pathway with a gain element T1 and a second parallel pathway with a neural integrator. Its output is sent to both the eye plant producing the eye velocity signal and an internal model that predicts the eye velocity that will be produced by these motor commands. The internal estimation of future eye velocity is combined with the estimation of the sensory RS and stored in memory to compute the internal representation of target motion for the future trials. The eye plant approximates the dynamics of the eyeball, the extraocular muscles, and the tissues that surround them. Its transfer function is composed of a second-order function, with time constants T1 and T2 fixed to 170 and 13 ms (Robinson, 1976; Zee and Robinson, 1979; Robinson et al., 1986). There is no noise added to the eye plant (no motor noise simulated).
Sensory pathway
The noisy RS as input.
The RS is estimated from the slip of the image of the target on the retina, i.e., by subtracting eye velocity from target velocity (Fig. 2). Information about RS appears to be noisy (Osborne et al., 2005, 2007; Stein et al., 2005; Osborne and Lisberger, 2009; Gold and Watanabe, 2010). In the middle temporal area (MT), some neurons are tuned for speed (Maunsell and Van Essen, 1983; Nover et al., 2005). The tuning curves are broader for neurons coding higher speed (Maunsell and Van Essen, 1983; Nover et al., 2005). Therefore, representation of higher speed might be more variable than lower speed.
Visual processing pathway. Eye velocity and target velocity are compared to compute the RS (RSdet; Eq. 1). Multiplicative and additive Gaussian noises are then added to generate the sensory signals sent to the pursuit system, before a Kalman filter estimates the sensory RS, 
Sens.
To obtain a noisy sensory input, we computed the actual RS (RSdet) and added both additive and multiplicative noise to the signal. With this simplified computation, we make the assumption that the problem of motion integration is solved. A Bayesian account of the motion integration stage has been proposed in other studies (Dimova and Denham, 2009; Bogadhi et al., 2011). Additive noise was used to specify a baseline sensory noise level, and multiplicative noise was used to simulate signal-dependent noise (Harris and Wolpert, 1998). This resulted in the following expression:
with RSdet the deterministic value of RS (current target velocity minus current eye velocity), δadd ∼N(0, σsens,add2), and δmult ∼N(0, σsens,mult2). This noisy signal was then processed (visual processing box in Fig. 1) to obtain an estimate of the RS (
Generative model of the sensory world.
To compute an optimal estimate of the sensory RS through Kalman filtering, a generative model about the evolution and observation of this variable is needed. In this framework, the evolution of the RS signal is described as a random walk by the following equation:
where RSSens represents the internal estimate of the RS, and θk is an additional process noise with θk ∼N(0, Q2) representing the state variability.
In addition, the brain gets some observation of RSObs that represents a noisy version of RSSens:
where γk and vk are the multiplicative and additive measurement noises with γk ∼N(0, D2) and vk ∼N(0, R2). Equations 2 and 3 represent the generative model. The noise characteristics (Q, D, and R) of the generative model are critical for the implementation of the Kalman filter (see below, Eqs. 5, 6).
Optimal estimation of the sensory RS.
This section details the actual implementation of the Kalman filter that is used to estimate the sensory RS. The Kalman filter aims at estimating a hidden variable from a noisy input. In our simulations, the observed input is RSNoisy computed following Equation 1. We hypothesize that σsens,mult and σsens,add (Eq. 1) are equal to D and R (Eq. 3). That is, the brain knows the statistics of the noise that perturbs the retinal signal, which is required to tune the Kalman filter. However, an exact knowledge of those noise parameters is not crucial, as we will discuss at the end of Results (Beck et al., 2012).
Given the observed value of the RS (RSkNoisy) and the previous estimate of the RS (
where RSkNoisy is the noisy signal of RS observed by the brain at time k (with a sensory delay of 80 ms), and ηt is the internal noise of the estimation with ηt ∼N(0, Ωn2).
Following Kalman theory and because the different noises are Gaussian and uncorrelated, the optimal value for Kk can be computed as follows (Todorov, 2005; Izawa et al., 2008):
with
where ΣkSens is the estimated error variance. This estimation will be used to assess the uncertainty associated with the estimation of
Predictive pathway
One hundred fifty milliseconds into the future.
The output of the predictive pathway corresponds to the measured RS that is expected 150 ms later if eye velocity did not change. This 150 ms period gives the predictive pathway an advance of 70 ms with respect to the actual RS. The remaining 80 ms compensate for the visual feedback delays. Target velocity is computed through Kalman filtering in a way similar to the estimation of RS. To do so, we also need current eye velocity (ėkeff) from an efference copy signal that can be estimated through an internal model of the eye plant (Skavenski and Robinson, 1973; Robinson, 1981). Therefore, the output of the predictive pathway at time k is:
where
Noisy target velocity as input.
In the absence of visual information (e.g., blanking of the target), there is good evidence suggesting that a prediction of future target motion can be used to drive the smooth pursuit system (for review, see Barnes 2008). In our model, the predictive pathway receives noisy target velocity as an input:
where TV is the actual value of target velocity, εadd ∼N(0, σpred,add2), and εmult ∼N(0, σpred,mult2). The actual value of TVk is not available to the system and is approximated by summing
Generative model of target motion.
To compute an optimal estimate of target velocity through Kalman filtering, a generative model about the evolution and observation of this variable is needed. This model represents the current knowledge of the statistics of the target velocity, i.e., how target velocity will evolve over time:
where TVkPred is target velocity, and uk represents the change in velocity from time k to k + 1 that is computed from the best prediction of target motion currently available (see Eq. 14), with αk ∼N(0, Qpred2) being the process noise and Bint = 1.
The predicted dynamics of target velocity (Eq. 8) has an additional term (uk) compared with the dynamics for RS (Eq. 2). This term represents the knowledge that future target velocity will vary with time. The control signal uk represents the expected change (i.e., computed from memory) in target velocity from time k to time k + 1 (Eq. 9).
As in Equation 3, the internal prediction of target motion needs to be compared with the actual value of target velocity. We hypothesize that the brain knows this observation is corrupted by noise, thus
with βk ∼N(0, Rpred2) and φk ∼N(0, Dpred2), the additive and multiplicative noises.
Optimal estimation of target motion.
This section details the actual implementation of the Kalman filter that is used to estimate target motion. The Kalman filter aims at estimating target motion from a noisy input. This estimate of target motion is not used during the current trial but is stored in memory (Fig. 1) and will be used for the next trial (see below, Eq. 14).
In our model, the observed input is TVNoisy that is computed following Equation 8. It is again hypothesized that σpred,add = Rpred and σpred,mult = Dpred.
Given the observed target motion (TVkNoisy) and its estimate (
where 
The variance of the prediction ΣkPred will be used to combine predicted and observed RSs in the motion pathway (see below).
Using optimal estimate in the next trial.
Because 
where δmultPred ∼N(0, Intmult2) and δaddPred ∼N(0, Intadd2). The content of the memory is used to drive smooth pursuit after combination with the output of the sensory pathway (see below). It is also used to build the internal representation of target motion that will be used on the next trial (Eq. 11).
When simulating the first appearance of a moving target, a default internal representation will be used (i.e., with value zero for the first 150 ms and directly based on
Motion pathway.
The inputs of the motion pathway come from two different sources of information [Fig. 3: sensory (i.e., retinal) and internal (i.e., extraretinal) signals]. These two sources of information are weighted in function of their estimated variability (ΣSens and ΣMem) in a Bayesian way. We approximated ΣMem by ΣPred because we noticed that ΣPred did not vary a lot from trial to trial. The optimal weighting of such stochastic variables (Ernst and Banks, 2002; Vaziri et al., 2006) is given by the following:
The model needs an initial condition for the uncertainty of the sensory and internal signals. This initial condition is based on previous experience. At movement onset, the visual feedback will be favored if the internal representation is absent or inaccurate (ΣPred is large). Practically, this is implemented by increasing the value of QPred (the process noise), which will increase the noise of the internal representation and then favor sensory information. Changing the initial conditions given to the sensory variance (e.g., after target disappearance ΣSens = ∞ as a result of the absence of retinal feedback) can thus lead to an abrupt change of relative weighting between sensory and internal signals.
Motion pathway. Sensory and predictive RS are weighted according to their uncertainty and give the estimated RS (
k). This signal is processed by a classical pursuit model (velocity motion pathway from Krauzlis and Lisberger, 1994). The output is then sent to a (leaky) integrator.
In the final steps of our model, the weighted 
The output of this image velocity motion pathway was then sent to a (leaky) integrator (called “positive efferent copy feedback loop” by Krauzlis and Miles, 1996) that is characterized by two variables: Gint and τ. Gint determines the gain of the pursuit system (gain ∼Gint2). It is set to
In the present model, the memory used for prediction stores a representation of target motion, which is dynamically updated by the Kalman filter based on past information. This contrasts with models in which short-term memories store efferent copies of eye motor commands that are replayed to replace the retinal information for predictive pursuit (Krauzlis and Miles, 1996; Bennett and Barnes, 2003; Madelain and Krauzlis, 2003). Therefore, the motion pathway is active during both visually guided pursuit and in the absence of retinal signals.
Simulation parameters.
Our model was implemented in MATLAB/Simulink (Mathworks). We ran all simulations with a fixed step size of 1 ms using the fourth-order Runge–Kutta method from Simulink (MATLAB) for numerical integration.
The SD of the different stochastic variables (Eq. 1) was tuned to produce a qualitatively correct pursuit response in a wide variety of contexts. Note that the set of used parameters represents only one possible choice inside a large manifold. Parameters were kept constant for all stimulations. The variances of the additive and multiplicative noise of the sensory signals were set to σsens,add = 10, σsens,mult = 1.5 (Eq. 1). In most instances (except for the sensitivity analysis; see Fig. 14), the variances of the additive and multiplicative noise fed to the Kalman filter matched the actual variances: R = σsens,add and D = σsens,mult. For the predictive pathway, these values were decreased by a factor 2: σpred,add = 5 and σpred,mult = 0.75 (Eq. 8) and RPred = σpred,add and DPred = σpred,mult (Eq. 10). The noise of the Kalman estimation was the same for the sensory and predictive Kalman filters: Ωn = Ωe = 0.3 (Eqs. 4, 11). From trial to trial, the internal representation of target motion was slightly corrupted with additive (Intadd = 1) and multiplicative Gaussian noises (Intmult = 0.1; Eq. 14) to simulate memory decay. Initial conditions of noise variances were set to ΣSens = 1 and ΣPred = 1. None of these parameters changed during simulations; Bint = 1 (Eq. 9).
For all trials in which no predictions were available (Figs. 4, 5, first half-cycle in Fig. 6, first pursuit trial in Fig. 8), the noises for the system dynamics (Q and QPred) of RS and TV were set to 1 (Eq. 9). For other trials when target motion prediction was available (Fig. 6 for all half-cycles except the first one, Figs. 7, 8 for all pursuit trials except the first one, Figs. 9, 10), the noise added to the dynamics of the system for TV (Eq. 9) was reduced (QPred = 0.3) because we assumed that the knowledge about target velocity decreased the associated uncertainty.
In simulations or human data, pursuit onset was detected by fitting (mean least squares regression) a piece-wise linear function on the eye velocity trace measured during an interval of 300 ms starting at stimulus onset, as follows:
where t is the time (seconds), T is the time of pursuit onset (seconds), A is the level of eye velocity before pursuit onset (degrees per second), and B is the mean acceleration during pursuit initiation (degrees per square seconds). The constants A, B, and T were considered as the free parameters of the function.
For sinusoidal pursuit (Fig. 7), the pursuit gain for a half-cycle was computed as the peak eye velocity divided by the peak target velocity. The phase was obtained as the difference in time (expressed in angular degrees) between those two peaks.
When the target was blanked (Figs. 10⇓–12), we set ΣSens = ∞, because there was no more visual input to the sensory system. This choice allows us to cancel any influence of the visual pathway on smooth pursuit response during target blanking. Therefore, we have
Results
The Kalman filtering approach allows us to simulate smooth pursuit behavior in a wide variety of contexts that range from purely visually guided pursuit to purely predictive pursuit in the absence of any visual input. In the first part of Results, we will describe how the sensory Kalman filter can account for visually guided smooth pursuit dynamics. Then, we will show how our approach of predictive pursuit allows us to unify the different aspects of predictive smooth pursuit within one simple mechanism, i.e., pursuit during sinusoidal motion, anticipatory pursuit before target motion onset, and pursuit during transient target disappearance. Finally, we will demonstrate that our simulations are very robust to changes in the parameters of the Kalman filter.
Simulations of visually guided pursuit: comparison with human data
The model performance in response to the sudden motion of a visible target is illustrated in Figure 4A. After a fixation period (∼500 ms), the target underwent a “step ramp” motion (20°/s). Different simulations are shown in gray, showing the variability of the model output attributable to the simulated sensory noise.
Simulations of visually guided pursuit initiation. A, Simulations of pursuit velocity (in gray). Target velocity (20°/s) is represented by the black dashed line. B, Relationship between sensory RS and mean eye acceleration in the interval between 80 and 180 ms after pursuit onset. Gray squares correspond to experimental data from the dataset of de Brouwer et al. (2002). Black disks correspond to simulation data (average of 30 simulations). SD is shown with vertical bars.
One of the feature landmarks of pursuit initiation is a very stereotypical initial acceleration component (Lisberger et al., 1981; Carl and Gellman, 1987; de Brouwer et al., 2002). Therefore, we analyzed pursuit acceleration as predicted by the model by simulating different target velocity steps. We compared the simulated results with experimental results from human subjects (data were reanalyzed from de Brouwer et al., 2002). This comparison is shown in Figure 4B. We measured the acceleration as the mean acceleration in the interval between 80 and 180 ms after pursuit onset (see Materials and Methods). Note that this model did not contain a specific saturation function for pursuit acceleration, as was the case in previous models (Bennett and Barnes, 2003, 2006). Despite the absence of an explicit saturation function, the model simulations closely matched smooth pursuit acceleration in humans for values of RS up to 50°/s and also exhibited a saturation effect of acceleration at ∼150°/s2. This saturation arises from the signal-dependent noise that perturbs the sensory input.
The latency of the pursuit response was 120 ± 14 ms (mean ± SD). These values were similar to those measured by Tarnutzer et al. (2007) and Rasche and Gegenfurtner (2009). The SD observed when initiating pursuit at 20°/s (500 ms after target onset) was ∼2°/s (10%). This value was also similar to those observed in previous studies (Osborne et al., 2005; Rasche and Gegenfurtner, 2009). Thus, our new model reproduced human visually guided pursuit dynamics with high fidelity only using a simple Kalman filter mechanism.
Simulations of visually guided pursuit: comparison with previous models
To directly compare our model predictions to previously published models, we simulated two classical models in Figure 5 (Robinson et al., 1986; Krauzlis and Lisberger, 1994). Simulations of pursuit initiation to a target moving at a constant velocity generated with the pursuit model of Robinson et al. (1986) were very similar to those produced with the pursuit model of Krauzlis and Lisberger (1994); pursuit acceleration and the overshoot at the end of the initiation phase were comparable. Our model did not reproduce the ringing behavior present in these previous models. However, our model appears consistent with many recent studies that did not exhibit a clear ringing behavior (Orban de Xivry et al., 2006; Spering and Gegenfurtner, 2007; Medina and Lisberger, 2009). It is very likely that the context of the experiments plays a major role in this ringing behavior (e.g., repetition of similar target trajectories).
Comparison with previous models. Initiation of pursuit in response to targets moving at 5, 10, 20, and 30°/s. Target velocity is represented in blue, simulations from the model of Robinson et al. (1986) are shown in continuous black traces, simulations from the model of Krauzlis and Lisberger (1994) are shown in dashed black traces, and simulations from our model are shown in red (average over 30 simulations).
It is worth noting that the current pursuit model does not contain an “image acceleration motion pathway,” as was the case in previous models (Krauzlis and Lisberger, 1994; Krauzlis and Miles, 1996). This image acceleration motion pathway was necessary in these previous models to generate persistent oscillations of eye velocity during sustained pursuit and to avoid large overshoots at the end of the pursuit initiation phase. In our model, it is not necessary because the weight of the signals from the predictive pathway increases after pursuit initiation. The predictive pathway limits the overshoot at the end of pursuit initiation (because the predictive system uses an estimation of the RS with an advance of 150 ms, predictive pursuit velocity will not overshoot target velocity).
Simulations of smooth pursuit in response to a sinusoidally moving target
Tracking a sinusoidally moving target results in a smaller phase lag than when tracking unpredictable target motion (Dallos and Jones, 1963; Yasui and Young, 1984; Barnes et al., 1987, 2000; Barnes and Ruddock, 1989) because the smooth pursuit system can exploit the predictable nature of target motion. Our model can achieve such small phase lags by using the memory acquired during half a cycle of target motion as an internal representation of future target motion for the next half-cycle. As can be observed in Figure 6, initially, there is no internal representation of future target motion (first half-cycle). During this period, the model relies solely on sensory signals (
Simulation of smooth pursuit velocity (gray traces) when tracking a target (dashed black line) moving with a sinusoidal velocity, with peak velocity of ±6.7°/s and frequency of 0.4 Hz.
After several such cycles, the phase lag and pursuit gain reached similar levels to those reported in human subjects. We quantified this behavior in Figure 7, comparing the pursuit gain and phase lag from our model to the ones measured experimentally in Barnes et al. (1987). Our model closely matched the experimental findings without any change in parameters.
Pursuit gain and phase lag as a function of the frequency of the sinusoidal moving target. Model predictions are represented by the gray disks. Data were obtained by averaging 30 different simulations over three half-cycles (from the 4th to the 6th half-cycle). Experimental data (black dashed lines) are adapted from Barnes et al. (1987) (their Fig. 2).
Simulations of anticipatory pursuit before target motion onset
Building an internal representation of target motion from past experience is advantageous for tracking sinusoidally moving targets but also for anticipating future target motion. Indeed, it is well established that, when the direction of target motion and the timing of its onset are known, subjects are able to move their eyes before target onset (Kowler and Steinman, 1979a,b). Similarly, in our model, after a few trials of repeated identical target motion, the memory contains enough information to construct a reliable internal representation of the future target velocity. Therefore, the model exhibits anticipatory pursuit responses to the expected future target motion, i.e., the eyes start moving before target motion onset. Figure 8A illustrates a reactive pursuit response at first target presentation with visually guided eye movements, and Figure 8B–D shows anticipatory eye movements in consecutive trials using the prediction of the future target velocity (attributable to the internal memory Kalman filter estimating target motion 150 ms into the future). As can be observed in Figure 8, there was a marked increase of anticipatory pursuit over the course of trials. In our model, anticipation arises from the use of the internal representation of target motion stored in memory that is replayed with an advance of 150 ms.
Anticipatory pursuit eye movements based on the learning of past motion trajectories. Dashed black traces represent target velocity. The estimation of target velocity given by the internal Kalman filter (TVPred) is shown in red. The internal representation of target velocity that comes from the previous trial (TVMem) is shown in blue. TVMem directly drives smooth pursuit, whereas TVPred will be used for the next trial (Eq. 14). Gray boxes represent periods when the target was blanked (gap periods of 300 ms). A, No prediction available: a visually guided pursuit eye movement is initiated (black trace), and an estimation of target velocity is computed (red trace). A default internal representation is used (blue trace). B, The previous estimation of target motion is used as the internal representation (
kMem in Eq. 7; blue trace), but because the associated uncertainty is quite important (data not shown), the anticipatory eye velocity is small. However, the estimation of target velocity is improved for the next trial (red trace). C, The anticipatory eye movement is larger than in B, and the internal estimation of target motion is still refined. D, The anticipatory pursuit movement begins earlier and is more prominent because of the larger reliability of the internal estimation of future target velocity. In all simulations, Gint was set to 0.6 during the gap periods and increased to 1 once the visual feedback of the moving target was available (80 ms after target motion).
The update of the internal representation of target motion after each trial will also naturally lead to a “history effect” (Kowler et al., 1984). This effect is observed when target motion varies from trial to trial and results in a weighted average of target motion from multiple previous trials in the internal memory. As a consequence of this history effect, anticipatory eye velocity will be stronger after two consecutive trials with fast target motion than after one trial with low and one trial with fast target motion (Poliakoff et al., 2005). Thus, our model was able to capture all aspects of anticipatory smooth eye movements thanks to an internal representation of target motion that was refined over the course of trials by a Kalman filter.
Simulations of learning by observation
The building of the internal representation can be achieved without any eye movements. Indeed, it is known that anticipatory smooth pursuit eye movements can arise after viewing but not pursuing a moving target (Barnes et al., 1997). In our model, eye movements can be inhibited during target motion by setting Gint to 0. After viewing a moving target three times, the internal representation that was built can give rise to a clear predictive smooth pursuit response (Fig. 9, solid black line). This predictive response differs from the pursuit response observed in the absence of an internal representation (Fig. 9, dashed line).
Anticipatory smooth pursuit eye movements after viewing but not pursuing the moving target. The internal representation (gray trace) was built after three repetitions of target motion without any eye movements (Gint = 0). The internal representation gave rise to anticipatory eye movements (solid black line) compared with when the internal representation of target motion is absent (dotted black line). Time of target onset is 0.5 s. Target speed is 20°/s.
Simulations of smooth pursuit during target blanking
Another important instance of predictive smooth pursuit eye movements can be observed when the moving target is temporarily blanked for several hundred milliseconds (Orban de Xivry and Lefèvre, 2007; Orban de Xivry et al., 2008). We simulate this case of predictive pursuit in Figure 10 in a 1 s target blanking paradigm.
Predictive smooth pursuit simulations with target blanking. Simulated pursuit traces (gray traces) when the target disappeared behind an invisible occluder (blanking period; gray area). Target velocity (dashed black trace) was 20°/s. A, Two typical simulations are highlighted (blue and red traces) with a large and no predictive recovery, respectively. For the simulations, Gint was set to 1 when the target was visible and then instantly decreased to 0.6 during target blanking. Gint was smaller than 1 when the target reappeared. It was reinstated to 1 when visual feedback of the target was available (i.e., 80 ms after target reappearance). B, Simulations of smooth pursuit eye movements with different levels of residual velocity. Varying the gain Gint during the blanking period illustrates the range of variability of the residual velocity during the blanking (values for Gint are 0.9, 0.84, 0.77, 0.69, 0.6, 0.5, and 0.4).
As in human subjects, simulated eye velocity decreased after target blanking until reaching an asymptotic velocity plateau. Both τ and Gint in the motion pathway (Fig. 3) determine the rate at which pursuit velocity decays toward the residual velocity plateau (Becker and Fuchs, 1985). The time constant is
Two different simulations are highlighted in blue and red (Fig. 10A), differing only in the behavior near target reappearance. The red trace illustrates a trial in which there was no predictive recovery, i.e., the pursuit model did not anticipate target reappearance. On this trial, the large acceleration occurring after target reappearance was attributable to sensory signals (visually guided acceleration). However, experimentally, there is a clear predictive reacceleration before the target reappears on most trials of such paradigms (Bennett and Barnes, 2003). Such a predictive recovery is illustrated by the blue trace in Figure 10A. This recovery results from an increase in Gint before target reappearance. The time of this increase was varied from 250 to 50 ms before target reappearance. Late increase in Gint did not yield strong predictive recovery, whereas early increase in Gint did. The existence of such an independent time control mechanism was confirmed by the specific impairment of predictive smooth pursuit eye movements in frontotemporal dementia patients (Coppe et al., 2012).
Different residual pursuit velocities have been reported in the literature for an individual subject (Orban de Xivry et al., 2006, their Fig. 8). Our model can reproduce these differences by changing the value of Gint. In Figure 10B, each simulation had a different value of Gint (ranging from 0.9 down to 0.4; see Materials and methods), resulting in different residual velocities.
Given that the model possesses a dynamic internal representation of target motion (Orban de Xivry et al., 2008), it was also able to reproduce the behavior during the blanking of an accelerating target (Bennett and Barnes, 2006; Bennett et al., 2007; Fig. 11) or a sinusoidally moving target (Whittaker and Eaholtz, 1982; Kveraga et al., 2001; Fukushima et al., 2002; Fig. 12). When the target was accelerating during the blanking period, the residual eye velocity was slightly increasing, as is the case in human recordings (Fig. 11). Note that, in our simulations, the value of Gint was constant during the occlusion period, whereas other models with a static internal representation increased the value of the gain to obtain the eye acceleration during the blanking (Bennett and Barnes, 2006). Instead, in our model, the acceleration during the blanking period was attributable to the internal representation of target motion trajectory.
Anticipatory eye movements and tracking of an accelerating target. Simulations (in gray) are compared with experimental data (black trace) with the same target conditions (target motion is represented by the dashed line). The target underwent a velocity step of 10°/s and a constant acceleration of 8°/s2 for 2 s, after the fixation period. It was blanked for 800 ms (gray box). Experimental data are adapted from Bennett and Barnes (2006).
As mentioned above, the use of a static internal representation would fail to provide appropriate target velocity signals for predictive tracking if the eye velocity is zero when the target is blanked (Orban de Xivry et al., 2008). In this case, a pursuit model with a static memory would not be able to increase its pursuit velocity, because there is no more retinal information and no velocity stored into the short-term memory. A simple example of this situation is shown in Figure 12 in which the sinusoidally moving target disappears a bit before eye velocity is zero. When the visual information of target disappearance was available (80 ms after its disappearance), the eye velocity was equal to zero. In our model, the pursuit velocity increased after target blanking thanks to the dynamic internal representation of target motion. During the blanking period, the mean pursuit gain was not as large as before (because of a decrease of Gint from 1 to 0.7 during the blanking period), but pursuit was clearly larger than zero and synchronized with the target motion, showing that the internal dynamic representation of the target motion (i.e., based on past information of the target motion) correctly updated the RS information. In contrast, a static memory would predict zero eye velocity during the blanking period (Fig. 12, dashed trace). To the best of our knowledge, our new model is the first capable to simulate this behavior.
Blanking of a sinusoidally moving target (peak velocity of ±6.7°/s and frequency of 0.4 Hz). Simulations of predictive pursuit of a sinusoidally moving target (thin dashed trace). The gray area represents the blanking period. The simulations of our model (black solid traces) are compared with a model with a static memory (i.e., 
kMem = TV0, where TV0 corresponds to target velocity immediately before the blanking; Orban de Xivry et al. 2008) that resulted in no eye velocity during blanking (thick dashed line). For the simulations, Gint was set to 0.6 until 80 ms after target reappearance (when sensory information was available). Then, it was reinstated to its initial value of 1.
In summary, the predictive pathway that is based on a trial-to-trial refinement of a dynamic internal representation of target motion allowed us to simulate the three different instances of predictive smooth pursuit that were thought to rely on different systems and to be primarily disconnected (Barnes, 2008), i.e., anticipatory smooth pursuit eye movements before target motion onset (Barnes and Asselman, 1991), predictive pursuit of a sinusoidal target motion (Stark et al., 1962), and predictive pursuit during transient target disappearance (Mitrani and Dimitrov, 1978).
Weighting of the sensory and predictive response
The model is able to modulate the weights of the sensory and predictive pathways when there is a conflict between them. For instance, an internal representation of a target moving at 10°/s can be in conflict with a visual target moving at 15°/s (Fig. 13). In this case, the smooth pursuit response will be biased by the most reliable representation. If the sensory RS has normal noise, it will bias the smooth pursuit response toward the actual target velocity (Fig. 13, solid black line). In contrast, if the target is dimmed, the internal representation will primarily dominate the smooth pursuit response (Fig. 13, dashed black line). The noise parameters are critical for determining the level of eye velocity at the end of the trial. This reweighting of the sensory and predictive pathways arises from the optimal combination of their signals (Eq. 15).
Online changes in weighting between the sensory and predictive pathways. For these simulations, an internal representation of target motion was first built by observation (10 repetitions) for a target velocity at 10°/s. For the simulation, a target with a velocity of 15°/s was then presented. Its visual characteristics were either normal (solid black line; R = 10 and D = 1.5) or the target was dimmed (dotted black line; R = 50 and D = 7.5). For these simulations, QPred = 3.
Sensitivity to noise
The introduction of stochastic noise was one of the most important features of our new model. Changing the amplitude of the noise present in the system will obviously influence the pursuit response, but changing the noise can also be interpreted differently depending on whether the brain is aware of this change or not. For example, if one increases the noise present in the system (σsens,add and σsens,mult in Eq. 1) and the estimated noise by the brain (D and R in Eq. 3) in the same way, then the pursuit response will simply become slower, with a lower acceleration. This is attributable to the Kalman filter effectively taking longer to estimate target motion as a result of the increase in noise. However, if one only changes the system noise (Eq. 1) without adjusting the estimated noise, the story is different. In this case, the brain does not correctly estimate the noise present in the system, leading to suboptimality (Beck et al., 2012). Figure 14 illustrates how much a suboptimal estimation of the noise can affect the pursuit response. When the noise in the model was overestimated, the pursuit variability decreased (Fig. 14B, blue) but the average pursuit response was little affected (Fig. 14A, blue). In contrast, pursuit variability was dramatically amplified when the brain underestimated the noise present in the system, whereas the average pursuit response was again only slightly modulated (Fig. 14, red). Thus, our results would argue that, practically speaking, the brain only needs a reasonable approximation of the noise of the system but does not require precise knowledge of it.
Effect of suboptimal noise estimation. A, Simulation responses to a 20°/s moving target (dashed black trace). Black trace shows the model simulation with correct noise estimation (σsens,add = 10, σsens,mult = 1.5). The blue dashed trace shows the pursuit response when the system noise was overestimated by the brain (4-fold noise overestimation: σsens,add = 5 and σsens,mult = 0.75). The red trace shows the pursuit response when the brain underestimated the noise present in the system (4-fold noise underestimation: σsens,add = 20 and σsens,mult = 3). In all simulations, R and D kept their original value. One hundred simulations were performed to compute the mean response and the SD around the mean for each of the three different cases (normal noise, noise underestimation, and noise overestimation). Shaded areas surrounding these traces represent the SD around the mean. B, Pursuit responses evaluated 300 ms after target onset (gray vertical bar in A) for the three simulations. The blue (respectively, red) error bars show fourfold noise overestimation (respectively, underestimation), whereas the black error bar represents the simulation with correct noise estimates. Means ± SD are represented.
Discussion
We propose a novel model integrating sensory and predictive signals that was tested on smooth pursuit eye movements. In this model, Kalman filters provide an estimate of the retinal and extraretinal inputs and their associated uncertainty. This uncertainty is used to combine the sensory and internal memory signals in a weighted manner (Bayesian integration). Kalman filtering allows our model to naturally produce both predictive and sensory-driven pursuit characteristics using a single mechanism.
Kalman filtering to estimate and combine retinal and extraretinal inputs
Previous studies (Osborne et al., 2005, 2007) have highlighted the importance of noise in the sensory signals, explaining most of the variability in the initiation of pursuit eye movements. It is also known that the nervous system lowers the impact of noise on the sensory estimation (Karmali and Merfeld, 2012). In the present model, noisy retinal inputs induce variability on pursuit latency and acceleration during pursuit initiation. During initiation, the integration of the noisy inputs through Kalman filtering helps reduce the noise and refine the internal representation of target motion.
The uncertainty associated with the output of the Kalman filter yields a straightforward way for combining retinal and extraretinal signals in a statistically optimal manner (Ronsse et al., 2009). This is consistent with experimental studies demonstrating the continuous presence of predictive signals despite the randomization of the target parameters (Kowler and McKee, 1987; Kao and Morrow, 1994). It argues against the existence of a switch between the sensory and predictive pathways (Barnes, 2008). This optimal weighting of two different noisy sources of information is a hallmark of brain function (van Beers et al., 1999; Ernst and Banks, 2002; Hillis et al., 2002; Körding and Wolpert, 2004; Vaziri et al., 2006), even at the single-neuron level (Fetsch et al., 2012). For the first time, our model combines continuous flow of sensory and predictive information to drive motor behavior, whereas a previous model used Kalman filtering for online prediction only (Shibata et al., 2005). A possible implementation of Kalman filters in the brain has been proposed previously (Denève et al., 2007), but a full model based on Kalman filters that integrates aspects of predictive and reactive motor control had never been developed. Our particular, simple model based on Kalman filters could account for all the main dynamics of predictive and reactive smooth pursuit eye movements without switching between submodels and with the same model parameters.
In the Kalman filtering approach, it is hypothesized that the brain knows the properties of the noise (i.e., its variance; Orbán and Wolpert, 2011; Gallistel and Matzel, 2013), yet if the estimated noise properties are incorrect, the estimate of the sensory input might be either biased or overly variable (Beck et al., 2012). This perspective suggests that the processing of visual information through Kalman filtering might be unreliable because of the required knowledge of noise properties. In contrast, we demonstrate here that, in the pursuit system, the dynamics are quite robust with respect to this hypothesis and that incomplete knowledge of noise properties mostly influences the variability of the response but not its average. Therefore, Kalman filtering appears to be a robust solution for the processing of sensory inputs even when the noise properties are not known accurately.
Using a dynamic representation of the expected target motion
Previous models always used a static (constant in time) memory module for the predictive component, which only allowed to make predictions at a fixed horizon (Shibata et al., 2005; Soechting et al., 2010) or which only allowed for constant velocity inputs (Bennett and Barnes, 2003; Churchland et al., 2003; Madelain and Krauzlis, 2003). For instance, Shibata et al. (2005) used a Kalman filter for online prediction of target trajectory. In contrast, in the present model, Kalman filtering was also used to build a dynamic memory from past information, even in the absence of active pursuit of target motion (Barnes et al., 1997). Indeed, the history of target motion can be used to predict future target motion when tracking a sinusoidally moving target or a similar periodic pattern (Dallos and Jones, 1963; Barnes et al., 1987; Barnes and Ruddock, 1989), when anticipating the timing of target motion (Barnes and Asselman, 1991; Wells and Barnes, 1998; Barnes and Donelan, 1999; Barnes et al., 2000), or when scaling the anticipatory eye velocity with the previous target velocity (Barnes and Collins, 2008). Our model demonstrates for the first time that all these features can stem from the existence of an internal representation of target motion and do not require separate mechanisms. The existence of such an internal dynamic representation of target motion is supported by target blanking experiments with nonlinear target motion (Orban de Xivry et al., 2008, 2009). It allowed us to simulate pursuit responses when tracking an accelerating target (Bennett et al., 2007, 2010) or a sinusoidally moving target (Fig. 11). In our simulations, a default internal representation was improved on a trial-to-trial basis thanks to Kalman filtering. This trial-to-trial refinement has been used to explain the content of motor memories for motor learning (Krakauer et al., 2006; Kording et al., 2007). The presence of such dynamic internal representation could also explain the predictive ability of human subjects in ball catching (Hayhoe et al., 2005; Diaz et al., 2013) or hitting (Land and McLeod, 2000).
In our model, the efference copy signal is essential to build the internal representation of target motion (Fig. 1). Schizophrenic patients cannot process the efference copy signal properly (Pynn and DeSouza, 2013). This impairment results in abnormalities in the awareness of actions (Frith et al., 2000) and in a deficit of sensory prediction (Shergill et al., 2005; Synofzik et al., 2010). The dependence of the internal representation on the efference copy signal might explain why schizophrenic patients are impaired in predictive smooth pursuit (Sereno and Holzman, 1993; Thaker et al., 1998, 2003; Trillenberg et al., 1998; Hong et al., 2005; Nagel et al., 2007; Adams et al., 2012) and in prediction of target trajectory (Hooker and Park, 2000; Spering et al., 2013).
The output of the predictive pathway had a lead of 150 ms with respect to the sensory signal (70 ms in advance of the actual target motion given 80 ms of sensory delay), yet we do not believe that a single time horizon for this prediction exists. Rather, we hypothesize that there are many predictions of the target motion for different time horizons, such as those used for simulations of walking (Azevedo et al., 2004; Aftab et al., 2012). Such optimal prediction of future target motion could ideally replace our prediction made on this arbitrary 150 ms horizon, although our results suggest that it might be a reasonable value.
Neural substrate of the model
Sensory areas appear to filter noise coming from sensory inputs to improve the reliability of the input over time. A model based on Kalman filtering can account for neuronal activity in visual areas during vision of natural images (Rao and Ballard, 1997). In these areas, the prior is represented by spontaneous activity (Berkes et al., 2011). Neurons in the temporal cortex also accumulate evidence for face recognition (Perrett et al., 1998). Their response to a new sensory input is biased by previous sensory inputs, i.e., by a prior (Jellema and Perrett, 2003). This is true for motion processing areas as well. Area MT plays a major role in the integration of local motion signals (for review, see Born and Bradley (2005)). Osborne et al. (2004, 2007) found evidence of information accumulation in the activity of a large population of MT neurons. The dynamics of neural activity in area MT might therefore reflect the iterative computation performed by the Kalman filter (Pack and Born, 2001; Osborne et al., 2004). However, MT neurons did not seem to discharge when blanking the target (Newsome et al., 1988), which suggests that this area is located within the visual pathway of our model.
The predictive pathway could be part of a parietofrontal network as suggested for manual interception tasks in monkeys (Merchant and Georgopoulos, 2006). For these tasks, area 7a of the posterior parietal cortex and the motor cortex appear to encode different parameters of target motion. This internal representation of target motion is then used in a predictive manner by the motor cortex (Merchant et al., 2004). For eye movements, neurons in the frontal eye field (FEF) are active during transient disappearance of the tracking target and might represent some form of internal representation of target motion for manual target interception or saccades (Barborica and Ferrera, 2003, 2004; Xiao et al., 2007; Ferrera and Barborica, 2010). The part of the FEF dedicated to smooth pursuit has been hypothesized recently to store previous expectations of target motion that can be used for smooth pursuit initiation (Yang et al., 2012). Alternatively, this prior could be interpreted as the dynamic internal representation of past target motion and could thus bias smooth pursuit initiation in the same way, as postulated in our model. In summary, the interaction between area MT and FEF might be critical to form an estimate of the RS incorporating both sensory and predictive signals.
Finally, the cerebellum exerts a large influence on the predictive pathway. First, it acts as a forward model for eye movements (Ghasia et al., 2008). Second, it may also contain an internal representation of target motion (Miles and Fuller, 1975; Suh et al., 2000; Cerminara et al., 2009) that does not depend on retinal signal (Stone and Lisberger, 1990; Suh et al., 2000).
Conclusion
Kalman filters elegantly describe in a functional manner how the brain may deal with noisy sensory inputs, how it may form memories, and how it may take advantage of uncertainty estimation to combine different signals optimally. These three principles allowed us to accurately describe visually guided and predictive smooth pursuit dynamics observed in a wide variety of tasks within a single theoretical framework.
Footnotes
This work was supported by Natural Sciences and Engineering Research Council of Canada, Canada Foundation for Innovation, the Botterell Fund (Queen's University, Kingston, Ontario, Canada), Ontario Research Foundation, Fonds National de la Recherche Scientifique, Fondation pour la Recherche Scientifique Médicale, the Belgian Program on Interuniversity Attraction Poles initiated by the Belgian Federal Science Policy Office, Actions de Recherche Concertées (French Community, Belgium), and the European Space Agency of the European Union. J.J.O.d.X. is supported by the Brains Back to Brussels program from the Brussels Region.
The authors declare no competing financial interests.
- Correspondence should be addressed to Philippe Lefèvre, Avenue Georges Lemaitre, 4, B-1348 Louvain-La-Neuve, Belgium. philippe.lefevre{at}uclouvain.be
References
In this issue
