Abstract
To evaluate the nature and possible sources of variation in sensory–motor behavior, we measured the signal-to-noise ratio for the initiation of smooth-pursuit eye movements as a function of time and computed thresholds that indicate how well the pursuit system discriminates small differences in the direction, speed, or time of onset of target motion. Thresholds improved rapidly as a function of time and came close to their minima during the interval when smooth eye movement is driven only by visual motion inputs. Many features of the data argued that motor output and sensory discrimination are limited by the same noise source. Pursuit thresholds reached magnitudes similar to those for perception: <2–3° of direction, ∼11–15% of target speed, and 8 ms of change in the time of onset of target motion. Pursuit and perceptual thresholds had similar dependencies on the duration of the motion stimulus and showed similar effects of target speed. The evolution of information about direction of target motion followed the same time course in pursuit behavior and in a previously reported sample of neuronal responses from extrastriate area MT. Changing the form of the sensory input while keeping the motor response fixed had significant effects on the signal-to-noise ratio in pursuit for direction discrimination, whereas holding the sensory input constant while changing the combination of muscles used for the motor output did not. We conclude that noise in sensory processing of visual motion provides the major source of variation in the initiation of pursuit.
- visual motion
- signal-to-noise ratio
- sensory discrimination
- motor noise
- extrastriate cortex
- MT
- information theory
Introduction
One hallmark of the visual system is its ability to distinguish reliably among subtly different signals. In the extreme, humans can detect and count single photons against a background of thermal noise in our photoreceptors (Hecht et al., 1942; Barlow, 1956), or detect misalignments of a few arc seconds in an array of objects (Westheimer, 1981; Klein and Levi, 1985). In flies, single central neurons can represent visual motion with a precision close to the limits set by diffraction and photon shot noise (Bialek et al., 1991). These and other perceptual phenomena demonstrate that visual computation can operate at a level of reliability set by the noise in early sensory processing (Barlow, 1981). Time is a critical factor in achieving such good performance: the brain can integrate over time to suppress noise and achieve more reliable decisions (Green and Swets, 1966; Roitman and Shadlen, 2002). Still, perceptual experiments offer a limited window on the precision of underlying sensory processing because they require discrete responses to indicate decisions. For example, each time point in a perceptual experiment must be derived from a separate observation, and there is no chance to evaluate the impact of correlations across time. Continuously varying motor outputs, in contrast, provide a direct and graded view of the relationship between early sensory signal processing and behavior, potentially allowing us to observe directly the evolution of sensory and motor precision with time.
Smooth-pursuit eye movements provide a continuously graded motor output that is ideal for understanding the neural basis for sensory and motor precision. Analysis of the mean behavior of pursuit has revealed some properties of the sensory input that drives the behavior (Lisberger and Westbrook, 1985; Masson et al., 1997) and has shown that the first ∼100 ms of the response is driven by the responses of the middle temporal visual area, MT (Newsome et al., 1985). Just as the mean initial pursuit response reveals properties of the sensory input that drive the behavior, so might trial-by-trial variation. Recent evidence suggests that variation in pursuit arises mainly from noise in sensory representations: the transformation of sensory signals to motor output in pursuit may add little noise (Osborne et al., 2005). If so, then just as we can “see through” human perceptual decisions about dim flashes to the noise induced by the random arrival of photons and the thermal isomerization of rhodopsin (Barlow, 1956, 1981), we should be able to analyze the trajectories of pursuit eye movements in a way that reveals the properties of sensory noise and their evolution over time.
In the present study, we examine how well small differences in target direction, speed, or onset time can be discriminated based on the evoked smooth eye movements. By estimating the time course of the signal-to-noise ratio (SNR) of the initiation of pursuit, we quantify discrimination thresholds over a wide range of sensory and motor parameters. Our findings demonstrate that we can use measures of motor output to derive the temporal evolution of sensory variables. The time course and magnitudes of the estimates are similar to those derived from perceptual experiments, providing evidence in favor of a sensory origin for variation in the initiation of pursuit.
Materials and Methods
Data acquisition.
Eye movements were recorded, using an implanted scleral search coil (Ramachandran and Lisberger, 2005), from six male rhesus monkeys (Macaca mulata) that had been trained to fixate and track visual targets. Experiments lasted 2–3 h, during which the monkeys sat in a specialized primate chair with their heads immobilized and received juice or water rewards for accurately tracking visual targets presented on a screen in front of them. All procedures had been approved by the University of California at San Francisco Institutional Animal Care and Use Committee and were in compliance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals.
Vertical and horizontal eye velocity signals were generated by passing voltages proportional to position through an analog circuit that differentiated frequencies <25 Hz and rejected higher frequencies with a roll-off of 20 dB per decade. Eye position and velocity signals were sampled and stored at 1000 samples per second on each channel. Before analysis, each trial record was inspected and rejected if a saccade occurred within the time window chosen for analysis. Data sets consisted of eye velocity responses to at least 30 and typically >100 repetitions of target motion for each condition.
Experimental design.
Bright, high-contrast visual targets were presented in a dimly lit room against the dark screen of a high-resolution analog display oscilloscope that subtended horizontal and vertical visual angles of 48 × 38°. We used several target forms. “Spot” targets were a cluster of typically 12 oscilloscope pixels subtending a 0.5–0.8° square. “Patch” targets were 4 × 4° squares with 50 illuminated pixels (“dots”) randomly positioned within the square or 7 × 7° squares with 99 dots, depending on whether the patch was located eccentric in the visual field or was centered on the position of fixation, respectively. When the dots moved, the aperture usually moved with them, creating the impression of a moving texture. In a small number of experiments, we initiated motion of the dots within a stationary aperture for 125 ms, before starting to move the aperture with the dots. This created a brief interval when the texture moved behind a stationary window, followed by a moving stimulus. Our analysis interval was chosen to ensure that we measured the response to dot motion rather than aperture motion when the dots started moving within a stationary aperture.
Experiments were presented as a series of trials, each delivering a single stimulus form and motion trajectory. Each trial began with the monkey fixating a stationary target at the center of the screen for a random interval of 700–1200 ms, providing 500 equally likely starting times for target motion. The target then underwent step-ramp motion (Rashbass, 1961) with steps of 2.5–3.7° and ramps at constant speed. For experiments designed to measure direction discrimination, target speed was 10, 20, or 30°/s, and target direction was randomly selected for each trial among a cluster of values that ranged from typically ±8–9° around a central direction in 2 or 3° steps. For example, an experiment might have a central direction that is horizontal and targets that move horizontally as well as at angles of ±3, ±6, and ± 9° relative to horizontal. In a typical experiment with different directions of target motion, all similar trajectories were preceded by the same small step of target position so that there would be no location cue for motion direction. For experiments designed to measure speed discrimination, target direction was rightward or leftward directly toward the position of the extinguished fixation point. The size of the target step was the same for all speeds, and target speeds differed by ±10, ±20, and often ±30%, around a base speed of 10, 20, or 30°/s; often all three base speeds were included in an experiment. In all experiments, target directions were balanced with the opposite motion, and trials were presented in random order so that the monkeys could not anticipate the direction or speed of the upcoming target motion.
Analysis of pursuit discrimination thresholds.
Pursuit eye velocity is a vector in time that we have recorded in horizontal and vertical components: v(t) ≡ (vH (t), vv(t)). Because we record velocity at millisecond precision, the pursuit response over an interval of time from 1 to time T creates a vector of dimension 2T. In practice, however, the vector could be reduced to a dimension of T because only one component of the response would contribute significantly to the SNR. Dimension reduction made our estimates less prone to errors attributable to limited data sample sizes. For experiments that comprised targets moving in closely spaced directions, we projected the horizontal and vertical eye velocity vectors onto vectors parallel and orthogonal to the central direction of the cluster of target motions. After verifying that the parallel component of the response did not contribute to direction discrimination, we analyzed the orthogonal component of the pursuit responses. For experiments that comprised targets moving at closely spaced speeds, we analyzed only the component of eye velocity parallel to target direction because it distinguished best among different target speeds and the orthogonal component provided mainly random noise. Finally, the component of eye velocity parallel to target direction was most suitable for determining the precision with which pursuit estimated the time of onset of target motion. Therefore, we were able to simplify the eye velocity response as a collection of T time points of the appropriate component: s ≡ {si}, where s is either vH or vV, and i indexes time.
For each speed (v) and direction (θ) of target motion within an experiment, we subtracted the average signal across trials for that target motion [〈s(θ,v)〉] from the eye velocity on each trial to form an eye velocity noise vector, δs. From the full set of noise vectors, we formed the covariance matrix of fluctuations about the mean eye velocity vector in time: where i and j index points in time and 〈… 〉θ,v is an average over all trials with target motion in direction θ at speed v. For the small changes in direction or speed in our experiments, the covariation of fluctuations about the mean trajectory did not depend on θ and v, so we obtained a better estimate of the covariance matrix by lumping all of the trials together.
The fluctuations δsi were distributed very nearly as Gaussian random variables, implying that the covariance matrix provides a complete description of their statistics (Osborne et al., 2005). We represent the mean eye velocity signals at a given time (i) for target motion of a given direction (θ) and speed (v) as mi(θ,v) ≡ 〈si〉θ,v. From the mean eye velocity signals and the covariance matrix of the noise, we define the SNR for discrimination between, for example, two different target directions, θ and θ′:
Here, C−1 is the inverse of the covariance matrix, i and j are index time (as above), and Δmi(θ,θ′) = mi(θ,v) − mi(θ′,v) is the difference in the mean eye velocity at time i for target motion in two different directions.
In principle, the SNR is defined exactly by Equation 2. In practice, inverting the full covariance matrix may introduce systematic errors when data sets are small. To deal with this problem, we used our largest data sets to test our ability to estimate the SNR using both the exact expression and a series of approximations. The crudest approximation neglected the fact that there are strong correlations across time in the behavior and treated each time point in isolation: where σi2(θ) denotes the variance of δsi across trials with target direction θ. Estimates using Equation 3 did not capture the true SNR of the data early in the pursuit response in our region of interest. Next, we allowed for the influence of temporal correlations in the noise by using a covariance matrix, but in a reduced form. We approximated the inverse of the covariance matrix by using just one dominant eigenvalue or, guided by the results of Osborne et al. (2005), the three dominant eigenvalues: where the uμ ≡ {uiμ} are the eigenvectors that correspond to the eigenvalues, λμ. These successive approximations led to increasingly stable estimates of the SNR, suggesting that we were indeed capturing more of the available information. Analysis of our largest data sets confirmed that approximating the inverse covariance matrix using Equation 4 yielded robust estimates of the full SNR for our smallest sample sizes. We therefore applied this approximation to the analysis in the rest of the study, satisfying the desire to take account of the temporal correlations in the initiation of pursuit while avoiding potential errors introduced by using the full covariance matrix.
In our experiments with targets moving in different directions, the angular difference between motions was small, and our data showed that the direction of eye velocity at the end of the open-loop interval rotated nearly perfectly with the target direction such that Δm(θ,θ′) ∝ θ − θ′. Given Equation 2, this predicts a relationship that we found experimentally, namely that the SNR for pairs of angles (θ,θ′) scales as the square of the angular separation of the pair: where Kθ(T) is a function of time that does not depend on the pair of angles. We used this relationship to rewrite the SNR as an effective threshold for reliable discrimination of target direction from the eye movement vector, defining threshold as the difference in direction |θ − θ′| = Δθthresh(T) that would generate SNR = 1 (equivalent to 69% correct in a two-alternative forced-choice paradigm): Similarly, for small differences of target speed about some reference speed v0, we expect a relationship that we found experimentally: Δm(v,v′) ∝ (v − v′)/v0. As a consequence, and where Δvthresh(T) is measured as a fraction of the base speed.
The time of target motion onset was randomized in all experiments, forcing the monkey to use target motion onset to trigger the initiation of pursuit. Thus, the time of pursuit initiation could be used to determine an effective behavioral threshold for discriminating the time of motion onset. Variation in the estimates of the time of target motion onset (t0) of the brain should cause the entire eye velocity trajectory to be shifted forward or backward in time (Osborne et al., 2005). Therefore, the eye velocity vector for a response that estimates a motion onset time of t0, m(t0) = 〈s〉t0, should be related to the trajectory for a response that estimates a slightly different onset time, tt′0, such that Δm(t0,t′0) =(t0−t′0)·dm(t0)/dt0. The SNR relevant to discriminating timing differences, by analogy to Equations 5 and 7, is given by the following: and the target motion onset timing difference just discernable from pursuit is as follows: For each parameter of target motion (direction, speed, and timing), K(T) was computed by averaging across the values computed for all pairs of target motions in a given experiment. Error bars on the SNR and quantities derived from it (i.e., threshold and K) were computed across different pairs of target trajectories and by bootstrapping from randomly drawn fractions of the data (Strong et al., 1998).
Information theoretic analysis.
The value of threshold at each moment in time can be taken as a measure of the overall level of internal noise in pursuit and can be used to compute the information capacity of the system for the whole range of possible target motions. If we assume that noise has a Gaussian distribution, then we can translate behavioral threshold into a measurement of information in bits about the target motion present in the eye movement. For example, if target motion were drawn uniformly around a circle of 360° and we measure the threshold Δθthresh(t) in degrees, then the information about target direction present in the eye velocity at time t can be approximated as follows (Rieke et al., 1997): We formed similar equations for information about speed and onset time by substituting appropriate ranges in the first term and threshold values in the second term of Equation 11. Information estimates were corrected for finite data sample effects according to methods of Strong et al. (1998).
Noise models.
In almost all of our data analysis, we created a covariance matrix from the eye velocities measured during the initiation of pursuit, without attempting to subtract any “background noise” such as found during fixation. In a single analysis shown in Figure 5, we isolated contributions to threshold from different internal noise sources by recomputing threshold using different models of pursuit noise generated from the data itself. In the study by Osborne et al. (2005), we found that variation in eye movement appears to arise from two separable noise sources, one that is specific to pursuit and another that is background noise present during fixation as well as during pursuit. Velocity fluctuations during fixation are small and have a structure that is consistent with a normally or Gaussian-distributed stationary noise process that we quantified by forming the covariance matrix of eye velocity fluctuations during the fixation interval that precedes target motion onset (Cbkgrnd). We tested the assumption that noise during fixation is an adequate model for the background noise component in pursuit by subtracting Cbkgrnd from the covariance matrix of pursuit noise (C in Eq. 1). This procedure isolated the low dimensional component of velocity fluctuations (ΔC = C − Cbkgrnd) that is present during pursuit but not during fixation. We then created inverse covariance matrices based on Cbkgrnd and ΔC, respectively, and used these to compute the contributions to the SNR from the two noise components individually. The inverse covariance matrices were computed as follows: where CX was either Cbkgrnd or ΔC, and the vectors uμ were the eigenvectors of C as above. We then computed the SNR and threshold for each noise model in the same way as we had for the data. This creates predictions of thresholds for two models with the same temporal structure, namely that given by the eigenvectors that represent the eye movements during pursuit. However, the scale of the noise in the two models is derived from the noise present either during fixation or during pursuit.
Results
In response to a target that has begun moving at a constant speed in a fixed direction, pursuit has a latency of ∼100 ms after which the eyes accelerate smoothly until eye velocity approximates target velocity. An example of this behavior appears in Figure 1, where the target moved to the right and slightly upward at a speed of 20°/s. The variation in the response to the same target motion is illustrated in Figure 1, B and C, for the first 300 ms after the onset of target motion. Each horizontal line in the color maps shows the response in a different trial over the course of a day's experiment, and eye velocity is indicated by the color of each pixel. Inspection of the graphs indicates that the initiation of pursuit is somewhat variable from trial to trial and that the variation is correlated across time within each individual trial: responses that start with lower or higher eye speeds tend to maintain lower or higher eye speeds throughout the first 200 ms of the response.
We will analyze the first 200–300 ms of eye movements, focusing on the initial open-loop interval of pursuit. The open-loop interval is the part of the response that is driven by the initial visual motion before there has been any opportunity for visual feedback. Previous analyses have indicated that the duration of the open-loop interval is somewhat longer than the latency of the pursuit, ranging from 93 to 139 ms after the initiation of pursuit (Lisberger and Westbrook, 1985). We confirmed these numbers in three of our monkeys. To ensure that we captured the entire relevant interval and to standardize our analysis across monkeys, we have chosen an analysis interval of 125 ms that may go slightly beyond the end of the open-loop interval in some data sets. However, none of our results or conclusions depended strongly on whether we made our measurements 100 or 125 ms after the onset of pursuit.
Quantitative analysis of variation in the initiation of pursuit
For each experiment, we formed the covariance matrix of fluctuations about the mean movement trajectories in time. The matrix in Figure 2A displays the temporal correlations in the vertical component of eye velocity from an experiment in which targets moved in directions that were ±3, ±6, and ± 9° relative to horizontal. The variance of eye velocity was small in the 100 ms interval before the onset of target motion, during fixation. During this interval, the covariance matrix has a structure that is consistent with a stationary noise process, with ∼80 eigenvalues that are significantly different from zero and temporal correlations restricted mostly to ∼10 ms (Osborne et al., 2005). After the onset of target motion, as pursuit begins, correlations between velocity fluctuations extend across longer times, and this appears as structure in the covariance matrix at greater and greater distance from the diagonal (Fig. 2A). Analysis of the variation after the onset of target motion revealed a low-dimensional structure indicative of longer temporal correlations. Normalizing all eigenvalues of the full covariance matrix so that their sum was 1 and ranking them in order of size shows that the covariance matrix during pursuit is dominated by the three largest modes (Fig. 2B, large symbols): these accounted for ∼90% of the variance of the pursuit response 125 ms after pursuit onset. As expected from the observation that there are long temporal correlations in the variation of pursuit during initiation (Fig. 1), the time courses of the three main eigenvectors of the covariance matrix are fairly broad (Fig. 2C).
As described in Materials and Methods, we performed our analyses on the component of eye movement that yielded the most variation with target movement: the component orthogonal to the central direction of target motion in experiments designed to determine thresholds for discriminating target direction, and the component parallel to target motion for experiments or analyses designed to determine thresholds for discriminating target speed and motion onset timing. For our largest data sets, we could verify that this choice made no difference to the estimate of the SNR, but for smaller data sets, the reduction in dimensionality reduced our susceptibility to systematic errors.
SNR and threshold for discriminating target direction, speed, and onset time
In the same sense that perceptual thresholds for discriminating different directions, speeds, and onset times of target motion can be determined through analysis of discrete actions such as verbal reports, we have determined thresholds for sensory discrimination as seen through measurements of a continuous motor output. We have computed the SNR to estimate how small a difference in target direction, speed, or onset time can be discriminated by observing the pursuit response and to establish the time courses over which thresholds are refined. The temporal correlations in the trial-by-trial variation of pursuit necessitate the treatment of eye velocity as a vector in time and argue for use of covariance matrices to determine the SNR (Eqs. 4–6) versus the more conventional approximation of computing the SNR on the basis of the mean and variance at each time point (Eq. 3).
Figure 3A plots the SNR as a function of time for an experiment with seven target directions that differed by ±3, ±6, and ±9° from a central direction that was horizontal. Initially, the SNR was computed separately for each pair of target directions. We then refined our estimates of the SNR by averaging the individual responses across pairs of different target directions with the same directional spacing. The resulting family of curves shows that the SNR was larger for pairs of target directions that differed by larger angles. As predicted from Equation 5, the functions SNR(t) for different directional separations collapsed into a single function [Kθ(t)] when normalized for the square of the directional separation for each pair of target directions (Fig. 3, continuous curves), validating the use of Equation 6 to compute the time course of the effective threshold for discrimination of directions of target motion by pursuit [Δθthresh(T)]. Figure 3C shows an example of the time course of the direction threshold for pursuit for one monkey (black curve), along with an error surface (gray) showing SDs. In this and most examples, thresholds were lower and less noisy when estimated from the three largest eigenvectors of the covariance matrix (Fig. 3C, solid curve with gray error surface) than when estimated from temporally local variance using Equation 3 (Fig. 3, dotted curves) or from the single largest eigenvector (Fig. 3C, dashed curve), particularly early in the movement. All methods tended to yield similar threshold values later in the movement for reasonably sized data sets.
Figure 4A–C shows effective thresholds obtained using pursuit to discriminate target direction, speed, and motion onset time in single experiments for one monkey. Figure 4D–F superimposes mean-value threshold curves for many experiments in multiple monkeys. For all three parameters, thresholds were undefined until after pursuit was initiated, ∼90–100 ms after the onset of target motion. Thresholds for discriminating when the target began to move (Fig. 4C,F) reached a steady value just after pursuit initiation, implying that eye movement initiation is tightly locked to target motion onset and that subsequent features of the pursuit trajectory do not add information about onset time. For target speed (Fig. 4B,E) and direction (Fig. 4A,D), in contrast, thresholds remained high and uncertain during the first 20–40 ms of the movement, because eye velocity is small compared with background noise levels. Then, thresholds dropped quickly as movement progressed. Smaller differences in the direction or speed of target motion could be resolved as the movement was observed over longer intervals. Thresholds reached or were close to their minima by the end of the open-loop interval, ∼225 ms after the onset of target motion. Thereafter, thresholds for direction and speed improved only very slowly. Note that there are only tiny differences in thresholds 200 versus 225 ms after the onset of target motion (100 versus 125 ms after the onset of pursuit).
The superimposed threshold curves across all six monkeys in our experiments showed a fairly consistent time course (Fig. 4D–F) and reached similar values at the end of the analysis interval (Tables 1⇓–3). One animal (Au) had higher thresholds for all motion parameters. Another animal (Zp), who had previously participated in experiments on learned timing in pursuit, had the lowest threshold for detecting the time of target motion onset. Compared with direction and onset timing thresholds, those for speed were less consistent across animals, but fewer data sets were collected. The time course of behavioral performance indicates that timing precision can be better than 10 ms based on even the earliest eye movements. Direction precision is better than 3°, and speed precision is as small as 11–15% of mean target speed as the open-loop period ends.
Why do thresholds improve as a function of time during the first 125 ms of pursuit? Is noise reduced by integrating across time, or do small differences in mean pursuit trajectory become more discriminable with time simply because the mean eye velocities grow? To distinguish these two possibilities, we separated the variation in our data into “background” and “pursuit-related” components of the noise (co)variance and asked how discrimination thresholds would vary if only one or the other component were present (see Materials and Methods). Here, we compare the threshold computed from the measured eye velocity in the initiation of pursuit (Fig. 5, black curve) with those predicted by models computed separately for the background noise and the noise restricted to pursuit. We find (Fig. 5) that the dramatic decline in discrimination thresholds in the first 20–30 ms of the actual pursuit response (black curve) is of the form expected from the background noise alone (blue curve). Early in the open-loop interval, discrimination thus improves simply because the mean eye velocity is rising above the background noise. Very quickly, however, the overall behavior in the pursuit data (black curve) crosses over to what we expect from the pursuit-related noise alone (red curve), as if the background noise had become irrelevant. This pattern of results implies that some of the approximate twofold improvement in pursuit thresholds over time cannot be attributed solely to growth of eye velocity relative to background noise, but rather reflects motor system access to estimates of target motion that improve with time.
Information about target motion from pursuit
We next used the concepts of information theory to convert our measurements of threshold into estimates of the overall capacity of the pursuit system to respond to differences in target motion. Converting each threshold to a measure of behavioral information in bits using Equation 11 also allowed us to evaluate discrimination of direction, speed, and timing in the same units and to compare time courses directly. Estimating the information capacity of pursuit requires assumptions about the range from which target motion parameters can be drawn, but these assumptions affect only the absolute information levels and not their time course or information rate. We assumed that target direction can range over 360°, speed can be drawn uniformly over a decade range (e.g., from 3 to 30°/s), and motion onset time could occur uniformly in the 500 ms window defined by the amount of randomization of target onset time in our trials.
Information about the parameters of target motion was low and uncertain early in the response but grew rapidly (Fig. 6). Timing information reached its asymptote within 112 ms (n = 3 monkeys; 16 data sets) after the onset of target motion. Information about target speed and direction rose more gradually. Exponential fits to the direction and speed information curves in Figure 5 yielded time constants of 43 and 94 ms, respectively, implying that they had reached 90 and 78% of their maxima 200 ms after the onset of target motion. At this time, pursuit provides >12 bits of information about the parameters of target motion, corresponding to 212 or ∼4000 distinguishable sensory-driven movements.
Systematic errors in pursuit
In the presence of noise, strategies for estimation and action inevitably involve a tradeoff between systematic and random errors. The most familiar example is that when we filter to suppress noise, we also smooth out details of the real signal, and the optimal filter involves a compromise between these different kinds of error. As a complement to our discussion of random errors in pursuit, then, we need to assess the systematic errors, because both must be quantified to give a complete characterization of the system.
We analyzed direction (in)accuracy on the basis of the eye velocity in the first 260 ms after target motion onset, averaged over repetitions of 14 different directions of target motion (Fig. 7A). Data are presented in a polar form, where vertical eye velocity is plotted as a function of horizontal eye velocity. Time is not explicitly represented in the graph, but the symbols on the traces indicate the eye velocity 50 and 100 ms after the initiation of pursuit. Figure 7A shows that the eye movements are approximately in the direction of target motion, at least after the first few milliseconds of pursuit. To quantify directional accuracy, we computed the difference between the direction of mean eye movement and target direction as a function of time. In the example in Figure 7B, the trajectory of the curves shows evidence of improvements in sensory estimates of target motion over time, just as we found for thresholds in Figure 5. There were systematic directional errors early in the response because the initial direction of pursuit was biased toward purely rightward motion. Yet, the direction of pursuit became quite accurate within 50 ms after the onset of target motion and almost perfect 25 ms later. Averages across multiple experiments in each of three monkeys (Fig. 7C) show a similar trend, with two of the three monkeys reaching asymptotic direction errors of just over 1° ∼75 ms after the onset of pursuit. Singular value decomposition (Press et al., 1992) of traces like those in Figure 7A confirmed the conclusions suggested by Figure 7, B and C. When comparing responses to 14 directions of target motion, almost all of the variance (98 ± 1%; n = 9 experiments) in the shape of the mean eye movements could be accounted for if eye velocity responses were rotated by the amount of the differences in the direction of target motion.
Systematic errors in eye speed showed a similar time course to those in eye direction. Figure 7D illustrates the time course of average eye velocity for targets that moved at ±10, ±20, and ±30% around a base speed of 20°/s. By analogy with direction error, we might think of speed error as the mean difference between eye speed and target speed. However, this intuitive measure of speed error would be dominated by the fact that the eyes accelerate from zero to target speed over the first 100–200 ms of the response. Instead, we have asked whether that time course is fixed for small changes in target speed: an ideal eye speed of a tracker should scale with target speed by the same factor at all times throughout the response; fractional changes in target speed should result in identical fractional changes in eye speed at any point during the open-loop interval of pursuit. We have computed deviations from ideal tracking as fractional speed error: where ė(t) is the fractional speed error, Ṫ0 and <Ė0(t)> are the base target speed and the average response to it, Ṫ and <Ė(t)> are the comparison target and mean eye speed, and the use of <X> is averages of X across multiple repetitions of the same target motion. If mean eye speed scaled perfectly with target speed, then the fractional speed error of Equation 13 would be zero; if eye speed did not scale at all with target speed, then the error would be 1 minus the fractional difference in target speed.
In the example experiment shown in Figure 7E, speed error declined as a function of time, and eye speed scaled nearly perfectly with target speed by the end of the open-loop interval, 125 ms after the onset of pursuit. Averages across experiments for two monkeys show a similar trend, with average speed error declining to <3% of base target speed by the end of the open-loop interval (Fig. 7F). Singular value decomposition of traces like those in Figure 7D confirmed the conclusions of Figure 7, E and F. Almost all of the variance (97 ± 1%; n = 7 experiments) in the shape of the mean eye movements evoked by target motion at different speeds could be accounted for by scaling responses in proportion to the ratio of target speeds.
We compared the scale of systematic and random errors directly by evaluating the average directional error and threshold across multiple experiments for three monkeys. At short times after the onset of pursuit, thresholds were much larger than directional errors (Fig. 8A). Thresholds remained higher than error at the end of the open-loop interval, but the values were much closer. In Figure 8B, each symbol compares directional error and threshold 125 ms after the onset of pursuit in individual experiments, revealing that systematic error was almost always less than random error. The fact that systematic and random errors are comparable over much of the time course, however, does suggest that the strategies of estimation and action involved in pursuit may be shaped by the need to minimize the combination of these errors, as noted at the start of this section.
Motor versus sensory sources of variation in the initiation of pursuit
Variation in the initial phase of pursuit could arise primarily from noise in the sensory representations of target motion, in the creation and execution of the motor command itself, or both, perhaps over different time scales. The goal of the remaining sections of our paper is to evaluate pursuit signal, noise, and thresholds under a variety of conditions, looking for data that might tip the weight of evidence toward motor or sensory sources of noise. We do so by analyzing data from experiments that varied the form of the target and the parameters of its motion to evaluate whether changes in motor or sensory parameters have a greater effect on thresholds.
Effect of target form on direction and speed discrimination in pursuit
To vary the pursuit stimulus without changing the motor task, we varied target form (small spots vs larger patches of texture) and determined the effect on the time course of the direction, speed, and timing thresholds of pursuit. In monkey Pk (Fig. 9A,D), for example, latency was shorter, and thresholds for both speed and direction were lower for patch targets (dotted and solid lines) than for spot targets (dashed lines). We obtained similar data across monkeys (Tables 1⇑–3). It did not matter whether the patch targets were configured so that the entire texture moved across the screen just as did the spot targets (patch targets; solid lines) or the texture moved within a stationary aperture for 125 ms before the entire texture began to move en bloc (“within” targets, dotted lines).
Part of our attempt to separate sensory and motor noise contributions exploits a key feature of motor noise, namely that its variance depends on the mean motor output (Harris and Wolpert, 1998). In general, the relationship between mean motor output (muscle tensions) and eye movements is complicated, but the conditions of our experiments allow several simplifications. Because our experiments are done with the eye close to the center of the orbit and eye position changed by <1.5° over the first 125 ms of pursuit, differences in the forces generated by the eye muscles will be related mainly to differences in eye velocity. The brief eye velocity responses to brief inputs to the eye muscles (Broussard et al., 1992) imply that the temporal integration time of the eye plant for this kind of input is ∼10 ms. In contrast, the mean trajectories of pursuit develop on a ∼100 ms time scale, and fluctuations are correlated over almost this full range of times. Therefore, for the special conditions of our experiments, the mechanical dynamics of the plant simplify to an effectively instantaneous relationship between force and velocity: if the dominant source of noise in the initiation of pursuit depends on the mean motor output, this should be measurable as a dependence on mean eye velocity. We should be able to test for a motor source of noise in pursuit by asking whether thresholds are related to eye speed even when the same speed is induced by very different sensory inputs.
To test the motor noise hypothesis, we plotted the normalized SNR (K) versus the mean instantaneous eye speed for each target type. Recall from Equation 5 that K is proportional to the SNR and is independent of the difference in the direction (or speed) of different pairs of target motions. If a sensory noise source sets the SNR, then K will depend on time and on stimulus parameters. But if the dominant source of noise is in the motor system, then the plots of K(t) for different stimulus conditions should collapse onto a single relationship if they are plotted versus the mean motor output, eye speed. In fact, the data support the sensory but not the motor prediction: plotting Kθ (Fig. 9B,C) or Kv (Fig. 9E,F) versus the mean eye speed reveals that K depends on the structure of the sensory input and not just the motor output.
Effect of target speed on pursuit direction and speed discrimination
To enable comparison with the results of similar determinations for perception, we evaluated the direction and speed threshold of pursuit over a threefold range of base target speeds (Fig. 10A,B). Direction thresholds decreased as a function of target speed, as shown both in the curves plotting threshold as a function of time for monkey Pk (Fig. 10A) and in the graphs summarizing the threshold 125 ms after the onset of pursuit as a function of target speed for two directions in three monkeys (Fig. 10D). Direction thresholds 125 ms after the onset of pursuit were an average of 84% higher (n = 3 monkeys) when targets moved at 10°/s compared with 30°/s.
Speed thresholds did not depend as strongly on target speed as did direction thresholds. For rightward pursuit in the example monkey shown in Figure 10C, the threshold curves were primarily superimposed for target speeds of 10, 20, and 30°/s. Graphs of speed threshold as a function of base target speed (Fig. 10D) confirm that speed threshold was independent of central target speed for both spot and patch targets in two monkeys. The increased speed threshold for monkey Yo at the slowest base target speed of 10°/s provides the only hint that pursuit speed threshold might depend on target speed, at least in the narrow range of target speeds we tested.
Effect of target direction on pursuit direction discrimination
Eye movements in different directions are controlled by different combinations of extraocular muscles. If pursuit variation arises from motor noise, then changes in the central target direction, and in the relative contributions of ocular muscle pairs, might lead to differences in directional thresholds. Figure 11 summarizes the direction discrimination thresholds of four monkeys pursuing target directions clustered in 2–3° increments around different mean directions. Each symbol represents the end point of a vector with a length indicating directional threshold 125 ms after the average onset of pursuit and an angle indicating the central direction of the target motions used to derive the threshold. Direction thresholds did not vary consistently as a function of the central direction of target motion, although they did vary, to some degree, across individuals. The picture was similar for spot targets (Fig. 11A) and patch targets (Fig. 11B). To obtain the data summarized in Figure 11, target directions were always presented so that they were balanced around the origin. For example, an experiment described as having five directions clustered around a central direction of 15° actually included trials that delivered five directions clustered around four central directions that were rotated ±15° relative to rightward and leftward. For presentation, we plotted results separately for targets with leftward versus rightward components of motion, but we averaged responses to target motions with upward and downward components; thus, the threshold value plotted at 15° is the average of those measured for target motion in the +15° and −15° directions.
Discussion
Origins of behavioral variation
In principle, variation in sensory–motor output could arise from noise introduced at any stage, including during estimation of sensory parameters, movement choice and planning, or movement execution. Previous analyses of reaching arm movements and saccadic eye movements have focused on sources of variation that arise in the motor part of the neural circuits (Harris and Wolpert, 1998; Jones et al., 2002). However, quite different conclusions have been drawn in other behavioral systems. In the well understood behavior of photon counting, for example, quantitative analysis has established that the dominant source of noise arises in the very first step of signal detection in the retina (Barlow, 1956).
Our work addresses the source of motor variation directly in smooth-pursuit eye movements. Pursuit is ideal for this analysis because it allows us to study sensory–motor transformations in isolation in the first hundred milliseconds of the response, before there has been an opportunity for sensory feedback to intervene. Furthermore, pursuit is sufficiently variable to provide interesting data, yet sufficiently accurate and precise that quantifying its precision has a chance of revealing fundamental properties of neural signal processing. Our goal was to quantify the degree of precision and accuracy in pursuit, leveraging evaluation of both random and systematic errors in performance to understand how precision and accuracy are controlled by the nervous system and to draw conclusions about the origins of the limits to performance.
The present study goes well beyond our previous one (Osborne et al., 2005), which considered the first 125 ms of pursuit en bloc. It found that the variation in smooth eye velocity was low dimensional, with 95% of the trajectory variance accounted for by errors in sensory estimations of the direction, speed, and time of onset of target motion. Now, we have data that allow a direct and more thorough comparison of pursuit and perceptual thresholds, evaluating how each is improved by the integration of sensory information across time and how thresholds vary over a wide range of target forms and motions. Our basic strategy in the present study was to ask how well we could use pursuit eye trajectories to estimate the parameters of the sensory input, by computing the time course of thresholds for discriminating small differences in the direction, speed, or time of onset of target motion. We realize that our analysis places upper bounds on threshold, leaving open the possibility that the pursuit system may have still slightly lower thresholds than the ∼2–3° of direction, ∼11–15% of target speed, and ∼8 ms of motion onset timing reported here.
Evidence favoring a sensory source of variation in pursuit
If a target moves directly to the right and the evoked eye movement starts by moving slightly upward relative to right, then the source of the error could be an incorrect sensory estimate of the direction of target motion, inaccurate planning of motions based on the sensory estimates, or imprecise implementation of these plans by the motor system. Perceptual tasks, in contrast, usually involve two-alternative forced-choice paradigms in which noise in the motor side of the system does not contribute to variation in responses. To the extent that the thresholds characterizing the accuracy of pursuit match those for perceptual decisions, it is natural to conclude that the two behaviors are limited by the same source of noise (Kowler and McKee, 1987; Watamaniuk and Heinen, 1999; Stone and Krauzlis, 2003) and that the noise arises primarily in the sensory component of the sensory–motor circuits.
The sensory noise hypothesis leads to several predictions that are borne out in our data. One prediction is that the time course and value of thresholds derived from pursuit should agree with those derived from perceptual reports of the ability to discriminate small differences in target direction or speed. Furthermore, both should show evidence of improvements because of integration across time. Figure 12 shows that the thresholds for pursuit and perception [data from de Bruyn and Orban (1988)] agree for target motion at 16–20°/s and that the agreement is clearest for durations of motion >50 ms. This is the interval where pursuit thresholds seem to be improving because of integration of sensory signals across time, and not merely because the eye velocity is increasing relative to fixed background noise. Figure 12 also shows that increases in target speed cause pronounced reductions in perceptual direction discrimination thresholds and smaller changes in perceptual speed discrimination thresholds, just as we found for pursuit. In Figure 12, the data for perception are plotted as a function of the duration of the motion exposure. The data for pursuit are plotted as a function of time from the initiation of pursuit, but we can think of the evolution of pursuit threshold as an effect of the duration of the preceding visual stimulus.
Two issues must be considered in interpreting Figure 12. One issue is that speed thresholds for perception vary among subjects in experiments on monkeys. In the study by Liu and Newsome (2005), for example, one monkey had a perceptual threshold versus target speed curve that was more like our pursuit data for monkey Pk, whereas the other had a curve more like our data for monkey Yo. With intersubject variability in mind, it would be optimal to compare perceptual and pursuit performance in the same animals, but this is beyond the scope of our work. The second issue is that pursuit and perception could have a common sensory noise source as well as separate, private noise sources (Gegenfurtner et al., 2003; Stone and Krauzlis, 2003). If the private noise sources dominate both pursuit and perceptual behavior, then we are forced to accept several coincidences: the low dimensionality of pursuit variation, the alignment of those dimensions with the parameters of sensory input (Osborne et al., 2005), and the agreement of the thresholds derived from pursuit and perception. A dominant sensory source for variation in both perception and the initiation of pursuit seems to be a more parsimonious conclusion. Motor sources of variation may make large contributions to variation in the later steady-state phase of pursuit, when visual inputs are less important.
A second prediction of the sensory noise hypothesis is that the time course of information about the direction of target motion derived from analysis of pursuit should agree well with that provided by the visual system to guide movement. Cortical area MT is a primary source for visual inputs that guide pursuit eye movements (Newsome et al., 1985; Groh et al., 1997; Born et al., 2000). In a previous study, we found that information about motion direction in single MT neurons rises quickly with the first few motion-driven action potentials (Osborne et al., 2004). If we normalize the direction information from each MT neuron and average, we find good agreement between the time course of average direction information in MT (Fig. 13B) and the time course of information about direction computed from pursuit behavior (Fig. 13A): the curves were fitted by exponentials with time constants of 59 and 52 ms for MT and pursuit, respectively (Fig. 13C). The similarity of time scales of information accumulation for pursuit behavior and MT neurons is encouraging and suggests that improvements in behavioral performance across the first 100 ms of pursuit are driven by an accumulation of sensory evidence. At the same time, the absolute levels of information in MT neurons are much smaller than the information capacity derived for pursuit (Osborne et al., 2004). The difference in information levels needs to be resolved by understanding how information is pooled across MT neurons to generate commands for pursuit.
Finally, for two of our experiments, we expect very different results if the system is dominated by sensory versus motor noise. If sensory noise were dominant, stronger sensory signals should generate more reliable pursuit behaviors; for a stimulus that consists of a patch of moving texture versus a single moving spot, the sensory noise hypothesis predicts lower thresholds for discriminating small differences in target motion, as well as faster time courses. This is what we found. If motor noise were dominant, then thresholds should change when the motor task is changed, for example, by altering the combinations of extraocular muscles that were most important. We accomplished changes in the motor task by using central target directions that were horizontal, vertical, or varying degrees of oblique. Thresholds did not change.
Pursuit as a window on sensory processing
We have shown that behavioral precision in pursuit eye movements evolves over a ∼100 ms time scale. The time dependence and parameter dependences all are consistent with the idea that motor output is limited by the same noise source as sensory discrimination. Traditionally, perceptual decision making has used the subject's brain for measuring the parameters of its sensory inputs. We have shown that the same approach can be used with sensory–motor performance. We can see the detailed properties of sensory processing in motor output, including manifestations of both the mean responses of neurons and their variability. This agreement is striking because of the enormous difference in dynamic range of the two kinds of behaviors: perceptual decisions generally provide 1-bit answers derived from two-alternative forced-choice tasks, whereas the sensory–motor behavior of pursuit can provide >10 bits of information on a scale of ∼100 ms. The pursuit system thus offers us literally one order of magnitude more bandwidth within which to explore the neural coding of the sensory information that guides perception and action.
Footnotes
-
This work was supported by National Institutes of Health Grants EY03878 and T32-EY007120, the Howard Hughes Medical Institute, the Alfred P. Sloan Foundation, and the Swartz Foundation. We thank Stefanie Tokiyama, Elizabeth Montgomery, and Karen MacLeod for assistance with animal monitoring and maintenance and Scott Ruffner for computer programming. W.B. thanks the Sloan-Swartz Center and the Department of Physiology at the University of California at San Francisco for their extended hospitality at the start of this collaboration.
- Correspondence should be addressed to Dr. Leslie C. Osborne, Department of Physiology, Box 0444, 513 Parnassus Avenue, Room HSE-802A, University of California at San Francisco, San Francisco, CA 94143-0444. osborne{at}phy.ucsf.edu