Abstract
We asked whether the dynamics of target motion are represented in visual area MT and how information about image velocity and acceleration might be extracted from the population responses in area MT for use in motor control. The time course of MT neuron responses was recorded in anesthetized macaque monkeys during target motions that covered the range of dynamics normally seen during smooth pursuit eye movements. When the target motion provided steps of target speed, MT neurons showed a continuum from purely tonic responses to those with large transient pulses of firing at the onset of motion. Cells with large transient responses for steps of target speed also had larger responses for smooth accelerations than for decelerations through the same range of target speeds. Condition-test experiments with pairs of 64 msec pulses of target speed revealed response attenuation at short interpulse intervals in cells with large transient responses. For sinusoidal modulation of target speed, MT neuron responses were strongly modulated for frequencies up to, but not higher than, 8 Hz. The phase of the responses was consistent with a 90 msec time delay between target velocity and firing rate. We created a model that reproduced the dynamic responses of MT cells using divisive gain control, used the model to visualize the population response in MT to individual stimuli, and devised weighted-averaging computations to reconstruct target speed and acceleration from the population response. Target speed could be reconstructed if each neuron’s output was weighted according to its preferred speed. Target acceleration could be reconstructed if each neuron’s output was weighted according to the product of preferred speed and a measure of the size of its transient response.
What we do is often guided by what we see. Sensory-motor systems must therefore transform visual signals into commands for accurate movement. Smooth pursuit eye movements provide an excellent opportunity to investigate the neural circuits that perform visual-motor transformations. The basic neuroanatomy of pursuit is known, the sensory inputs and motor outputs are well understood, and the behavior itself has been studied extensively.Eckmiller (1987), Lisberger et al. (1987), Tusa and Ungerleider (1988),Leigh (1989), Kowler (1990), and Keller and Heinen (1991) have provided reviews of these issues.
The particular visual signals needed to control pursuit are related to the motion of visual targets (Rashbass, 1961), and previous work has implicated extrastriate visual area MT as a major source of these visual motion signals. Lesions of MT cause deficits in the initiation of pursuit for targets moving in any direction across the part of the visual field represented at the site of the lesion (Newsome et al., 1985). Electrical stimulation of MT affects the initiation of pursuit if the stimulation coincides with the motion of a tracking target (Groh et al., 1997). Single neurons in MT are selective for the direction and speed of the motion of small targets and prefer speeds in a range that is relevant to pursuit (Maunsell and Van Essen, 1983; Albright, 1984).
Pursuit is configured as a negative feedback control system; its visual input is target motion with respect to the (potentially moving) retina, defined as “image motion.” As a result, the visual input for pursuit varies as a function of time according to the pattern illustrated in Figure 1. For this single-pursuit trial, the stimulus was a ramp of target position from an eccentric starting point, which provides a step of target velocity. After the onset of the step, the eye remained still for ∼100 msec, accelerated for ∼150 msec, and then oscillated around target velocity at a frequency of ∼6 Hz. To estimate the image velocity, we subtracted eye velocity from target velocity. This reveals that the visual input during pursuit is, sequentially, a step increase in image velocity, constant image velocity for 100 msec, a ramp decrease in image velocity toward zero, and small oscillations around zero at ∼6 Hz.
Schematic diagram showing typical pursuit eye movements and the visual image motion that drives them. Fromtop to bottom, the tracesare as follows: superimposed eye and target position, superimposed eye and target velocity, image velocity computed as target velocity minus eye velocity, and image acceleration computed as the low-pass filtered derivative of image velocity. The image acceleration at the onset of target motion is represented by a brief square pulse that has been clipped as a reminder that it is an impulse of acceleration. Theshort horizontal dashed line below the eye positiontrace shows the position of a fixation point that went out when the eccentric tracking target started to move.
Several laboratories have made computer models that reproduce the eye movements illustrated in Figure 1 on a millisecond time scale. An issue that distinguishes many of these models is whether the dynamics of the biological system are generated by direct sensory drive or by feedback from motor computations. For example, the model of Krauzlis and Lisberger (1994) requires that the discharge of visual neurons providing input to pursuit represents image velocity and image acceleration, both of which are shown in Figure 1. In contrast, the models of Robinson et al. (1986) and Ringach (1996) assume that the visual inputs to pursuit encode only image velocity and that the dynamics of pursuit arise in motor circuits.
The present experiments were designed to record the dynamics of visual motion signals in area MT, to determine whether those dynamics could represent image acceleration, and to explore ways to extract information about image velocity and acceleration from the population responses in area MT. Thus, we used trajectories of stimulus speed that varied in the same way as image velocity and acceleration vary during pursuit. Our recordings showed that many individual MT neurons have transient responses that can provide information about image acceleration. Individual neurons, however, do not carry an invariant acceleration-related signal across all image speeds, so the true value of image acceleration can be represented only by the activity of a population of MT cells. We developed a model of MT neuron responses that simulates the responses of the population of MT neurons for a variety of stimuli, and we used the model to visualize the distributed representation of target motion in MT and to reconstruct target velocity and acceleration from this distributed representation.
Parts of this paper have been published previously (Movshon et al., 1990; Lisberger and Movshon, 1991, 1994; Lisberger et al., 1995).
MATERIALS AND METHODS
Surgical preparation and maintenance. We recorded the activity of single units in area MT in 10 hemispheres of eight macaque monkeys (six fascicularis and two nemestrina). The monkeys were prepared for acute single-unit recording using methods we have described in detail previously (Levitt et al., 1994; Kiorpes et al., 1996). They were premedicated with atropine (0.25 mg) and with acepromazine (0.05 mg/kg) or diazepam (Valium, 0.5 mg/kg). After induction of anesthesia with intramuscular injections of ketamine HCl (Vetalar, 10–30 mg/kg), cannulae were inserted into the trachea and the saphenous veins, the monkey’s head was fixed in a stereotaxic frame, and surgery was continued under intravenous anesthesia with the opiate anesthetic sufentanil citrate (Sufenta, 4–8 gm · kg−1 · hr−1). Infusion of the surgical anesthetic continued throughout the recordings.
To minimize eye movements, paralysis was maintained with an infusion of vecuronium bromide (Norcuron, 0.1 mg · kg−1 · hr−1) in lactated Ringer’s solution with dextrose (5–20 ml/hr). Monkeys were artificially ventilated with room air or a mixture of 50–70% N2O in O2. Peak expired CO2 was maintained near 4% by adjusting the tidal volume of the ventilator. Rectal temperature was kept near 37°C with a thermostatically controlled heating pad. Monkeys received daily injections of a broad-spectrum antibiotic (Bicillin, 300,000 units) to prevent infection, as well as dexamethasone (Decadron, 0.5 mg/kg) to prevent cerebral edema. The electrocardiogram (EKG), EEG, autonomic signs, and rectal temperature were monitored continuously to ensure the adequacy of anesthesia and the soundness of the monkey’s physiological condition.
Tungsten-in-glass microelectrodes (Merrill and Ainsworth, 1972) were introduced by a hydraulic microdrive through a small guide needle. To obtain the most consistent access into the portions of MT representing the central visual fields, we used a vertical approach to MT through the anterior bank of the superior temporal sulcus. After the electrode was in place in the cortex, the exposed dura was covered with warm agar. Action potentials were amplified conventionally, displayed, and played over an audio monitor. The recording sessions lasted between 72 and 108 hr.
Physiological optics. The pupils were dilated, accommodation was paralyzed with topical atropine, and the corneas were protected with +2D gas-permeable hard contact lenses. When necessary, supplementary lenses were chosen by direct ophthalmoscopy to make the retinas conjugate with the display screen. The power of the lenses was then adjusted as necessary to optimize the visual responses of recorded units. Contact lenses were removed periodically for cleaning. At this time, the eyes were rinsed with saline and infiltrated with a few drops of ophthalmic antibiotic solution (gentamycin). At least once a day, the locations of the foveae were recorded using a reversible ophthalmoscope.
Characterization of receptive fields and stimulus presentation. We initially mapped the receptive fields of single MT neurons by hand on a tangent screen using small black-and-white geometric targets. For each neuron, we recorded the location and size of the minimum response fields and determined its selectivity for the direction of motion. With the exception of a small group clustered around 30° eccentric, our neurons had receptive field centers that were fairly evenly distributed between 1 and 17° from the fovea. After receptive fields had been determined, we positioned a mirror to place the preferred eye’s receptive field on the face of a display oscilloscope that subtended 10–15° at the monkey’s eye. For most experiments, textures consisting of several hundred randomly placed bright dots were generated and moved with 100% coherence under computer control. The mean luminance of the random-dot displays was between 5 and 10 cd/m2, and the frame rate was 128 or 250 Hz. In some experiments, we also used a separate display generated by a raster frame buffer to measure responses to drifting gratings or plaid patterns. Methods for generating these stimuli are detailed elsewhere (Levitt et al., 1994). Gratings and plaids were presented at a frame rate of 107 Hz. Time-sampled motion stimuli can contain energy in the direction opposite to the motion, because of spectral replicas created by the temporal sampling (Watson et al., 1986; Britten et al., 1993). Because cells having high preferred speeds tend also to prefer low spatial frequencies (Levitt et al., 1988), the intrusion of energy from these spectral replicas was almost certainly outside the cells’ spatiotemporal sensitivity range for the speeds that we used.
Before starting quantitative analysis, we attempted to optimize the stimulus to elicit strong and reliable responses to stimuli moving at the optimal speed and direction. Many cells responded much better if dots moved through a small window inside a surround of stationary dots; this often meant presenting a surround of stationary dots around dots that moved through a window over the classical receptive field. For the remainder of the paper, the moving texture will be called a “target,” and we will refer to target speed and acceleration even though these attributes actually belong to the individual dots within the moving part of the texture stimulus. Because our recordings were made in anesthetized monkeys, target and image motion are interchangeable; this would not be the case during pursuit in awake monkeys.
Stimuli were presented as a series of trials with durations that ranged from 1024 to 3072 msec with intertrial intervals of ∼1 sec. Target motions consisted of steps and ramps of target speed of different final speeds and durations, double pulses of target speed at different interpulse intervals, and sinusoidal variation of target speed around baselines ranging from zero to several times the neuron’s preferred speed. Trials were blocked by stimulus type to generate coherent, controlled experiments while keeping each block small enough so that we would obtain useful data for at least part of our series if neuronal isolation were lost. Each block of trials was repeated 8–32 times, and the order of trials was shuffled between blocks. The seed used to initiate the pseudorandom sequence that placed the random dots was changed, and the dots were replotted for each trial; we cannot reconstruct the locations of the dots because we did not record the seed for each trial. Early experiments were controlled by a PDP11 computer, and later experiments were controlled by an 80486 personal computer using a DSP board with 16-bit DACs to generate random dots. Data were analyzed after the experiment by aligning the responses to identical target motions and compiling average response histograms. Details of the data analysis are provided in the descriptions of the relevant figures.
Reconstruction of recording sites. During recording, small electrolytic lesions were produced at locations of interest along the electrode tracks by passing DC current (2 μA for 2–5 sec; tip negative) through the electrode. At the end of the experiment, the monkeys were killed with an overdose of Nembutal and perfused through the heart with 2 l of 0.1 m PBS followed by 2 l of a solution containing 4% paraformaldehyde in 0.1m PBS. Blocks containing the region of interest were stored cold overnight in a post-fix solution of 4% paraformaldehyde and 30% sucrose, after which 40 μm sections were cut on a freezing microtome. Sections were stained for Nissl substance with cresyl violet or for myelin using the methods of Gallyas (1979). Most recordings were verified to lie within area MT, as defined by standard histological criteria (Van Essen et al., 1981). In the few cases in which we were unable to recover all the electrode tracks, we used the distinctive concentration of direction-selective neurons and the relatively small sizes of their receptive fields to identify recording sites as lying within MT (Desimone and Ungerleider, 1986).
RESULTS
Experimental results
Our results are presented in separate experimental and simulation sections. In the experimental results, we show first that MT cells vary substantially in the dynamics of their responses to motion. Some cells encode only target velocity, whereas others have large transient responses to changes in target motion. We will conclude that the distribution of transient responses across the population of MT cells has the potential to provide information about target acceleration, although no individual cell does. Second, we use paired pulses of target velocity at short intervals to characterize the mechanism that controls the time course of transient responses. We will conclude that a class of mechanisms that falls under the general term “adaptation” is responsible for the transients, although our data could be accounted for by any of several specific cellular or neural mechanisms. Third, we describe the responses of MT cells for sinusoidal oscillation of target speed at frequencies normally seen during pursuit eye movements.
Responses to steps of target speed
It is well known that MT cells are tuned for the direction and speed of target motion. Our first step was to look for dynamic features in the responses to target motion by measuring the time course of neuronal firing for target motions that had instantaneous onsets. Figure 2 shows the responses of two cells to steps of target speed for stimuli that moved in the neurons’ preferred directions. For these experiments, the target initially was visible and stationary for 256 msec, then moved at one of 8–10 speeds for 512 msec, and finally stopped and remained stationary for 256 msec. For the cell in Figure 2A, steps of target speed evoked a large transient pulse of activity followed by a sustained response. Both the transient and sustained responses were tuned for speed, with a preferred speed of ∼9°/sec for the transient response and ∼5°/sec for the sustained response. The offset of target motion caused only a slight transient response, in contrast to the large transients emitted by this cell for the onset of motion. For the cell in Figure 2B, a step of stimulus speed caused little or no transient response at the lower speeds, only a small transient response at speeds >19°/sec, and a clear sustained response with a preferred speed of ∼19°/sec. In our sample of ∼100 cells, we always observed a speed-tuned sustained response, but we observed a range of transient responses. The example in Figure2A is near one end of the range, whereas that in Figure 2B is in the middle of the population.
Representative responses of MT cells to steps of target speed. A, Responses of a neuron (321r10) with one of the largest transient responses we recorded. B, Responses of a neuron (324r4) with a more typical transient response. Each histogram shows the accumulated response of the neuron to steps of target speed from zero to the final speed indicated at the left of A. Thehistogram labeled stationary shows each neuron’s response to the appearance of a stationary dot field that did not move. Bin width was 8 msec. The traces at thebottom of the figure show the time course of target speed, which always started at zero. Histograms were accumulated from nine repeats of each stimulus in A and eight repeats in B.
For each MT cell, speed tuning and direction tuning were similar for the transient and sustained responses. Figure3A illustrates speed-tuning curves for the transient (open symbols) and sustained (filled symbols) responses of the typical MT cell whose responses to steps of target speed are shown in Figure2B. For each speed, the size of the sustained response was computed as the mean firing rate in the interval from 256 to 512 msec after the onset of target motion. The size of the transient response was estimated as the largest mean firing rate in a 24 msec window within 120 msec of the onset of target motion. The spontaneous firing (Fig. 3A, horizontal dashed line) was taken as the sustained firing rate for a target that remained stationary throughout the trial. We fitted each speed-tuning curve (with spontaneous firing subtracted) with the function:
Equation 1where Rs
is the response at each speed, a is the amplitude, b is the preferred speed, c is the tuning bandwidth, d is a parameter controlling the skew of the curve, and s is target speed. As shown in Figure 3B, there was a strong correlation between the preferred speeds of the transient and sustained responses (r = 0.92), with a tendency for the preferred speed for the transient response to be slightly higher than that for the sustained response.
Quantitative analysis of the responses of 104 MT neurons to steps of target speed. A, Firing rate plotted as a function of target speed for neuron 324r4, which also appears in Figure 2B. Open and filled symbols show the transient and sustained responses, respectively. The horizontal dashed line shows the spontaneous firing in the presence of stationary dots. The solid curves show the best-fitting function based on Equation 1. Thevertical arrows labeled aT and aS are positioned at the preferred speeds of the transient and sustained responses, respectively, and show the peak responses, which were used to compute the transient/sustained ratio. B, Comparison of preferred speed for the transient and sustained response. Each symbol summarizes the response of one MT neuron. C, Representative speed-tuning curves showing the range of preferred speeds of the sustained responses. The curves were normalized so that each is plotted on the scale shown by the calibration bar on the lower right but were shiftedvertically to facilitate viewing. The horizontal dashed line at the right of eachcurve shows the baseline for that neuron, and thenumber next to the curve gives the amplitude of the sustained response at the preferred speed. The threevertical dashed lines were drawn at speeds of 1, 10, and 100°/sec to facilitate comparison of the different tuningcurves.
We computed the transient/sustained ratio asaT /aS , whereaT and aS are the fitted values of a for the transient and sustained responses, respectively. For the two cells illustrated in Figure 2, the transient/sustained ratio was 7.8 and 1.9. Among 104 cells that were studied during steps of target speed, the transient/sustained ratio varied from 1.08 to 9.4 with a median of 1.77 and a mean of 2.1. The size of the transient/sustained ratio was not correlated with any of the other parameters we measured, including laminar location of the recorded neurons, preferred speed, sustained response strength, spatial or temporal tuning for sine wave gratings, or pattern versus component characteristics during stimulation with moving “plaids” (Movshon et al., 1985; Movshon and Newsome, 1996).
The stack of speed-tuning curves in Figure 3C illustrates the range of observed speed-tuning characteristics for the cells’ sustained responses, with preferred speeds varying over two orders of magnitude from 0.6 to 65°/sec. Every cell we recorded showed speed selectivity when the range of testing speeds went as low as 0.125°/sec. Note that the three cells plotted at the bottom of Figure3C would have been classified as low-pass (Lagae et al., 1993) had they been tested only at speeds >1°/sec.
Latency of response
We measured the latency of the neuronal response for steps of target speed; to avoid contaminating our data with noise, we did not make measurements if a stimulus evoked sustained firing that was <10% of the sustained response at the preferred speed. Latencies were measured manually from histograms with 4 msec bin widths by following the rising phase of the averaged response back in time until the average firing was within the fluctuations in spontaneous rate. Because of the low or absent spontaneous rates and the brisk rises of the initial responses to steps of target speed, this procedure almost always yielded an estimate of latency that had an error of 8 msec or less. In 80% of the responses, the manual analysis yielded latencies in perfect agreement with those obtained by an objective procedure that found the time after which the firing rate remained >1 SD above the base rate. We elected to use the manual analysis throughout, however, because it was clearly superior to the objective analysis in the remaining 20% of responses, in which the objective analysis was unreliable because of response variability.
In almost all cells, the latency was quite long for low target speeds, decreased with increases in target speed, and became as short as 40 msec at the highest speeds. We have plotted the latency of response as a function of the inverse of target speed (Fig.4A) because straight lines in these graphs have a clear functional interpretation if response latency has two components—a variable component that requires the target to traverse a given distance before the response is initiated and a fixed delay affecting all responses. They-intercept latency from the linear model estimates the fixed minimum latency. The slope of the line has the units of degrees, estimating a “space constant” that measures the distance the target has to traverse before a response is initiated. We have not analyzed the linear model rigorously, but linear fits were generally excellent for speeds as low as 0.5°/sec (Fig. 4A). Latency often failed to follow the linear model for speeds <0.5°/sec, which we excluded from the linear regression for this reason.
Determinants of response latency in MT neurons.A, Plots of latency versus 1/speed for three representative MT cells. Symbols show the measurements, and the straight lines show the result of regression analysis with the equation: latency = b +a/speed, where a is the space constant and b is the fixed, minimum latency. B, Space constant (a converted to minutes of arc) plotted as a function of preferred speed of the transient response.C, Intercept latency (b) plotted as a function of the transient/sustained ratio for steps of target speed. D, Intercept latency (b) plotted as a function of preferred speed of the transient response. InB–D, each symbol shows the response of one MT neuron.
The regular relationship between latency and inverse speed appeared to be robust despite wide variations in response strength across speed. For example, note the difference in latency between the weak responses of each of the example neurons to targets of low and high speeds. The fixed component of latency showed no convincing (or statistically significant) relationship to either the transient/sustained ratio (Fig.4C) or the preferred speed (Fig. 4D), suggesting that cells preferring different speeds did not differ in their temporal properties. In contrast, the graph in Figure4B shows that the space constant was larger for cells that had higher preferred speeds. This relationship can be understood by noting that neurons with different speed preferences can be constructed by variation in either their spatial or temporal properties. In agreement with the previous finding that cells responding well to high speeds tend to prefer lower spatial frequencies (Levitt et al., 1988), our results suggest that spatial variations are more important than temporal variations in differentiating neurons with different preferred speeds.
Stimulus-dependent variations in latency provide information about the neural mechanisms that lead to cellular responses, whereas the latency of the population constrains how the neuronal responses in MT might be decoded by downstream circuits. Figure 5summarizes the distribution of latencies in all cells in which we measured latencies at target speeds of 1, 8, and 64°/sec. Both the distributions and the medians (Fig. 5, vertical arrows) shifted toward shorter latencies as speed increased. For speeds >1°/sec, we recorded some latencies as short as 40 msec; the median latencies were 88, 72, and 65 msec at target speeds of 1, 8, and 64°/sec, respectively.
Distribution of response latency for MT neurons with measurable responses at three selected speeds: 1, 8, and 64°/sec. Vertical arrows show the median latency for each speed.
Responses to ramps of target speed
In this section, we show that the firing rate of some MT neurons is influenced by target acceleration. The data do not support a conclusion that the firing of individual MT neurons encodes target acceleration. Instead, these data provide the basis for the final section of the paper, in which we demonstrate that target acceleration can be reconstructed from the population response in MT.
To determine whether the firing of MT cells was influenced by target acceleration and how, we used stimuli in which the target either accelerated or decelerated through a given range of speeds. Figure6 shows the time course of firing rate during these stimuli for the two cells whose responses to steps of target speed appear in Figure 2. As shown by the target speed traces at the bottom of the figure, the target was initially stationary and visible for 256 msec. Target speed then increased at a constant rate for 128 msec up to final speeds indicated by the numbers at the left of each histogram in Figure 6A, moved at constant speed for 512 msec, decelerated at a constant rate for 128 msec back to zero velocity, and finally remained stationary and visible for 256 msec. In our later experiments, the entire sequence of constant acceleration, constant speed, and constant deceleration was presented in single trials like those illustrated in Figure 6. In our earlier experiments, we presented the accelerations and decelerations in separate trials and then spliced the averages together. The results were identical for the two methods.
Representative averaged responses of MT neurons to ramp increases and decreases in target speed. A, MT neuron (also shown in Fig. 2A) that had one of the largest transient responses and one of the largest asymmetries and that was recorded between ramp increases and decreases in target speed.B, MT neuron (also shown in Fig.2B) that fell approximately in the middle of our sample in terms of transient response and asymmetry between ramp increases and decreases in target speed. For each neuron, the different firing rate histograms show responses for ramp increases to and decreases from different target speeds, given by thenumbers at the left of A. Target speed always started at zero. The traces at thebottom of A and B show the time course of target speed. The vertical dashed linesdelimit two analysis intervals in the first 256 msec after the onsets of ramp increases and decreases in target speed. Histograms were accumulated from 9 repeats of each stimulus in A and 16 repeats in B. Bin width is 8 msec.
In principle, the response of a cell to this stimulus could be determined entirely by the sequence of speeds traversed by the target during ramp accelerations and decelerations, or it might also be influenced by features of motion other than target speed. If a ramp acceleration took target speed from zero through the cell’s preferred speed to a final speed well above preferred speed, then we would expect to see a large transient response because of the speed tuning of the cell. However, we would expect to see approximately the same transient as ramp deceleration took target speed from well above preferred speed back through preferred speed down to zero. Thus, MT cells that give symmetric responses to ramp increases and decreases in target speed probably do not provide information that could be used to determine target acceleration. On the other hand, an asymmetric response to ramp increases and decreases in speed would show that a cell’s response could carry information about target acceleration as well as speed.
The cell whose responses appear in Figure 6A shows one extreme of a continuum of neuronal behavior for ramp increases and decreases in target speed, whereas the cell in Figure6B is near the median. For all six final ramp speeds shown here, the cell in Figure 6A showed a pronounced transient response during ramp increases in target speed and either none or much less of a transient during ramp decreases in target speed. The cell in Figure 6B showed a clear transient response during ramp increases in target speed to 74°/sec as well as to other speeds >18°/sec. However, this cell also emitted a large transient for ramp decreases to zero target speed from these higher speeds. Thus, inspection of the histograms demonstrates, at least for ramps of target speed that started at zero, that the firing of the cell in Figure 6A differentiates between target acceleration or deceleration for all final speeds, whereas the firing of the cell in Figure 6B did so only over a middle range of speeds. Inspection of Figure 6, A and B, reveals that the latency of the peak response during ramp increases in target speed becomes shorter as final target speed increases. This suggests that the transients of firing during ramp increases in target speed occur as the target passes through the cell’s preferred speed and leads to the question (addressed below) of why similar transients are not always evident for ramp decreases in target speed.
We quantified the asymmetry in each cell’s responses to ramps of target speed by measuring the peak firing rates over a 24 msec window in the two 256 msec intervals starting at the onset of the ramp increases and decreases in target speed (Fig. 6, intervals marked byvertical dashed lines). The analysis intervals always included the peak of the responses without including other peaks that occasionally occurred later. The results of this analysis appear in Figure 7 for the two example cells in Figure 6. Figure 7, A and B, plots the peak firing during ramp increases (open symbols) and decreases (filled symbols) in target speed as a function of the magnitude of target acceleration. As was clear in the histograms, the cell in Figures 6A and 7Ashowed a large asymmetry, whereas that in Figures 6Band 7B shows a consistent but smaller asymmetry. To gain an impression of the size of the asymmetry in relation to each cell’s sustained and transient responses to steps of target speed, we computed the difference between the peak firing for target accelerations to and decelerations from a given target speed and plotted the difference firing rate as a function of target acceleration (Fig.7C,D). For the cell in Figure 7C, the difference firing between ramp acceleration and deceleration increased consistently as a function of target acceleration up to an asymptote of ∼320 impulses/sec. The asymptote was much larger than the peak of the sustained response to steps of target speed (Fig. 7C,bottom horizontal dashed line). For the cell in Figure 7D, the difference firing rate was clearly tuned and reached a peak that was less than the maximal sustained response for steps of target speed. Comparison with the maximal sustained response is useful because it reveals the size of the asymmetry, which may provide information about target acceleration, relative to the size of the same neuron’s responses to sustained speed. We conducted this analysis on all 89 cells that were studied during steps and ramps of target speed, and for each cell, we fitted the difference firing with Equation 1, with speed supplanted by acceleration. Examples of the fits are shown by the smooth curves in Figure 7, C and D.
Quantitative analysis of the asymmetry between responses to ramp increases and decreases in target speed.A, B, Open andfilled symbols plot the peak firing rate in the first 256 msec after the onset of ramp increases and decreases, respectively, in target speed as a function of the magnitude of target acceleration. The horizontal dashed lines show the maximum sustained firing rate, computed as the maximum sustained response plus the spontaneous firing rate. C,D, Open symbols plot the difference between the peak firing rates for ramp increases and decreases in target speed as a function of target acceleration. The solid curves plot the best fit obtained with Equation 1, with target acceleration substituted for target velocity. The two horizontal dashed lines show the largest transient and sustained responses for steps of target speeds, with spontaneous firing subtracted.A, C, Data are for cell321r10, which also appears in Figures2A and 6A.B, D, Data are for cell324r4, which also appears in Figures2B, 3A, and 6B.
The selection of curves in Figure8A shows the diversity of the asymmetries among the cells that had the largest responses to ramp acceleration and deceleration. These examples were selected from the 40 cells that had transient/sustained ratios of 1.8 or larger during steps of target speed and that were studied during ramps of target speed. Each curve plots the difference firing rate described in the previous paragraph, normalized for the largest transient response evoked by steps of target speed, as a function of the value of the ramp acceleration. We chose to normalize for the largest transient response during steps of target speed so that most of the normalized points would have values between 0 and 1 and different cells could be compared easily. The curve labeled e in Figure8A is from the neuron that was used to construct Figure 7, A and C, and shows an asymmetry evident across the full range of accelerations we tested.Curve g in Figure 8A is from the neuron used to construct Figure 7, B and D, and shows an asymmetry tuned for a middle range of accelerations. Other cells showed tuning for a narrow range (Fig. 8A,curve h) or a broad range (curve f) of accelerations or had high-pass characteristics with positive values of difference firing rate starting at low (curves c,d), medium (curve b), or high (curve a) accelerations. None of the 39 cells that were tested with target accelerations as low as 0.9°/sec2 showed low-pass characteristics. The presence of large differences in the response to ramp accelerations and decelerations demonstrates that the firing of some MT cells is influenced by target acceleration. The diversity of the tuning across the population suggests that target acceleration across different speed ranges is probably represented by different MT cells.
Asymmetry between responses to ramp increases and decreases in target speed. A, Family ofcurves showing the relationship between the difference firing rate (peak during acceleration minus peak during deceleration) and the magnitude of target acceleration for eight MT neurons. Difference firing rates are normalized to the peak transient response for steps of target speed and are plotted according to the calibration bar on the lower right of the graph. Thecurves for different cells have been shiftedvertically to facilitate viewing. The horizontal dashed lines show difference firing rates of zero.Curves e and g are from the two neurons that appear in Figure 7. B, Comparison of the transient/sustained ratio for ramps and steps. C, Comparison of individual neurons’ preferred stimulus acceleration for ramps and preferred speed for steps of target speed. The solid line is the type 2 regression line for the neurons plotted withopen circles. In B and C,open circles are for cells with transient/sustained ratios for ramps >1.8, and small x symbols are for cells with transient/sustained ratios <1.8. Because the ramp increases in target speed always started from zero and were always 128 msec in duration, the two y-axes in C are proportional and differ only by the factor of 0.128−1 used to convert the change in target speed (left y-axis) into target acceleration (right y-axis).
Two facts argue against the possibility that the transient responses of MT neurons to accelerating and decelerating targets might arise because the apparent contrast of rapidly moving stimuli is lower than that of stationary targets. First, this effect should be the same for both increases and decreases in target speed, because MT cells usually respond equally to increases and decreases in contrast (Maunsell and Van Essen, 1983) (J. A. Movshon, unpublished observations). Thus, it cannot explain the asymmetries we observed; if anything, it would attenuate them by adding a response at the point of both increasing and decreasing target speed. Second, an effect of apparent contrast on the transient responses of MT neurons predicts that the transient responses should be unselective for target speed. Figures 3A and8A contradict this prediction.
Comparison of responses to ramps and steps of target speed
To summarize the ramp asymmetry for each cell, we computed the transient/sustained ratio for ramps, defined as (aR +aS )/aS , whereaR is the peak value of the curve fitted to the difference firing rate from graphs like Figure 7, Cand D, and aS is the peak value of the curve fitted to the sustained firing rate during steps of target speed. The transient/sustained ratio for ramps of target speed was 4.9 for the cell in Figures 6A and 7, A andC, and was 1.6 for the cell in Figures 6Band 7, B and D. Figure 8B shows that the transient/sustained ratio for steps and ramps of target speed agreed well for almost every cell we recorded (r = 0.87). The different symbols indicate cells with transient/sustained ratios for ramps greater than and less than 1.8 (compare Fig.8C). Type 2 regression analysis on the log10 of the data in Figure 8B, under the assumption that the two values of transient/sustained ratio were equally well estimated from the data, gave a slope of 0.74. As expected, there was also excellent agreement between the peak sustained responses for steps and ramps of target speed (r = 0.96; type 2 regression slope = 1.19).
Figure 8C illustrates that there was also a correlation between the preferred speed of the transient response to steps of target speed and the preferred acceleration from the fits to the difference curves for ramps of target speed. The values plotted along the y-axis (Fig. 8C) were all obtained for 128 msec ramps from zero to final speed, creating a proportionality between the final speed (left y-axis) and the acceleration of the preferred ramp (right y-axis). The scatter plot suggests a more consistent relationship for neurons with transient/sustained ratios >1.8 (Fig. 8C, open circles, r = 0.61) than for those with smaller transient/sustained ratios (small x symbols, r = 0.16). For the neurons with transient/sustained ratios of 1.8 or greater, type 2 regression analysis on the log10 of the data in Figure 8C(solid line) revealed a regression slope of 1.0, indicating that the derived values on the y- andx-axes are proportional. The constant of proportionality of approximately four implies that most cells could contribute most effectively to a distributed representation of target acceleration when the target accelerates from zero to final speeds higher than the cell’s preferred speed.
Responses to spatially restricted targets
To establish the generality of the response characteristics that we measured in MT cells stimulated with speed steps and ramps for random-dot texture targets, we studied the responses of 38 of the cells to the same motion trajectories of small spots and textures. These experiments were complex to design and execute, because the use of small targets mandated that we explore the effect of varying the receptive field position at which steps and ramps of target speed occurred. The analysis was similarly complex, because the sensitivity profile of the neuron’s receptive field had to be taken into account in evaluating the responses. We do not present a detailed analysis of these experiments here, but the results showed that the important dynamic features of MT cells’ responses to textures could also be discerned in their responses to small targets. We are therefore confident that our measurements with textures represent adequately the response of MT cells for the small targets normally used to study pursuit eye movements.
Effect of base velocity and ramp duration on the responses to target acceleration
The results in Figures 7 and 8 imply that many MT cells could provide information about the direction and possibly the magnitude of target acceleration for targets that are initially stationary and accelerate smoothly through the range of speeds to which the cell is sensitive. However, individual MT cells cannot signal acceleration for all initial target speeds. Figure9A–D shows the responses of an example MT neuron to different combinations of acceleration and initial target speed. This cell had large transient responses during 128 msec ramps of target speed from 0 to 2.25, 4.5, or 9°/sec (Fig.9A–C). The transient response was completely absent when the ramp started at the preferred speed for the sustained response, which was ∼4.5°/sec for this cell, and increased to 9°/sec (Fig.9D). Thus, a target acceleration of 35°/sec2 caused a large transient in firing when the target speed started at 0°/sec (Fig. 9B) and no transient when the target speed started at 4.5°/sec (Fig.9D). We obtained the same result on all 15 cells that were tested. The converse experiment, of varying ramp duration and therefore target acceleration while keeping the initial and final target speed the same, also emphasized the nature of the relationship between firing rate and target acceleration. When the initial target speed was 0°/sec and the final target speed was 9°/sec, the response of the cell in Figure 9 was essentially the same when the target acceleration was 2240°/sec2 (Fig. 9H, ramp duration 4 msec), 560°/sec2 (Fig. 9G, ramp duration 16 msec), 280°/sec2 (Fig. 9F, ramp duration 32 msec), 140°/sec2 (Fig.9E, ramp duration 64 msec), or 70°/sec2(Fig. 9C, ramp duration 128 msec). These results show that individual MT neurons can contribute to a distributed representation of target acceleration only for a limited range of target speeds.
Examples showing that MT neurons are sensitive to target acceleration only for some stimulus trajectories. Eachpanel shows a histogram of firing rate obtained from nine repeats of the target speed trajectory shown by thelower trace. The column ofhistograms shows the responses for different initial and final target speeds. The row ofhistograms shows responses to different accelerations produced by different duration ramps between zero and twice the preferred speed. A, 0–2.25°/sec in 128 msec.B, 0–4.5°/sec in 128 msec. C, 0–9°/sec in 128 msec. D, 4.5–9°/sec in 128 msec.E, 0–9°/sec in 64 msec. F, 0–9°/sec in 32 msec. G, 0–9°/sec in 16 msec. H, 0–9°/sec in 4 msec.
Responses to double pulses of target velocity at different interpulse intervals
Our data on responses of MT cells to steps and ramps of target speed (Figs. 2, 6) revealed that many neurons have large transient responses with complex dynamics. Because the size and time course of these transients are dependent on stimulus speed and acceleration, such simple transient-forming mechanisms as linear high-pass temporal filters (e.g., spike frequency adaptation) probably cannot provide a complete account of the data. To account for these complex dynamics, we guessed that neuronal excitability might be influenced by a time-dependent adaptation of the responses to a given stimulus. Adaptation might arise either from complex synaptic or cellular mechanisms or from neurons outside those providing excitatory input to the cell. We conducted the following experiments to probe adaptation, even though we realize that they do not distinguish possible mechanisms rigorously and that adaptation could be implemented by division or subtraction. Thus, we use the term adaptation to encompass a number of possible neural and cellular mechanisms.
As an initial probe of adaptation, we used a series of condition-test experiments in which we measured the response to a test pulse of target speed as a function of time after an identical conditioning pulse. As shown in Figure 10, targets were stationary and visible for 256 msec before undergoing one or two 64 msec pulses of motion of the preferred speed and direction. In Figure10, top, for example, the histogram (Firing rate) shows the response of one cell to two 64 msec pulses of target speed that were separated by 64 msec. As a control, the solid firing rate trace shows the response of the same cell to a single pulse of target speed. The response to the second pulse can then be estimated by subtracting the response to the first pulse from the response to two pulses (Fig. 10, middle, Difference firing rate).
Example histograms showing the experimental design and data analysis for experiments that presented two pulses of target speed at different interpulse intervals.Top, The histogram shows the accumulated response to 10 repeats of two 64 msec pulses of target motion in the neuron’s preferred direction at its preferred speed with an interpulse interval of 64 msec. Bin width is 8 msec. The solid trace superimposed on the histogram is the average response of the same neuron to 40 repeats of just the first pulse.Middle, The difference firing rate in thehistogram shows the response to two pulses minus the response to the first pulse. Bottom, Thesolid and dashed traces show the time course of target speed for the double- and single-pulse stimuli, respectively.
Figure 11 shows two examples that relate the results of the two-pulse condition-test experiments and the responses of MT cells to steps of target speed (traceslabeled Step). The data in Figure 11A came from a cell that had a clear transient response to 512 msec steps of target speed, whereas the data in Figure 11B came from a cell whose response was almost purely tonic. For each cell, the traces labeled Pulse (Fig. 11) show the response to the test pulse alone, and the traces labeled with different times show the responses when the onset of the test pulse followed the offset of the conditioning pulse by the specified delay. For each histogram in the left columns of Figure 11, A and B, the companion histogram in the right columns shows the difference firing rate, obtained by subtracting the response to the conditioning pulse from the response to two pulses and aligning the differences on the onset of the test pulse. For the response to the test pulse alone (Fig. 11,Pulse), the difference histograms were obtained by subtracting the spontaneous firing recorded during presentation of a stationary texture.
Comparison of transient behavior during steps of target speed with response attenuation in two-pulse experiments for two representative neurons. A, A neuron with a large transient during steps of target speed and strong attenuation of the response to the second pulse at short interpulse intervals.B, A neuron with no transient and no attenuation of the response to the second pulse. In A and B, the histogram labeled Step shows the accumulated firing for a 512 msec step of target speed to the same speed used for the two-pulse experiments. For thehistogram labeled Pulse inA and B, the left columnis the accumulated firing for a single pulse of target speed, and theright column is the same response with the spontaneous firing for a stationary dot pattern subtracted. For the lower six rows of A and B, theleft column shows the response histogramfor two pulses of target speed at the interval indicated by thenumber to the left of thehistogram. The right column shows the difference firing rate obtained by subtracting the response to the first pulse and aligning the resulting histograms on the onset of the second pulse of target speed. The vertical dashed line shows the approximate start of the response. Bin width is 16 msec for the response to steps of target speed and is 8 msec in all of the other histograms. The traces beloweach histogram show the time course of target speed. For each neuron, we performed this experiment for pulses of target speed that were near the preferred speed. The pulses and step of target speed were 2°/sec in A and 4°/sec in B. InA, the step and single-pulse responses were obtained from 40 repeats of each stimulus, and the otherhistograms were from 10 repeats. In B, the step and single-pulse responses were obtained from 60 repeats of each stimulus, and the other histograms were from 16 repeats.
For the cell with a transient response to steps of target speed (Fig.11A), there was a clear effect of the time between the pulses on the response to the second pulse. The second response was attenuated for interpulse intervals <64 msec and returned to control values when the interpulse interval reached 256 msec. The gradual return of the amplitude of the response to the second pulse can be seen in the histograms of the responses to two pulses (Fig.11A, left column ofhistograms) but is clearer in the difference firing rates obtained by subtracting the response to the first pulse (Fig.11A, right column ofhistograms). The difference firing rates are aligned on the time of onset of the second pulse, revealing that the latency of the response to the second pulse (Fig. 11A,vertical dashed line) did not vary consistently as a function of the interpulse interval. Note also that these difference rate histograms suggest that the response to the second pulse at short interpulse intervals is a scaled-down replica of the unconditioned response (Fig. 11A, top traces); these scaled replicas are a characteristic signature of a mechanism that controls response gain and are quite different from the abbreviated (“iceberg”) responses that would result from delivering double-pulse stimuli to a linear high-pass filter.
For the cell that lacked a transient response to steps of target speed (Fig. 11B), the two-pulse experiment yielded a different result. The response to the second pulse did not depend strongly on the interpulse interval and was nearly the same as that of the control even for interpulse intervals as short as 32 msec.
To quantify the relationship between the attenuation of the response to the second pulse at short interpulse intervals and the transient responses of MT cells to steps of target speed, we determined the latency of the response to a single pulse, defined the next 64 msec as an analysis interval, and measured the mean difference firing rate in the analysis interval for each of the six condition-test intervals. We then computed a “response attenuation index” defined as the mean difference firing rate for an interpulse interval of 32 msec divided by the average of the firing rates for interpulse intervals of 128 and 256 msec. Figure 12 shows the relationship between the transient behavior of firing rate for steps of target speed and the amount of attenuation revealed in two-pulse experiments. Each point in this plot shows results from one of the 22 cells studied using two pulse stimuli. The x-axis plots the response attenuation index derived above, and the y-axis plots the transient/sustained ratio for steps from stationary to the target speed used for the two-pulse experiment. In general, cells that had little or no transient response for steps of target speed also had attenuation indices near 1.0 for two pulse stimuli, indicating no attenuation. Cells with large transient responses for steps of target speed had attenuation indices as small as 0.1. Although we have elected not to show the data here, we obtained very similar results from the same cells when we tested them with a 64 msec pulse of target speed at different intervals after the offset of a 512 msec step of target speed. We interpret the correlation between the existence of a transient response to steps of target speed and attenuation of responses at short interpulse intervals as evidence that adaptation shapes the transient responses of MT cells. The recovery of the response to the test pulse at longer interpulse intervals reflects recovery from adaptation. In a later section of the paper, we will implement adaptation as divisive gain control to create transient responses in a model that reproduces the dynamics of MT cell responses.
Quantitative analysis of the relationship between response attenuation in two-pulse experiments and the transient behavior for steps of target speed. Each point shows the response of one MT neuron. The x-axis plots the response attenuation index for two-pulse experiments: the mean firing rate for an interpulse interval of 32 msec divided by the average of the responses for interpulse intervals of 128 and 256 msec. The y-axis plots the transient response divided by the sustained response for a step from zero to the target speed used for the two pulse stimuli. Note that this is different from the transient/sustained ratio computed earlier as the peak transient response divided by the peak sustained response across target speeds.
To determine whether the influence of the conditioning pulse was direction selective, we tested seven cells with a variant of the double-pulse experiment in which the first 64 msec pulse of target speed provided motion in the null direction and the second pulse was in the preferred direction. Figure 13shows one example of the responses, for the same cell whose responses are also shown in Figure 11A. This cell gave a brisk response to a 64 msec pulse of target motion in the preferred direction (Fig. 13, top pair of traceslabeled On-direction) and was inhibited slightly when the same pulse of target speed was delivered in the null direction (trace labeled Null-direction). The histograms of firing rate for the double pulse stimuli reveal that this cell responded well to the pulse of target speed in the preferred direction for the shortest condition-test intervals, even though the same cell showed almost complete attenuation of the response to the second pulse at short intervals when the two pulses were in the same direction (see Fig. 11A). We again isolated the response to the second pulse by computing the difference between the responses to two pulses and to the test pulse alone and aligning the difference firing rate on the onset of the second pulse.
Example of a two-pulse experiment showing the effect of conditioning motion in the null direction on subsequent responses to target motion in the preferred direction. The same neuron’s responses are also shown in Figure 11A. For the histograms labeledOn-direction, the left column is the response histogram for a single pulse of target speed in the preferred direction, and the right column is the same response with the spontaneous firing for a stationary dot pattern subtracted. The histogram labeledNull-direction shows the response to a single pulse of preferred target speed in the null direction. For thelower six rows, the left column shows the accumulated firing for two pulses of target speed at the interpulse interval indicated by the number to theleft of the histogram. The right column shows the difference firing rate obtained by subtracting the response to the first, null-direction pulse and aligning the resulting histograms on the onset of the second, preferred-direction pulse of target speed. The vertical dashed line shows the approximate start of the response to a single pulse. The traces below each histogramshow the trajectory of target speed, which was 2°/sec for this experiment and was close to the neuron’s preferred speed in all experiments. In all histograms, the bin width is 8 msec. The on-direction and null-direction histograms were accumulated from 40 repeats of the same stimulus; the otherhistograms were from 10 repeats.
In each of the seven cells tested, conditioning motion in the null direction affected responses to subsequent test motion in the preferred direction, but in a different way than did conditioning motion in the preferred direction. There was no response attenuation at short latencies. Instead, the dynamics of the response were affected. In Figure 13, for an interpulse interval of 0 msec, the response to the preferred-direction pulse was delayed by almost 24 msec and had a sharper time course and a larger amplitude than did the response to the control pulse (histogram labeledOn-direction). The effect on the latency of the response to the preferred direction was absent when the interpulse interval increased to 96 msec. But inspection of Figure 13 hints that null-direction motion may have had a rather long-lasting effect on the shape and amplitude of this cell’s responses to subsequent motion in the preferred direction, even for an interpulse interval of 256 msec. We have confirmed this effect with more detailed observations on a larger sample of cells (Priebe et al., 1998). However, our limited sample of seven cells provided enough data to show that the adaptation in MT cells is direction selective.
Responses to sinusoidal modulation of target speed
Traditional approaches to understanding dynamics have often relied on sinusoidal-forcing functions to obtain estimates of response gain and phase as a function of frequency. In a linear system, such analysis provides the same information as the time domain analysis we have presented so far. In this instance, however, it seemed useful to analyze the responses of MT cells to sinusoidal modulation of target speed, partly because their responses are clearly nonlinear and partly to obtain data relevant to the performance of the smooth pursuit system, which has been analyzed frequently with sinusoidal target motion (e.g., Fuchs, 1967; Lisberger et al., 1981; Goldreich et al., 1992).
Because of the nonmonotonic relationship between firing rate and target speed, we expected that the firing elicited by sinusoidal modulation of target speed would depend critically on the base speed around which the oscillations occurred. For example, if target speed during the oscillation were confined entirely to speeds on the rising phase of the cell’s speed-tuning curve, then we would expect firing rate to be modulated approximately sinusoidally with a peak response near peak on-direction target speed. If, however, target speed were greater than preferred speed throughout the full sinusoidal oscillation, then we would expect firing rate to be modulated at the frequency of the stimulus but with peak firing at minimum rather than maximum target speed in the preferred direction. Finally, if the oscillation of target speed were centered on the peak of the speed-tuning curve, then we would expect firing rate to decrease for both increases and decreases in target speed, and the modulation of firing rate would be at twice the frequency of the sinusoidal stimulus.
With these expectations in mind, we customized the parameters of sinusoidal modulation of target speed for each cell according to the strategy summarized in Figure 14. The graph on the right shows the speed tuning for a hypothetical cell by plotting firing rate in the preferred and null directions as a function of target speed. The four sine waves on the left show target speed, on the same axis as the speed-tuning curve, as a function of time for the four conditions. For DC = 0, the baseline speed was zero, and target speed oscillated between preferred speed in the preferred and null directions. This was the only sinusoidal modulation of target speed that delivered motion in the null direction. For DC = 0.5, target speed oscillated between zero and the preferred speed. For DC = 1, target speed was centered on the preferred speed and oscillated approximately between the two speeds that caused half-maximal responses. For DC = 2, target speed oscillated along most of the arm of the relationship between firing rate and target speed that was above preferred speed. In practice, we did not always achieve the goals outlined in Figure 14, partly because the parameters of target motion had to be estimated during recording for each neuron and partly because skew in the speed-tuning curves of real MT cells made it impossible to achieve the ideal reflected by Figure 14. When our stimuli did approach the goal envisaged in Figure 14, however, the general features of the neuronal responses in MT conformed to our expectations.
Schematic diagram of the experimental design for presenting sinusoidal modulation of target speed on different base speeds. The graph on the right plots target speed on the y-axis, with preferred-direction motion upward, and sustained firing rate on the x-axis, with increased firing rate plotted to the right. Thecurve shows a schematic speed-tuningcurve for an MT neuron. The four sine waves on theleft of the figure show target speed as a function of time. The speed calibration for the sine waves is the same as that for the speed-tuning curve at the right. The sine waves labeled DC = 0, DC = 0.5, DC = 1, and DC = 2 show schematically the situation we tried to achieve when we selected the baseline speeds and amplitudes of modulation of target speed used to study the responses of each MT cell.
Figure 15 shows data from a cell that exemplifies all the basic features of the responses to sinusoidal modulation of target speed. When the baseline speed was zero (Fig.15A, DC = 0), firing rate was strongly modulated at frequencies up to 8 Hz. Modulation of firing rate increased somewhat as the frequency of target speed oscillation increased from 1 to 8 Hz but was small at 16 Hz. The attenuation of modulation at 16 Hz is verified by the cycle histograms on the upper right of each long histogram. The cycle histograms were constructed with 24 bins per cycle by averaging the responses across the last second of sinusoidal modulation. For all frequencies, MT neurons emitted a pulse of firing for the first cycle of target motion at DC = 0. When target speed oscillated between zero and the preferred speed (Fig. 15B, DC = 0.5), the modulation of firing rate was considerably weaker than when target speed oscillated around a baseline speed of zero. Again, there were responses at 1, 2, 4, and 8 Hz, but it is difficult to see any modulation of firing rate at 16 Hz in spite of an increase in the sustained firing of this MT cell.
Responses of a representative MT neuron to sinusoidal modulation of target speed. The figure consists of 15 pairs of target velocity traces and histogramsaccumulated from six repeats of the same target motion. Theinset on the right above eachhistogram is a cycle histogram showing the modulation of firing rate averaged over the last second of the sinusoidal speed modulation; the baseline of these cyclehistograms has a duration equal to one period of the sinusoid. Cycle histograms have 24 bins per cycle. Full-stimulus histograms have a bin width of 10 msec.A, The amplitude of modulation of target speed was 3°/sec, and the base speed was zero. B, The amplitude of modulation was 1.5°/sec, and the base speed was 1.6°/sec.C, The amplitude of modulation of target speed was 10.5°/sec, and the base speed was 13.5°/sec. The preferred speed of this neuron was 3.2°/sec. From top tobottom in A–C, the frequency of the sine wave was 1, 2, 4, 8, and 16 Hz. Data are from neuron 405r09.
When the baseline target speed was above the preferred speed so that target speed oscillated on the descending limb of the relationship between firing rate and target speed (Fig. 15C,DC = 2), the response of the cell was again modulated. As expected, the phase of the neuronal responses was now reversed relative to DC = 0 and DC = 0.5. For example, at 1 Hz, increased firing occurred during the upward deflection of the sinusoidal component of target speed for DC = 0 and DC = 0.5 but during the downward deflection for DC = 2. The single-cycle histograms illustrate the effects of oscillation frequency on phase shift. At each frequency, the response phase is similar for DC = 0 and DC = 0.5 but reversed for DC = 2, just as described above for oscillations at 1 Hz. For Figure 15, A–C, response phase lag increases with frequency, as expected of a response with a latency that is a substantial fraction of the period of the higher frequency oscillations. Responses for DC = 1 were quite small and, when present, were dominated by the second harmonic of the oscillation frequency (data not shown).
For target speed oscillation around a base speed of zero, the relationship between the modulation of firing rate and the frequency of oscillation varied widely from neuron to neuron. Figure16A plots the modulation of neuronal response as a function of the frequency of oscillation of target speed for 10 example cells. The examples were selected by ordering the 31 cells tested with sine waves according to the maximum modulation of firing rate at any frequency and plotting every third cell. For each cell, we normalized response modulation at all frequencies to the maximum. In Figure 16A, the functions are plotted on the normalized scale defined by the calibration bar on the bottom right of the graph but at arbitrary positions on the y-axis to facilitate viewing. The short horizontal dashed line on the right end of each curve shows zero modulation for that curve and demonstrates that the normalized modulation of neuronal firing at 16 Hz was always <0.1. Inspection of Figure 16A shows that many cells had increases in response modulation as the frequency of target oscillation increased from 1 toward 8 Hz, while a few had low-pass characteristics. Peak response modulation occurred at frequencies that ranged from 1 to 10 Hz in different cells, and response modulation was always small at frequencies >10 Hz.
Analysis of responses to sinusoidal target motion. A, Response modulation as a function of the frequency of sinusoidal target motion for base speeds of 0 (DC = 0) in 10 representative MT neurons. Eachcurve plots data from a different MT neuron selected according to the procedure outlined in the text. Eachcurve has been normalized for the maximum modulation of the individual neuron’s firing at any frequency of modulation of target speed and then shifted vertically to facilitate viewing. The calibration bar on the lower right of thegraph shows the full scale of each normalized curve. Thehorizontal dashed line on the right of each curve shows the baseline for thatcurve, and the number next to eachcurve gives the maximum amplitude of response modulation for that neuron at any frequency. B, Phase shift between target speed and firing rate plotted as a function of the frequency of sinusoidal target motion with base speeds of zero. Thecurves summarize the responses from 16 neurons. Thelarge filled circles show the phase predicted by a fixed time delay of 90 msec. C–F, Polar plots summarizing the responses of all the MT neurons that were studied with sinusoidal modulation of target speed at 1 Hz. Each line is a vector representing the response of an individual MT neuron; thesmall filled circles show the center of the plot, i.e., zero response; and the large outer circles show a response modulation of 25 impulses/sec. Vectors pointingto the right would be in-phase with on-direction target speed, and phase lag is indicated by counterclockwise rotation of the vectors.
The phase shift between the sine wave of target speed and neuronal firing showed a consistent relationship to the frequency and DC level of the sinusoidal stimulus motion. For each cell, we measured the phase difference between firing rate and target speed as the difference between the phase shifts of the fundamental components obtained from Fourier analysis. To correct for the fact that this difference was, by definition, <360°, we assumed that phase lag would increase monotonically and added 360° of phase shift whenever an increase in frequency caused a large decrease in phase difference between the two fundamental components.
Figure 16B plots the response phase for 16 cells as a function of stimulus frequency during sinusoidal target motion around a DC value of zero (DC = 0). The responses were extremely consistent across cells. Firing rate almost always lagged target speed. Phase lag was small at 1 Hz (mean, 27°; n = 31) and increased as a function of frequency. This behavior is expected if firing rate is determined by the response to stimulus speed. If firing rate were dominated by a response to image acceleration, then we would predict some phase lead, which we observed in only a few neurons. Two factors probably contribute to the absence of phase lead. (1) Target acceleration is small at the low frequencies in which phase lead would be most evident. (2) For target speeds below preferred speed, neuronal responses are related more closely to stimulus speed than to acceleration (compare Figs. 6, 8C).
It was possible to account for the increases in phase lag as a function of frequency with a fixed time delay. The relationship between response phase Φ and frequency ft
is described by the relation:
Equation 2where Δt is a fixed time delay. We applied Equation 2to the data plotted in Figure 16B and found that a Δt of 90 msec provides an acceptable fit to the data for DC = 0 (large filled circles). Similar analysis revealed that the data for DC = 0.5 were fit by a Δtof 85 msec (data not shown). There are a number of reasons why the time delay that accounts for the phase shifts in Figure16B is longer than the latency measured for steps of target speed (Figs. 4, 5). (1) The use of Fourier analysis to compute phases emphasizes the center-of-mass of the neuronal response rather than its early peak. (2) The phase shift was computed for steady-state stimuli and low target speeds, which gave longer latencies even for steps of target speed. (3) The effect of null-direction motion on the latency of responses to subsequent motion in the preferred direction would lengthen the latency and add phase lag for DC = 0 (compare Fig. 13).
Figure 16, C–F, shows that the responses to 1 Hz sinusoidal modulation of target speed conformed with our expectations when the sine wave was imposed on different baseline target speeds. These four polar plots represent the responses of each cell at 1 Hz as a vector, in which the length of the vector indicates the amplitude of modulation of the fundamental frequency component of firing rate and the angle indicates the response phase. For DC = 0 (Fig. 16C), the response magnitudes were uniformly large, and all but one of the vectors pointed to the right and slightly up, indicating that firing slightly lagged preferred-direction target speed. The plot for DC = 0.5 (Fig. 16D) was similar to that for DC = 0, although the mean response amplitude was slightly smaller (19.7 impulses/sec for DC = 0.5 vs 22.3 impulses/sec for DC = 0). For DC = 2 (Fig. 16F), many of the responses were still quite large (mean, 13.7 impulses/sec), but most of the vectors pointed to the left, indicating that firing lagged on-direction target speed by ∼180° and was approximately in-phase with off-direction target speed. Finally, for DC = 1 (Fig.16E), the response amplitudes were small (mean, 7.5 impulses/sec), and the vectors pointed haphazardly in all directions, indicating that the population as a whole showed no consistent response phase.
Simulation results
We began the experiments described above to determine whether the image acceleration signals used in some models of smooth pursuit eye movements were represented along with image velocity in the distributed response of neurons in MT. The previous section suggests an affirmative answer by showing that the firing of many MT neurons is influenced by image acceleration. The goal of the present section is to test the hypothesis that the population response in MT represents image velocity and acceleration. To accomplish this goal, we (1) create models to simulate the responses of all MT cells in our sample, (2) use the models to simulate the population response to a given set of stimuli, and (3) demonstrate computations that reconstruct image velocity and acceleration from the simulated population response.
We have simulated the responses of MT neurons using the model structure illustrated in Figure 17; the details of the implementation of this model can be found in the . The model (Fig. 17A) received an input that represented image velocity as a function of time. Image velocity was processed by a time delay that recreated the relationship between latency of MT responses and target speed, three parallel elements that used divisive gain control to model the responses to preferred-direction motion, a summing junction to add the outputs of the three elements, and a directional interaction to account for the effects of motion in the null direction on the subsequent responses of MT cells to motion in the preferred direction. The elements in Figure 17A were constructed according to the diagram in Figure 17B. Each consisted of a numerator pathway that produced output A (Fig.17B, top pathway), a denominator pathway that produced output B (bottom pathway), and a division junction so that the output from each element wasA/(1 + B). Each numerator pathway consisted of a low-pass filter with a single time constant and a speed-tuned nonlinearity. Each denominator pathway consisted of a time delay, a low-pass filter with a single time constant, and another speed-tuned nonlinearity. The tuned nonlinearities were of the form given by Equation 1, used above to describe the relationship between MT cell firing rate and target speed. The same model architecture was used for all cells, but different parameter values were needed for different cells. For each cell, we adjusted the parameters of the model within realistic limits [using the minimization program STEPIT (Chandler, 1965)] to optimize its fit to different subsets of the responses we recorded. For most cells, models containing only a single element were capable of fitting the data adequately, but the fit was almost always improved by using two or three elements in parallel.
Architecture of the model used to simulate the responses of MT neurons to the stimuli used in this paper.A, General structure of the model. Delayrefers to a time delay that depended on target speed. Elements 1, 2, and 3 each have the structure defined by panel B. The circlewith a plus sign in it is a linear-summing junction.Directional interaction is used to account for the effects of null-direction motion on subsequent preferred-direction motion. B, Structure of each element. The input is image velocity after the delay in panel A. Low-pass filters responded to a step input with an exponential increase in output. The two tuning functions were implemented using Equation 1. Thecircle with a division sign in it produces A/(1 + B) as its output. The four traces on the right show signals at different points within an element during a ramp of target speed like those used to study MT neurons. From top tobottom, the traces are delayed image velocity (I), the output of the numerator pathway (A), the output of the denominator pathway (B), and the output of the gain control element. The traces have been scaled arbitrarily along the amplitude axis to allow easy viewing. The vertical dashed line is aligned on the onset of the ramp increase in delayed image motion. The calibration bar on the right oftrace I shows the preferred speed of the nonlinearity in the numerator pathway for this element. vel., Velocity.
The denominator pathways were designed to produce two of the main features of the response dynamics of MT neurons in our experiments: transients in the responses to steps of target speed and an asymmetry between the responses to ramp accelerations and decelerations. These dynamics are present in neither the numerator nor the denominator pathways but instead arise from their interaction. For example, the stack of traces at the right edge of Figure 17B shows the time courses of the signals in a single element when delayed image velocity (I) follows the trapezoidal trajectory shown in the top trace of the stack, with sustained target speed twice the cell’s preferred speed. Because the ramp increase and decrease in target speed provide the same sequence of speeds in opposite order, they evoke similar small pulses that overshoot a substantial sustained response in the numerator pathway (trace labeled A). Similarly, the denominator pathway (trace labeled B) emits a response that is symmetrical for ramp increases and decreases in target speed. In this example, the output of the denominator pathway lacks overshoot because the tuned nonlinearity has a higher preferred speed in the denominator pathway than in the numerator pathway. Because of the extra time delay in the denominator pathway, its response (B) is delayed relative to the numerator pathway’s response (A), and the initial, large output from the model element is controlled entirely by the numerator pathway. When the output of the denominator pathway starts to increase, the output of the model declines toward its sustained response with a trajectory that is determined by the delay and low-pass filter in the denominator pathway. Because the nonlinearity in the denominator pathway need not have the same preferred speed, bandwidth, or skew as that in the numerator pathway, the effect of gain control on response dynamics in the model can vary with target speed. This feature of our data would not emerge from a model that used a single, linear high-pass filter to simulate the dynamics of MT neuron responses.
Performance of the model for steps and ramps of target speed
Figures 18 and19 illustrate the agreement between the output of the optimized model (bold traces) and the response histograms of two MT neurons with very different dynamics. For a cell with a large transient (Fig. 18), for example, the response of the model showed a realistic large transient at the onset of the response to a step of target speed (Fig. 18A) for targets at speeds below, near, and above preferred speed. The output from the model also showed the same large asymmetry between the responses to ramp increases and decreases in target speed as did the neuronal response (Fig. 18B). For double pulses of target speed (Fig. 18C), the output of the model showed large attenuation of the response to the second pulse when the interval between pulses was 32 msec (top trace). Attenuation decreased as the interval between pulses was lengthened. Finally, for sinusoidal modulation of target speed with DC = 0 (Fig.18D), the model reproduced the pulse at the onset of the response to each cycle of 1 Hz target motion, the slightly asymmetric pulse of response to each cycle of 4 Hz target motion, and the very small response to 16 Hz target motion.
Comparison of neuronal responses and model output for selected target motions in a neuron with large transient responses.A, Steps of target speed from 0 to 1, 4, or 16°/sec.B, Ramps of target speed from 0 to 1, 4, or 16°/sec.C, Double pulses of target speed. Each pulse was 64 msec in duration, and the interpulse interval was 32, 64, and 128 msec fromtop to bottom. D, Sinusoidal modulation of target speed at 1, 4, and 16 Hz with DC = 0. Histograms give the neuronal responses, andbold traces show the output from the optimized model. Bin width was 8 msec for A–C and 10 msec for D. Data are from unit 404r02.
Comparison of neuronal responses and model output for selected target motions in a neuron with relatively small transient responses. A, Steps of target speed from 0 to 0.25, 2, or 8°/sec. B, Ramps of target speed from 0 to 0.25, 2, or 8°/sec. C, Double pulses of target speed. Each pulse was 64 msec in duration, and the interpulse interval was 32, 64, and 128 msec from top tobottom. D, Sinusoidal modulation of target speed at 1, 4, and 16 Hz with DC = 0.Histograms give the neuronal responses, and bold traces show the output from the optimized model. Bin width was 8 msec for A–C and 10 msec forD. Data are from unit 405r20.
For an MT neuron with a much smaller transient response that depended on target speed (Fig. 19), the model again reproduced the responses accurately. The time course of the model output reproduced that of the neuronal response for steps (Fig. 19A) and ramps (Fig.19B) of target speed, double pulses of target motion (Fig.19C), and sinusoidal modulation of target speed with DC = 0 (Fig. 19D). Especially noteworthy is the ability of the model to produce quite different response time courses for steps or ramps of target speed to different sustained target speeds.
To quantify the performance of the model, we analyzed its output in the same way as our neuronal data. This revealed excellent agreement between the transient/sustained ratio of the model and the data for steps (r = 0.97; type 2 regression slope = 0.88) and ramps (r = 0.95; type 2 regression slope = 0.98) of target speed, except that the model consistently produced transient responses that were slightly too small. For two pulse stimuli, the agreement between the model and the data was excellent for interpulse intervals of 0 and 256 msec and less good but acceptable for the other interpulse intervals. For sinusoidal modulation of target speed, the performance of the model was excellent for frequencies of 1–8 Hz and DC = 0 and DC = 0.5. For DC = 2, however, the modulation of model output fell short of the modulation of neuronal firing by an average of 50%. For all DC values, the modulation of the model output modestly exceeded the modulation of neuronal firing at the highest frequencies (12 and 16 Hz). Thus, the model captured the basic features of each MT neuron’s response dynamics, even if a few details could not be simulated perfectly. Two features of the model were most important for allowing it to fit the data. The denominator pathway enabled it to emit transient responses for steps of target speed and to reproduce the asymmetry in the transient responses to ramp increases and decreases in target speed. The use of three parallel elements allowed the model to reproduce the wide variation in dynamics in the responses of individual MT neurons across target speeds.
Reconstruction of target velocity and acceleration from the population response in MT
After we had obtained excellent simulations of the responses of all MT cells in our sample, we could predict the population response to a single target motion and ask how to reconstruct the dynamics of the target motion from the population response. Figure20 shows the population response in area MT for single target motion that consisted of a 128 msec ramp of target speed from 0 to 8°/sec, 512 msec of motion at a constant speed of 8°/sec, and another 128 msec ramp back to 0°/sec. The 50 traces in Figure 20 show the responses of 50 different neurons, simulated by our model of MT cell responses, under the assumption that each cell had the same preferred direction. Each trace is plotted at x andy coordinates determined by the neuron’s preferred speed (x-axis) and transient/sustained ratio for steps of target speed (y-axis). This stimulus elicited measurable responses from almost every cell, but the pattern of response varied in a characteristic way according to each cell’s speed preference and response dynamics. Cells with preferred speeds near the maintained target speed (Fig. 20, vertical dashed line) had the largest sustained responses. Cells with higher values of transient/sustained ratio had larger transient responses and more pronounced asymmetries between the transients produced by ramp accelerations and decelerations. For neurons with the same transient/sustained ratio but different preferred speeds, the largest asymmetries appeared in neurons whose preferred speeds were well below sustained target speed.
Distributed response in MT to a particular target motion. The graph plots 50 neurons’ responses to a standard stimulus as a function of time. Each trace is plotted at a location determined by the preferred speed (x-axis) and the transient/sustained ratio for steps of target speed (y-axis) of the neuron that gave rise to the trace. Each trace was obtained by using the model fitted to the given cell to estimate how the cell would have responded to the standard stimulus and then by truncating the response to the stationary target at the start and end of each stimulus. Each trace is normalized to the cell’s maximum sustained response to steps of target speed. Thetrace at the top left labeledTarget speed shows the time course of the standard stimulus; it consisted of a 128 msec ramp from 0 to 8°/sec, 512 msec of motion at constant speed, and a 128 msec ramp back to 0°/sec. Thevertical dashed line is drawn at the sustained stimulus speed on the x-axis. The responses of 50 cells are plotted here. The other 39 cells have been omitted because theirtraces plotted on top of one of those on thegraph. A number of the traces have been displaced slightly to avoid crossed traces.
Salinas and Abbott (1994) showed that a center-of-mass computation is a useful method for pooling the responses of neurons in a population whose members are dispersed over a parameter space, as in Figure 20. We formulated their suggestion to calculate a pooled motion response as follows:
Equation 3where m[t] is the pooled response at timet, MTi[t] is the response at time t of the model fitted to the ith cell, wi
is the weight afforded the outputs from the ith cell, and ε has a small positive value to make the computation less sensitive to noise. The denominator of Equation 3 implements vector averaging of MT unit responses, consistent with evidence that the command for pursuit eye movements is created by vector averaging the outputs of the active MT neurons (Ferrera and Lisberger, 1997; Groh et al., 1997).
Figure 21 illustrates how the pooled response from Equation 3 depends on the way that the vector of weightsw is related to the response properties of each cell. For these computations, we first simulated the distributed representation in MT for trajectories of target speed consisting of a 128 msec ramp increase in target speed, motion at constant speed for 512 msec, and a 128 msec ramp decrease in target speed. We used four final target speeds of 4, 8, 16, and 32°/sec to create four different distributed representations for the same set of 89 MT neurons. We then explored different rules for setting the output weight of each neuron (wi ). If we setwi to be proportional to the preferred speed (PSi) of the ith cell (Fig. 21,traces labeled Preferred speed only), then the pooled responses for the four final target speeds provide reasonable reconstructions of the full trajectory of target speed. If we setwi to be proportional to the transient/sustained ratio (TSRi) of theith cell minus 2.1 (Fig. 21, traces labeledTSR only, the choice of 2.1 to be explained below), then the pooled responses reconstruct a transient in association with target acceleration but do not differentiate the different rates of target acceleration produced by the ramps for different final speeds. If we set wi to be proportional to both TSRi − 2.1 and PSi (Fig.21, traces labeled TSR and preferred speed), then the pooled responses reconstruct a transient during ramp increases in target speed; moreover, the transient scales with target acceleration. We conclude that a center-of-mass computation can reconstruct target acceleration from the distributed response in area MT, although imperfectly; inspection of these traces reveals an undesired, small sustained response for a final target speed of 32°/sec and, perhaps more importantly, shows that this approach fails to reconstruct target deceleration.
Results of pooling the distributed representation of target motion in MT under different assumptions about the weighting of the output from each neuron. The bottom set oftraces shows the profiles of target speed. Thetop three sets of traces show the pooled responses. From top to bottom, unit outputs were weighted in proportion to the preferred speed (PS) of the unit, the transient/sustained ratio (TSR) minus 2.1, or PS × (TSR − 2.1). Line weights indicate sustained target speed: light dashed, 4°/sec; light solid, 8°/sec; bold solid, 16°/sec; andbold dashed, 32°/sec.
Why do these different choices of wi provide the pooled responses they do? First, subtracting 2.1 from the transient/sustained ratio assigns negative weights to the outputs of MT neurons with a low transient/sustained ratio. The computation then rejects the sustained component of firing rate related to target velocity, which is approximately equal in all neurons with the same preferred speed, but retains the transient component, which is related to TSRi. Second, weighting by the product of the transient/sustained ratio and preferred speed capitalizes on a feature of the data that we mentioned above in reference to Figures8C and 20. Consider Figure 20 as a set of vertical arrays of cells, in which the cells in a given array have similar preferred speeds but a wide variety of transient/sustained ratios. For a ramp increase in target speed, the response profile within each array will depend on the relationship between the final target speed and the preferred speed of cells within that array. Within arrays of cells that have preferred speeds below the final target speed (Fig. 20, left side), there will be a range of transient responses in relation to the value of the transient/sustained ratio; weighting by TSRi − 2.1 will reveal transients. Within arrays of cells that have preferred speed above the final speed, the transient responses are quite attenuated; weighting by TSRi − 2.1 will not reveal large transients. It follows that information about the rate of target acceleration can be obtained by knowing the preferred speed of the array of cells that has a wide range of transient responses. Thus, reconstruction of target acceleration depends on weighting by TSRi to detect the transient responses and by PSi to make a reconstruction based on which vertical array of cells has large transients.
DISCUSSION
The representation of target motion by MT neurons
It has been known for many years that neurons in MT are tuned for the direction and speed of target motion and that any given target motion is represented by the distributed response of a large number of neurons in MT. We have now shown that the same distributed representation of target speed and direction may also use transient signals to provide information about target acceleration. The transients are seen at the onset of target motion at constant speed and as an asymmetry in the responses to smooth increases and decreases in target speed. Two important features of this asymmetry can be appreciated from the data alone. First, the firing of a given MT neuron is influenced by target acceleration only for targets that start from close to zero speed or at least well below preferred speed. Thus, it would be incorrect to state that MT neuron firing is “related” to target acceleration. Second, most MT neurons show the largest transient/sustained ratio for ramps to target speeds at least twice their preferred speed. As a result, the best information about target acceleration is available in the firing of neurons with preferred speeds that are traversed by the accelerating target.
A model that reproduces the dynamic responses of MT cells
We constructed a model of MT cell responses with the initial goal of simulating the distributed representation of image motion in MT. Only after completing the model were we able to show that target acceleration could be reconstructed from the population response of MT neurons. The same models will be useful for the next obvious step, which is to create a model of pursuit eye movements based on visual inputs that simulate the population responses of MT. For these two purposes, it did not matter whether the equations used to simulate cell responses were biologically plausible. Our goal in simulating MT cell responses was simply to obtain a good fit to the data that would generalize well to image motions that had not been presented to the cell, including the image motions normally seen during pursuit. The model presented here accomplished this initial goal satisfactorily, in spite of minor failings on some of the more complicated target motions we used when recording from MT neurons.
The main feature of our model that allows it to reproduce the transient responses of MT neurons is a nonlinearity configured to perform divisive gain control. The gain control provided by the denominator pathways in our model is similar to, indeed inspired by, the suggestion that gain control may be a general mechanism of cortical function (Bonds, 1989; Heeger, 1992; Heeger et al., 1996; Carandini et al., 1997). Of course, our modeling neither proves that gain control is used to create transient responses in MT nor constrains the mechanisms of gain control. Plausible mechanisms include the divisive normalization proposed for MT cells by Simoncelli and Heeger (1998), forms of short-term synaptic depression like those revealed by Varela et al. (1997), a nonlinear implementation of spike frequency adaptation [after Wang (1998)], or any other mechanism that acts like a suitably nonlinear high-pass filter.
After we realized that gain control made the model work well, we explored the time course of the adaptation mechanisms we had modeled as gain control by measuring neural responses to two pulse stimuli. This approach used the conditioning pulse to invoke adaptation and test pulses at different intervals to probe the recovery from adaptation. We found a good correlation between attenuated responses to the test pulse at short intervals and large transient responses to steps of target speed. This finding confirms the existence of adaptation that can be modeled by gain control to create transient responses. That adaptation was directional in our recordings makes it likely that its mechanism resides in MT, rather than in the primary visual cortex where contrast gain seems to be set by a signal that is insensitive to the orientation and direction of the stimulus (Carandini et al., 1997). The gain control we postulate for MT has a different outcome—the creation of transient responses—than that proposed by many other authors. Perhaps gain control has many functions in the cortex. These functions could range from normalization to render steady-state neuronal responses invariant with the contrast of a stimulus to sculpting the temporal response profiles in a way that creates transient responses capable of conveying information about the dynamics of the stimulus.
Image motion inputs to models of pursuit
It has been understood for some time that reconstruction of the direction and speed of target motion requires comparison of the responses of multiple neurons. The responses of any individual neuron are ambiguous; the neuron might fire at less than its maximum rate because target motion is below preferred speed, above preferred speed, or in a nonpreferred direction. The same is true of attempts to reconstruct target acceleration. Even with post hocinspection of the responses to a given ramp increase and decrease in target speed, individual neurons provide information primarily about the direction of acceleration rather than about the rate of acceleration. Thus reconstruction of target acceleration also requires neural computations based on the responses of multiple MT neurons.
We have demonstrated that it is possible to reconstruct both image velocity and image acceleration from the distributed response we recorded in MT. However, this does not achieve our long-term goal of using the population code in MT to drive a model of pursuit that reproduces the features shown in Figure 1: short latency, brisk smooth eye acceleration in the first 100 msec of tracking, lack of a large overshoot in the transition from initial eye acceleration to maintained eye velocity, and a relatively high frequency of eye velocity oscillations during sustained pursuit of target motion at constant speed. We have not yet evaluated this question fully because we are aware of two problems that must be solved before such a model is likely to succeed. (1) Although it is clear that Equation 3 can extract information about target acceleration from the distributed response in MT, it does not provide information about image deceleration, such as occurs during the initial, brisk eye acceleration toward target velocity. In the model of pursuit eye movements proposed by Krauzlis and Lisberger (1994), image deceleration is used to prevent eye velocity from overshooting target velocity excessively. (2) From what we know now, it is not clear exactly how to extract information about image acceleration from the population response in MT during sinusoidal modulation of target speed. Image acceleration information should cause MT neuronal discharge to lead sinusoidal target velocity. Phase lag, not phase lead, was revealed by our recordings. Furthermore, when eye velocity is oscillating sinusoidally around target velocity, the maximum image acceleration occurs at zero image speed, when MT cells are mostly silent. It will be difficult to extract meaningful information from silent neurons. In the model proposed by Krauzlis and Lisberger (1994), sinusoidal image motion provided an image acceleration signal that was essential for modeling the propensity of monkey pursuit to oscillate at ∼6 Hz during pursuit of target motion at constant velocity. To a first approximation, a negative feedback system like pursuit will oscillate at the frequency at which the total phase lag around the feedback loop is 180°. Some of the phase lag is caused by the 80 msec latency, some by 90° of phase lag inserted by a neural integrator, and some by the dynamics in visual processing. If visual processing introduces no lead or lag, then the system will oscillate at the frequency at which the latency introduces 90° of lag: 3.125 Hz. If visual processing introduces 90° of phase lead, because of image acceleration signals, then the system will oscillate at the frequency at which the latency introduces 180° of phase lag: 6.25 Hz.
Several approaches might be used to extract more information from the population code in MT. Image deceleration information may be extractable by a number of neurally plausible computations such as dividing along the transient/sustained ratio axis or weighting the outputs of each cell differently from the modified center-of-mass computation we have performed. The problem of silent MT neurons at maximum acceleration during sinusoidal image motion may be resolved by creating a true opponent representation in which baseline firing is increased by image motion in the preferred direction and decreased by image motion in the nonpreferred direction. It also is possible that the dynamics of MT neuron responses in awake monkeys will be richer than those in anesthetized monkeys and will support reconstruction of image acceleration and deceleration for the full range of image motions normally seen during pursuit eye movements.
General principles for reading a distributed neural code
Our success in reading image velocity and acceleration from a single distributed code suggests a principle that might provide a neurally plausible solution to this general problem. Suppose that multiple variables are represented in a single population response and that it is possible to parameterize the distributions of neural responses along multiple eigen-axes as we did in Figure 20 for the population response in area MT. Then, it should be possible to extract the different variables by center-of-mass computations along the eigen-axis of the variable that is being reconstructed. For the population response in MT, this approach reconstructed target velocity or acceleration by center-of-mass computations along the eigen-axes of preferred speed or the product of preferred speed and transience.Salinas and Abbott (1994) have shown for a single variable that this approach is close to optimal under reasonable assumptions about the statistics of the distributed responses in the population.
There are two major strengths of the center-of-mass approach to reconstructing individual variables from a distributed representation of multiple stimulus features. First, the center-of-mass computation is biologically plausible. In neural terms, Equation 3 suggests that the population response is pooled by summing each neuron’s output weighted according to some feature of the neuron’s response—i.e., preferred speed or the product of preferred speed and transience—and then normalizing according to the overall activity of the population. These features correspond to the age-old ideas of “labeled lines” or “The Law of Specific Nerve Energies” (Müller, 1840) and the more modern concepts of normalization, adaptation, and gain control (e.g., Heeger, 1992; Chance et al., 1998). Second, the existence of a plausible method for extracting different features from the distributed code means that a given sensory feature such as visual motion needs to be represented only once in the brain. Different functions, such as perception and motor control, need only to choose the weights used to pool the distributed code to reconstruct different features of motion according to their individual needs.
Appendix
The basic structure of the model is defined by Equations EA1 andEA2, the structure of each element (Ej
) is defined in Equations EA3-EA7, and the terms of the “directional interaction” used to convert R(t) into MT(t) are defined in Equations EA8 and EA9:
Equation A1
Equation A2
MT is the output of the model for a given MT neuron,R is an intermediate output, Ej is the output of the jth element, τv is the visual time delay, and tis time.
When there has not been any null-direction target motion for a long time, MT(t) is equal to R(t). As a result, the three elements (E1,E2, and E3) contained all the parameters that had to be adjusted to fit the data that included only motion in the preferred direction. The components of these elements are defined in Equations EA3-EA7.
The latency of the visual input is computed dynamically for each time point in the simulation as:
Equation A3
The gain control within each element is defined by:
Equation A4
In equation A4, gnj
andgdj
are nonlinear gain elements,Mj
and Lj
are filters, and τdj is a time delay for the denominator pathway. All except the time delay are defined below. The low-pass filter Lj
(x) has a single time constant Tj
and was simulated by a numerical solution to the differential equation:
Equation A5
The low-pass filter Mj
(x) has different time constants for increases and decreases in input and was simulated by a numerical solution to the differential equation:
Equation A6
The nonlinearity in the numerator and denominator pathways of each element is:
Equation A7
The extra time delay in the directional interaction, caused by previous motion in the nonpreferred direction, is computed as:
Equation A8
The gain enhancement in the denominator of Equation EA1 is defined by setting Gdi
equal to the lesser of 0.9 and the quantity computed in Equation EA8:
Equation A9
The full model, including directional interaction and the three elements, has 51 adjustable parameters, of which 20 are used to adjust the parameters of the five nonlinearities. There are 3 parameters in the velocity-dependent time delay (Eq. 3), 12 parameters in each gain control element, and 12 parameters in the directional interaction. Although this is a large number of parameters, it is small compared with the ∼5000 time points of image velocity and MT neuron firing rate that were used as the basis for adjusting the parameters.
Simulations were conducted on a DEC Alpha computer with a modification of A Simulation Program (ASP), first written by L. Optican and H. Goldstein. The model was conceived as a series of differential equations, time delays, and nonlinear gain elements. The equations for these elements were then described in the BOMOL language and converted into C code by the ASP software. The C program was compiled and linked into a simulation shell that could operate in either manual mode under operator control or automatic mode under the control of a gradient descent optimization algorithm [STEPIT (Chandler, 1965)]. In manual mode, the operator was able to adjust the parameters one at a time to try to get the model started in the correct direction. In automatic mode, we selected a subset of parameters for adjustment, specified the order of priority for adjusting them, indicated upper and lower bounds on parameter values, and provided a set of target velocities and neuronal firing rates as a desired output for the optimization algorithm. Error was defined as the sum over all time points of the mean squared difference between the output of the model and the firing rate of the neuron being fitted. We will be happy to make our simulation software available to anyone who wishes it.
Many iterations of manual and automatic adjustment were often needed to get the model to fit the neuronal responses well. Sometimes it was possible to simply feed the model the full set of data described in this paper. More frequently, however, it was necessary to add specific classes of target motions one at a time, allowing the optimization algorithm to adjust only the parameters that were intended to fit those target motions. On the one hand, we doubt that we have demonstrated either the optimal parameter set or the optimal model architecture for fitting the responses of MT cells to the set of target motions we used in our experiments. Thus, it would almost certainly be possible to reduce the computed error by further simulations. However, we were satisfied with the performance of our model because it succeeded in capturing all the basic response properties we had measured in MT neurons and because it could simulate closely a heterogeneous set of temporal responses merely by changing parameters.
Footnotes
This work was supported by the Howard Hughes Medical Institute, the McDonnell-Pew Program in Cognitive Sciences, National Institutes of Health Grants EY02017 and EY03878, and ONR contract N00014-94-1-0269. We are grateful to Maninder Kahlon, Henry Mahncke, Larry O’Keefe, and Nicholas Port for their assistance with the recording experiments. Suzanne Fenstemaker and Christine Brown did the histological processing and track reconstructions. We are grateful to Wyeth Bair and Jennifer Groh for their comments on an earlier version of this manuscript.
Correspondence should be addressed to Dr. Stephen G. Lisberger, Department of Physiology, Box 0444, University of California, San Francisco, 513 Parnassus Avenue, Room HSE-802, San Francisco, CA 94143.
