 |
||||
|
||||
|
||||
|
||||
The Journal of Neuroscience, July 2, 2003, 23(13):5650-5661
Previous Article | Next Article
The Neural Representation of Speed in Macaque Area MT/V5
Nicholas J. Priebe, Carlos R. Cassanello, andStephen G. Lisberger Howard Hughes Medical Institute, Department of Physiology, W. M. Keck Foundation Center for Integrative Neuroscience, the Neuroscience Graduate Program, and the Sloan-Swartz Center for Theoretical Neurobiology, University of California, San Francisco, California 94143
Abstract
Top
Abstract
Introduction
Tuning for speed is one key feature of motion-selective neurons in the middle temporal visual area of the macaque cortex (MT, or V5). The present paper asks whether speed is coded in a way that is invariant to the shape of the moving stimulus, and if so, how. When tested with single sine-wave gratings of different spatial and temporal frequencies, MT neurons show a continuum in the degree to which preferred speed depends on spatial frequency. There is some dependence in 75% of MT neurons, and the other 25% maintain speed tuning despite changes in spatial frequency. When tested with stimuli constructed by adding two superimposed sine-wave gratings, the preferred speed of MT neurons becomes less dependent on spatial frequency. Analysis of these responses reveals a speed-tuning nonlinearity that selectively enhances the responses of the neuron when multiple spatial frequencies are present and moving at the same speed. Consistent with the presence of the nonlinearity, MT neurons show speed tuning that is close to form-invariant when the moving stimuli comprise square-wave gratings, which contain multiple spatial frequencies moving at the same speed. We conclude that the neural circuitry in and before MT makes no explicit attempt to render MT neurons speed-tuned for sine-wave gratings, which do not occur in natural scenes. Instead, MT neurons derive form-invariant speed tuning in a way that takes advantage of the multiple spatial frequencies that comprise moving objects in natural scenes.
Key words: direction tuning; speed tuning; MT; visual cortex; visual motion processing; spatial frequency
Introduction
Many of our behaviors depend on accurate information about changes in our environment. To make appropriate responses for a moving object, like catching a baseball in a glove, we need accurate information not only about the shape of the object, but also about its motion. Ideally, the shape of the object should not interfere with the estimation of its motion, and its location or motion should not cause errors in identifying the object. We have studied the influence of object form on visual motion processing in the extrastriate middle temporal visual area (MT).In the visual system, it has been traditional to think of motion by characterizing the visual scene according to its spatial and temporal sine-wave components in Fourier space. For example, sine-wave gratings are characterized by "spatial frequency," defined in cycles per degree as the inverse of the width of a single cycle of the grating, and "temporal frequency," defined in cycles per second as the inverse of the time required for the intensity of a single pixel to go through a full cycle of sinusoidal modulation. The speed of a moving grating is the ratio of the temporal frequency and the spatial frequency. Although sine-wave gratings are commonly used in the laboratory setting to assess the response properties of neurons, moving real-world objects are more complex than sine-wave gratings and contain multiple spatial and temporal frequencies. In the present paper, we have tested whether motion processing by the brain is different for real objects containing a broad spectrum of spatial and temporal frequencies, rather than for the unnatural grating stimuli used in the laboratory.
In principle, motion-sensitive neurons could be truly tuned for speed, meaning that the tuning is independent of the form of the moving stimulus (Movshon, 1975
; Tolhurst and Movshon, 1975
In the present paper, we show that the neural processing of speed is both more complex and more interesting than implied by either of the alternatives outlined above. First, by correcting a flaw in the data analysis of Perrone and Thiele (2001
Materials and Methods
Physiological preparation. Extracellular single-unit microelectrode recordings were made in the MT of nine anesthetized, paralyzed monkeys (Macaca fascicularis). Anesthesia was induced with ketamine (5–15 mg/kg) and midazolam (0.7 mg/kg), and cannulae were inserted into the saphenous vein and the trachea. The animal's head was then fixed in a stereotaxic frame and the surgery was continued under an anesthetic regimen of isoflurane (2%) inhaled in oxygen. A small craniotomy was performed and the dura was reflected directly above the superior temporal sulcus (STS). The animal was maintained under anesthesia using an intravenous opiate, sufentanil citrate (8–24 µg · kg –1 · hr –1), for the duration of the experiment. To minimize drift in eye position, paralysis was maintained with an infusion of vecuronium bromide (Norcuron, 0.1 µg · kg –1 · hr –1; Oragnon, West Orange, NJ) and the animal was artificially ventilated with medical-grade air. The body temperature was kept at 37°C with a thermostatically controlled heating pad. The electrocardiogram, electroencephalogram, autonomic signs, and rectal temperature were monitored continuously to ensure the anesthetic and physiological state of the animal. The pupils were dilated using topical atropine and the corneas were protected with +2 diopters gas-permeable hard contact lenses. Supplementary lenses were selected by direct ophthalmoscopy to make the lens conjugate with the display. The locations of the foveae were recorded using a reversible ophthalmoscope.Tungsten-in-glass electrodes (Merrill and Ainsworth, 1972
All experiments followed protocols that had received prior approval by the Institutional Animal Care and Use Committee at University of California, San Francisco.
Stimulus presentation. After isolating a single unit in the MT, we mapped its receptive field on a tangent screen by hand. We recorded the spatial position of each receptive field and, for many of the neurons, the size of the minimum response field. All of the neurons reported in this paper had receptive field centers within 12° of the fovea. Visual stimuli were then presented on a video monitor (CCID 121; Barco, Poperinge, Belgium), and were generated by a video frame buffer (Cambridge Research, Kent, UK). The video system had a noninterlaced refresh rate of 100 Hz. The spatial resolution of the monitor was 1024 x 768 pixels and the screen subtended 33.6 cm horizontally and 25.2 cm vertically. Because the monitor was placed 65 cm from the monkey's eyes, there were at least 19 pixels per visual degree. The video monitor always had a mean luminance of 68 candelas per degree. We positioned a mirror so that stimuli presented on the video monitor fell within the receptive field of the isolated neuron.
Experiments consisted of a sequence of brief trials with an intertrial interval of
700 msec. All trials began with the appearance of a stationary stimulus surrounded by a gray background of the mean luminance. For all trials, the stimulus appeared and was stationary for 250 msec before starting to move. After the motion was completed, the stimulus remained visible for an additional 250 msec. For each neuron, we first assayed the preferred direction by recording the responses to a 1000 msec motion of a 32% contrast sine-wave grating in 16 directions. We then assessed the preferences of the neuron for spatial and temporal frequency by measuring the response to gratings moving in the preferred direction for all combinations of six spatial frequencies (0.125, 0.25, 0.5, 1, 2, and 4 cycles per visual degree) and nine temporal frequencies (0, 0.25, 0.5, 1, 2, 4, 8, 16, and 32 cycles per second). For 5 of 104 neurons, the response to the lowest spatial frequency was not measured. For many neurons, gratings were presented at two contrasts, 32 and 8%, in randomly interleaved trials. Next, we tested the responses of 48 neurons with stimuli that contained two spatially overlapping gratings. Dual-grating stimuli were created by temporally interleaving frames in which each of the gratings were displayed individually. Because the refresh rate of the monitor was 100 Hz, the refresh rate to display both gratings was 50 Hz. To allow fair comparison to the dual gratings, single sine-wave gratings were displayed with a temporal refresh rate of 100 Hz, but frames containing the grating were temporally inter-leaved with a blank (gray) stimulus of the same mean luminance.
For a subset of 71 neurons, we measured the speed tuning for random dots that moved within a window placed over the receptive field of the neuron (Priebe et al., 2002
Data acquisition and analysis. Experiments were controlled by a computer program running on a UNIX workstation and a Windows NT PC running the real-time extension RTX (VenturCom, Waltham, MA). The two computers were networked together: the UNIX workstation provided an interface for programming target motion and customizing it during recording from a neuron; the PC provided real-time control of target motion and data acquisition. The times of spikes were recorded by the PC and sent over the network to the UNIX workstation, which stored them for subsequent analysis, along with codes indicating the target motion that had been commanded. Each experiment consisted of a list of trials, in which each trial presented a different stimulus motion. Trials were sequenced by shuffling the list and presenting the trials in a random order until all the trials in the list had been presented. The list was then shuffled and repeated again until enough repetitions of each stimulus had been obtained. The mean number of repetitions for each trial condition was 15.3 ± 5.27 (mean ± SD).
Data were analyzed by aligning all the responses to identical trajectories of grating motion on the onset of motion and accumulating post-stimulus time histograms with a bin width of 1 msec. For presentation, the histograms were built with a bin width of 50 msec. Background responses were eliminated by creating a histogram for trials that presented a stationary stimulus, computing the mean firing rate during the interval when stimuli for other trials were moving, and subtracting this scalar from the firing rate in every bin of every other histogram. We then measured the firing rate from the background-corrected histograms to determine the stimulus selectivity of each neuron and to quantify the results of each experiment. Fits to the data were made by extracting the firing rate from each cell on a trial-by-trial basis. The extracted firing rates were then passed to a Matlab (MathWorks, Natick, MA) function ("nlinfit") that fit the data using the equations presented in the Results section of the paper. Confidence intervals for parameter estimates were computed from the Jacobian matrix and the residuals using the Matlab function "nlparci." Specific analyses are presented in the relevant places in the Results.
Results
Theoretical basis for the experiments
Most models of motion selectivity are based on a comparison of luminance or contrast signals across time and space. Although many models have been shown to extract the direction of motion accurately, the details of the filters in the models determine whether they signal the speed of motion independently of object shape or object contrast. For example, one of the most influential models of motion selectivity, the motion energy model originally developed independently by Adelson and Bergen (1985Because Figure 1, B and E, uses logarithmic axes, the contour lines of equal speed are parallel to one another (dashed lines).
(1)
View larger version (35K):
[in this window]
[in a new window]
Figure 1. Two models for the creation of motion-sensitive neural responses A, D, Response contours indicating the strength of responses as a function of the position of the stimulus and time. Red contours indicate regions of increasing response to dark stimuli, whereas blue contours indicate the response to light stimuli. The response fields in A and D are diagrams contrived to represent motion filters that would and would not have a dependence of speed tuning on the spatial frequency of sine-wave gratings. B, E, The contour lines show the receptive field of each model after transformation into Fourier space. The parallel, dashed lines indicate isospeed contours, with speed indicated by the number at the top right of each line. C, F, Each curve shows the speed tuning of the model at one spatial frequency; different colors indicate different spatial frequencies and the colors of the curves are coordinated with those of the arrows above B and E, The graphs inset at the top right of C and F plot preferred speed as a function of spatial frequency. C and F were derived exactly from the contour plots in B and E.
The Fourier transform of a nonoriented motion filter (Fig. 1B, contour lines) reveals independence along the spatial and temporal frequency axes: the temporal frequency that gives the best response is the same at each spatial frequency, and vice versa. As a consequence, speed tuning for sine-wave grating stimuli changes as a function of spatial frequency. As shown in Figure 1C, the best response occurs at a speed of 8°/sec when the stimulus has a low spatial frequency of 1/8 cycles per degree. As spatial frequency increases, the preferred speed decreases: when the spatial frequency is 4 cycles per degree, the best response occurs at 0.25°/sec. Thus, for each subunit of the motion–energy model, the preferred speed varies over a 32-fold range as a function of spatial frequency. In contrast, the Fourier transform of the oriented motion filter is tilted (Fig. 1E) so that its spatial and temporal frequency tunings are not independent: as the spatial frequency increases, the preferred temporal frequency also increases. In Fourier space, the long axis of the response field now lies on a contour of constant speed instead of crossing different speed contours for each spatial frequency. When the speed tuning is measured for each spatial frequency, shown in Figure 1F, the speed tuning curves overlap one another; thus, the preferred speed does not vary as the spatial frequency is changed.
In this paper, we will first ask whether the speed tuning of MT neurons resembles better the predictions of the model in the top or bottom row of Figure 1 by quantifying the relationship between preferred speed and spatial frequency of MT neurons: stimuli will consist of sine-wave gratings chosen to tile the parameter space of spatial and temporal frequency. We will next demonstrate that the responses to single gratings underestimate the true speed tuning of MT neurons: stimuli will consist of spatially overlapping pairs of sine-wave gratings, square-wave gratings, and textures.
Speed tuning of MT neurons for single sine-wave gratings
We recorded the responses of 118 MT neurons to gratings moving in the preferred direction; 104 neurons provided adequate data to allow a proper estimation of the response field in Fourier space. Each combination of spatial and temporal frequency yielded a single histogram, like those shown in Figure 2, which summarizes the responses of one MT neuron to all combinations of six spatial frequencies and eight temporal frequencies. Inspection of Figure 2 reveals that the largest responses occur for spatial frequencies of 0.5–1 cycles per degree and temporal frequencies of 8–32 cycles per second. However, inspection of the family of histograms does not reveal whether the response field is separable (Fig. 1B), oriented (Fig. 1E), or in between.
View larger version (28K):
[in this window]
[in a new window]
Figure 2. Responses of a representative MT neuron to moving sine-wave gratings with different combinations of temporal and spatial frequency. Each histogram shows the average firing rate in 50 msec bins for multiple repetitions of a single stimulus. The grating was visible throughout each histogram, and the bold horizontal lines below the bottom row of histograms indicate the time when the grating was moving.
We created response fields from the mean response to each combination of spatial and temporal frequency for each neuron like those shown for three representative neurons in Figure 3A1–C1. In these graphs, the amplitude of each steady-state response of the neurons to the combinations of spatial and temporal frequency is indicated by the diameter of the symbol. For the neuron in Figure 3A1, the response field is similar to the model filter shown in the top row of Figure 1: the spatial and temporal frequency profiles appear to be approximately independent. The same data have been plotted differently in Figure 3A2, now showing response as a function of stimulus speed separately for each spatial frequency. As expected for a neuron with a nonoriented response field, the preferred speed of the neuron changed as a function of spatial frequency: the preferred speed varied from 40 to 1.5°/sec, as the spatial frequency of the moving stimulus was changed from 0.125 to 4 cycles per degree. The data in Figure 3B, from the same neuron shown in Figure 2, show a somewhat tilted spatiotemporal response field (Fig. 3B1), and the plots of firing rate as a function of speed (Fig. 3B2) reveal that the preferred speed of the neuron decreased from 36 to 8°/sec as the spatial frequency of the moving stimulus increased from 0.125 to 1 cycles per degree. Finally, the neuron in Figure 3C shows an oriented spatiotemporal response field (Fig. 3C1) and no relationship between preferred speed and spatial frequency (Fig. 3C2). The speed tuning curves in Figure 3C2 show different amplitudes at different spatial frequencies, but peak at the same speed for each spatial frequency.
View larger version (30K):
[in this window]
[in a new window]
Figure 3. Effect of spatial frequency on the preferred speed of three MT neurons chosen to indicate the diversity of effects. A1–C1, Response fields in the coordinate system of temporal and spatial frequency. Each symbol indicates the response to one combination of spatial and temporal frequency, and the diameter of the symbol gives the size of the response. A2–C2, Symbols plot the response of each neuron as a function of speed; the different colors indicate gratings of different spatial frequencies. Curves show the result of fitting the data with Equations 2 and 3. As before, the colors are coordinated for the symbols and curves in the bottom row of graphs and the arrows above the top row of graphs. Error bars indicate SEM of the firing rate.
To quantify the dependence of speed tuning on spatial frequency, we used a variant of a two-dimensional Gaussian on logarithmic axes to fit spatiotemporal response fields like that illustrated in Figure 3, A1 to C1:
where tfp depends on spatial frequency and is defined as:
(2) Fits and statistical significance were derived from the responses to the full set of individual presentations of each stimulus, whereas the variance accounted for by the fits was based on how well they predicted the mean responses for each stimulus. Equations 2 and 3 have the advantage that they use a single equation to fit all the data, quantifying the slope of the relationship between preferred speed and spatial frequency using a single parameter, the exponent Q. When Q is 0, there is no relationship between spatial frequency and the preference of a neuron for speed, indicating that the neuron has speed tuning that is independent of spatial frequency (Fig. 1F, inset). When Q is –1, there is a strong dependence of the preferred speed on the spatial frequency: as the spatial frequency is increased by a log unit, the preferred speed of the neuron is decreased by a log unit (Fig. 1C, inset). A Q value of –1 indicates that the spatial and temporal frequency tunings of the neuron are independent. The Q value assumes that the interaction between spatial frequency and preferred speed is linear in logarithmic space, following a power law in linear frequency space. For the example neurons shown in Figure 3A–C, Q was –0.95, –0.55, and –0.05, indicating a strong, medium, and weak dependence of preferred speed on spatial frequency.
(3) The distribution of the parameter Q calculated for our population of 104 MT neurons is unimodal and peaks near the mean Q value of –0.52 (Fig. 4). To compare with other studies, we classified the neurons according to whether the 95% confidence intervals of Q overlapped 0 or –1: if they overlapped –1, then we classified the neuron as "spatiotemporally independent" (26 of 104) (Fig. 4, black bars); if the confidence intervals overlapped 0, we classified the neuron as speed tuned (25 of 104) (Fig. 4, white bars); if Q was between –1 and 0 but the confidence intervals overlapped neither, we called the neuron unclassed (49 of 104) (Fig. 4, gray bars), although it had features of both speed tuning and spatiotemporal independence. A few neurons (4 of 104) had Q values >0 and confidence intervals that did not overlap 0, indicating that their speed tuning shifted with spatial frequency, but in the opposite direction predicted by a spatiotemporal-frequency-independent model. For the remainder of the paper, we have considered these neurons as part of the speed-tuned group. The model defined by Equations 2 and 3 provided excellent fits to the spatial and temporal frequency tuning of MT neurons, accounting for the majority of the variance in their mean responses (94.8 ± 3.6%; mean ± SD).
View larger version (20K):
[in this window]
[in a new window]
Figure 4. Summary of the effect of spatial frequency on preferred speed across the population of MT neurons. The histogram plots the distribution of the value of Q (Eq. 2) for all 104 neurons in our sample. A Q value of –1 indicates spatial and temporal frequency independence. A Q value of 0 indicates no relationship between spatial frequency and preference for speed. The dark bars indicate neurons whose 95% confidence intervals for Q overlapped with –1. The white bars indicate neurons whose 95% confidence intervals for Q overlapped with 0. Gray bars indicate the neurons whose confidence intervals lie between –1 and 0, whereas the light gray bars indicate neurons whose Q values and confidence intervals were >0. The values above the corresponding portions of the histogram indicate the number of cells falling into each classification.
As additional independent tests of speed tuning we used two alternative analysis methods. First, we refitted the relationships between firing rate and speed (Fig. 3A2–C2) separately for each spatial frequency and then performed regression analysis for the log of the preferred speed as a function of the log of the spatial frequency. This analysis confirmed the validity of the assumption of a linear relationship between the logarithms of the spatial frequency and the preferred speed. It yielded values of Q that were nearly the same in individual neurons (r = 0.91) but were slightly closer to –1 (less speed-tuned) than for the fits based on Equations 2 and 3. Because fitting the data for each spatial frequency separately used more parameters, we did obtain the expected slight improvement in how well the equations fitted the data.
Second, we used a variant of a method devised by (Levitt et al., 1994
(4) where Rindep and Rspeed are the partial correlations of the response field with the independent and speed-tuned predictions, ri is the correlation of the data with the independent prediction, rs is the correlation of the data with the speed-tuned prediction, and ris is the correlation of the two predictions. We then plotted Rspeed as a function of Rindep (Fig. 5D) and divided the population of neurons according to whether they were speed-tuned, spatiotemporally independent, or unclassed. The results in Figure 5D are similar to those shown by the distribution of values of Q in Figure 4B: the population included 28 speed-tuned neurons, 25 spatiotemporally independent neurons, and 51 unclassed neurons, Finally, there was a strong correlation between the classifications based on using the correlation analysis and the Q analysis (r = 0.91).
(5)
View larger version (20K):
[in this window]
[in a new window]
Figure 5. A correlation-based analysis to classify the speed tuning of MT neurons (after Levitt et al., 1994
Our conclusions about the speed tuning of MT neurons disagree with those in a recent paper of Perrone and Thiele (2001
A nonlinearity that improves speed tuning of MT neurons for pairs of sine-wave gratings
Cognizant that objects comprise multiple spatial frequencies, we next asked whether speed tuning became form-invariant for stimuli that contained multiple spatial frequencies. We tested MT neurons with dual-grating stimuli consisting of two superimposed gratings of the same orientation. We paired gratings of the same speed from either the higher or lower pairs of four spatial frequencies (Fig. 6A, connected by black or gray lines). Predicting the responses to the two gratings in each pair by applying the linear model (adding the responses to each grating presented alone) implied that we should find distinctly different preferred speeds for the different ranges of spatial frequency, as shown by the open symbols and dashed curves in Figure 6B. However, recording the responses to dual gratings yielded speed-tuning curves with similar peaks for the different ranges of spatial frequency, as shown by the filled symbols and solid curves in Figure 6B (top).
View larger version (33K):
[in this window]
[in a new window]
Figure 6. Demonstration of a nonlinear mechanism that creates better speed tuning for dual gratings than for single gratings. A, Response field of an example MT neuron to single gratings of varying spatial and temporal frequency, as in Figure 3. The black and gray arrows indicate the spatial frequencies used for the two groups of dual grating stimuli. Responses connected by black or gray lines indicate dual-grating stimuli in which each grating provided motion at the same speed. B, Top, The open symbols and dashed curves plot the speed tuning predicted by the linear sum of the responses to individual gratings, for the spatial frequency pairs of stimuli in A comprising the lower (black) or higher (gray) frequencies. The filled symbols and continuous curves plot the actual speed tuning for dual gratings. Arrows plotted along the curves indicate the preferred speeds for predicted responses and the actual responses. B, Bottom, black and gray symbols plot the normalized gain of the response to dual gratings for the lower and higher ranges of spatial frequency pairs. The black arrow shows the mean of the preferred speeds for the actual responses over the two ranges of spatial frequencies. Gain was not computed for a given dual-grating stimulus unless the predicted response, in the denominator, was at least 10% of the maximum response. C, The population average of normalized response gain is plotted as a function of speed, in which the data of each neuron has been normalized so that preferred speed was 1. Black and gray curves show responses for low and high ranges of spatial frequencies. Error bars indicate SEM. D, Comparison of the actual and predicted difference in preferred speed between the dual-grating stimuli comprised of different pairs of spatial frequencies. Each symbol shows data for a single MT neuron. E, Comparison of the actual and predicted speed tuning for the dual grating stimuli. Each symbol indicates the tuning width for a single MT neuron. As detailed in Results, predicted responses were obtained by summing the responses to each component grating presented singly.
To evaluate this effect in the population of MT neurons, we fitted the predictions and data with Gaussian functions and used the midpoint and
2 from the fits to estimate the preferred speed and the tuning width for each function. We then computed the absolute difference in preferred speed between the higher and lower pairs of spatial frequencies for both the linear predictions and the data, and plotted the data from each neuron as a point in Figure 6D. All but two of the points fell below the line of slope 1, indicating that the difference in preferred speed for dual-grating stimuli over the two ranges of spatial frequencies was consistently smaller in the data than predicted by the linear model (means of 0.73 and 1.35 octaves, respectively). Figure 6E shows that the tuning width for the dual-grating stimuli was narrower in the data than predicted, for almost every neuron. Population averages were: linear prediction, 1.69 ± 0.60; data, 1.28 ± 0.53 (mean ± SD). This experiment indicates the presence of nonlinear mechanisms based on multiple spatial frequencies and suggests that the nonlinearities could create form-invariant speed tuning for real object motion.
To characterize the nonlinear mechanism, we calculated "gain," defined as the actual response to each dual-grating stimulus divided by the prediction from summing the responses to the two gratings singly. Figure 6B, bottom, plots the gain separately for the high and low ranges of spatial frequencies: gain was highest near the preferred speed for this neuron (upward black arrow). To summarize the nonlinearity across our sample of MT neurons, we normalized the speed axis in the gain plots for each neuron so that preferred speed had a value of 1. We then averaged the curves that plotted gain as a function of normalized speed for all 30 neurons tested with dual-grating stimuli. The average gain (Fig. 6C) was largest near the preferred speed of the neurons and declined as speed moved away from preferred. Furthermore, the average gain curves were similar for the higher and lower ranges of spatial frequencies (Fig. 6C, gray vs black curves).
To further investigate the source of the nonlinear interaction that yielded more form-invariant speed-tuning curves, we presented stimuli composed of two overlapping gratings whose spatial and temporal frequencies were adjusted independently according to the design illustrated in Figure 7, A and B. For each MT neuron presented with the dual-grating stimulus, we first determined its preferred spatial and temporal frequencies. We then chose pairs of spatial and temporal frequencies surrounding its preference, yielding four sine-wave gratings. The way these gratings were selected and combined is indicated by the position of the four histograms for single gratings in Figure 7A. Each histogram is placed as if it were in a plot of spatial versus temporal frequency, so that the two histograms outlined in gray had the same speed and, when combined into a dual-grating stimulus produced the "same-speed" response that is also outlined in gray. For this neuron, the actual response to the same-speed dual grating was twice as large as that predicted by adding the mean responses to the two gratings presented singly (horizontal dashed line). The two histograms outlined in black represent responses to two gratings that had the same spatial and frequency components as the same-speed gratings, but now in different combinations so that the two gratings moved at different speeds. For this neuron, the response to the two different-speed gratings (outlined in black) was approximately the same amplitude as that predicted by adding the mean responses to the two gratings presented singly (horizontal dashed line). The example neuron illustrated in Figure 7B is more typical of the responses we found. The response to the same-speed dual grating is slightly smaller than predicted by linear summation, whereas the response to the different-speed dual-grating stimulus is half as large as predicted by the linear summation of the mean responses to the two gratings presented singly.
View larger version (23K):
[in this window]
[in a new window]
Figure 7. Additional analysis of the nonlinear response to dual grating stimuli. A, B, Poststimulus time histograms showing the responses of two example neurons to both single gratings and dual gratings. In each panel, the four responses grouped in a square were obtained from single gratings with all combinations of two spatial and temporal frequencies. The numbers in each box indicate the speed of the stimulus. The two histograms at the top and bottom right of each panel show the responses to dual gratings composed of gratings moving at the same speed (gray boxes) or different speeds (black boxes). The dashed line shows the response level predicted by adding the mean firing to the two component grates presented alone. C, Group data for all 48 neurons whose responses were measured to dual gratings. Each symbol summarizes data from one neuron and shows the average gain of the response to same-speed dual gratings as a function of that for different-speed dual gratings. Gain is defined as the actual response divided by the linear prediction of the response. The filling of each circle indicates the classification of the neuron according to the effect of spatial frequency on preferred speed for single grating: black, independent; gray, unclassed; open, speed-tuned.
We again quantified the response to the dual gratings by computing the gain, defined as the actual response divided by that predicted by summing the responses to the two component gratings singly. For the population, the gain for the same-speed dual grating was almost always greater than the gain for the different-speed dual-grating stimulus. When the gain along the same-speed axis was plotted as a function of that along the different-speed axis (Fig. 7C), 75% of the recorded neurons (36 of 48) showed higher gains along the same-speed axis, indicating the presence of a nonlinearity that favored the same-speed gratings. The rest of the neurons were grouped around the line of slope 1, indicating that the nonlinearity did not favor same-speed gratings, although the gains were almost always <1.0, indicating the presence of a nonlinearity. The gains averaged 0.79 and 0.56 along the same-speed and different-speed axes. For same-speed dual gratings there was a slight difference in the change in the gain of response that depended on the degree that speed tuning was affected by spatial frequency. Using the broad classifications described above, neurons described as spatiotemporal-frequency-independent (Fig. 7C, black circles), unclassed (gray circles), or speed-tuned (open circles) had average gains of 0.86, 0.82, and 0.65, respectively. For different-speed dual gratings, gain averaged 0.53, 0.58, and 0.53 for the three classes of MT neurons.
The movement of the two overlapping gratings contained both a first-order component of motion, defined by luminance changes in time, and a second-order component, defined by contrast changes in time (Cavanagh and Mather, 1989
Additional evidence for a nonlinearity that improves speed tuning of MT neurons
If the presence of multiple spatial frequencies in the stimulus acts through the kind of nonlinearity described in the previous section to improve speed tuning, then speed tuning should vary in a number of consistent ways depending on the exact form of the stimulus. The presence of multiple spatial frequencies should render speed tuning both less dependent on form and more narrowly tuned than predicted by the summation of the responses to the component gratings singly. Neurons might be closer to speed-tuned for high-contrast than for low-contrast gratings if the former cause some saturation earlier in the visual pathways. Neurons should show better speed tuning for square-wave than sine-wave gratings, because the former comprise multiple spatial frequencies. The speed tuning for moving random-dot textures should be narrower than predicted from the responses to sine-wave gratings. In the present section, we test each of these predictions.Effect of contrast on spatiotemporal frequency response fields
Lowering the contrast of the moving gratings moved the responses in MT substantially toward spatiotemporal independence and away from speed tuning, without altering the preferred spatial or temporal frequency of individual neurons. The effect of lowering contrast was measured in 61 MT neurons by using interleaved trials to present moving gratings composed of the high contrast used in the previous section (32%) (Fig. 8A1,B1) and a lower contrast (8%) (Fig. 8A2,B2). For the two neurons summarized in Figure 8, A and B, the spatiotemporal response field was visibly less oriented when the grating contrast was lower. Quantitative analysis showed that both neurons became less speed-tuned: reducing the contrast of the grating changed the value of Q from –0.46 to –0.86 in one neuron (Fig. 8A1,B1) and from –0.05 to –0.54 in the other (Fig. 8A2,B2). The population summary in Figure 8C shows that lowering the contrast of the stimulus caused the Q values of many MT neurons to shift toward spatiotemporal independence (i.e., toward –1), leading to a shift in the mean Q for this subset of our population of MT neurons from –0.45 to –0.79.
View larger version (25K):
[in this window]
[in a new window]
Figure 8. Effect of spatial frequency on preferred speed of MT neurons for high-versus low-contrast sine-wave gratings. A1, B1, Spatiotemporal frequency response fields of two example MT neurons for high-contrast gratings (32%), in the same format as Figure 3. A2, B2, Spatiotemporal frequency responses fields of the same two neurons for low-contrast gratings (8%). For each contrast, the diameter of the symbols was normalized relative to the peak response at that contrast. The numbers in the bottom right of each graph give the value of Q obtained by fitting Equations 2 and 3 to the data in that graph. C, Comparison of the values of Q for single sine-wave gratings of high versus low contrast. In the graph, each symbol summarizes the responses of an individual neuron and the dashed line indicates a line of slope 1 and y-intercept of 0. The histograms above and to the right of the scatter plot show the distributions of Q for single sine-wave gratings of 32% and 8%, respectively. Neurons were included in the histograms only if they were tested with gratings of both contrasts. The arrows in both histograms indicate the mean values across our sample of MT neurons.
Responses to square-wave gratings
When the stimulus consisted of a square-wave grating instead of a sine-wave grating, the responses of MT neurons were essentially speed-tuned. Comparison of the spatiotemporal response fields for two example neurons illustrates responses that are much more nicely oriented for square-wave gratings (Fig. 9A2,B2) than for sine-wave gratings (Fig. 9A1,B1). For the two sample neurons, the values of Q obtained by applying Equations 2 and 3 were –0.88 and –0.58 for sine-wave gratings and 0.02 and –0.20 for square-wave gratings. The same effect appeared in almost all 20 MT neurons we tested with sine-wave and square-wave gratings. As summarized in Figure 9C, the values of Q for square-wave gratings grouped around 0 and were less than the values of Q for sine-wave gratings in all but two neurons.
View larger version (29K):
[in this window]
[in a new window]
Figure 9. Comparison of the effect of spatial frequency on preferred speed for sine-wave versus square-wave gratings. A1, B1, Spatiotemporal frequency response fields of two example MT neurons for high-contrast sine-wave gratings (32%), in the same format as Figure 3. A2, B2, Spatiotemporal frequency responses fields of the same two neurons for square-wave gratings. For each class of stimulus, the diameter of the symbols was normalized relative to the peak response for that stimulus. The numbers in the bottom right of each graph give the value of Q obtained by fitting Equations 2 and 3 to the data in that graph. C, Comparison of the values of Q for sine-wave versus square-wave gratings. D, Comparison of the values of Q fitted to the responses to square-wave gratings with those predicted by a linear sum of the responses to sine-wave gratings representing the frequency components in the square wave: the fundamental frequency (F) plus 3F, 5F, 7F, 9F, and 11F. In C and D, each symbol summarizes the responses of an individual MT neuron and the dashed line has a slope of 1 and an intercept of 0.
The use of the spatiotemporal response fields for square-wave gratings is formally incorrect because square-wave gratings are composed of multiple sine-wave gratings including the fundamental spatial frequency (F) and higher harmonics (3F, 5F, 7F, etc). The spatial and temporal frequency content of a moving square-wave grating is a series of points aligned along an isospeed contour in the Fourier space. Because the stimulus itself is oriented in plots of temporal versus spatial frequency, the responses to square-wave gratings also should appear to be more oriented, when plotted according to the fundamental frequency, as shown in Figure 9. Thus, the critical question is not whether MT neurons appear more speed-tuned for square-wave gratings, as they do, but rather whether the shift toward speed tuning can be accounted for using a linear summation of the responses to the fundamental and harmonic components of the stimulus, after adjusting for the contrast change attributable to the amplitude reduction. For each neuron tested, we predicted the value of Q for square-wave gratings responses using the parameters of Equations 2 and 3 fitted to the responses to sine-wave gratings at 32% contrast. On average, the linear prediction did not predict the full extent of the shift toward speed tuning for square-wave gratings: across the 20 neurons the average value of Q changed from –0.53 to –0.08 when the stimulus was changed from sine- to square-wave gratings, whereas the linear model predicts a more modest change to –0.30 (Fig. 9D).
The relationship between spatial and temporal frequency tuning speed tuning to random dots
For a subset of our sample of neurons (71 of 104), we were able to compare the speed tuning for a random-dot stimulus to the spatiotemporal frequency response field. The random-dot stimulus contains a broad spatial frequency spectrum; our strategy, again, was to compare the speed tuning of MT neurons for random-dot stimuli with predictions based on the responses of each neuron to single sine-wave gratings of different spatial and temporal frequencies. To predict the response to random dots, we averaged the actual responses to sine-wave gratings along contours of equal speed, as indicated by the dashed lines in Figure 10A. In the example of Figure 10B, the preferred speed of the predicted speed tuning curve (black symbols and curve) is similar to that for the actual response to the random dots (gray symbols and curve). However, the width of the speed tuning predicted by the sine-wave gratings is greater than that for the actual responses to random-dot motion. These two features were representative of our sample population. The actual and predicted preferred speeds for moving random dots were highly correlated (r = 0.86) (Fig. 10C). However, in most cells the tuning was narrower for the dot stimuli than predicted by applying the linear model to the responses to sine-wave gratings (Fig. 10D). Across the population,
View larger version (34K):
[in this window]
[in a new window]
Figure 10. Comparison of MT neuron responses to sine-wave gratings versus random-dot textures. A, A diagram of the method used to predict speed tuning for textures from response to gratings. For each speed of texture motion, the response of the neuron was averaged for sine-wave gratings that fell along the isospeed contours indicated by the dashed lines. B, Black versus gray symbols and curves show predicted and actual responses and turning curves from an example MT neuron as a function of texture speed. In C–F, each symbol shows measurements made from an individual neuron. C, D, Comparison of predicted and actual preferred speed (C) and tuning width (D) for the population of neurons. The dashed line shows the expected relationship if the predicted and actual tuning matched. The values plotted in these graphs were taken from the Gaussian curves fitted to the data for each neuron. E, F, Comparison of the preferred speed to random-dot textures with the preferred spatial frequency (E) or temporal frequency (F) for sine-wave gratings. Preferred spatial and temporal frequencies were obtained from the parameters that gave the best fits of Equations 2 and 3 to the data.
For each neuron in our sample population we also used the fits of Equations 2 and 3 to estimate the spatial and temporal frequency combination that would elicit the best response. For high-contrast gratings (32%), the peak spatial frequency varied from 0.125 to 4 cycles per degree (mean, 0.55 cycles per degree; SD, 1.1 octaves). The preferred temporal frequency of MT neurons ranged from 0.75 to 25 Hz (mean, 3.94; SD, 1.02 octaves). Reducing the contrast of the moving gratings from 32 to 8% caused the preferred spatial and temporal frequencies to decrease by an average of 0.05 log unit. Comparison of the preferred parameters for the two contrasts in each individual yielded correlation coefficients of 0.82 and 0.92 for spatial and temporal frequency. There was no statistically significant effect of neuron categorization (spatiotemporally independent, unclassed, or speed tuned) on the preferred spatial or temporal frequency. Although it did not change the tuning for spatial or temporal frequency, reducing the contrast of the gratings from 32 to 8% did reduce the peak firing of MT neurons by an average of 42%.
Figure 10, E and F, compares the preferred spatial and temporal frequencies of MT neurons with their preferred speed to dot motion. The correlation between preferred spatial frequency and preferred speed was very high (rsf =–0.81), whereas the relationship was less strong between preferred temporal frequency and preferred speed (rtf = 0.53). Correcting for the small correlation between preferred spatial and temporal frequency (r = –0.25) changed the estimated correlations between preferred spatiotemporal frequency tuning and preferred speed only slightly (rsf = –0.82; rtf = 0.54). Thus, it appears that spatial frequency tuning dominates temporal frequency tuning in determining the speed tuning of MT neurons. The dominance of spatial frequency tuning in determining preference for speed is consistent with the psychophysical finding that there are more channels for spatial frequency than for temporal frequency (Watson and Robson, 1981
The response of MT neurons to moving plaids
Previous reports (Movshon et al., 1986
View larger version (18K):
[in this window]
[in a new window]
Figure 11. Comparison of the responses of MT neurons to moving plaids with the degree of form-invariant speed tuning. A, Direction tuning of two example MT neurons for moving plaids and single sine-wave gratings. A, Top, Component-selective neuron; bottom, pattern-selective neuron. Black squares show the response to single sine-wave gratings, whereas the gray circles indicate the response to the plaid stimulus. Error bars indicate the SEM response for each direction of plaid movement. The plaid components were separated by 135°. B, The distribution of pattern versus component selective neurons, based on the correlation assay defined by Movshon et al. (1986
We used the correlation analysis developed by Movshon et al. (1986
(6) where Rc and Rp are the partial correlations of the direction tuning curves for plaids with the component and pattern predictions, rc is the correlation between the response to plaids and the model prediction for a component neuron, rp is the correlation to the modeled pattern neuron response, and rcp is the correlation of the two predictions. Rp was then plotted as a function of Rc (Fig. 11B) for each of the 61 neurons we studied with plaids, and the neurons were classified as component, unclassed, or pattern according to the criteria indicated by the solid curves. Consistent with previous reports,
(7) Figure 11C evaluates our sample of MT neurons to see whether there is any correlation between the classification of individual neurons along the separate axes of speed-tuned/spatiotemporally separable (y-axis) and pattern/component responses to plaids (x-axis). For each entry in the table, the diameter of the symbol indicates the number of neurons that fell in that class. The largest group was unclassed along both dimensions, but there was neither a visible nor a statistically significant correlation among neurons that were classed along one or both axes (Spearman rank correlation; r = 0.067; p > 0.5).
Discussion
Are MT neurons tuned for the speed of sine-wave gratings?
Current theories about the creation of motion-sensitive neurons revolve around two extremes that make different predictions about how motion is represented in the brain. At one extreme, motion-sensitive neurons are tuned independently for spatial and temporal frequency. Speed selectivity then becomes a consequence of the fact that they show their largest response for a specific pair of spatial and temporal frequencies, defining preferred speed as preferred temporal frequency divided by preferred spatial frequency. In this model, speed tuning would depend significantly on the spatial frequency of the stimulus. At the other extreme, neurons are tuned for speed and therefore show some covariance in their spatial and temporal frequency tuning, eliminating or minimizing the dependence of speed tuning on spatial frequency.As we illustrated in Figure 1, one way to diagnose whether the response of a neuron shows spatiotemporal independence or speed tuning is to measure the responses to single sine-wave gratings and ask whether the response field is tilted in Fourier space. In a study similar to ours, Perrone and Thiele (2001
How do MT neurons become tuned for the speed of real-world objects?
Sine-wave gratings provide a tool for the analysis of neural responses, but have the drawback that they do not occur frequently in natural visual scenes (Field, 1987We think that the speed-tuning nonlinearity is more than linear summation followed by the half-wave rectification created by a threshold, because the responses to a given set of stimuli are both more reliably and more narrowly speed-tuned than predicted by a linear summation based on the response of the neuron to single sine-wave gratings. Dual-grating stimuli not only altered the amplitude of the response relative to that predicted by linear summation, but it also caused the speed tuning to shift toward the single value that was revealed by testing with random-dot textures. These phenomena would result from the kind of excitatory interactions proposed in models by Simoncelli and Heeger (1998
MT neurons seem to derive form-invariant speed tuning in a way that takes advantage of the fact that moving objects in natural scenes comprise multiple spatial frequencies. The nonlinearity revealed by our dual-grating experiments would make the response field appear to be oblique (Fig. 1E) when multiple spatial frequencies are present, even if it was cardinal for single sine-wave gratings (Fig. 1B). It does this by allowing strong responses when stimuli fall along the preferred speed line in spatiotemporal frequency plots, while suppressing responses along speed lines above or below the preferred speed. It enables the desired result (a form-invariant assessment of speed) without creating receptive fields that are tuned for the speed of artificial stimuli such as sine-wave gratings.
Implications for models of speed tuning in MT
Our data do not support strongly any particular model of motion-selective neurons. Rather, they raise a number of important, related issues about future attempts to model motion-selective neurons, especially those in area MT. First, new models of MT neurons should have the goal of creating a population of MT responses that reflect the diversity of speed tuning rather than the two extremes. In contrast, previous models of motion-sensitive neurons have focused on creating responses at the two extremes. An important issue that will need to be addressed is whether direction-selective neurons in the primary visual cortex show the same diversity, or if they lie closer to spatiotemporal independence as suggested by Tolhurst and Movshon (1975Relationship between the coding of speed in MT and motion psychophysics
We turn now to the problem of decoding the population response in area MT to reconstruct speed for use in generating perceptions and actions. Although neurons in area MT may not encode the speed of motion independently of spatial frequency, they may still contribute to our sensation of motion. In fact, psychophysical data indicate that the spatial frequency of a grating does affect the perception of speed: low spatial frequencies bias human observers to perceive faster speeds (Campbell and Maffei, 1981Finally, one might ask why it is important to have a speed-tuning nonlinearity when it is possible to obtain a reasonable estimate of speed by simply adding the responses to the component gratings of the visual stimulus without using the nonlinearity. We imagine two reasons. First, form-dependent speed tuning could cause serious misjudgments of object speed, especially if a small object is moving toward the observer, becoming larger and changing spatial frequency content as it looms. Second, as indicated by our data, one important function of the speed-tuning nonlinearity is to narrow the speed tuning of MT neurons. Narrower tuning of individual neurons means that a smaller population of neurons is activated by any given stimulus. Thus, the nonlinearity contributes to the creation of a sparse code, thereby increasing
