Abstract
How the visual system reconstructs global patterns of motion from components is an important issue in vision. Conventional studies using plaids have shown that approximately one-third of neurons in cortical area MT respond to one-dimensional (1D) components of a moving pattern (component cells), whereas another third responds to the global two-dimensional (2D) motion of a pattern (pattern cells). Conversely, studies using spots of light or random dots that contain multiple orientations have seldom reported directional tuning that is consistent with 1D motion preference. To bridge the gap between these studies, we recorded from isolated neurons in macaque area MT and measured tuning for velocity (direction and speed) using random dot stimuli. We used the “intersection of constraints” principle to classify our population into pattern-direction-selective (PDS) neurons and component-direction-selective (CDS) neurons. We found a larger proportion of PDS cells (68%) and a smaller proportion of CDS cells (8%) compared with prior studies using plaids. We further compared velocity tuning, measured using random dot stimuli, with direction tuning, measured using plaids. Although there was a correlation between the degree of preference for 2D over 1D motion of the two measurements, tuning seemed to prefer 2D motion using random dot stimuli. Modeling analyses suggest that integration across orientations contributes to the 2D motion preference of both dots and plaids, but opponent inhibition mainly contributes to the 2D motion preference of plaids. We conclude that MT neurons become more capable of identifying a particular 2D velocity when stimuli contain multiple orientations.
Introduction
One of many important issues in motion perception is the integration of one-dimensional (1D) to two-dimensional (2D) motion (Born and Bradley, 2005; Bradley and Goyal, 2008). When vision is restricted to a particular portion, motion detection is limited to motion perpendicular to an observed edge in cases where only one edge can be observed through the aperture. Because neurons have small receptive fields (RFs), the brain is effectively confronted with this so-called aperture problem. One way to solve this problem is to add motion originating from another edge. Observation of two edges, in theory, can resolve the aperture problem for rigid motion in 2D space (Bradley and Goyal, 2008). Thus, the visual system must integrate across various orientations to determine true 2D motion.
In the primate visual system, neurons in the primary visual cortex first respond to motion stimuli in a direction-selective manner (Hubel and Wiesel, 1968). They respond, however, to motion components (i.e., 1D motion) and do not combine across orientations (Movshon et al., 1985; Khawaja et al., 2009). Approximately one-third of MT neurons, on the other hand, respond to pattern motion (i.e., 2D motion) by combining motion information across orientations (e.g., Movshon et al., 1985; Rust et al., 2006; Khawaja et al., 2009). These conclusions resulted from experiments using plaids.
Conversely, many studies of area MT have used random dot stimuli. Random dot motion contains various orientation and spatiotemporal frequency components. That most MT neurons are strongly tuned to motion direction and speed using random dots indicates that they convey information about 2D motion. However, the existence of component cells predicts that some neurons should have broad or bimodal direction tuning when presented with random dot stimuli at high speed (Hammond and Smith, 1983; Simoncelli et al., 1996; Simoncelli and Heeger, 1998). Indeed, Okamoto et al. (1999) found such neurons, although Rodman and Albright (1987) did not in a similar experiment. In both studies, responses of MT neurons were measured using a single dot. Moreover, it is still not fully understood how pattern-motion selectivity, measured using plaids, is related to that using random dots that contain multiple orientation and spatiotemporal frequency components on a single-neuron basis.
Here, we measured the direction and speed tuning of MT neurons using random dot stimuli. We found that only a minority of neurons behaved in accordance with predictions of component cells when random dots were used. We further compared, on a single-neuron basis, pattern-motion selectivity using random dots with that using plaids. Although there was a weak correlation between the degree of preference for 2D over 1D motion of the two measurements, tuning seemed to prefer 2D motion using random dot stimuli. Analyses using a standard computational model (Simoncelli and Heeger, 1998) suggested that integration across orientations contributes to the 2D motion preference of both dots and plaids, but opponent inhibition mainly contributes to the 2D motion preference of plaids. Our results suggest that MT neurons become more capable of identifying a particular 2D velocity when stimuli contain multiple orientations.
Materials and Methods
The general experimental procedures were as described previously (Uka and DeAngelis, 2003; Kumano and Uka, 2010). Here, we briefly summarize aspects relevant to the present study. All animal care, training, and experimental procedures were in accordance with the National Institutes of Health guidelines and were approved by the Juntendo University Animal Care and Use Committee.
Subjects and surgery.
We used two Japanese macaque monkeys (Macaca fuscata): one female (Monkey P; 6 kg) and one male (Monkey K; 8 kg). Animals were prepared for experiments using standard aseptic surgeries. A post for head restraint and a recording chamber were implanted chronically in each monkey. To monitor eye movements, we implanted scleral search coils into both eyes for Monkey P and one eye for Monkey K (Judge et al., 1980). A cylindrical recording chamber was mounted over the occipital cortex ∼17 mm lateral and 14 mm dorsal to the occipital ridge, at an angle of 25° above the horizontal. Area MT was accessed by passing through the striate cortex and extrastriate visual areas in the lunate sulcus.
Task and visual stimuli.
The behavioral task and data acquisition were controlled using a commercial software package (TEMPO, Reflective Computing). The monkeys were seated with their heads restrained in a primate chair. A DLP projector (Mirage S+2K, Christie Digital Systems) back-projected a visual stimulus on a tangent screen positioned 57 cm in front of the monkeys' eyes. The screen subtended a visual angle of 122° × 91°, and the frame rate was 100 Hz. The monkeys viewed random dot stimuli while maintaining fixation on a yellow dot (0.15°) on the screen. The monkeys received a drop of water as a reward when their conjugate eye position remained within a 2.0° × 2.0° electronic window around the fixation point during stimulus presentation (500 ms). If the monkeys broke fixation during the trial, the trial was terminated, the data were discarded, and the monkeys were not rewarded.
Random dot stimuli were presented using an OpenGL accelerator board (Quadro FX 1400, NVIDIA). Each random dot stimulus was presented within a circular aperture. The dot density was 64 dots per square degree per second, with each dot subtending ∼0.1°. The starting position of each dot was newly randomized for each trial. The random dot stimulus consisted of red dots (5.3 cd/m2) presented on a black background (0.10 cd/m2). Smooth motion was achieved by plotting dots with subpixel resolution using anti-aliasing provided by the OpenGL board.
Electrophysiological recordings.
We used a tungsten microelectrode (FHC) with impedance values between 0.5 and 2.0 MΩ (at 1 kHz) for recording the extracellular activity of single neurons. The electrode was advanced through the cortex via a transdural guide tube using a pulse motor micromanipulator (MO-951, Narishige) mounted on the recording chamber. Raw signals from the electrode were amplified and bandpass filtered (200–10,000 Hz) using conventional electronic equipment (Bak Electronics). We isolated single neurons using a voltage–time window discriminator (Bak Electronics). Times of action potential and trial event occurrences were stored to disk with 1 ms resolution. Eye position was monitored using a magnetic search coil system (Sankeikizai) and stored to disk at 250 Hz. Area MT was identified on interpreting the pattern of gray matter and white matter encountered during electrode penetration and on the physiological response properties (direction, speed tuning, RF location, and size) of single neurons and multiunit clusters.
Experimental protocols.
After isolating a single MT neuron, we qualitatively explored RF size and location, as well as the tuning properties (direction, speed) of the neuron using a small circular patch of coherently moving random dots. Next, we conducted a set of quantitative preliminary tests to measure direction tuning, speed tuning, RF location, and size tuning (area summation) of each MT neuron. Each of these measurements was performed in a separate block of randomly interleaved trials, with each unique stimulus presented at least three times. During these tests, a tuning curve or RF map was constructed online, and the preferred stimulus parameter was used in subsequent tests. First, direction tuning was measured by presenting eight motion directions 45° apart. Speed tuning was then measured by presenting dots that drifted at 0, 1, 2, 4, 8, 16, 32, and 64°/s toward the preferred direction. Next, we mapped the RF by presenting a small (∼0.25 × approximate RF diameter) patch of random dots drifting at the preferred velocity at each location on a 4 × 4 grid that covered the entire RF. A 2D Gaussian function with an identical radius along two cardinal axes was fitted to this RF map. After determining the RF center, we assessed size tuning by presenting dots within circular apertures at the RF center. We presented aperture sizes of 1, 2, 4, 8, 16, 32, and 64° in diameter, with all other parameters optimized.
Measurement of 2D velocity tuning using random dot stimuli.
After preliminary tests, we measured neuronal responses to random dot stimuli moving in various directions at various speeds. Responses to a combination of eight motion directions 45° apart and seven speeds (1, 2, 4, 8, 16, 32, and 64°/s) as well as those to static and flickering random dot stimuli (all dots were randomly replotted every fourth video frame) were recorded. For neurons with high-pass speed tuning, we also tested 128°/s speed and for low-pass speed-tuned neurons, 0.5°/s speed. The RF location and aperture size were kept constant across trials. All stimulus conditions were pseudo-randomly interleaved within a block. Data were discarded if single-unit isolation was lost before three repetitions. The median number of trial repetitions across the range of accepted datasets was five.
Measurement of direction tuning using Gabor and plaid.
For a subpopulation of neurons, we also measured direction tuning using a Gabor patch and a plaid stimulus (see Fig. 5A) and determined whether the neuron could be classified as a component cell or a pattern cell. First, we measured spatiotemporal frequency tuning using a Gabor moving in the preferred direction determined from the direction-tuning curve using random dot stimuli (described above). The Gabor had one of six carrier spatial frequencies (0.0625, 0.125, 0.25, 0.5, 1, or 2 cycles/degree) and one of eight temporal frequencies (0.1953125, 0.390625, 0.78125, 1.5625, 3.125, 6.25, 12.5, or 25 Hz). The envelope was a 2D Gaussian with an SD of 1.33 or 2.66°, depending on eccentricity. Next, direction tuning of a Gabor and a plaid was measured at the preferred spatial and temporal frequency. The plaid contained two Gabors 120° apart. Both the Gabor and the plaid were high contrast (0.9) and moved in one of 24 directions 15° apart. Data were discarded if the neuron was lost before three repetitions. The median number of trial repetitions across the range of accepted datasets was five.
Data analysis.
Neuronal responses to each stimulus condition were defined as the mean firing rate across trial repetitions in the time window of visual stimulus duration. The spontaneous firing rate was calculated from spiking activity during presentation of a blank screen.
For both 2D velocity tuning and plaid experiments, we sought to classify each MT neuron as a pattern-direction-selective (PDS) or component-direction-selective (CDS) neuron. We created predictions of ideal PDS and CDS neurons for 2D velocity tuning and direction tuning for plaids, as described in Results. We computed partial correlations (Rp and Rc) of the measured responses with the pattern and component predictions as follows:
where rp is the correlation of the data with the pattern prediction, rc is the correlation of the data with the component prediction, and rpc is the correlation between the two predictions. The component correlation (Rc) can be obtained by exchanging the subscripts p and c in the above equation. These correlation values were converted into Z-scores using Fisher's r-to-Z transformation using the following equation:
where n is the number of stimulus conditions. The Z-scored component correlation (Zc) can be computed analogously using Rc. Each of the Z-scored correlations was tested for significance. As in previous studies, we used a criterion of 1.28, equivalent to p = 0.90 (Smith et al., 2005; Khawaja et al., 2009; Jazayeri et al., 2012). The difference in Z-scored correlations (Zp − Zc) yields the pattern index, with negative values indicating component-motion selectivity and positive values indicating pattern-motion selectivity.
Modeling responses of MT neurons.
Several computational models have been proposed to describe the computation of MT neurons (Simoncelli and Heeger, 1998; Rust et al., 2006; Nishimoto and Gallant, 2011). To determine the kinds of mechanisms that might account for our results, we implemented a publically available code for neuronal responses of area MT (Simoncelli and Heeger, 1998). Model direction-selective V1 neurons were represented as a blob at a particular spatiotemporal frequency in 3D spatiotemporal frequency space. The model MT neuron linearly summed the outputs of a set of direction-selective V1 neurons that were equally distributed on a ring-like structure on a plane corresponding to the preferred direction of the model MT neuron in the 3D spatiotemporal frequency domain (Simoncelli and Heeger, 1998, their Fig. 3B). In addition to summation, the model MT neuron subtracted the outputs of V1 neurons that lay off this plane. In the analysis of model responses, we varied the number of V1 neurons that fed into the model MT neuron from 1 (no integration across orientations; i.e., CDS neurons) to 8 (V1 neurons separated by 22.5°). For each integration bandwidth, we also modified the balance of excitation and inhibition by multiplying the inhibitory V1 afferent weight by a coefficient ranging from 0 to 1 in steps of 0.2. We measured responses of the model MT neuron with each of these parameters to random dot motion, Gabors, and plaids. When using random dot motion, for each unique stimulus condition, we calculated responses for 100 different dot patterns. These dot patterns were used across all parameters of the model to compare the responses across different parameters. The preferred speed of the model MT neuron was ∼8°/s.
Results
We recorded from a total of 168 neurons from 2 monkeys (109 from Monkey K and 59 from Monkey P). For each neuron, we measured 2D velocity tuning using random dot motion at a combination of eight directions and seven or eight speeds. For our population, most of the neurons were selective (p < 0.05, ANOVA) for direction at the optimal speed (165 of 168, 98%) and for speed in the optimal direction (160 of 168, 95%). Of the 168 neurons, 75 (52 from Monkey K and 23 from Monkey P) were available for direction-tuning measurements with a Gabor and a plaid. In the following text, we first considered the predictions of 2D velocity tuning for ideal CDS and PDS neurons. We then examined 2D velocity tuning of MT neurons, measured using random dot motion, and compared these data with direction tuning, measured using a Gabor and a plaid.
2D velocity tuning of ideal CDS and PDS neurons
We first considered how ideal CDS and PDS neurons respond to random dot motion across directions and speeds. The random dot stimulus contains components of all orientations. According to the intersection of constraints principle (Adelson and Movshon, 1982), for a particular object velocity, the speed of each orientation component is determined by the angle between the component direction and the direction of object motion. Figure 1A illustrates an example of this relationship for an upward-moving object (red arrow). The speed of the orientation components (length of blue arrows) is given as follows:
where Sc is component speed, So is object speed, and θ is the angle between the object and component-motion directions. Now, assume that one of the blue arrows in Figure 1A is the preferred velocity of a CDS neuron. The diagram shows that the CDS neuron should respond to object motion (red arrow) faster than the preferred speed of the CDS neuron at a nonpreferred direction.
Characterizing pattern-motion selectivity using random dot motion. A, Random dot stimuli contain components of all orientations. The velocity of a random dot stimulus (object velocity, red arrow) and the velocity of each orientation component (component velocity, blue arrow) are related by a cosinusoidal function. B, A single component velocity is consistent with two-object velocity at a faster speed. This indicates that CDS neurons respond to two directions away from the preferred direction at faster speeds. C, The direction-tuning curve of an ideal CDS neuron selective for upward motion. Different shadings represent different motion speeds. For speed ≤ optimal speed, the direction-tuning curve has a single peak (brighter lines). For speed > optimal speed, the direction-tuning curve is bimodal (darker lines). D, These direction-tuning curves across all speeds are converted to the 2D velocity-tuning map by plotting responses in polar coordinates. There is elongation of the response region orthogonal to the preferred direction. E, An ideal PDS neuron is selective for the same direction across all speeds. F, The 2D velocity tuning of an ideal PDS neuron does not show elongation.
For a given component velocity, there are two possible pattern-motion directions consistent with the component velocity when considering objects moving at a particular speed (Fig. 1B). A CDS neuron thus responds to two directions away from its preferred direction at faster speeds (Fig. 1B, red arrows). The directional difference from the preferred direction is determined by solving Equation 1 for θ, as follows:
This equation suggests that the direction-tuning curve of a CDS neuron for speed faster than the preferred speed (Sc) is bimodal and that the two peaks become more separated for faster speeds.
Based on this scheme, for an ideal CDS neuron tuned to upward motion, we predicted the direction-tuning curves across multiple speeds in Figure 1C. Here, we assumed that the direction-tuning curve at the preferred speed can be approximated by a circular Gaussian (von Mises) function (Liu and Newsome, 2003; Kohn and Movshon, 2004). For speeds slower than the preferred speed (brighter lines), the direction-tuning curve was invariant. For faster speeds (darker lines), the direction-tuning curve was the sum of two circular Gaussian functions, the center positions of which were shifted away from the preferred direction, according to Equation 2. The peak and trough responses of the direction-tuning curves were approximated to match the speed preference of typical MT neurons. Figure 1D shows the 2D velocity tuning of the ideal CDS neuron by plotting the responses in 2D polar coordinates, with direction mapped on the angular axis and speeds on the radial axis. The response map was elongated along a line orthogonal to the preferred direction.
An ideal PDS neuron, on the other hand, had invariant direction-tuning curves across all speeds, with the peak responses matching the speed preference of typical MT neurons (Fig. 1E). The 2D velocity tuning of the ideal PDS neuron had a single response peak, corresponding to the preferred velocity (Fig. 1F). The extent and shape of the response map depended on the width of the direction- and speed-tuning curves. Even for a PDS neuron with extremely broad direction tuning, however, the response map did not elongate along a line orthogonal to the preferred direction, as the CDS neuron did. This is because direction was mapped on the iso-speed angular axis in the 2D velocity space. If an MT neuron preferred the fastest speed we examined, we could not classify the neuron as PDS or CDS based on this method. In this case, the PDS and CDS predictions would be identical.
2D velocity tuning of MT neurons
We next examined the 2D velocity tuning of MT neurons and evaluated pattern-motion selectivity using the predictions of the 2D velocity tuning of the ideal CDS and PDS neurons. Figure 2A shows direction-tuning curves across all speeds for a representative MT neuron classified as a PDS neuron, as described below. First, we interpolated direction-tuning data with a periodic spline at the optimal speed where the maximum response occurred (4°/s for this neuron). This interpolated curve was used as a template for the prediction of the ideal CDS and PDS neurons. For prediction of the ideal CDS neuron, the direction-tuning curves for speeds equal to or less than optimal speed were the same as the interpolated tuning curve. For speeds greater than optimal speed, the predicted direction tuning was computed as the sum of two interpolated curves, each shifted by an amount determined from the ratio of the optimal speed to each speed (Eq. 2). Finally, the predicted peak and minimum responses at each speed were normalized to the peak and minimum values obtained from the raw direction-tuning curves. The resulting direction-tuning curves of the CDS prediction were bimodal at fast speeds (Fig. 2C). The predicted 2D velocity-tuning map exhibited elongation along a line orthogonal to the preferred direction (Fig. 2D). For prediction of the ideal PDS neuron, the interpolated spline curve at the optimal speed was used across all speeds. As for the CDS prediction, the peak and minimum responses were normalized to the corresponding values obtained from the raw direction-tuning curves (Fig. 2E). The 2D velocity map of the ideal PDS neuron showed a single peak and a nearly round shape (Fig. 2F).
An example PDS neuron. A, Direction-tuning curves across all speeds. Different colors denote different speeds, as indicated in the inset. Error bars indicate SEM. B, 2D velocity-tuning map. C, For CDS predictions, a periodic spline curve was interpolated to the direction-tuning data at the optimal speed. The direction tuning for a speed higher than the optimal speed was computed as the sum of two interpolated curves, each shifted by an amount determined from the ratio of the optimal speed to each speed. D, 2D velocity-tuning map of CDS predictions. E, For PDS predictions, the interpolated curve was used across all speeds. F, 2D velocity-tuning map of PDS predictions. The pattern and component correlations are noted on the top right of the 2D velocity-tuning maps.
The 2D velocity map of this example neuron (Fig. 2B) seemed to match the prediction of the ideal PDS neuron (Fig. 2F), but not that of the ideal CDS neuron (Fig. 2D). This impression was quantified by computing the partial correlation between the actual responses and the predictions of the PDS and CDS responses. For this example neuron, the pattern correlation (Rp) was 0.98, and the component correlation (Rc) was −0.2. These correlation values were converted into Z-scores (Zp and Zc) using Fisher's r-to-Z transformation. The difference in Z-scores (Zp − Zc) yielded the pattern index, for which negative values indicate component-motion selectivity and positive values indicate pattern-motion selectivity. The pattern index of this example neuron was 18.3. Because the pattern index was >1.28 (corresponding to p = 0.90), this neuron was classified as a PDS neuron.
Figure 3 shows data from an example neuron classified as a CDS neuron. The direction-tuning curves for speeds ≤ optimal speed (16°/s) were unimodal. For faster speeds, the direction-tuning curves broadened, and two peaks were evident at the fastest speed of 64°/s (Fig. 3A). This bimodality led to the elongation of the response map in 2D velocity space (Fig. 3B). These characteristics were consistent with the prediction of the ideal CDS neuron (Fig. 3C,D), but not with that of the ideal PDS neuron (Fig. 3E,F). The component correlation was 0.80, and the pattern correlation was 0.66. These values resulted in a pattern index of −2.11, suggesting component-motion selectivity.
An example CDS neuron. Conventions are the same as in Figure 2.
Of the 168 neurons we recorded from, 37 were excluded from this analysis because the maximum response occurred at the fastest speed we examined, and thus the CDS and PDS predictions were identical. Figure 4 plots Zp against Zc across the remaining 131 neurons. The distribution formed a continuum from component-motion selectivity (bottom right sector) to pattern-motion selectivity (top left sector). Previous studies on plaids demonstrated approximately equal numbers of PDS and CDS neurons in MT (Movshon et al., 1985; Smith et al., 2005; Rust et al., 2006; Khawaja et al., 2009; Jazayeri et al., 2012). In contrast, most of our MT neurons (113 of 131, 86%) were classified as PDS neurons, 6 (5%) as CDS neurons, and 12 (9%) as unclassified, based on the responses to random dot motion. The median pattern index was 6.58.
The distinction between PDS and CDS neurons based on this method relied on the differential predictions for speeds faster than the optimal speed (Fig. 1C,E, dark lines). Thus, classification of MT neurons that preferred a high speed could be problematic. To avoid this issue, we examined only MT neurons with preferred speeds ≤ 8°/s. Even for this subpopulation (n = 69), the predominance of PDS neurons remained; 62 of 69 (90%) neurons were classified as PDS neurons, one (1%) as CDS neurons, and six (9%) as unclassified. The median pattern index was 8.90.
We further examined whether the number of trials affected our results. We found no correlation between the pattern index and the number of trials (rs = 0.09, p = 0.28), suggesting that the prevalence of pattern-motion selectivity was not the result of the relatively small number of trials per stimulus (median, 5). Additionally, for neurons that were measured for 10 repetitions, we subsampled (with replacement) responses for a given number of trials and asked how the pattern index depended on the trial number. The mean estimated pattern index tended to increase when the trial number increased from three (median, 5.3) to 10 (median, 6.0). Thus, we did not overestimate, but rather underestimated the pattern index when the trial numbers were small.
Direction tuning for Gabor and plaid
Given the predominance of PDS neurons with random dot motion, we sought to compare pattern-motion selectivity for random dot motion and pattern-motion selectivity using the conventional plaid method. For 75 of 168 neurons, we measured direction tuning to Gabor patches and plaids that were constructed by superimposing two Gabors drifting 120° apart (Fig. 5A). A CDS neuron responds to the direction of the component Gabors and not to the overall motion of the plaid pattern. Thus, the direction-tuning curve of CDS neurons for plaids is bimodal, with each peak corresponding to one of two component Gabors drifting in the preferred direction. In contrast, a PDS neuron signals the motion direction of a plaid pattern, not individual component Gabors, and has nearly identical direction-tuning curves for Gabors and plaids. We characterized the pattern-motion selectivity of each neuron in our population using a standard method (Smith et al., 2005). Consistent with previous studies, we found each type of MT neuron, including CDS neurons (Fig. 5B) and PDS neurons (Fig. 5C). As in the analysis of 2D velocity tuning, we quantified pattern-motion selectivity for plaids by calculating the pattern index as the difference between the Z-scored pattern correlation and Z-scored component correlation. For our population, 17 (23%) neurons were classified as PDS neurons, 37 (49%) as CDS neurons, and 21 (28%) were unclassified (Fig. 5D). The median pattern index was −1.01.
Analysis of direction tuning for a Gabor and a plaid. A, For each neuron, we measured the direction-tuning curve for a Gabor and a plaid that was constructed by superimposing two Gabors drifting 120° apart. B, An example CDS neuron. The polar plots represent the direction-tuning curve for Gabors (left), plaids (middle), and the two predictions (right). The pattern prediction (red) was the same as the direction-tuning curve for Gabors. The component prediction (blue) was computed as the sum of two direction-tuning curves, each shifted by 60°. C, An example PDS neuron. D, Pattern and component-motion selectivity for plaids. Z-scored pattern correlations were plotted against Z-scored component correlations. Based on the correlation values, each neuron was classified as a PDS neuron (red), a CDS neuron (blue), or unclassified (black).
Relationship between pattern-motion selectivity for random dot motion and plaids
For the subpopulation of 75 neurons where both random dot motion and plaid data were available, we compared pattern-motion selectivity between the two stimuli. Nineteen neurons were excluded from this comparison because these neurons preferred the fastest speed we examined and thus were not usable for calculation of pattern-motion selectivity using random dot motion. Figure 6A plots the pattern index for random dot motion against the pattern index for plaids for the remaining 56 neurons. The color of the data points denote the classification based on plaids. Of the 28 neurons classified as CDS, based on plaids (blue points), 21 neurons were PDS when tested with random dot motion, and only one neuron was classified as CDS. The majority of PDS neurons based on plaids (red points) were also judged as PDS for random dot motion (10/11). As a result, the pattern index for random dot motion was significantly larger than the pattern index for plaids (p = 3.4 × 10−10, Wilcoxon signed-rank test). We also found a weak but significant positive correlation between the two pattern indices (rs = 0.38, p = 0.004). This suggests that integration across orientations observed using plaids was at least partially responsible for the prevalence of pattern-motion selectivity measured using random dot stimuli.
The distribution of pattern indices for random dot motion and plaids. A, The pattern index for random dot motion was plotted against the pattern index for plaids. Color denotes the classification, based on plaids. Many blue points (classified as CDS for plaids) lie above the diagonal line, suggesting pattern-motion selectivity. B, The pattern index for random dot motion (ordinate) was recalculated using direction tuning at the optimal speed and a speed just above it. The pattern index for plaids was also recalculated using the subsampled direction-tuning data at 45° and spline interpolation. C, The pattern indices for random dot motion and plaids were calculated using a standard computational model of MT neurons. The bandwidth of integration across orientations and inhibitory weights on V1 afferents were modified. Each line represents pattern indices for a unique inhibitory weight. For each inhibitory weight, from the top right to bottom left, the integration bandwidth across orientations decreased from 8 to 1. Gray dots represent a subpopulation of neurons from A whose preferred speed was <8°/s.
We further analyzed the pattern-motion selectivity for random dot motion in a slightly different way to perform a fair comparison with the data for plaids. Specifically, we used only two direction-tuning data at the optimal speed and at the speed just above it (e.g., the optimal speed of 4°/s and 8°/s). We interpolated direction-tuning data with a periodic spline at the optimal speed, and the interpolated curve was used to create CDS and PDS predictions at the speed just above the optimal speed. The speed ratio of the two resulted in a directional difference of 120° between the two component motions (Eq. 2), which was the same as in the plaid experiment. We further analyzed the direction tuning for Gabors and plaids using a spline interpolation. We also downsampled the direction-tuning data at 45° for Gabors and plaids because the increment in the direction for the 2D velocity data was 45°. For each neuron, we interpolated the direction-tuning curve for the Gabor with a periodic spline. A CDS prediction was created by summing two interpolated curves, each shifted by 60°. A PDS prediction was identical to the interpolated curve. The pattern index for random dot motion calculated using two speeds was plotted against the pattern index for plaids calculated with spline interpolation, as shown in Figure 6B. Even in this analysis, the pattern index for random dot motion was significantly larger than the pattern index for plaids (p = 4.0 × 10−9, Wilcoxon signed-rank test).
Model MT responses
We implemented a computational model of MT neurons provided by Simoncelli and Heeger (1998) to determine the mechanisms that might underlie the observed prevalence of pattern-motion selectivity for random dot motion (Fig. 6A). A model MT neuron linearly summed the outputs of a set of direction-selective V1 neurons that were equally distributed on a ring-like structure on a plane corresponding to the preferred direction of the model MT neuron. In addition to summation, the model MT neuron subtracted the outputs of V1 neurons that lay off this plane. Here, we varied the number of V1 neurons that fed into the model MT neuron from 1 (no integration across orientations; that is, CDS neurons) to 8 (larger than the original: V1 neurons were separated by 22.5°). For each integration bandwidth, we also modified the balance of excitation and inhibition by multiplying the inhibitory V1 afferent weight by a coefficient ranging from 0 to 1 in steps of 0.2. We measured the responses of the model MT neuron with each of these parameters to random dot motion, Gabors, and plaids.
Figure 6C shows the pattern indices for the model MT neuron overlaid on the pattern indices for real MT neurons. Because the model prediction depended on the preferred speed of the model MT neuron, the model prediction was overlaid on the pattern indices for neurons with preferred speeds of ≤8°/s (gray symbols; subpopulation of Fig. 6A). Each solid line connects the pattern indices for adjacent integration bandwidths for each inhibitory weight condition. As expected, with increased orientation integration, pattern indices increased for both random dots and plaids under all inhibitory weight conditions. In contrast, increasing the inhibitory weights shifted the trajectory of pattern indices toward the right (Fig. 6C, from green to orange), suggesting that inhibition contributes to pattern selectivity of plaids more strongly than that of dots. The model predictions were above the diagonal in Figure 6C, suggesting that the basic framework of the MT model (Simoncelli and Heeger, 1998) partly accounted for the observed prevalence of PDS neurons for random dot motion. Additionally, we suggest that part of the discrepancy between pattern selectivity for plaids versus dots was the result of opponent inhibition.
Relationship between pattern-motion selectivity and spatiotemporal separability
Random dot stimuli contain multiple orientations and spatiotemporal frequencies. Previous studies have shown that some MT neurons possess speed tuning independent of stimulus spatial frequency (Perrone and Thiele, 2001; Priebe et al., 2003). These speed-tuned neurons could summate inputs along a line that bypasses the origin in the 3D spatiotemporal frequency volume. We examined the relationship between pattern-motion selectivity and the dependence of speed tuning on spatial frequency. As in the study by Priebe et al. (2003), we analyzed the response map obtained with a Gabor stimulus in 2D spatial and temporal frequency space. To quantify the dependence of speed tuning on spatial frequency, we fitted a variant of the 2D Gaussian function on log-log coordinates of the 2D frequency response map as follows:
where tfp depends on spatial frequency and is given by the following:
The fitted parameter Q quantifies the dependence of speed tuning on spatial frequency (Priebe et al., 2003). Specifically, when Q is zero, the response map on 2D frequency space is elongated such that speed tuning does not depend on spatial frequency. A Q value of −1 indicates that the 2D response map is separable, and the preferred speed decreased with increasing spatial frequency.
The response map on 2D frequency space was available for 124 neurons (88 from Monkey K and 36 from Monkey P). The distribution of Q values from this population was unimodal, with a mean value of −0.53, consistent with that found by Priebe et al. (2003). Both the 2D velocity tuning for random dots and the spatiotemporal frequency tuning were available for 94 neurons. Of these, 25 neurons preferred the fastest speed we examined; thus, the pattern index could not be calculated. Consistent with Priebe et al. (2003), we found no correlation between the pattern index for random dot motion and Q values (rs = 0.13, p = 0.29, N = 69) and no correlation between the pattern index for plaids and Q values (rs = 0.03, p = 0.79, N = 75).
Discussion
Our results show that so-called “component cells” in area MT can give rise to 2D motion tuning when stimuli are broadband in orientation. The modeling analysis confirmed that the basic framework of MT computation, in which weighted linear summation of V1 outputs is followed by normalization, can partly account for our results. Furthermore, we suggest that this characteristic is partly attributed to MT neurons with weak inhibitory inputs from V1 afferents. Although we have not tested this explicitly, most MT neurons presumably have the ability to encode 2D velocity during normal viewing because objects in our world typically contain multiple orientations and spatiotemporal frequencies.
Pattern-motion selectivity using random dot stimuli
In this study, pattern-motion selectivity for random dot stimuli was predicted based on the shape of the direction-tuning curve at each speed. In theory, if a neuron responds purely to motion components, the 2D velocity map should be elongated along a line perpendicular to the preferred direction. In reality, responses gradually fall off, but the way in which they decrease may be difficult to predict. Direction tuning is usually described by a Gaussian function (Britten et al., 1993; DeAngelis and Uka, 2003). Speed tuning can be described by a log-Gaussian function, consistent with Weber's law (Nover et al., 2005). If direction tuning is Gaussian and speed tuning is log-Gaussian for CDS neurons, then the 2D velocity map should be elongated along a line perpendicular to the preferred direction, with a tail elongating toward faster speed. However, applying a log-Gaussian function in Cartesian coordinates is difficult, especially because the prediction for static stimuli is ambiguous. For PDS neurons, a likely prediction would be that the 2D velocity map is the product of a Gaussian direction-tuning curve and a log-Gaussian speed-tuning curve. However, it is unclear how these would arise from CDS neurons. For these reasons, we decided not to make precise predictions using mathematical formulations but instead just predicted the preferred direction at each speed. We also assumed that the shape of direction tuning did not change with speed. Although it is unclear whether these assumptions are valid, they nonetheless provided a fairly good description of the data.
To critically assess our CDS and PDS predictions, we performed several control analyses. First, we predicted pattern selectivity from direction-tuning curves at two speeds (Fig. 6B). This tended to underestimate the pattern index, presumably because of undersampling of the data, pointing to the importance of mapping the entire pattern of direction/speed tuning. Second, we tested how the number of trials might affect our results and found that we might have underestimated the pattern index when the trial numbers were small. Finally, we focused on neurons with a slow speed preference. Pattern indices for this population were actually larger than those for the whole population. Because CDS and PDS predictions become less distinguishable for neurons with high-speed preferences, the pattern indices might have been underestimated in neurons that prefer high speeds. Therefore, it is possible that pattern indices for the population of neurons with a slow speed preference represent the true distribution of pattern indices.
Relationship to previous studies measuring direction/speed tuning
Several previous studies have measured direction/speed tuning using stimuli with multiple orientations. Hammond and Smith (1983) were the first to show bimodality of direction tuning at high speeds in the cat visual cortex. In the monkey MT, Rodman and Albright (1987) used a dot stimulus and found that direction tuning for most neurons can be explained by a change in gain, depending on speed. Okamoto et al. (1999), on the other hand, found neurons with bimodal peaks at fast speeds. There are two potential issues with these studies. First, because these stimuli only consisted of one element, the time during which the stimulus was contained in the RF could have changed depending on the speed. Second, inaccurate mapping of the RF may have led to stimulation of different parts of the RF depending on direction. These issues might have contributed to the apparent discrepancy between the two studies, where Rodman and Albright (1987) reported no bimodal direction tuning with fast speeds, whereas Okamoto et al. (1999) reported a fairly substantial population (6 of 35, 17%).
Random dot stimuli contain multiple elements and thus can continue to stimulate the same location regardless of direction and speed. In their preliminary report, Simoncelli et al. (1996) found that some component cells have bimodal direction-tuning curves at high speeds for random dot stimuli. However, they reported neither the pattern indices nor the incidence of bimodality. By quantifying the degree of pattern selectivity using random dot stimuli, our results show that the incidence of bimodality lies between those of the two studies described above. We found a large proportion of PDS neurons (86%) and a small proportion of CDS neurons (5%). Thus, our results confirm those of Okamoto et al. (1999), although the percentage of CDS neurons using multioriented stimuli was smaller than their population (17%). Moreover, our results are similar to Okamoto et al. (1999) in that only a minority of neurons classified as CDS using bioriented stimuli (plaids or crosses) had bimodal direction tuning at fast speed using multioriented stimuli (1 of 28 in our study, 6 of 15 in Okamoto et al., 1999).
Explanations for the difference between pattern-motion selectivity for random dot stimuli and that for plaid
Many studies using plaids have confirmed that approximately one-third of MT neurons are PDS neurons, and another third comprises CDS neurons (Movshon et al., 1985; Smith et al., 2005; Rust et al., 2006; Khawaja et al., 2009; Jazayeri et al., 2012). Conversely, studies using random dot stimuli have shown sharp direction and speed tuning (Britten et al., 1993; DeAngelis and Uka, 2003; Nover et al., 2005), suggesting that neurons encode 2D motion. Here, we estimated pattern-motion selectivity using random dot stimuli and compared this directly with measurements using plaids. Our results show a striking difference in pattern-motion selectivity depending on stimuli and suggest that MT neurons are more capable of identifying a particular 2D velocity when stimuli contain multiple orientations. Although the weak, but significant, correlation between pattern index for random dot stimuli and that for plaids suggests that the same mechanisms are in play for both types of stimuli, additional explanations are necessary to account for the stronger pattern-motion selectivity for random dot stimuli.
Analyses of models of MT neurons (Simoncelli and Heeger, 1998) suggest that the basic framework of MT computation in which weighted linear summation of V1 outputs is followed by normalization can partly account for our results (Fig. 6C, orange line). We further found that, by reducing the strength of inhibitory weights from V1 afferents, the trajectory of simulated pattern indices shifted leftward (Fig. 6C, from orange to green). This suggests that the variation in the balance between excitation and inhibition of V1 inputs additionally accounts for our results. This is also important because inhibition from opposing directions is important for plaid tuning to appear pattern-like (Rust et al., 2006). Our results suggest that opponent inhibition does not contribute in the same way for stimuli with multiple orientations as it does for plaids. The trajectory of simulated pattern indices did not pass through a population of neurons in which the pattern index for random dots was large and that for plaids was small (Fig. 6C, neurons above green line). Additional nonlinearities, such as end-stopping at V1 (Hubel and Wiesel, 1965), presumably function for these neurons. Indeed, by combining the motion energy detector (Adelson and Bergen, 1985) with end-stopping, the true motion of tilted bar stimulus can be signaled by component direction-selective neurons (Tsui et al., 2010). Because our plaids comprised two Gabor patches, not gratings with sharp edges, end-stopping might have been less effective for our plaids compared with random dot motion, leading to the discrepancy between pattern selectivity of random dot motion and plaid motion.
Our results show that the majority of MT neurons had a narrower integration bandwidth across orientation and a weaker inhibitory weight than the standard model of pattern cell. This is consistent with a recent study by Nishimoto and Gallant (2011), who reported that the spectral RFs of MT neurons tend to lie on a single plane in 3D spatiotemporal frequency space, corresponding to optimal velocity coding (Simoncelli and Heeger, 1998), although the spectral RFs of typical MT neurons showed a partial ring-like structure and did not span the whole velocity plane. This RF structure ensures that tuning for stimuli that contain multiple orientations and spatiotemporal frequencies appears pattern-like.
Footnotes
This work was supported by a Grant-in-Aid for Scientific Research on Innovative Areas from the Ministry of Education, Culture, Sports, Science and Technology of Japan (23135527) to T.U., H.K. was supported by a High Technology Research Center Grant from MEXT. We thank Kazushi Maruya and Shin'ya Nishida for programming the Gabor and plaid stimuli and Sanae Hosotani for technical and surgical assistance.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Takanori Uka, Department of Neurophysiology, Graduate School of Medicine, Juntendo University, 2-1-1 Hongo, Bunkyo, Tokyo 113-8421, Japan. uka{at}juntendo.ac.jp