 |
||||
|
||||
|
||||
|
||||
Previous Article | Next Article
After the monkey had 3-6 months of training on the discrimination task, a beveled CILUX recording chamber (Crist Instruments, Hagerstown, MD) was attached to the monkey's skull at an angle of 25° above the horizontal, and was located over the occipital cortex approximately 17 mm lateral and 14 mm dorsal to the occipital ridge. A second eye coil was implanted into the other eye at this time to allow measurements of vergence posture. After 1-2 weeks of recovery time, the animal underwent an additional training period in which vergence angle was monitored and enforced to be accurate to within ±0.25°; subsequently, we started electrophysiological recordings in MT. All animal care and experimental procedures were approved by the Institutional Animal Care and Use Committee at Washington University and were in accordance with NIH guidelines.
The Journal of Neuroscience, April 15, 2003, 23(8):3515
Contribution of Middle Temporal Area to Coarse Depth Discrimination: Comparison of Neuronal and Psychophysical Sensitivity
Takanori Uka and Gregory C. DeAngelis Department of Anatomy and Neurobiology, Washington University School of Medicine, St. Louis, Missouri 63110
ABSTRACT
TOP
ABSTRACT
Introduction
Recent work suggests that the middle temporal (MT) area contributes to depth perception in addition to its well established roles in motion perception. To determine whether single MT neurons carry disparity signals with sufficient fidelity to account for depth perception, we have compared neuronal and psychophysical sensitivity to disparity while monkeys discriminated between two coarse disparities (near vs far) in the presence of noise. The strength of the visual stimulus was titrated around psychophysical threshold by varying the percentage of binocularly correlated dots in a random dot stereogram. We find that the average MT neuron has sensitivity equal to that of the monkey, as was reported previously for direction discrimination in MT. We further address some important factors that could bias the neuronal/psychophysical sensitivity comparison, including the possibility that monkeys reach a decision before the end of the stimulus presentation. Unlike the predictions of a simple model that uses Poisson spiking statistics, the sensitivity of many MT neurons has little dependence on the time interval over which spikes are counted to compute a neuronal threshold. Thus the response properties of many MT neurons appear to be adapted for rapid discrimination of depth, and we describe how temporal variations in both signal and noise contribute to this effect. We therefore predicted that psychophysical thresholds should exhibit little dependence on viewing duration in our task, and this was confirmed by additional behavioral experiments. Overall, our findings show that MT is well suited to provide sensory signals that form the basis for perceptual judgments of depth.
Key words: visual cortex; extrastriate; stereopsis; binocular disparity; neuronal sensitivity; depth discrimination
Introduction
Horizontal binocular disparities are used by the visual system to reconstruct three-dimensional (3-D) scene structure from two-dimensional retinal images. Many areas in primate visual cortex contain disparity-selective neurons, including V1, V2, VP, V3/V3A, V4, MT, MST, CIP, and IT (Hubel and Wiesel, 1970
; Poggio and Fischer, 1977
Several techniques have been used to establish firm links between neuronal activity in the middle temporal (MT) area and perception of visual motion (for review, see Parker and Newsome, 1998
If MT plays an important role in depth perception, then MT neurons should be sufficiently sensitive to account for the ability of monkeys to discriminate depth. We tested this hypothesis by recording from single MT neurons while monkeys performed a depth discrimination task identical to the one used by DeAngelis et al. (1998)
Comparison of neuronal and behavioral sensitivity is fraught with assumptions and practical difficulties. A second major goal of our study was to determine how some of these factors could affect our estimates of neuronal/psychophysical threshold ratios. Specifically, we examined the effects of trial-to-trial stimulus variation, variability in psychophysical performance, and the monkeys' decision (integration) time. Our analyses place firm bounds on how much each of these factors affects the overall results, providing new insights into how the brain uses information coded by single neurons to form a perceptual decision. Preliminary results have been reported previously (Uka and DeAngelis, 2001
Materials and Methods
Subjects and surgery
Physiological experiments were performed with two male rhesus monkeys (Macaca mulatta) weighing 5-6 kg. The animals were prepared for daily training and recording sessions by using standard surgical procedures described in detail previously (Britten et al., 1992Visual stimuli
The monkeys sat in a primate chair and faced a flat-screen 22 inch color monitor (Sony GDM-F500) placed at a viewing distance of 57 cm. The display subtended a visual angle of 40 × 30°, had a resolution of 1152 × 864 pixels, and was refreshed at 100 Hz. Visual stimuli were generated by a Dual CPU workstation running Windows 2000. Random dot stimuli were programmed in Microsoft Visual C++ by using the OpenGL libraries and were displayed by an OpenGL accelerator board with quad-buffered stereo support (Oxygen GVX1, Creative Labs, Milpitas, CA). Each random dot stereogram (RDS) was presented within a circular stimulus aperture. Dot density was 64 dots per square degree/sec, with each dot subtending ~0.1°. The starting position of each dot within the aperture was newly randomized for each trial (VAR condition) except for some trials, specifically noted in the text, in which the dot patterns were identical across trials (NOVAR condition). Precise disparities and smooth motion were achieved by plotting dots with subpixel resolution, using the hardware anti-aliasing capabilities of the OpenGL accelerator board. Stereoscopic images were displayed by presenting the left and right half-images alternately at a refresh rate of 100 Hz. The monkey viewed the display through a pair of ferroelectric liquid crystal shutters (DisplayTech, Longmont, CO) that were synchronized to the video refresh such that one shutter was closed while the other was open. Ghosting effects were minimized (stereo crosstalk was <3%) by presenting red dots on a black background, because the decay of the red phosphor is much faster than that of the green or blue phosphors. The position of each dot in a moving stimulus was updated every video frame (rather than every pair of frames) to avoid unwanted changes in the binocular disparity of the stimulus with variations in the direction or speed of motion. All dots within the RDS moved coherently (100% motion coherence) at a velocity tailored to each MT neuron. Thus dots did not disappear until they reached the boundary of the circular aperture, after which point they resumed motion from the opposite side of the aperture. In the discrimination task described below (Fig. 1), the disparity signal was titrated by manipulating the percentage of binocularly correlated dots in the RDS. Correlated (i.e., signal) dots were assigned one of two fixed disparities (crossed vs uncrossed) during each trial, and the remaining (noise) dots were assigned random disparities within the range from2 to 2° (Fig. 1C). Dots retained their identities (signal or noise) throughout a trial; hence the distribution of noise disparities was fixed within a given trial. For each binocular correlation level the exact distribution of noise disparities varied across trials from one repetition to the next, except where explicitly noted in the text (NOVAR condition).
For a given neuron the location and size of the circular RDS aperture did not vary with the disparity or binocular correlation of the dots, thus eliminating monocular cues to depth. For one monkey and one human we verified that the task could not be performed at all under monocular viewing conditions. As a result of the fixed RDS aperture there was a fringe of binocularly uncorrelated dots along the edges of the stimulus. The aperture size generally was chosen to be slightly larger than the classical receptive field such that this uncorrelated fringe lay outside the receptive field. Stationary background dots (in fixation trials) or flickering background dots (in discrimination trials) were presented at zero disparity to help anchor the monkey's vergence posture (gray dots in Fig. 1A).
View larger version (20K):[in this window]
[in a new window]
Figure 1. Depth discrimination task. A, A random dot stereogram was presented within a circular aperture slightly larger than the receptive field (RF) of the neuron, and dots moved at the preferred velocity (arrow) of the neuron. Filled and open dots represent left and right half-images, respectively. Regions of the 40 × 30° screen outside the receptive field were filled with flickering zero-disparity background dots (gray). Saccade targets were located 5° above and below the fixation point, corresponding to far and near choices, respectively. B, Time course of a discrimination trial. The fixation point (FP) first appeared along with the background (Bgnd) dots. After the random dot stereogram was presented for 1.5 sec, the fixation point and dots were extinguished, and two choice targets appeared. Monkeys reported the depth of the stimulus by making a saccade to one of the two targets. C, Manipulation of task difficulty. The strength of the depth signal was adjusted by varying the binocular correlation. At 100% binocular correlation (left) all dots within the receptive field were presented at either the preferred disparity of the neuron (short horizontal line inside gray oval) or the disparity that elicited a minimal response (null disparity). At 50% binocular correlation (middle) one-half of the dots have random disparities, thus forming a 3-D cloud of disparity noise. At 0% correlation (right) all dots are assigned random disparities, making the stimulus ambiguous.
Tasks and behavioral training
Behavioral tasks and data acquisition were controlled by a commercially available software package (Reflective Computing, St. Louis, MO), and on-line data analyses were done with MATLAB (MathWorks, Natick, MA). The positions of both eyes were sampled at 1 kHz and stored at 250 Hz. Monkeys were trained first on a fixation task in which they were required to fixate on a yellow spot (0.15 × 0.15°) within a 1.6 × 1.6° electronic window. Monkeys received a water or juice reward for maintaining fixation throughout a 1.5 sec trial. When the monkey's conjugate eye position left the fixation window prematurely, the trial was aborted immediately, reward was withheld, and a brief time-out period ensued before the next trial. After fixation training the monkeys subsequently were trained on the depth discrimination task (Fig. 1). An RDS containing signal dots at one of two fixed disparities was presented, and the monkeys were required to report whether the signal dots were near (crossed) or far (uncrossed) by making a saccade to one of two targets (located 5° below and above the fixation point, respectively) that appeared 200 msec after offset of the RDS. The saccade had to be made to one of the two targets within 1 sec after their appearance, and the saccade endpoint had to remain within 2.5° of the target for at least 150 msec to be considered a valid choice. Correct responses were rewarded with a drop of water or juice. Discrimination training began with 100% binocular correlation trials, and lower correlations were introduced gradually after monkeys reached at least 75% correct. The range of correlation levels then was pushed downward gradually over many weeks of training until the monkeys' performance reached a plateau and would not improve further. In early stages of training the monkeys often exhibited strong choice biases, choosing one target on most of the discrimination trials. To discourage these biases, we used a staircase procedure in which the stimulus probabilities could be altered on the basis of the recent history of the monkey's choices. A block of staircase trials began with the highest binocular correlation value. After a correct choice the binocular correlation was lowered (usually by one-half) with a probability of a, and the disparity of the signal dots changed sign with a probability of b. After an incorrect choice the binocular correlation increased (usually by a factor of two) with a probability of c, and the disparity changed signs with a probability of d. A typical set of training parameters was {a, b, c, d} = {0.33, 0.6, 0.66, 0.1}. Note that, after an error, there was a large probability (1Electrophysiological recordings
We recorded extracellular activity of single neurons from two monkeys. A tungsten microelectrode (Frederick Haer, Bowdoinham, ME; tip diameter 7-15 µm, impedance 0.2-1 Mat 1 kHz) was advanced into cortex through a transdural guide tube, using a micromanipulator (MO-951C, Narishige, East Meadow, NY) mounted on the recording chamber. Single neurons were isolated by using a conventional amplifier, bandpass filter (500-5000 Hz), and window discriminator (Bak Electronics, Mount Airy, MD). Times of occurrence of action potentials and trial events were stored to disk with 1 msec resolution. Area MT was recognized on the basis of several criteria. First, the patterns of gray and white matter transitions along electrode penetrations, especially the gap between extrastriate visual areas in the anterior bank of the lunate sulcus and MT, were verified. Next the direction, speed, and disparity tuning properties of single units and multiunit clusters, along with the relationship between receptive field size and eccentricity, were measured and identified to be typical of MT responses (see DeAngelis and Newsome, 1999
Experimental protocol
After isolating an MT neuron, we used a custom software interface to map carefully the receptive field and to estimate the preferred direction, speed, and horizontal disparity of the neuron. Next we measured quantitatively the direction, speed, size, and horizontal disparity tuning of each neuron. First, a direction tuning curve was obtained by presenting eight directions of motion, 45° apart. In cases in which the tuning width was unusually narrow, the sampling range was reduced accordingly. Then we measured a speed tuning curve for each neuron after adjusting the stimulus to the preferred direction of the neuron. Typically, we presented speeds of 0, 0.5, 1, 2, 4, 8, 16, and 32°/sec. Next we measured a size tuning (i.e., area summation) curve at the preferred direction and speed of the neuron. In most cases we presented aperture sizes of 1, 2, 4, 8, 16, and 32° diameter. This test was used to determine the smallest stimulus patch that yielded the maximal response and to assay for the presence of surround inhibition. Subsequently, we measured a disparity tuning curve with all of the other parameters optimized. In most cases disparities were tested fromData analysis
Quantitative tuning measurements. For off-line analysis the responses were calculated from the firing rate during the 1.5 sec stimulus presentation period. Spontaneous activity was calculated by using the response to a blank screen. Direction tuning data were fit with a Gaussian of the form:
where R0 is the baseline level of the curve, A is the amplitude,
(1) 0 is the location of the center of the Gaussian (i.e., the preferred direction of the neuron), and
is the SD. We fit this curve to the individual trial responses of the neuron, using the constrained minimization tool, fmincon, in MATLAB. To homogenize the variance of the neural responses across different directions, we minimized the difference between the square root of the neural responses and the square root of the Gaussian (see Prince et al., 2002a
where s is the stimulus speed and R0, A,
(2) ,
, and n are free parameters. The gamma distribution varies in shape from an exponential to a Gaussian depending on the value of the exponent, n. The denominator term normalizes the curve to have amplitude specified by A. This formulation provided excellent fits to speed tuning curves, accounting for 98% (median) of the response variance across the population. Each area summation curve was fit with the integral of a Gaussian (or error function, erf) having the form:
where w is the stimulus size, R0 is the baseline response level, A is the amplitude, and
(3)
where
(4) ) is the size of the inhibitory surround. Note that this formulation constrains the inhibitory region to have a size greater than that of the excitatory center. A sequential F test was used to determine whether the DoE function provided a better fit than the single error function, indicating that the neuron exhibited significant surround inhibition (p < 0.05). If so, the optimal size and percentage of surround inhibition were derived from the DoE fit. Otherwise, the receptive field size was taken as 1.163*
where d is the stimulus disparity, R0 is the baseline response level, A is the amplitude, d0 is the center of the Gaussian envelope,
(5) is the phase of the sinusoid (relative to the center of the Gaussian). Because the disparity frequency, f, often is poorly constrained by the data at the low-frequency end, this parameter was allowed to vary only within ± 10% of the peak of the Fourier transform of the raw tuning curve. We found that this constraint considerably improved the convergence of the optimization (see also Prince et al., 2002a
where Rmax and Rmin are the maximum and minimum responses, respectively. To keep this index restricted to the range from zero to one, we did not subtract spontaneous activity from Rmax or Rmin. Finally, to characterize the ability of the neurons to discriminate between the preferred and null disparities, we used the DDI (see Cumming and DeAngelis, 2001
(6)
where SSE is the sum squared error averaged across disparities, N is the number of observations (trials), and M is the number of disparities tested. This index differs importantly from the DTI in that it takes into account the variability of the neural responses. Both DTI and DDI were computed from the square root of firing rate on each trial. Analogous discrimination indices were computed for direction and speed tuning curves also. Calculation of neuronal thresholds. To characterize the sensitivity of MT neurons in our depth discrimination task, we used ROC analysis to calculate neuronal thresholds on the basis of the anti-neuron formulation used by Britten et al. (1992)
(7)
where c is the binocular correlation of the stimulus, p is the proportion of correct responses,
(8)
Results
Neuronal database
Data were drawn from a sample of 170 MT neurons (94 from monkey B and 76 from monkey J) for which preliminary tests of direction, speed, size, and disparity tuning were completed and for which the preferred speed of motion was <32°/sec. Monkeys were trained for speeds up to this value only because of technical limitations on stimulus generation. We estimate that at most 5-10% of cells were excluded because of speed preference. Of the original 170 neurons, 104 (52 from each monkey) were included into the final database for this report on the basis of the following criteria. (1) Good isolation of the action potential of the neuron had to be maintained through at least 10 repetitions of the discrimination task. Forty-one of 170 neurons were excluded because isolation was lost prematurely or because the monkey ceased working. (2) The neuron had to exhibit some disparity tuning such that we could reasonably define a preferred and null disparity for the discrimination task. This judgment was always made from on-line visual inspection of the disparity tuning curve (plotted with error bars), and only 10 of 170 neurons were excluded because of lack of tuning. Cells were never excluded on the basis of qualitative assessments of tuning. (3) The preferred and null disparities had to have opposite signs (one near, one far), because monkeys were not trained to discriminate between two disparities of the same sign (i.e., the task was always to judge near vs far relative to the plane of fixation). Because most MT neurons have disparity tuning curves with odd symmetry around zero disparity (Cumming and DeAngelis, 2001
Aside from these criteria we recorded from all of the MT neurons that we could isolate, including several neurons with very weak disparity tuning. In fact, post hoc testing showed that 2 of 104 neurons did not have statistically significant disparity selectivity (ANOVA, p > 0.05). Consequently, our selection criteria did not strictly match those of Britten and colleagues, who chose neurons for which the "distribution of response amplitudes evoked by preferred direction motion (100% correlated stimuli) did not overlap with the distribution evoked by null direction motion" (Britten et al., 1992
Receptive field eccentricities ranged from 1.9 to 15.7° (median, 6.3°). Preferred speeds ranged from 0.0 to 32.0°/sec (median, 5.4°/sec), and receptive field sizes ranged from 3.5 to 20.0° (median, 7.5°). Direction preferences were distributed uniformly.
Comparison of neuronal and psychophysical sensitivity
Figure 2 shows data from an individual experiment. This neuron was tuned strongly for near disparities, as shown in Figure 2A. Note that the response to a binocularly uncorrelated stimulus (labeled U) lies approximately midway between the maximum and minimum responses to disparities presented at 100% correlation. By visually inspecting this tuning curve on-line, we chose
View larger version (29K):[in this window]
[in a new window]
Figure 2. Analysis of an example data set. A, Disparity tuning curve for a near-tuned neuron from monkey B. Filled circles show mean responses (± SE) to five repetitions of each disparity. The solid curve is a Gabor fit. Arrowheads denote the preferred (
To quantify neuronal sensitivity, we calculated an ROC curve for each binocular correlation, as shown in Figure 2C (Green and Swets, 1966
ROC values are plotted as a function of binocular correlation in Figure 2D to create a neurometric function (filled circles). These data were fit with a Weibull function (solid curve) to extract an 82% correct threshold and a slope. For this neuron the threshold was 10.7% binocularly correlated dots, and the slope was 1.15. These values now could be compared with the performance of the monkey, which was derived from the psychometric function shown in Figure 2D (open circles, dashed curve). By fitting the behavioral data using identical methods, we obtained a psychophysical threshold of 15.2% and a slope of 1.32. Thus this particular neuron exhibited slightly greater sensitivity (lower threshold) than the monkey, although this difference was not statistically significant (p > 0.05) on the basis of a bootstrap analysis (see Materials and Methods). The difference in slope was also not significant.
Figure 3 shows data for four additional MT neurons that illustrate the range of effects that we observed. Like many MT neurons, those illustrated in Figure 3, A and B, had neurometric functions that closely matched the monkey's psychometric function in both threshold and slope. Other MT neurons had thresholds that were significantly higher (Fig. 3C) or lower (Fig. 3D) than the psychophysical threshold.
View larger version (21K):[in this window]
[in a new window]
Figure 3. Disparity tuning curves (left) and neurometric/psychometric functions (right) for four additional MT neurons. Error bars denote ±SE. A, For this data set the neurometric and psychometric functions were nearly identical. Thresholds were 15.6 and 16.6%; slopes were 1.82 and 1.74. Neither of these differences was significant (p > 0.05). B, Another example in which neurometric and psychometric functions were statistically indistinguishable (p > 0.05). Thresholds were 17.6 and 19.5%; slopes were 2.06 and 2.18. C, An example in which the neuronal threshold (51.0%) was significantly larger than the psychophysical threshold (16.6%; p < 0.05). Slopes of the two curves (2.60 and 1.41) were not significantly different (p > 0.05). D, An extreme example for which the neuronal threshold (9.73%) was significantly smaller than the psychophysical threshold (57.7%; p < 0.05). Slopes of the two curves (1.36 and 1.34) were not significantly different (p > 0.05).
Figure 4 summarizes our results for a population of 104 MT neurons. Figure 4A shows the comparison between neuronal and psychophysical thresholds, whereas Figure 4B shows the comparison between slopes. For more than one-half of the data sets (61 of 104) there was no significant difference between neuronal and psychophysical thresholds (bootstrap, p > 0.05). Among the remainder, 23 of 104 had neuronal thresholds significantly smaller than the corresponding psychophysical thresholds (p < 0.05), and 20 of 104 exhibited neuronal thresholds that significantly exceeded behavioral thresholds. Overall, the neuronal/psychophysical (N/P) threshold ratio was distributed around unity, with a geometric mean of 0.979 (1.03 for monkey B and 0.925 for monkey J). Thus the modal MT neuron matched the performance of the animal. Moreover, 5 of 104 neurons had thresholds lower than the best psychophysical threshold exhibited by either monkey (11.7% correlation). There is a weak, but significant, correlation between neuronal and psychophysical thresholds (r = 0.35; p < 0.001), which we will discuss later. As for the slopes of the neurometric and psychometric functions, Figure 4B shows that these did not differ significantly for most (89 of 104) neurons (p > 0.05). The geometric mean of the N/P slope ratio was 1.16 (0.989 for monkey B and 1.37 for monkey J). These results are similar to those of Britten and colleagues (1992)
View larger version (17K):[in this window]
[in a new window]
Figure 4. Population data from 104 MT neurons (52 from each monkey). A, Comparison of neuronal and psychophysical thresholds. Filled symbols indicate cases in which the neuronal and psychophysical thresholds are significantly different (p < 0.05). Circles and triangles indicate data from monkeys B and J, respectively. The histogram (top right) shows the distribution of neuronal-to-psychophysical threshold ratios. B, Comparison of slopes of neurometric and psychometric functions in the same format as A.
In a comparison of neuronal and psychophysical thresholds there are a number of factors that are not well constrained. Some of these, including the algorithm used by the ideal observer, the size of the neuronal population contributing to decision making, and the contribution of neurons with nonoptimal stimulus preferences, have been considered previously (Shadlen et al., 1996
Effect of trial-to-trial stimulus variations on N/P threshold ratios
In general, both the starting location of dots within the circular aperture and the binocular disparity of noise dots were randomized for each trial (see Materials and Methods). If MT neurons are sensitive to this randomization, it would increase the variance of response distributions for the preferred and null disparities (Fig. 2B) and subsequently increase neuronal thresholds computed by using ROC analysis. To assess the magnitude of these effects, in 61 of 104 experiments we divided the 0% binocular correlation trials into two groups; one group had identical random dot patterns for every trial (NOVAR condition), and the other group had the normal randomization of dot patterns across trials (VAR condition). For each neuron we calculated the mean and variance of the spike count across trials for each of these two conditions.
Figure 5A plots the trial-to-trial variance against mean spike count for each neuron that was tested under the VAR and NOVAR conditions. There was no significant difference in mean spike counts between the two conditions (paired t test, p = 0.91), but the variance was significantly smaller for the NOVAR condition (paired t test, p < 0.0001). Thus MT neurons were sensitive to trial-to-trial variations in the random dot stimuli, most likely because of variations in the mean disparity of noise dots in the VAR condition. We fit separate lines with a slope of 1 to the data for the VAR and NOVAR conditions after confirming that separate slopes for the two lines would not improve the overall fit (sequential F test, p > 0.05). From these fits on log-log scales we calculated the variance-to-mean ratio (VMR; the y-intercept at x = 1) for each condition. The VMR was 1.40 for the NOVAR condition (similar to what other studies have found: Tolhurst et al., 1983
View larger version (14K):[in this window]
[in a new window]
Figure 5. Effects of stimulus variations on neuronal and psychophysical thresholds. A, Trial-to-trial variance in spike count is plotted against mean spike count for 0% binocular correlation conditions. Each of 61 neurons is represented by two data points. Filled circles show data obtained by using an identical random dot pattern for each trial (NOVAR condition); open symbols show data from interleaved trials when the random dot pattern changed from trial to trial (VAR condition). Data for each condition were fit with a unity slope line (solid line for NOVAR condition; dashed line for VAR condition) to estimate the variance-to-mean ratio. B, The variance of response distributions for each data set was scaled down by a factor of 1.51 derived from the analysis shown in A. Simulated neuronal thresholds based on reduced variance (i.e., simulated NOVAR condition) are plotted against the original measured psychophysical thresholds (VAR condition) for all 104 neurons (circles for monkey B; triangles for monkey J). C, Comparison of psychophysical thresholds for VAR and NOVAR conditions. These data were obtained from monkey B after all of the recording experiments were completed.
To estimate the effect of stimulus variations on neuronal sensitivity, we recalculated the neuronal threshold for each MT unit after scaling down the variance of the response distribution (at each binocular correlation and disparity) by the factor of 1.51 (2.11/1.40) derived from the above analysis, while keeping the mean response constant:
Rorig and Rscaled are the responses before and after variance scaling, and Rmean is the mean response for each disparity at each correlation level. Figure 5B shows the recalculated neuronal thresholds plotted against the original values. Variance scaling reduced the threshold for all MT neurons, with the average effect being a reduction of 17.4%. This estimate assumes that trial-to-trial stimulus variation produces the same increase in VMR for all binocular correlations, an assumption that we did not test because of limitations of recording time.
(9) If MT provides critical sensory signals for performance of our task, then the above analysis predicts that psychophysical performance should improve by ~20% under the NOVAR condition. We tested this prediction (after all neurophysiological experiments were completed) by remeasuring the psychophysical thresholds of monkey B under both VAR and NOVAR conditions (randomly interleaved). For the NOVAR condition a fixed random dot pattern was used for each disparity at each binocular correlation such that all repeats of each stimulus condition were identical. Because the simulated neuronal thresholds (Fig. 5B) were calculated under the assumption of variance scaling but no change in mean responses, we forced the noise dots to have zero mean disparity for each NOVAR stimulus. This matches the mean disparity of the NOVAR stimuli to the mean disparity (across repetitions) of the VAR stimuli and thus minimizes changes in the mean response of MT neurons between the two conditions.
We obtained psychophysical thresholds under VAR and NOVAR conditions for all sets of stimulus parameters used in the 52 recording experiments done with monkey B. Figure 5C shows that NOVAR and VAR thresholds are correlated significantly (r = 0.61; p < 0.0001), but the average NOVAR threshold (19.2%) is significantly lower than the average VAR threshold (24.9%) (paired t test, p
0.0001). The average reduction in psychophysical threshold under the NOVAR condition was 22.8%, not far from the 17.4% reduction observed for the simulated neuronal thresholds (Fig. 5B). Thus stimulus variation has similar effects on neuronal and psychophysical thresholds in our task.
Retesting of psychophysical thresholds
If the conditions of the recording experiments interfere with peak performance of the task, this will produce N/P threshold ratios that are artificially low. In a study of V1 neurons during performance of a stereoacuity task, Prince and colleagues (2000)
To address this potential concern, we remeasured psychometric functions by using stimulus parameters that were identical to those of each recording experiment. For the handful of neurons that required a range of binocular correlations larger than the standard range used for each monkey (see Materials and Methods), we retested psychophysical performance by using the standard range. These repeat behavioral measurements were taken after all recording experiments were completed. For monkey B these data were obtained in blocks of trials with interleaved NOVAR conditions, as discussed in the previous section.
A comparison of the original psychophysical thresholds and retested thresholds is shown in Figure 6. For monkey J there was a modest 14% reduction in the average psychophysical threshold after retesting, and this difference was significant (paired t test, p < 0.001). In contrast, the average psychophysical threshold for monkey B increased by 11% during retesting, although this difference was only marginally significant (paired t test, p = 0.02). Combined across the two animals, there is a fairly strong correlation between original and retested psychophysical thresholds (r = 0.62; p
View larger version (23K):[in this window]
[in a new window]
Figure 6. Behavioral retesting of psychophysical thresholds. Using stimulus parameters identical to those for each recording session, we remeasured psychophysical thresholds in separate behavioral experiments. Retested psychophysical thresholds are plotted against the original thresholds for both monkey B (filled circles) and monkey J (filled triangles).
The data of Figure 6 suggest that the psychophysical performance of both monkeys had reached a stable plateau when recording sessions commenced, and this is confirmed by the fact that there was no significant trend for psychophysical thresholds to decline across recording sessions for either monkey (monkey B: r = 0.09, p = 0.52; monkey J: r = 0.14, p = 0.32). There are two likely reasons why the improvement in psychophysical performance after retesting was much smaller than the 61% reduction seen by Prince et al. (2000)
When we use retested psychophysical data, the average N/P threshold ratio for monkey J increases from 0.93 to 1.07, and the average N/P threshold ratio for monkey B decreases from 1.03 to 0.93. Combined across monkeys, the average N/P threshold ratio is 1.001 when retested psychophysical thresholds are used. Thus variations in behavioral performance had little influence on our estimates of N/P threshold ratios.
Effect of integration time on neuronal thresholds
Another important issue to consider when comparing neuronal and psychophysical thresholds is the point in time at which the animal reaches his decision during a trial. In all of our physiology experiments the visual stimuli were presented for 1.5 sec, and firing rates were computed over this entire interval. Although the monkeys were not allowed to indicate their decision until the end of the trial (ours was not a reaction time task), monkeys may have reached a decision much sooner and subsequently ignored the visual stimulus. If so, our estimates of N/P threshold ratios would be too low because the monkey would be integrating neuronal activity over less time than the ideal observer (ROC). Because we do not know the monkey's decision time in the simultaneous behavioral/physiological experiments, we first addressed this issue by computing neuronal thresholds over a range of integration times.
For each MT neuron we calculated neuronal thresholds by counting spikes over a variable epoch beginning at stimulus onset and ending at times ranging from 200 to 1500 msec after stimulus onset (in 100 msec steps). We could not calculate thresholds reliably at short integration times for insensitive MT neurons. Thus we restricted this analysis to neurons having a threshold (using the full 1500 msec integration window) less than one-half of the largest binocular correlation used in the measurements. Figure 7A shows neuronal thresholds as a function of integration time for the 75 neurons that met this criterion. Not surprisingly, neuronal thresholds generally decreased with integration time, but the trend was more gradual than we had expected. Strikingly, many MT neurons, including some of the most sensitive units, showed thresholds that were approximately independent of integration time. Data for nine such neurons are shown in Figure 7B; several others are not shown to avoid overcrowding. Summary data for the population of 75 neurons are shown in Figure 8. For each neuron we normalized all thresholds to the value obtained by using a 1500 msec integration window. The thick solid line in Figure 8 shows the median normalized threshold versus integration time, and the thinner solid lines indicate the 25th and 75th percentiles. For a 200 msec integration time the median threshold rises by 83% relative to that at 1500 msec; at 500 msec the median threshold is elevated by only 28%.
View larger version (23K):[in this window]
[in a new window]
Figure 7. Analysis of neuronal thresholds as a function of integration time. A, Neuronal thresholds were computed over a range of time windows (abscissa) beginning at stimulus onset. Each curve plots the change in threshold of an individual MT neuron as a function of the length of this analysis window (integration time). Data from 29 of 104 neurons with high thresholds, using a 1500 msec integration window, were excluded from this analysis. B, Integration time curves are shown for nine MT neurons for which there was no clear change in neuronal threshold with integration time. C, Simulated neuronal thresholds, based on a Poisson model (see Results), are plotted as a function of integration time for the same 75 neurons.
View larger version (21K):[in this window]
[in a new window]
Figure 8. Summary curves for integration time analysis. Thresholds for each neuron were normalized by dividing by the threshold obtained at an integration time of 1500 msec. The thick solid curve shows the median normalized threshold as a function of integration time for the MT responses, with thin solid curves indicating the 25th and 75th percentiles. The thick dashed curve shows the median normalized threshold for the Poisson simulations, with thin dashed curves indicating the 25th and 75th percentiles.
The shallow decline in neuronal threshold with integration time for many of our MT neurons appears at odds with similar data reported by Britten et al. (1992)
To assess whether the integration time behavior of MT neurons differs from the Poisson expectation, we performed the simulations of Figure 7C. For each trial of each data set we generated spike trains by using an inhomogeneous Poisson process. We first generated a poststimulus time histogram (PSTH) with 1 msec resolution for each binocular correlation at each disparity and then smoothed it by using a boxcar filter with a width of 20 msec. Spikes were generated randomly for each 1 msec bin, with a probability determined from the smoothed PSTHs. These simulated spike trains have firing rate variations that match the real data, have a fixed VMR equal to 1, and have random local structure. We then performed ROC analysis on the synthesized Poisson spike trains in an identical manner to that used for the real spike trains. Figure 7C shows the results of this simulation for each of the 75 neurons that were analyzed in Figure 7A. Clearly, the simulated neuronal thresholds decline more steeply with integration time than those of the actual neurons, and none of the simulated data sets showed an integration time curve that was flat. The median normalized threshold versus integration time for the simulations is shown by the thick dashed curve in Figure 8, along with the 25th and 75th percentiles for the simulations (thin dashed lines). For a 200 msec integration time the median threshold rises by 227% relative to 1500 msec integration time; at 500 msec the median threshold is elevated by 92%.
What factors might allow MT neurons to have such a diminished dependence on integration time relative to the Poisson expectation? One possibility is that the variability of MT responses is not fixed throughout the trial epoch. If responses are more reliable in the early part of the trial, this could flatten the relationship between neuronal threshold and integration time. To address this possibility, we computed the VMR of each neuron within consecutive 200 msec time windows spanning the trial epoch. Each VMR was obtained by computing the mean and variance across trials for each different correlation level and disparity; then these data were plotted on log-log scales and fit with a unity slope line (as in Fig. 5A). Figure 9A (filled circles) shows the average VMR as a function of time for the same 75 neurons that were analyzed in Figures 7 and 8. There is a significant increase in VMR over the first 500 msec of the trial (ANOVA, p = 0.028), with the average VMR increasing by 26%.
View larger version (29K):[in this window]
[in a new window]
Figure 9. Factors that contribute to the shallow integration time curves of MT neurons. A, Time course of the average variance-to-mean ratio (VMR), disparity signal, and noise correlation. Filled circles and dashed curve show the average VMR (± SE) calculated in successive 200 msec time windows during stimulus presentation. Solid curve shows the time course of the normalized difference in response (± SE) between preferred and null disparities (derived from the highest binocular correlation). For each unit the responses were normalized by the peak response to the preferred disparity; then the difference curves were averaged across neurons (in 20 msec bins). Open circles and short dashed curve show the average correlation (± SE) between z-scored firing rates for neighboring 200 msec time bins (for details, see Results). B, Changes in neuronal sensitivity with integration time are correlated positively with temporal variations in the VMR. The ordinate is the neuronal threshold, which is calculated as the threshold for an integration time of 300 msec (0-300 msec from stimulus onset) divided by the threshold normalized for the entire trial (0-1500 msec). The abscissa is the VMR computed from the early portion of the response (100-300 msec after stimulus onset) divided by that computed from the later portion of the response (1300-1500 msec after onset). Circles and triangles indicate data from monkeys B and J, respectively. C, Changes in neuronal sensitivity with integration time are correlated negatively with changes in the differential response rate. The abscissa is now the preferred-null response difference from the early portion of the trial (100-300 msec) divided by the response difference from the late period (1300-1500 msec). D, Changes in neuronal sensitivity with integration time are related inversely to the strength of noise correlations. The abscissa is the average noise correlation between neighboring 200 msec time bins for each neuron.
If changes in VMR contribute to flattening of integration time curves, neurons with the flattest integration time curves should show the largest increases in VMR and vice versa. Figure 9B shows that this expectation was confirmed. To construct this scatterplot, we divided the neuronal threshold for a 300 msec integration time by the threshold for a 1500 msec integration time (as in Fig. 8). This normalized threshold is plotted against the ratio of VMR between the early (100-300 msec) and late (1300-1500 msec) portions of the response. There is a significant positive correlation (r = 0.31; p = 0.0079) such that neurons with flat integration time curves (values near unity on the ordinate) tend to have the largest increases in VMR during the trial epoch. This confirms that changes in VMR over time do contribute to flattening the integration time curves of MT neurons.
Another factor that could flatten the integration time curves of MT neurons is a change in the differential disparity signal (preferred-null) during the time course of the response. If the mean response difference between preferred and null disparities declines over time, neurons would be more sensitive in the early portion of the trial. Because the firing rate variations of MT neurons were modeled accurately in our Poisson simulations (Fig. 7C), this factor cannot account for the difference seen in Figure 8. To shed further light on this, Figure 9A (solid curve) shows the time course of the preferred-null response difference averaged across the population of 75 neurons. At the population level there is indeed no significant change in the disparity signal after 100 msec following stimulus onset (ANOVA, p = 0.99). Thus changes in the disparity signal over time do not contribute to flattening of integration time curves, on average. Nevertheless, we find that changes in the disparity signal over time do explain variability in the slope of the integration time curve from neuron to neuron. Figure 9C shows the normalized neuronal thresholds plotted against the ratio of preferred-null response differences in the early (100-300 msec) versus late (1300-1500 msec) segments of the trial. There is a significant negative correlation between these variables (r =
A third factor that may contribute to flat integration time curves is the statistical dependence of firing rates between one brief time period and the next. For a Poisson process the number of spikes that occur within one short epoch of a trial is not correlated with the number of spikes that occur within the next epoch. As a result, counting spikes over a longer period of time yields a better estimate of the true mean firing rate and thus a lower neuronal threshold (Fig. 7C). If trial-to-trial variations in spike counts are correlated between neighboring epochs, however, then the expected improvement in neuronal threshold with integration time will be reduced substantially. To examine this possibility, we again divided the trial into seven nonoverlapping 200 msec time bins. Within each bin the responses were z-scored and combined across binocular correlations and disparities. For each pair of neighboring bins we then computed the correlation coefficient (across trials) between the normalized responses. Figure 9A (open circles) shows the average noise correlation as a function of time for 75 MT neurons. The correlation is constant throughout the trial (ANOVA, p = 0.93), consistent with the fact that integration time curves of MT neurons are flatter than Poisson simulations at every point within the trial epoch (Fig. 8).
To obtain a single noise correlation value for each neuron, we averaged the correlation values across all pairs of neighboring time bins. Figure 9D shows that there is a significant inverse correlation between normalized neuronal thresholds and overall noise correlation (r =
In principle, positive noise correlations could be either stimulus-driven or intrinsic to neuronal connectivity in MT. To assess this, we compared data from the VAR and NOVAR conditions at 0% correlation, because the NOVAR condition removes stimulus variability. The average noise correlation is significantly larger for VAR (0.29) than NOVAR (0.16) trials (paired t test, p
Effect of integration time on psychophysical thresholds
If MT neurons underlie performance of our depth discrimination task, then the results of Figures 7 and 8 suggest that psychophysical thresholds should exhibit little dependence on how long the monkey scrutinizes the visual stimulus. Because we cannot know when the monkey reached his decision during the 1.5 sec trials used in the recording sessions, we performed additional psychophysical experiments to measure how behavioral thresholds depend on stimulus duration. These tests were performed on monkey B as well as another animal (monkey R) that was not used in the recording experiments (monkey J was engaged in other studies and could not be used for these additional tests). We used a fixed set of stimulus conditions (eccentricity, 5.5°; direction, 0°; speed, 7°/sec; size, 8°; disparity, ±0.5°) that were selected by averaging the stimulus parameters across the 52 recording experiments performed with monkey B. Psychophysical thresholds were determined by using a staircase procedure, as done by Britten et al. (1992)
Figure 10 shows average psychophysical thresholds as a function of viewing duration for monkeys B and R. Each datum is the average (± SE) across numerous identical blocks of 420 trials/block (20 blocks for monkey B and 14 blocks for monkey R). Surprisingly, there is no significant dependence of psychophysical threshold on viewing duration for either monkey (ANOVA; monkey B, p = 0.85; monkey R, p = 0.62), indicating that sufficient information was available to the monkeys in the first 200 msec of the trial for adequate task performance. In contrast, Britten and colleagues (1992)
View larger version (16K):[in this window]
[in a new window]
<
