Abstract
In the macaque extrastriate cortex, robust correlations between perceptual choice and neuronal response have been demonstrated, frequently quantified as choice probabilities (CPs). Such correlations are modest in early visual cortex, suggesting that CPs may depend on the position of a neuron in the hierarchy of visual processing. However, previous studies have not compared neurons with similar precision in equivalent tasks.
We investigated the role of cortical hierarchy on CP using a task for which significant CPs have been described previously for middle temporal area (MT). We measured CPs in disparity-selective neurons from both V1 and V2. The stimulus was a dynamic random dot stereogram, presented with a near or a far disparity, masked by varying numbers of binocularly uncorrelated dots. Two macaque monkeys reported whether they perceived a circular patch in front or behind a surrounding annulus in a forced choice task.
For V2 (n = 69), CP was on average 0.56, the first demonstration of systematic CPs in a visual area as early as V2. In V1 (n = 74), average CP was at chance level (0.51). The pattern was similar in a subgroup of neurons selected such that the statistical precision in the task was on average identical to that reported for MT (mean CP, 0.51 for V1, n = 33; 0.58 for V2, n = 54). This difference between V1 and V2 could not be explained by eye movements, stimulus size relative to the receptive field, or differences in disparity tuning. Rather, it seems to reflect a functional difference (at least in disparity processing) between striate and extrastriate cortex.
Introduction
Single-unit recording combined with threshold psychophysics in awake animals has revealed trial-to-trial correlations between neuronal firing and perceptual reports, not explained by the visual stimulus. These observations may reflect a causal link between the activity of single neurons and sensory decisions. Several groups quantified these correlations with choice probabilities (CPs), allowing comparison across cortical areas (Celebrini and Newsome, 1994; Britten et al., 1996; Uka et al., 2005) and visual tasks (Britten et al., 1996; Dodd et al., 2001; Uka and DeAngelis, 2004; Purushothaman and Bradley, 2005; Uka et al., 2005).
Such comparisons suggest that one important prerequisite for observing significant CPs is that the neurons carry signals sufficiently reliable for the task. Several studies found systematic correlations between CP and the precision of task-relevant information (Celebrini and Newsome, 1994; Britten et al., 1996; Parker et al., 2002; Uka and DeAngelis, 2004). These observations are readily explained as the result of stochastic variation in neuronal firing contributing to the animals' decisions, if one assumes appropriate correlations between neurons in the pool (Shadlen et al., 1996). Other observations are less easily explained in this scheme: Even within the middle temporal area (MT), there is substantial variation in the size of CPs observed for different tasks despite comparable precision of neuronal signals (Britten et al., 1996; Parker et al., 2002; Uka and DeAngelis, 2004), and similar CPs are observed in tasks for which there are very substantial differences in neuronal precision (Britten et al., 1996; Purushothaman and Bradley, 2005). One comparison of two stereo tasks found that a modest reduction in neuronal precision for one task was associated with the loss of significant CPs (Uka and DeAngelis, 2003b).
One way to account for these differences is to suggest that a second important factor determining CP is the location of a cortical area in the visual hierarchy that leads to perception (in the particular task): areas closer to the stage at which decisions are formed may show larger CPs. This could explain why CPs for motion discrimination are larger in MST (medial superior temporal area) than MT (Celebrini and Newsome, 1994; Britten et al., 1996), and a similar metric devised for detection tasks is larger in VIP (ventral intraparietal area) than MT (Cook and Maunsell, 2002b). This might also account for why correlations between neuronal firing and choice are modest or absent in V1 (Leopold and Logothetis, 1996; Grunewald et al., 2002). However, in both V1 studies, interpretation is complicated: V1/V2 responses were pooled (Leopold and Logothetis, 1996) or neuronal precision differed between V1 and extrastriate cortex (Grunewald et al. 2002).
To examine the relationship between cortical hierarchy and choice probability, we studied responses of neurons in V1 and V2 in a task (disparity discrimination in weakly correlated random dot stereograms) for which substantial CPs have already been demonstrated in MT. The similarity of the disparity signals in V1, V2, and MT means that responses of neurons with nearly identical precision to equivalent stimuli can be used to evaluate how CP depends on the location of a cortical area in the hierarchy of visual processing in a particular visual task.
Materials and Methods
Animals.
We examined responses of disparity tuned units in visual areas V1 and V2 in two male macaque monkeys (Macaca mulatta). Both animals were implanted with scleral search coils in both eyes (Judge et al., 1980), head fixation posts, and a recording chamber over the operculum of V1 under general anesthesia. A detailed description of the methods is given by Cumming and Parker (1999). All procedures were in agreement with the Public Health Service policy on the humane care and use of laboratory animals, and all protocols were approved by the Institute Animal Care and Use Committee.
Recording.
We recorded extracellular activity from single units using glass-coated platinum–iridium electrodes (FHC, Bowdoinham, ME) or tungsten in glass electrodes (Alpha Omega, Nazareth, Israel). The signal from the electrode was amplified (Bak Electronics, Mount Airy, MD), filtered (200 Hz to 2 kHz), digitized (32 kHz), and stored to disk. Unit isolation was rechecked off-line using software developed in the laboratory. All spike waveforms whose peak voltage exceeded a threshold value, but that did not fall inside the single-unit cluster were counted as multiunit spikes.
Electrodes were introduced through the dura into the operculum of V1 on each day of experiments using a custom-made microdrive. For recordings in V1, we waited at this point for ∼30 min before advancing the electrode and starting to isolate units. For V2 recordings, we plotted the V1 minimum response field by hand using a bright bar at approximately preferred orientation. We then moved the electrode through the entire gray matter of V1 until it entered the white matter separating the deep layers of V1 and V2. At this transition, we did not move the electrode for ∼60 min. Subsequently, we advanced it further and started searching for units. Units were identified as being in V2 if the electrode had clearly passed through a layer of white matter, if the location of the minimum response field had shifted laterally with respect to the location we had initially recorded in V1, and if the size of the receptive field was larger than the size of the V1 minimum response field. We also checked that the locations of V2 receptive fields across all penetrations formed a consistent topographic map. On isolating a single unit, we determined the receptive field position (see below) and centered all subsequent stimuli over the receptive field center.
The horizontal and vertical positions of both eyes were measured with a magnetic scleral search coil system (C-N-C Engineering, Seattle, WA) and digitized at 800 Hz. For much of the study, fixation was required to remain within 1° of the target, but later the vertical limit was reduced (to as little as 0.1°) to prevent animals from making small vertical saccades before making their choice-related saccades.
Stimulus presentation.
Stimuli were generated by a Silicon Graphics (Mountain View, CA) workstation at a mean luminance of 42 cd · m−2, maximum contrast of 99%, and at a frame rate of 96 Hz, and presented on two EIZO (Ishikawa, Japan) Flexscan F980 monitors at a viewing distance of 89 cm. The monkeys viewed the stimuli through two small mirrors positioned at 45° ∼1.5 cm in front of their eyes (Wheatstone stereoscope).
All stimuli were dynamic random dot stereograms (RDSs) (50% black and 50% white dots of 99% contrast; dot density, generally 40%; dot size, generally 0.09 × 0.09°). In a few cases, dot density or dot size was altered to improve response rates. A new dot pattern was presented on each video frame. Stimulus duration was 2 s when the animal was performing a psychophysical task. For all other experiments, four stimuli lasting 420 ms, separated by 100 ms, were presented on each 2 s trial.
Measurements of disparity selectivity.
To measure selectivity for disparity, each RDS consisted of a central circular patch whose disparity varied pseudorandomly from trial to trial, and of a surrounding annulus at zero disparity. The width of the surrounding annulus (1.5–2°) always had to exceed the largest absolute disparity value applied to the central patch. In this way, we avoided monocularly detectable changes in the stimulus, and vergence eye movements by the monkeys were reduced. The size of the central patch of the RDS was chosen to cover the entire receptive field. Its diameter was 2° for V1 units, and 2–5° (mean, 3.9°) for V2 units.
Measurement of receptive field size.
For most V2 and some V1 neurons, we quantified the size of the minimum response fields by measuring responses to thin stimulus patches as a function of position (Read and Cumming, 2003). If the stimulus was an RDS strip (as it usually was for V2 cells), we measured responses as a function of horizontal and vertical position. If the stimulus was a grating patch, the grating was at the preferred orientation, and the displacements were applied orthogonal to or parallel to this axis. For V2, the patch size was typically 0.25 × 6° or 0.5 × 6°, whereas for V1 this was 0.12 × 6° or 0.25 × 6°.
Task.
If a unit showed selectivity for disparity, we recorded its activity while the monkey performed a disparity discrimination task (“near” vs “far”). Stimuli were circular dynamic RDS with most parameters (annulus width, diameter, dot size, dot density, contrast) identical to those used for the previously recorded disparity tuning curve. On each trial, the central patch contained one of two disparities (one near, one far disparity), close to the preferred and null disparities of the unit. The disparity values were not necessarily symmetrical about zero, but one disparity always had to be negative and the other always positive. The percentage of the dots correlated between the two monocular images varied from trial to trial (typical percentages were 0, 6.25, 12.5, 25, and 50%). The dots in the surrounding annulus were always 100% correlated, at zero disparity. Thus, although the formal structure of the psychophysical task was a discrimination task, the threshold we measure is a correlation threshold for detecting the disparities.
In the initial experiments, the disparity of the central patch was applied as a horizontal shift to the monocular images (equal and opposite in the two eyes). This meant that dots on one side of the central patch were shifted into the region of the surrounding annulus of zero disparity, thereby replacing the dots of the annulus in this region of overlap. As a consequence, the width of the surrounding annulus along the horizontal axis was reduced by one-half the absolute value of disparity added to the central patch. For experiments in which the negative and positive disparities were asymmetrical about zero, the surrounding annulus was therefore different between near and far disparity stimuli. (In total, 14 experiments were done this way, with disparities asymmetrical.) In principle, this might allow the animal to identify the nominal disparity, even when the central part of the stimulus was uncorrelated. Although the animals' psychophysical data did not indicate that they were doing this, we modified the task in later experiments to avoid this possibility. The aperture formed by the surround patch was fixed, and dots in the central region were displaced as if behind an occluder.
Animal training.
Before training for the task of the present study, both monkeys had been fully trained on a stereoacuity task, similar to that described previously (Prince et al., 2000), and had been psychophysical subjects in a study of stereoacuity as a function of interocular delay (Read and Cumming, 2005). The monkeys viewed the RDS stimulus while fixating on a 0.2 × 0.2° fixation square. After the 2 s stimulus presentation, they indicated a forced-choice decision as to whether the central circular region of the RDS appeared in front or behind its surrounding annulus (zero disparity, 100% correlation). The report was made by a vertical saccade to one of two targets (3° above or below the fixation marker, lower target indicating near), within 500 ms of the stimulus disappearance. Task difficulty was controlled by varying the percentage of the dots in the central region of the RDS that were correlated between the two monocular images. From the beginning of training for this task, both monkeys performed >90% correct for 100% correlation stimuli at disparity values symmetrical about zero. Training for the present task started by choosing disparity values well above threshold and symmetrical about zero, and by adding stimuli of lower interocular correlation. Stimuli of different disparities and correlation values were presented pseudorandomly using the method of constant stimuli. Once performance reached a plateau, we introduced disparity values asymmetrical about zero. Both monkeys had a bias during training. A modified staircase procedure identical to that described by Uka and DeAngelis (2003a) helped to overcome these biases effectively. We subsequently trained monkeys at various stimulus positions (eccentricities, ∼2.5–7°), stimulus sizes (center size, 2–5°), disparities, and disparity asymmetries. This staircase procedure was only used for training, not during neuronal recordings.
Analysis of disparity selectivity.
Entry criteria into this study for all neurons were that their responses were significantly modulated by the disparity in the RDS stimulus (1% level on a one-way ANOVA with main effect for disparity) and that they fired at least 10 spikes/s at the preferred disparity. The strength of disparity selectivity was quantified with the disparity discrimination index (DDI) (Prince et al., 2002), which contrasts the amplitude of the response modulation with its variability as follows:
where Rmax and Rmin are the highest and lowest
Analysis of choice probabilities.
Trial-to-trial correlations between neuronal firing and perceptual choice were quantified as “choice probabilities” (Britten et al., 1996) based on signal detection theory. For each disparity and level of correlation, firing rates (mean firing rate during the entire 2 s stimulus presentation) were divided into two groups: those for which the monkeys signaled the target to be in front of the surround (“near choices”) or behind the surround (“far choices”). From these two distributions of firing rates, we calculated a receiver-operating-characteristic (ROC) curve. The choice probability for this disparity and level of binocular correlation was defined as the area under the ROC curve. Data for correlation levels of 25% and below and for which the monkey had made at least five near choices and five far choices were included in the “grand choice probability.” For this, the firing rates at each disparity and correlation level were first normalized to remove the stimulus evoked mean response and variance (responses were z-scored by subtracting the mean and dividing by the SD of the firing rates at the respective disparity and correlation level). The z-corrected responses were then pooled and the CP calculated from this pool gave the grand choice probability. In a permutation test (Britten et al., 1996; Uka and DeAngelis, 2003a), we determined whether the correlation of firing with choice was significantly different from chance (CP significantly different from 0.5). For each of the 1000 permutations, firing rates were randomly assigned to the choices (while maintaining the distributions of both firing rates and choices), and a CP was calculated for these permuted values. These permuted CPs correspond to the distribution of CPs that would have arisen by chance given the distributions of choices and of firing rates. Choice probabilities that lay outside the 95% interval of the distribution of permuted CPs were considered significant.
Analysis of neurometric and psychometric functions.
Psychometric curves measured the percentage of near choices as a function of signed binocular correlation, where negative and positive correlation values corresponded to those trials for which the signal dots in the stimulus were at a near or far disparity, respectively. We fitted these curves with cumulative Gaussians (estimated by maximum likelihood), and used the SD as our estimate of threshold (which corresponds to the 84% correct level). The mean of these cumulative Gaussian functions was a free parameter in the fit to measure the monkey's bias. (Mean |behavioral biases| in all experiments were 6.1% binocular correlation for monkey ruf and 6.0% binocular correlation for monkey duf.) Importantly, allowing this mean to be fit has the effect that small biases in the monkey reports do not affect our estimate of threshold. For comparison with neuronal data, the responses can be replotted in terms of percentage of correct responses for each unsigned correlation value (see Fig. 1), although such plots obscure any response bias. Neuronal thresholds were calculated based on ROC analysis (applied to mean firing rates during the entire 2 s trial) using a “neuron–antineuron” formulation (Britten et al., 1996; Uka and DeAngelis, 2003a). (This formulation assumes a theoretical “antineuron” with opposite disparity tuning, but otherwise identical responses to that of the actual neuron recorded.) For each level of binocular (nonzero) correlation, firing rate distributions in response to the preferred disparity were compared with those in response to the null disparity (representing the responses of the antineuron) by constructing ROC curves. The area under the ROC curve then corresponds to the percentage of stimuli an ideal observer would correctly discriminate based on the firing rate of each trial, given previous knowledge of the distributions of firing rates. These results can then be expressed in a way that is identical to the psychometric function: the percentage of near responses, as a function of signed correlation, and then the threshold can be calculated with a cumulative Gaussian fit in the same way. Expressed in this way, the neurometric data are inevitably symmetrical about zero, so the mean of the cumulative Gaussian is always zero. Neuronal thresholds were defined as the SD of the cumulative Gaussian function. Only data pairs for which both cumulative Gaussian fits accounted for at least 65% of the variance were included in the comparison of neuronal and psychophysical thresholds.
Interneuronal correlation.
As an estimate of the noise correlation between neurons (also referred to as spike count correlation) (Bair et al., 2001; Kohn and Smith, 2005), we analyzed the noise correlation between the single-unit and the multiunit activity for the experiments during which the monkeys performed the task. To remove the effect of slow fluctuations in the mean firing rate, we first removed slow trends in rate from both the single-unit and the multiunit activity. For all responses to a given stimulus type, we subtracted the running mean of four consecutive trials to this stimulus type (excluding the response to the current trial). We then z-corrected all single and multiunit responses and calculated the interneuronal noise correlation as the correlation coefficient between these z-scores. Interneuronal signal correlation was calculated using the Pearson correlation coefficient between the disparity tuning curve for the multiunit and the single-unit activity, analogous to Bair et al. (2001).
Results
We recorded from 195 and 190 single units in V1 and V2, respectively, 77 and 82 from monkey duf, and 118 and 108 from monkey ruf. For V1, 127 of 195 (65%), and for V2, 114 of 190 (60%) units were significantly tuned for disparity (1% level on a one-way ANOVA with main effect for disparity) and fired >10 spikes/s at the preferred disparity. Note that, for V2, disparity selectivity was frequent in some penetration locations and missing in others, consistent with the observation that disparity selectivity is compartmentalized in V2 and mainly found in the thick cytochrome oxidase stripes (Roe and Ts'o, 1995; Ts'o et al., 2001). The population statistics given for V2 should therefore not be interpreted as representing an unbiased sample.
For the disparity-selective neurons, we analyzed neuronal responses recorded while the monkeys performed a disparity discrimination task in weakly correlated RDS. Behavioral inclusion criteria for all subsequent analyses were that the performance exceeded 85% correct at one of the highest correlation levels for both disparities and that the monkeys did not choose one target >75% of the time in response to the uncorrelated stimulus. For each dataset to be included, we also required at least five stimulus presentations each for both disparities at the lowest three correlation levels. (Seventy-four V1 neurons and 69 V2 met these criteria.) We first examine the precision of the neuronal signals related to the stimulus disparity and compare this with the simultaneously measured psychophysical performance.
Statistical precision of V1 and V2 neurons for signaling the disparity in the stimulus
We gathered psychophysical data while recording from disparity-selective neurons, and fit the psychometric curves (percent near choice as a function of signed binocular correlation) with cumulative Gaussian functions. The mean of the cumulative Gaussian fit estimates the animal's response bias, and the SD of the fit measures the animal's threshold, in a way that is unaffected by response bias. This threshold is the correlation level for which the monkey could discriminate 84% of the stimuli if he had no bias. These data are replotted in terms of percentage of correct reports in Figure 1A (filled squares; data collected while recording from a V1 neuron), for comparison with the neuronal data. Although such plots obscure any response bias, it is important to note that our quantitative analysis is unaffected by this. Figure 1B plots the mean responses of a disparity-selective V1 neuron as a function of binocular correlation. The two curves represent trials on which the signal disparity corresponded to the preferred (squares; solid lines) or null disparity (circles; dotted lines) of the neuron, respectively. For the correlation level of 6.25%, the mean response for the preferred and null disparity are similar, whereas for correlations >12.5% the responses are well separated. To quantify how reliably the neuron discriminated between the two disparities without making any assumptions about the distribution of the neuronal firing rates, we used an ROC analysis identical to that in previous studies (Newsome et al., 1989; Britten et al., 1992; Uka and DeAngelis, 2003a). Separately for each value of binocular correlation, we plotted an ROC curve from the distributions of firing rates in response to the null disparity and the preferred disparity stimulus. The area under the ROC curve then corresponds to the probability with which an ideal observer knowing the distributions of firing rates would be able to differentiate the two stimuli from the response on a given trial. For the uncorrelated stimulus, the distributions are identical (because the stimulus is identical) and the probability of a correct response is by definition 50%. The probability correct derived from the spike counts calculated in this way is plotted as a function of binocular correlation in Figure 1A (open circles). The data are fitted with a cumulative Gaussian function in the same way as the psychometric data and neuronal threshold is defined as the SD of this fit. For the neuron in Figure 1, the neuronal threshold is 16% binocular correlation, which is similar to the psychophysical threshold during that experiment, 17% binocular correlation.
Calculating the neuronal and psychophysical threshold. In A, the statistical reliability (expressed as percentage correct) of an example V1 neuron (see Materials and Methods and Results) for discriminating between the preferred and the null disparity is plotted as a function of percentage binocular correlation (% correlation, logarithmic scale; open circles). Superimposed is the psychometric curve from the same experiment (filled squares). Both curves are fitted with cumulative Gaussian functions (solid line for neurometric; dotted line for psychometric curve). The neuronal (16%) and psychophysical (17%) thresholds are defined as the SD of the cumulative Gaussian (see Materials and Methods). B shows the mean responses (in spikes per second; error bars depict SEs) of the same V1 neuron as a function of the level of binocular correlation (% correlation). Responses to stimuli containing the preferred and null disparity are represented by the solid line (squares) and dashed line (circles), respectively.
A previous study using an identical task compared neuronal thresholds in disparity-selective MT neurons and simultaneously measured psychophysical thresholds (Uka and DeAngelis, 2003a). These authors found a mean ratio of neuronal over psychophysical threshold of unity. Figure 2examines this relationship for those neurons where our quantitative measures were reliable (59 of 74 V1 neurons and 58 of 69 V2 neurons, for which the cumulative Gaussian functions to both the neurometric and psychometric curves explained >65% of the variance). Figure 2, A and B, plots the psychophysical threshold against the neuronal threshold for V1 and V2, respectively. (Circles correspond to data from monkey duf, and squares to data from monkey ruf.) The diagonal represents unity. The mean neuronal threshold in V1 is 50 ± 26% (SD) and 40 ± 19% in V2, and the mean psychophysical thresholds for the experiments in both areas are 31 ± 11% (V1) and 34 ± 11% (V2). These thresholds are somewhat larger that those reported previously by Uka et al. (2003a). We believe that this is mainly a consequence of the smaller stimulus size used in our experiments. The neuronal and psychophysical thresholds were significantly correlated for V2 (Spearman's rank correlation, rs = 0.44, p < 0.01 for V2; for comparison, rs = 0.19, p = 0.15 for V1). This correlation is primarily attributable to interanimal differences (monkey ruf had lower neuronal and psychophysical thresholds) and stimulus parameters such as eccentricity and the value of the stimulus disparity (for V2, the correlation between neuronal threshold or psychophysical threshold and the larger absolute value of the two stimulus disparities is rs = 0.29, p < 0.05, and rs = 0.43, p < 0.001, respectively; for V1, these correlations are rs = 0.32, p < 0.05, and rs = 0.30, p < 0.05). Consequently, when correcting for the stimulus- and animal-dependent correlation, the partial correlation between neuronal and psychophysical threshold reduces to 0.26 (p > 0.05) for V2 (partial correlation, 0.09 for V1; p > 0.5).
Comparing neuronal and psychophysical thresholds. Data from V1 (n = 59) and V2 (n = 58) are drawn in A and B, respectively. The circles correspond to data from monkey duf, and the squares correspond to data from monkey ruf. Neuronal thresholds on the abscissa are plotted against the psychophysical threshold on the ordinate (both in percentage binocular correlation). The diagonal (solid line) is the identity line. Frequency histograms of the ratio (neuronal threshold)/(psychophysical threshold) are represented in the top right of each scatterplot. The mean N/P ratio (μ) for V1 is 1.51, and is 1.05 for V2 (open triangles). The filled symbols and filled bars in the histogram correspond to the subpopulation of neurons whose mean N/P ratio (μ) equals that for MT neurons (1.00 for V1 and 0.99 for V2; filled triangles).
More relevant for the current purpose is that the V1 and V2 data are approximately scattered around the unity line, suggesting that a substantial fraction of the neurons provided signals about the stimulus sufficiently precise to account for the behavioral performance. The histograms in the upper right of each scatterplot depict the distribution of the ratio of neuronal over psychophysical threshold (N/P ratio). The mean N/P ratio is 1.51 in V1 and 1.05 in V2. These values are similar to the mean N/P ratio reported previously in MT (0.97) (Uka and DeAngelis, 2003a). The central goal of the current study was to examine trial-to-trial correlations between neuronal firing and perceptual choice in populations of neurons in V1 and V2, which were just as useful for solving this task as disparity-selective MT neurons. To ensure that the small discrepancy in N/P ratio could not account for any differences, we selected a subgroup of neurons from each area for which the N/P ratio was identical, on average, as in MT neurons (Uka and DeAngelis, 2003a). These selected subpopulations of neurons (n = 54 for V2; n = 33 for V1) are represented by the filled symbols, and filled bars in the histograms. We will next examine whether the responses of neurons in these subpopulations were correlated on a trial-to-trial basis with the monkeys' perceptual choices in the same way as previously found for MT neurons (Uka and DeAngelis, 2004).
Trial-to-trial correlations between neuronal firing and perceptual choice
Figure 3 depicts typical examples of the neuronal responses of one V1 (left column) and one V2 neuron (right column). The top row shows the disparity tuning curves of the cells. Both neurons are strongly tuned for a near disparity and have similar degrees of disparity selectivity as quantified with the disparity discrimination index (DDI, 0.71 and 0.70 for the V1 and V2 neuron, respectively). These tuning curves were used to select the disparities for the psychophysical discrimination task in a given experiment (Fig. 3A,B, filled circles). For each signal strength, the histograms in Figure 3 plot the mean firing rates in each 2 s trial on the abscissa. The filled bars correspond to trials on which the monkey chose the preferred disparity, and open bars to trials with choices to the null disparity. For the V1 neuron stimulated with 0% correlation, the distribution of firing rates is very similar regardless of choice. For higher levels of binocular correlation, the proportion of choices to the two targets diverges as the monkey chooses the correct target more frequently. Nonetheless, the distribution of firing remains similar for both choices. This indicates that whether the monkey chooses the target at the preferred or the null disparity is not reflected in the mean firing rate on a trial-by-trial basis. Accordingly, the CP of the neuron is not significantly different from chance. For the neuron in V2, the behavior is different. For the 0% correlation stimulus, the distributions of firing rates for choices to the preferred disparity target and to the null disparity target fall into two groups with different mean rates, 20 and 28 spikes/s, respectively. This difference in mean response rates on a trial-by-trial basis is reflected in the CP of 0.76. The same pattern holds in response to stimuli for higher levels of binocular correlation: mean responses are on average higher on trials after which the monkey reports the preferred disparity than on those when he reports the null disparity of the neuron. Accordingly, the CP is >0.5 for each level of correlation and disparity. (For statistical reliability in the calculation of the CP, we required that the monkey made at least five choices in response to each stimulus type to both targets.)
Trial-to-trial covariation between neuronal firing and perceptual choice in two example neurons. The left column shows data obtained from V1, and the right column shows data from V2. Disparity tuning curves are plotted in A and B. The filled circles correspond to the disparity values chosen for the correlation threshold task. The filled square depicts the response to an uncorrelated RDS. The disparity selectivity is similar for both neurons (DDI, 0.71 for the V1 neuron; DDI, 0.70 for the V2 neuron). In C and D, frequency histograms of the responses are shown for different levels of binocular correlation (in percentage). Trials on which the monkey chose the preferred disparity are shown by filled bars, and null disparity choices are represented by open bars. Note that, for the V1 neuron, the response distributions are similar for both types of choices, resulting in CPs ∼0.5, whereas for the V2 neuron, the trials on which the monkey chose the preferred disparity target tended to yield higher responses. This is reflected in the CPs that are >0.5 for all correlation values.
In both the V1 and in the V2 neuron, the CPs were fairly similar across correlation levels and disparity. Figure 4shows that this was also true in the population. Average CP values are plotted for each level of signed binocular correlation, where negative and positive values correspond to the correlation level at the null disparity and preferred disparity of the neuron, respectively. Averages are built separately for all V1 (n = 74) and V2 (n = 69) neurons for correlation levels ≤25% for which the monkey made at least five choices each to both targets. For V1 (circles), the average CP at all levels of correlation is ∼0.5, and for V2 (squares) is ∼0.56 (the differences in CPs across correlation levels are not statistically significant, for either V1 or V2, in a one-way ANOVA). Given the similarity across correlation levels, we collapsed data for different correlation levels and for both disparities (after z-scoring) (see Materials and Methods) to obtain a single “grand choice probability” (Britten et al., 1996). In this grand choice probability, data for which the correlation level was ≤25% and for which the monkey had made at least five choices to both targets were included. For each neuron, we determined whether the CP was significantly different from chance by permutation (Britten et al., 1996). The distributions of this grand choice probability for all neurons in V1 (n = 74) and V2 (n = 69) are shown in Figure 5, A and B, respectively.
Mean CP as a function of signed binocular correlation (negative and positive values represent stimuli at the null and preferred disparity, respectively). The average CP across all neurons (n = 74 for V1, circles; n = 69 for V2, squares) is plotted. Error bars depict SE.
Distribution of grand choice probabilities. Data from 74 V1 neurons are shown in A, and data from 69 V2 neurons are shown in B. The filled bars correspond to cells whose CP was significantly different from 0.5. The mean CP for V1 is 0.51, and for V2 is 0.56 (both indicated by the triangles).
For V1, only 16 of 74 neurons (22%) have CPs that are individually significant from 0.5, and the mean grand choice probability is 0.51 ± 0.009 (SD). This is not significantly different from 0.5 (by resampling). In contrast, the mean grand choice probability in V2 is 0.56 ± 0.013, highly significantly larger than 0.5 (p < 0.001, by resampling). The mean grand choice probabilities are significantly greater than 0.5 also in both monkeys separately [0.59 (n = 34) and 0.54 (n = 35) for monkeys duf and ruf, respectively]. For 27 of 69 (39%) of the V2 neurons, the CPs were individually significant. The distribution of CPs in V2 is significantly shifted toward higher values than that in V1 (p < 0.001, by resampling). When restricting this analysis to the subpopulation of neurons in V1 (n = 33) and V2 (n = 54) for which the mean N/P ratio is the same as that documented for MT (Fig. 2, filled symbols; compare previous section), these distributions remain similar (Fig. 6). The mean CP for V1 is 0.51 (Fig. 6A), not significantly different from 0.5 and for V2 with 0.58 highly significantly larger than 0.5 (Fig. 6B) (p < 0.001, by resampling). This value is close to that reported previously for MT (0.59) (Uka and DeAngelis, 2004). The mean CP in V2 is significantly higher than in V1 (p < 0.001, by resampling). These results indicate that despite very similar mean N/P ratios (suggesting equivalent statistical reliability for the task), the CPs differ substantially between V1 and V2 (or MT). Next, we will examine whether eye movements, differences in the shape of disparity tuning, or the relationship between stimulus size and receptive field size might underlie this discrepancy in CP between V1 and V2. We find that none of these can.
Grand choice probabilities for the V1 and V2 subpopulations with mean N/P ratios close to 1 (as reported for MT neurons) (Uka and DeAngelis, 2003a). All symbols are identical with those in Figure 5. Mean CPs for these subpopulations are 0.51 for V1 (n = 33) and 0.58 for V2 (n = 54).
Effect of eye movements
Although the animals were required to maintain fixation during each entire trial, small eye movements may nonetheless affect response rates. There are three ways in which such eye movements might influence our measures of CP. First, changes in vergence eye movement could alter the retinal disparity of the stimulus, and hence affect firing rates. Second, microsaccades can modulate neuronal firing rate (Gur et al., 1997; Bair and O'Keefe, 1998; Leopold and Logothetis, 1998; Martinez-Conde et al., 2000). Any change in microsaccade frequency associated with choice could influence CP measures. Third, any systematic change in mean eye position, by changing the stimulus location relative to the receptive field, may alter response rates. We address each of these possibilities in turn.
Vergence eye movements
We examined whether there was a relationship between vergence angle and the monkey's choice by calculating the mean vergence angle for each trial included in the calculation of grand choice probability. For 33 of 69 V2 neurons, the vergence angle was significantly (Wilcoxon's rank sum test) different on up choices than on down choices, with a mean difference (vergence angle on up choice minus vergence angle on down choice) of 0.04°. The sign of this difference indicates that animals tended to converge on trials when they reported the stimulus as “near.” For V1, the vergence angle was significantly different (Wilcoxon's rank sum test) in 56 of 74 neurons, and the mean vergence difference was 0.14°. Both mean vergence differences were significantly larger than 0° (by resampling). However for both visual areas, the difference in vergence angle was similar across all values of CP (correlation between grand choice probability and difference in vergence angle was rs = 0.03, p = 0.84 for V2, and −0.11, p = 0.37 for V1), suggesting that the CPs in V2 did not simply result from the changes in vergence with choice.
More importantly, changes in vergence angle should have no effect on response rates when the stimulus consists entirely of uncorrelated dots. Because there is no systematic disparity in such a stimulus, a small change in vergence angle does not change this. The only effect of vergence changes in such a stimulus is to change the locations on each retina at which the uncorrelated dots are presented. To verify that vergence did not affect response rates in these stimuli, we examined the correlation between mean firing and vergence for all trials on which the stimulus was uncorrelated. (Vergence data and firing responses were both corrected for nonstationarity. Included were neurons on which the monkeys made at least five choices to both targets for the uncorrelated stimulus; n = 68 for V2 and n = 72 for V1.) The mean value of these correlations (where positive correlation means increased firing with vergence that displaces the stimulus in the preferred direction) were 0.0015 for V2 and 0.014 in V1, neither significantly different from 0 (by resampling). There was no relationship between the strength of this correlation and the CP. We therefore calculated CP only in response to this completely uncorrelated stimulus, where we can be confident that vergence changes do not affect the result. We compared the values of CPs in response to the uncorrelated stimulus only, with the grand choice probabilities for the neurons in V1 (n = 72) and in V2 (n = 68) (Fig. 7). The Spearman's rank correlation was 0.74 for V1 and 0.78 for V2, both highly significant (p < 10−6 in both areas). For V2, mean CP for the uncorrelated stimulus was 0.56, which was statistically indistinguishable from the distribution of grand choice probabilities (p = 0.79, Wilcoxon's rank sum test), and the pairwise comparison did not yield a statistically significant difference (p = 0.70, paired t test). These comparisons were also nonsignificant for the V1 neurons (mean choice probability for uncorrelated stimuli was 0.51; the distribution was not significantly different from the distribution of grand choice probability, p = 0.87, Wilcoxon's rank sum test; no statistical difference on a paired t test, p = 0.98). Thus, all of the characteristics of the CPs in V1 and V2 areas were also present when the data analysis was restricted to those trials for which the responses could not be affected by differences in vergence angle. The CPs we observed can therefore not be explained by systematic differences in mean vergence between the two types of choices.
Vergence eye movements do not affect CP. A plots V1 data (n = 72), and B plots V2 data (n = 68). The circles and squares represent data from monkeys ruf and duf, and the filled and open symbols denote cells with significant (p < 0.05) and nonsignificant grand choice probabilities, respectively. Grand choice probability on the abscissa is compared with the CP calculated only for the responses to the uncorrelated stimulus (which does not depend on vergence eye movement) on the ordinate. Choice probabilities calculated in both ways are highly correlated, and the values are not systematically different (paired t test).
Microsaccades
To identify microsaccades, we calculated horizontal and vertical eye velocity, and took the magnitude of the vector sum to estimate instantaneous speed of eye movement. When this exceeded 12°/s during fixation, a miscrosaccade was deemed to have occurred. The frequency per trial with which the monkeys made fixational microsaccades was not associated with choice in either monkey for V2 (Wilcoxon's rank sum test; the mean frequency of saccades per 2 s trial was 3.88 and 3.83 for monkey duf and 1.22 and 1.18 for monkey ruf, for up-choice and down-choice trials, respectively). However during the recordings from V1, both monkeys made significantly more fixational saccades on up-choice trials (p < 0.05, Wilcoxon's rank sum test; mean frequency of saccades per 2 s trial was 3.35 and 2.96 for monkey duf, and 2.31 and 1.60 for monkey ruf). Averaging across all trials, we found that these microsaccades caused a weak transient decrease in firing, similar to that reported by Leopold et al. (1998). The effect of the increased microsaccade frequency for far choices should be to increase CPs for near-preferring neurons and to decrease CPs for near-preferring neurons. However, we found no difference between the mean CPs between the two classes of neurons in V1. In particular, CPs in near-preferring neurons were not significantly larger than 0.5. It therefore seems unlikely that this slight difference in microsaccade frequency accounts for any of our CP findings.
Small changes in mean eye position
The monkeys indicated their choices with vertical saccades at the end of each trial. Upward saccades (“up choices”) denoted that the center disparity was far, downward saccades (“down choices”) that it was near. As the study proceeded, both monkeys developed a tendency to make upward microsaccades during stimuli for which they subsequently made an up-choice saccade. This led to a systematic difference in the mean vertical eye position as the trial progressed, which depended on choice. We therefore calculated the vertical eye position for up choices or down choices (relative to the vertical eye position within the first 10 ms) as a function of time. These averages include all trials used for the calculation of grand choice probability. Because this analysis revealed a different time course in the two monkeys, we discuss the data for each animal separately.
For monkey duf, the choice-dependent difference in vertical position was only evident after the first second of the trial, reaching a peak difference of 0.17° at the end of the trial for the V1 experiments. For the V2 experiments, this difference appeared only after 1.5 s, and reached a maximum of 0.16° by the end of the trial. To obtain an estimate of CP uncontaminated by this systematic difference in vertical eye position, we therefore recalculated all CPs for monkey duf by including only the initial 1 s of the response on each trial. Figure 8A compares these measures with CPs calculated from the responses during the second second of each trial (which are contaminated by the difference in vertical eye position). For both V1 and V2, the two values of CPs are highly correlated (V2, rs = 0.82, p < 0.001, filled circles; V1, rs = 0.74, p < 0.001, open circles), and there is little change in the mean (0.58 vs 0.59 for V2; 0.51 vs 0.52 for V1). Paired t tests were not significant for the comparison in either V1 or V2. The significant CPs in V2 in this monkey can therefore not be attributed to vertical eye movements that are systematically related to choice.
No significant effect of vertical eye position on CP. A plots data for monkey duf. The filled and open symbols depict data from V1 (n = 34) and V2 (n = 34), respectively. Grand choice probabilities calculated from response in the first second of each trial (abscissa) are compared against CPs obtained from the response during the second one-half of each trial. Both for V1 and for V2, there is no systematic difference between the CPs during the first and the second half of each trial (paired t test). Mean CPs are 0.51 vs 0.52 for V1, and 0.58 vs 0.59 for V2. B represents data for monkey ruf. The filled symbols show V2 data (n = 20), and the open symbols show V1 data (n = 9). In these experiments, small differences in the vertical position of the fixation point were applied. This allowed us to factor out the animal's small vertical eye movements, comparing responses only across trials with similar vertical eye positions. The grand choice probability calculated after the correction for vertical eye position (abscissa) is compared against the uncorrected grand choice probability (ordinate). Both values are significantly correlated (V2 values, rs = 0.51, p < 0.05; V1 values, rs = 0.7, p < 0.05), and the values are not systematically different for V2 or V1 (paired t test).
For monkey ruf, the difference in mean vertical eye position emerged earlier in the trial (starting after ∼200 ms). This then does not permit us to eliminate the artifact simply by analyzing the first one-half of the trial. This monkey underwent additional training using a fixation window with a vertical extent of only 0.24°. All V2 experiments in this monkey were recorded using a fixation window whose height measured between 0.22 and 0.5°. The average difference in vertical eye position at the end of the trials for the V2 experiments was therefore only 0.11°. To control for this remaining effect, we varied the vertical position of the fixation marker (FP) between trials (usually 0 ± 0.1°). We recalculated CPs by comparing neuronal firing rates on up-choice trials (FP at 0°) with those on down-choice trials (FP at 0.1°), or analogously down-choice trials (FP at 0°) with up-choice trials (FP at −0.1°). The comparison of the CPs obtained with and without this correction for 20 V2 neurons and 9 V1 neurons produces a significant correlation (rs = 0.51, p < 0.05, and rs = 0.70, p < 0.05, respectively) (Fig. 8B). Mean CP is 0.54 before and 0.53 after correction (V2), and 0.52 before and 0.53 after correction (V1). Paired t tests for both areas were not significant. This suggests that the CP in the remaining neurons for which we did not collect the data to correct for vertical eye movements in this monkey are not attributable to his systematic vertical eye movements within the fixation window.
The analogous analysis of horizontal eye position did not show any significant differences as a function of choice for the V1 or the V2 experiments in either monkey. The averaged differences of the eye positions did at no time during the trials exceed 0.03° (for both monkeys).
Effect of disparity tuning symmetry
In the previous study examining CP in the identical task in MT (Uka and DeAngelis, 2004), the authors reported that neurons whose disparity tuning curve was more asymmetrical about 0° were associated with higher CPs. Symmetry was quantified as the phase of the Gabor fit to the disparity tuning curve whose mean was constrained to 0°. For many disparity tuning functions in V1 and V2, this constraint on the mean leads to poor fits. Furthermore, for disparity tuning functions with the shape of broadband Gabor functions, common in V1, phase is an unreliable estimate of symmetry (Prince et al., 2002; Read and Cumming, 2004). We therefore used a slight modification of the symmetry phase (Read and Cumming, 2004) to quantify symmetry of disparity tuning both in V1 and V2: the disparity tuning curves were fit by Gabor functions (required to explain 65% of the variance; n = 54 for V1; n = 64 for V2). The extent to which this fitted curve was even or odd symmetric (about zero disparity) was then expressed as a phase angle between −90 and 90°. The distributions of |symmetry phase| for V1 and V2 were not significantly different (p = 0.07, Kolmogorov–Smirnov). The mean values were 54 ± 20° in V2 and 51 ± 18° in V1, not statistically different (p = 0.17, Wilcoxon's rank sum). Moreover, we did not find that CP in V2 depended on symmetry phase (rs = 0.08; p = 0.55). Differences in disparity symmetry between V1 and V2 are therefore unlikely to be the explanation for the lack of significant CP in V1.
Effect of stimulus size relative to the receptive-field size
For a larger stimulus, the population of neurons whose receptive field is covered by the stimulus is larger, so the pool of neurons that contributes to the monkey's choice in a certain visual area is likely to be larger. One consequence might be that this results in a smaller CP for the individual neuron in this larger pool (Shadlen et al., 1996). With this reasoning, one might expect a negative relationship between CP and the size of the stimulus relative to the receptive field size of the neuron (provided that the stimulus is equal or larger than the receptive field). Consequently, if the relative size of the stimulus to the receptive field size had been systematically larger for the V1 than for the V2 experiments, this might explain why the average CP in V1 was at chance level, unlike those in V2 or MT.
To examine this possibility, we measured the size of the receptive field of 68 of 69 V2 neurons using a long, rectangular RDS (typically 0.5 × 6°) positioned at different vertical or horizontal locations on the receptive field (see Materials and Methods). The averaged responses then represented a horizontal and vertical response profile of the receptive field. As a model-free estimate of the horizontal or vertical extent of the receptive field, we calculated the equivalent width (w) (Bracewell, 1986) in both vertical and horizontal directions. If A is the area under the receptive field profile minus the baseline, and h is the maximal response of the receptive field profile plot, then w = A/h. We took the diameter of the central region of the stimulus, divided by the mean of the vertical and horizontal equivalent widths, as our measure of stimulus size relative to the receptive field. We compared this relative size (mean, 2.2) with the CPs for 68 V2 neurons (Fig. 9). (Note that, for a Gaussian receptive field profile, w corresponds to 2.5 SD. Thus, for a receptive field with a Gaussian profile, a relative size of 1.6 is required to cover 95%.) The relative size of the stimulus to the receptive field was not correlated with CP (rs = −0.15; p = 0.22), suggesting that at least over the range used in the experiments, it did not affect the CP of the respective neuron considerably. We only collected quantitative measures of V1 receptive field sizes in 18 of 74 neurons, but this sample was similar to a much larger sample recorded in separate experiments [189 V1 neurons at a similar mean eccentricity (4.85°) compared with the 4.44° mean eccentricity for the V1 neurons in the current study]. For those 189 neurons recorded in the same monkeys, the mean equivalent width (measured orthogonal and parallel to the preferred orientation of the respective neuron) (see Materials and Methods) was 0.56°; for the 18 neurons from this study, this value was 0.46°. The diameter of the inner patch of the stimulus in the V1 experiments was always 2°. Note that at a stimulus disparity of, e.g., 0.5°, this meant that the width of the region containing matched dots in both eyes was only 1.5°. Furthermore, in a previous study, we found that the above measurements of receptive field size correspond to the lower limit of the area over which V1 neurons integrate disparity in a RDS (Nienborg et al., 2004). We thus deemed a 2° stimulus size the lower limit in order to prevent small eye movements moving the disparity-varying portion of the stimulus out of the receptive field. This led to a somewhat larger ratio for (stimulus size)/(receptive field equivalent width) (mean, 3.6; compared with 2.2 for the V2 experiments). The lack of correlation between receptive field size and CP for V2 suggests that this difference in relative stimulus size cannot explain the lack of CP in V1. As an additional check, we calculated CP for a subpopulation of V2 cells whose relative stimulus size matched that used for V1 (3.6; n = 24). For these neurons, the mean grand choice probability was 0.55, significantly larger than 0.5 (p < 0.05, by resampling). These analyses suggest that the lack of significant trial-to-trial correlations with choice for the V1 neurons cannot be explained by the modest difference in stimulus size relative to receptive field size.
CP does not depend on the relative size between the stimulus and the receptive field in 68 V2 neurons. The symbol convention is identical with that of Figure 7. Grand choice probability is plotted against the ratio of the stimulus size (in degrees) over the receptive field size (quantified as the mean of the horizontal and vertical equivalent width in degrees).
Interneuronal correlation
The previous analyses show that the difference in CPs we observe between V1 and V2 cannot be accounted for by experimental differences or artifacts but seem to reflect at least partially genuine differences between these two areas. We now examine whether noise correlations between neurons may explain this difference.
It has been pointed out previously that the CP measured for neurons in a given population depends crucially on the extent to which noise is correlated between neurons in that population (Zohary et al., 1994; Shadlen et al., 1996). It is important not only that neurons with similar stimulus selectivities show correlation, but also that these are stronger than correlations between neurons with dissimilar tuning (it is only then that the correlations give rise to a systematic choice-related difference in mean firing in one population of neurons). Thus, if the activity of disparity-selective V2 neurons with similar tuning were systematically more strongly correlated than that in V1, this could explain the differences in CP we observed between these two brain areas. We estimated the interneuronal correlation between nearby neurons by calculating the noise correlation (Bair et al., 2001) between the single-unit activity and the multiunit activity (see Materials and Methods) for each neuron recorded. The mean noise correlation was 0.24, and 0.35 for V1 and V2, respectively. [Both of these values are higher than values of interneuronal correlation between pairs of neurons (Zohary et al., 1994). This is expected because in contrast to the previous studies we are measuring the correlation between the activity of one neuron with the multiunit activity, presumably consisting of many weakly correlated neurons.] To assess whether this correlation is stronger between similarly tuned neurons, we weight the noise correlation by the signal correlation (Bair et al., 2001) between the multiunit and the single-unit activity. We find a significant correlation between noise correlation weighted by signal correlation, and CP. The correlation is stronger in V2 than in V1 (rs = 0.28, p < 0.05 for V2; rs = 0.15, p = 0.2 for V1) (Fig. 10). It is, however, only individually significant for the monkey with the higher CP (rs = 0.55; p < 0.001). The fact that the weighted noise correlation was correlated with CP is in line with the idea that, for sufficiently large neuronal pools, CP arises essentially as a result of correlated activity within the sensory pools (Shadlen et al., 1996).
Relationship between interneuronal noise correlation (weighted by signal correlation) and CP. Each noise correlation [correlation between single unit (SU) and multiunit (MU)] is weighted by the MU–SU signal correlation (both expressed as Fisher's Z-scores), so that large values mean that the noise correlation was high and the tuning curves were similar. A and B depict V1 and V2 data, respectively. The symbols are as in Figure 7. Both metrics are correlated for V2 (n = 69; rs = 0.28; p < 0.05) and not for V1 (n = 74; rs = 0.15; p = 0.2).
What gives rise to this difference in interneuronal correlation is unclear. One commonly invoked source for such correlations is shared input between neurons (Gawne and Richmond, 1993; Zohary et al., 1994; Shadlen et al., 1996; Bair et al., 2001; Kohn and Smith, 2005). Thus, similar disparity-selective neurons in V2 may share more common input than disparity-selective V1 neurons. This might arise if neurons with similar disparity selectivity are more clustered in V2. To investigate this possibility, we compared the preferred disparity of each single unit with that for the multiunit activity recorded simultaneously in both areas. Each response function of disparity was fitted with a cubic spline, and the preferred disparity was defined as the maximum of the fit. The comparison of the preferred disparities for single-unit and multiunit activity was restricted to those recordings for which both the single-unit and the multiunit response were selective for disparity (one-way ANOVA for disparity as main effect, p < 0.01). Figure 11, A and B, depicts this comparison of the preferred disparity for the single-unit with that for the multiunit for 62 recordings in V1, and 90 recordings in V2, respectively. Although there is only a weak correlation between these values in V1 (rs = 0.18; p = 0.16), which is comparable with that reported previously (Prince et al., 2002), the correlation is tight for V2 (rs = 0.84; p ≪ 0.0001). The extent of this correlation is similar to that found for MT (DeAngelis and Newsome, 1999). It is compatible with a columnar structure for disparity in V2 as suggested previously (Ts'o et al., 2001).
Similarity of preferred disparity for single-unit (SU) and multiunit (MU) activity. V1 data are shown in A, and V2 data are shown in B. The circles depict data from monkey duf, and the squares depict data from monkey ruf. Both axes plot the peak disparity (in degrees) of the disparity tuning curve, for the SU activity on the abscissa and for the MU activity on the ordinate. The correlation between the peak disparity for SU and MU activity is weak in V1 (rs = 0.18; p = 0.16; n = 62) and much stronger in V2 (rs = 0.84; p < 0.001; n = 90).
This suggests that our finding of significant CPs in V2 but not in V1 may be the consequence of stronger interneuronal correlation between similarly tuned neurons in V2. The higher degree of interneuronal correlation in V2 in turn possibly results from a more columnar organization for disparity in V2 than in V1, perhaps more like that reported in MT (DeAngelis and Newsome, 1999). So far, all of the results we have described for V2 have been comparable with those reported for MT. We will next compare two more properties of V2 CPs to those reported in MT: time course and correlation with N/P ratio.
Time course of the response differences reflected in choice probabilities in V2
To appreciate how the choice-related response difference evolved over time, we plotted the time course of the average population response [in 10 ms bins, normalized as described by Uka and DeAngelis (2004)] for all 69 V2 neurons (Fig. 12) (the average for the first 10 ms bin is plotted at 5 ms, etc.). Superimposed is the averaged differential normalized response (right ordinate). The responses start separating with the onset of the visual response and the difference plateaus at ∼400 ms after the onset of the trial. Both of these features are qualitatively similar to the time course of the averaged responses in MT [Uka and DeAngelis (2004), their Fig. 9].
Time course of the response. For 69 V2 neurons, the average of the normalized response within each 10 ms bin is shown separately for trials on which the monkey chose the preferred (thin solid line) and null (dashed line) disparity target (left y-axis). Superimposed is the averaged difference of the responses (thick solid line; right y-axis).
Effect of other tuning properties on choice probabilities in V2
Previous studies consistently reported a negative correlation between CP and the neuronal precision for a given task (Celebrini and Newsome, 1994; Britten et al., 1996; Parker et al., 2002; Uka and DeAngelis, 2004). This correlation was interpreted as indicating that neurons carrying more precise signals are more relevant to the task, and hence are given more weight in forming the perceptual decision. Choice probabilities in V2 in our study were neither correlated with the neuronal threshold (n = 61; rs = 0.003; p = 0.98), nor with the extent of disparity selectivity (as quantified by the DDI) (see Materials and Methods) (rs = 0.02; p = 0.85; n = 69). We wondered whether the lack of correlation between CP and neuronal threshold was attributable to our strict inclusion criterion on disparity selectivity and repeated the comparison without any criterion regarding disparity selectivity. Even then the correlation is only significant in the monkey with the higher average CPs (rs = −0.33; p < 0.05; n = 40), whereas it is absent in the other monkey (rs = 0.28; p = 0.11; n = 35). This seems to result from the fact that the distribution of CPs was inhomogeneous even in the population of disparity-selective V2 neurons. Neurons whose preferred disparity was near tended to have large CPs, whereas for neurons whose maximal response was to a far disparity, the CP was clustered around 0.5. Because near disparities were defined as negative, this resulted in a significant negative correlation between CP and the preferred disparity of the neurons (rs = −0.42; p < 0.001). This feature may be a reflection of the monkeys' strategy in solving the task, and we are currently investigating this aspect of the data in more detail.
A previous study showed that a proportion of V2 neurons were specialized for signaling relative disparity (Thomas et al., 2002). We characterized relative disparity only in a small number of the V2 neurons (n = 17). Only four of these showed significant tuning for relative disparity, comparable with the previously reported proportion (Thomas et al., 2002). This small number does not allow us to draw definitive conclusions about the relationship between tuning for relative disparity and CP in V2.
Discussion
This study compares the extent of covariation between neuronal response and psychophysical judgments in neuronal populations from different brain areas. To make this comparison meaningful, we chose a stimulus and task (disparity discrimination in weakly correlated RDS) for which neurons in three different areas (V1, V2, and MT) show similar responses. Previous work has already demonstrated both a causal (DeAngelis et al., 1998) and a correlative (Uka and DeAngelis, 2004) relationship between neuronal firing and perceptual choice in this task, for disparity-selective neurons in MT. The magnitude of the correlation reported for MT neurons was quite large; the mean choice probability (CP, 0.59) was larger that that reported for direction discrimination in MT. For both V1 and V2, we were able to select a subpopulation of neurons with a similar average statistical precision for the task as MT neurons (Uka and DeAngelis, 2003a). From a purely informational point of view, these should be all equally useful to an animal attempting to solve the task. Despite this similarity, neurons in V1 and V2 displayed very different degrees of correlation between neuronal activity and psychophysical judgment. There was no significant correlation in V1, but the correlation for V2 neurons (0.58) was nearly as strong as that reported in MT. This difference cannot be explained by artifacts caused by eye movements, differences in disparity tuning symmetry or by the size of the stimulus relative to the receptive field size.
This is the first demonstration of systematic CPs (showing a consistent relationship between choice-related activity and stimulus-driven activity) this early in the visual hierarchy. The fact that CP is so similar in V2 to that reported in MT (0.58 vs 0.59) implies that cortical neurons at different levels in the visual processing hierarchy may show very similar CPs.
Conversely, the lack of a significant CP in disparity-selective V1 neurons indicates that other factors are also important, because a significant CP is not an inevitable consequence of a neuron's carrying signals with a certain statistical precision. CP may be absent in such a population for three reasons: (1) The neurons play no role in the perceptual decision (despite their suitability for the purpose); (2) CP does not merely reflect stochastic fluctuation in an input signal, but (at least partially) is a result of the choice itself (CP reflects a top–down process); (3) although the neuronal signals contribute to the decision in a bottom–up manner, some other property of the neuronal signals makes the CP undetectably small. The first explanation seems improbable: V1 is the likely source of disparity signals for neurons in V2. Furthermore, several limitations in stereoscopic performance (especially spatial and temporal resolution) seem to reflect processing in V1 (Prince et al., 2000; Nienborg et al., 2004, 2005). The second explanation would make it quite natural that CPs are not significant in V1: One need only suppose that the top–down signal does not reach all the way back to V1. The final possibility can be best appreciated in the context of the detailed simulations relating activity in a pool of neurons to psychophysical decisions in a bottom–up model (Shadlen et al., 1996). These showed that the extent of correlated noise in a pool of neurons profoundly affects the CP: if the activity of neurons in a pool is independent, then sizeable CPs are observed only with implausibly small pools. However, interneuronal correlation alone is not sufficient to produce sizeable CP: it is crucial that neurons that contribute to opposite decisions (near-preferring vs far-preferring neurons in this study) be less strongly correlated than neurons that contribute to a single decision pool. Thus, the lack of a significant CP in striate cortex could result if neurons with similar disparity preferences were more weakly correlated than such neurons in V2 and MT. Our indirect estimates of interneuronal correlation (correlating isolated units with multiunit activity) lend some support to this conclusion: these correlations were weaker in V1 than in V2. However, this observation must be interpreted with caution, because it seems clear that any columnar structure for disparity in V1 (Prince et al., 2002) is much weaker than in MT (DeAngelis and Newsome, 1999) or V2 (Ts'o et al., 2001) (Fig. 12). This has two important consequences. First, it means that a pool of neurons with similar disparity preferences must be spread over a wider cortical area in V1, so measurements of local correlations are less relevant. Second, it means that the pool of neurons contributing to the multiunit activity will be more heterogeneous in terms of its disparity selectivity, inevitably weakening the correlation. Thus, although our observations are compatible with weaker interneuronal correlations in V1, they do not demonstrate this.
In summary, two explanations for the lack of CP in V1 seem equally possible. In a bottom–up framework, it could be the result of weaker interneuronal correlations (between neurons with similar disparity selectivity) in V1 than in V2/MT. Alternatively, the CP could represent a top–down signal that does not reach V1. This suggests some similarity with top–down modulation by attentional signals, which also tend to be stronger in higher visual areas (Cook and Maunsell, 2002a). Although the top–down signal may share some of the same mechanisms as attentional modulation (Krug, 2004), it is a different phenomenon from those demonstrated previously. One has to propose that the attention is to a feature (in this case, near or far disparity) that is not present in the display itself (for a discussion of this point, see Krug, 2004). An important point to note is that such a top–down signal would itself give rise to an increased interneuronal correlation. Consider a top–down signal that is either related to choice or to feature-based attention. (And assume in the latter case that the monkey is more likely to choose the preferred disparity target, if it attends to that feature.) This top–down signal feeding into sensory neurons could directly cause the trial-to-trial correlations between the activity of neurons receiving the top–down input. Thus, significant CPs must be associated with correlations in activity between neurons in a pool that contribute to the decision (unless that pool is implausibly small). These correlations alone do not differentiate top–down and bottom–up models.
It is possible that the difference in CP between V1 and V2/MT is the result of the weak columnar structure for disparity in V1: the processing that leads to a columnar structure for disparity may also lead to increased correlations between neurons sharing disparity preferences. In a top–down scheme, it may be that sending signals to a set of neurons with the same disparity selectivity can better be achieved where there is a columnar organization for disparity. One aspect of cortical organization that may be important is how much subsequent processing is required before the decision stage. This is a difficult quantity to assess, because it may not be straightforwardly related to anatomy. One property of V1 neurons that suggests they are remote from the site that determines depth percepts is revealed by anticorrelated dot patterns: these produce substantial changes in neuronal firing rate that have no measurable perceptual consequences (Cumming and Parker, 1997). However, by this measure, MT and V2 neurons are equally remote from the decision stage (Krug et al., 2004; Allouni et al., 2005), despite the significant CPs in V2 and MT for the current task. Additional investigations are necessary to reveal what features of the cortical organization give rise to correlations between the activity of single neurons and behavioral decisions.
Two previous studies compared the covariation of neuronal activity and percept between striate and extrastriate cortex (Leopold and Logothetis, 1996; Grunewald et al., 2002). These studies reported covariation between neuronal activity and perceptual judgment in a small fraction of the neurons in V1. However, in both studies, this covariation between neuronal activity and perceptual judgment in striate cortex was not systematically related to the tuning of the neurons for the stimulus. This is quite different from the sort of systematic correlation quantified by a significant mean CP. Given only the responses of a population of neurons and a knowledge of their preferred stimulus, it is possible to predict an animal's choices, provided that the mean CP is >0.5. If correlations are equally strong in both directions (mean CP ∼0.5), this is no longer possible. These previous studies are also complicated by the fact that the comparison was confounded by other factors. In one case (Grunewald et al., 2002), the stimulus chosen was similar to that used in a previous study of MT (moving dots portraying a rotating three-dimensional cylinder) (Bradley et al., 1998; Dodd et al., 2001), but the selectivity of V1 cells for the feature of interest (direction of rotation) was poorer than in MT. In the other case (Leopold and Logothetis, 1996), responses to V1 and V2 were pooled. Furthermore, a study of MT using stimuli similar to those used in V1 also found no net correlation between activity and reported direction at the population level (Logothetis and Schall, 1989). Because such population-level correlations are present in MT for other stimuli, it is clear that comparing like stimuli is essential. In the present study, we used a very similar visual stimulus to that used previously in MT, and studied neurons with a similar degree of disparity selectivity. For the first time, this revealed significant CPs for area V2. That CP was much smaller in V1 than in V2 or MT under such circumstances establishes a new and important difference between striate and extrastriate cortex.
Footnotes
-
This work was supported by the Intramural Research Program of the National Institutes of Health–National Eye Institute. We also thank Chris Hillman, Mark Szarowicz, Carrie Silver, and Denise Parker for excellent animal care.
- Correspondence should be addressed to Hendrikje Nienborg, 49/2A50 Convent Drive, Bethesda, MD 20892-4435. hn{at}lsr.nei.nih.gov
References
- Allouni et al., 2005.↵
- Bair and O'Keefe, 1998.↵
- Bair et al., 2001.↵
- Bracewell, 1986.
- Bradley et al., 1998.↵
- Britten et al., 1992.↵
- Britten et al., 1996.↵
- Celebrini and Newsome, 1994.↵
- Cook and Maunsell, 2002a.↵
- Cook and Maunsell, 2002b.↵
- Cumming and Parker, 1997.↵
- Cumming and Parker, 1999.↵
- DeAngelis and Newsome, 1999.↵
- DeAngelis et al., 1998.↵
- Dodd et al., 2001.↵
- Gawne and Richmond, 1993.↵
- Grunewald et al., 2002.↵
- Gur et al., 1997.↵
- Judge et al., 1980.↵
- Kohn and Smith, 2005.↵
- Krug, 2004.↵
- Krug et al., 2004.↵
- Leopold and Logothetis, 1996.↵
- Leopold and Logothetis, 1998.↵
- Logothetis and Schall, 1989.↵
- Martinez-Conde et al., 2000.↵
- Newsome et al., 1989.↵
- Nienborg et al., 2004.↵
- Nienborg et al., 2005.
- Parker et al., 2002.↵
- Prince et al., 2000.↵
- Prince et al., 2002.↵
- Purushothaman and Bradley, 2005.↵
- Read and Cumming, 2003.↵
- Read and Cumming, 2004.↵
- Read and Cumming, 2005.↵
- Roe and Ts'o, 1995.↵
- Shadlen et al., 1996.↵
- Thomas et al., 2002.↵
- Ts'o et al., 2001.↵
- Uka and DeAngelis, 2003a.↵
- Uka and DeAngelis, 2003b.↵
- Uka and DeAngelis, 2004.↵
- Uka et al., 2005.↵
- Zohary et al., 1994.↵