Dissociation between Neural Signatures of Stimulus and Choice in Population Activity of Human V1 during Perceptual Decision-Making

Fine ring-size classification task

We devised a difficult one-interval two-alternative forced-choice task wherein observers viewed one of three different-sized rings that were always symmetrically centered around a central fixation mark and classified the ring as either “small” or “large” (Fig. 1A). To detect cortical signatures of stimulus and choice simultaneously in trial-to-trial population fMRI activity of V1, we optimized the task and stimulus parameters as follows.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Task, stimuli, and behavioral performance. A, An example sequence of trials and phases constituting a trial. Eight exemplar trials are shown, each belonging to one of the six possible classes (labeled by letter symbols at the top, “stimulus|choice”). The gray rectangles represent brief periods during which observers were warned of stimulus onset, viewed a ring stimulus (colored thick vertical bars), and made a choice at a particular time point (colored thin vertical bars). B, Examples of the three different-sized rings. The luminance polarity is reversed here for illustrative purpose. C, Distribution of threshold SC values (on the horizontal axis) and actual performances in the main fMRI experiment (on the vertical axis). The small circles represent individual observers, and the large circle and error bars represent their population average and standard deviation (SD) respectively.

As stimuli whose subtle differences are best resolvable with population fMRI measurements, we opted for concentric rings whose size was the feature dimension of relevance for perceptual decision (Fig. 1B). Because of its configuration, a concentric ring engages a large ensemble of neurons whose peak activities within the V1 retinotopic map will vary with the eccentricity of the ring's circumference (i.e., ring size). Because of this feature of concentric ring stimuli, we could exploit the fact that the retinotopic architecture of V1 is resolvable with a mesoscopic-scale analysis of fMRI (Lee et al., 2005; Dumoulin and Wandell, 2008; Kay et al., 2008; Park et al., 2013), allowing us to take advantage of population coding of subtle stimulus differences (Paradiso, 1988; Pouget et al., 2000; Jazayeri and Movshon, 2006). Also, these ring stimuli provide the additional benefit of encouraging observers to maintain central fixation, for this ensures optimal retinal stimulation for performance of the task: shifting fixation toward any selected portion of a ring inevitably images the remaining portions of the ring at even more eccentric areas of the retina with poorer spatial resolution.

We optimized the spatiotemporal parameters of the ring stimuli to generate an optimal level of uncertainty in perceived size of the ring, so as to observe cortical representations of choice information. The SC (Fig. 1B) between the rings was calibrated to be at a threshold level for each individual, based on the performance in prescan practice trials performed in the scanner (0.020 ± 0.009; mean ± SD across observers; see Materials and Methods). In addition, we created trials in which observers' choices would not correlate with stimuli by introducing a middle-sized ring (M-ring) whose radius (r_M = 2.84°) was halfway between the radii of the smallest (S-ring) and largest (L-ring) rings (Fig. 1B). Observers were not told there would be three different-sized rings; they were only told to classify each ring as “small” or “large.” The ring was shown for 0.3 s (Fig. 1A, “stimulus” period), a duration sufficiently long to produce reliable fMRI responses in V1 yet sufficiently brief to contribute to a degree of uncertainty in perceived size of the ring. This tailor-made calibration of stimulus size and duration succeeded at holding observers' performances during the fMRI scan sessions within a threshold range (73.7 ± 5.7%; mean ± SD across observers; Fig. 1C).

We adopted a sparse event-related design (Fig. 1A) to individuate trial-to-trial fluctuations of fMRI responses to repeated presentations of the rings. To encapsulate neural events associated with a single trial of perceptual decision-making within a short period of time, we forced observers to make a perceptual choice within 1.2 s after stimulus onset, which resulted in actual response times with the mean of 0.66 s and SD of 0.13 s (2825 trials, pooled across observers). To minimize carryover effects in fMRI signal between consecutive trials resulting from hemodynamic delay, individual trials were separated by 13.2 s. To stabilize eye position and to regulate cortical and cognitive states during this intertrial period, we required observers to maintain their gaze on the fixation dot (diameter 0.12°) and signaled an upcoming trial by increasing the size of the fixation dot slightly (diameter 0.18°) 2.2 s before stimulus onset. It is worth noting that stable, central fixation is essential for optimizing psychophysical performance, as mentioned above, and for successful measurement of high-resolution fMRI responses to the ring stimuli. To prevent unwanted feedback-related events from contaminating trial-locked fMRI measurements, we did not provide trial-by-trial feedback. Instead, observers were updated about their overall performance at the end of each scan run containing 26 trials.

Definition of eccentricity-tuning curves for individual voxels

While observers performed the ring-size classification task with parameters optimized as described above, we acquired time-series of fMRI measurements from a population of unit gray-matter volumes (voxels) in the V1 cortical surface whose width (0.5°–7.5°; a region marked by color spectrum in Fig. 2A) was larger than the site directly stimulated by the rings (2.72°–2.96°, i.e., the smallest and largest rings, respectively, used in the experiment across observers; dotted circle in the inset of Fig. 2A). To inspect trial-to-trial patterns of population responses in a feature dimension relevant to the perceptual decision task, we first mapped the coordinates of those individual voxels in visual eccentricity space.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Eccentricity-tuning curves in V1. A, Eccentricity map of V1 from Observer S08 shown on the flattened left occipital cortex. The white dot, dashed curve, and solid curve represent the V1 cortical sites representing the fovea, the upper vertical meridian, and the lower vertical meridian, respectively, in visuotopic space. The colors represent the eccentricities of the voxels with high goodness of fit by the tuning-curve model (R² > 0.4; see Materials and Methods). The black dotted circle in the color legend represents the eccentricity of the M-ring stimulus. B, Relationship between preferred eccentricity and tuning width. The gray lines plot tuning widths (the vertical axis) as a function of preferred eccentricity (the horizontal axis) for individual observers, and the black line is a pseudo linear regression of the eccentricity to the tuning width (see Materials and Methods), which was used to estimate the eccentricity-tuning curves for the 21 cortical bins (the black dots) shown in C. C, Population-averaged eccentricity-tuning curves. The horizontal axis specifies stimulus eccentricity, the vertical axis the estimated preferred eccentricity of a cortical bin, and the intensity corresponds to the normalized cortical activity. The solid blue and red vertical lines indicate the population averages of the eccentricities of the S- and L-rings, respectively. The dotted blue and red horizontal lines indicate the two cortical sites that show maximum responses to the S- and L-rings, respectively. The blue and red curves on the right are the predicted population responses to the S- and L-rings, respectively. D, Individual (thin gray) and averaged (thick black) HIRFs. E, Predicted fMRI responses to ring stimuli. In all three panels, the horizontal and vertical axes specify time relative to stimulus onset (indicated by the colored dots) and the preferred eccentricity of a cortical bin, respectively. The L- and S-ring panels represent responses to the L- and S-ring stimuli, respectively, which were predicted by convoluting the red and blue curves in C with the averaged V1 HIRF in D. The L-S panel represents the differences between the L- and S-ring panels, with hue and saturation representing the sign and magnitude of the differential responses, respectively.

By applying the model-based population receptive field estimation method (Dumoulin and Wandell, 2008) to fMRI time-series responses to an expanding/contracting annulus, we defined the eccentricity-tuning curves with a Gaussian function for individual voxels in each observer's V1 (Fig. 2A; see Materials and Methods). The range of estimated widths of the tuning curves (1.08 ± 0.51°; mean ± SD across 6379 V1 voxels with R² > 0.4, pooled across observers) and their positive correlation with preferred eccentricity (Pearson's r = 0.32 ± 0.08, 10⁻¹⁷ < p < 0.001; mean ± SD across observers; Fig. 2B) were consistent with previous studies (Dumoulin and Wandell, 2008; Kay et al., 2008; Harvey and Dumoulin, 2011; Park et al., 2013), supporting the validity of our estimation procedure. These tuning estimates from individual observers (Fig. 2B, thin gray lines) were merged and summarized by fitting a power function (Fig. 2B, black line; see Eq. 5) (Duncan and Boynton, 2003) to obtain a reference eccentricity map (Fig. 2C) with 21 eccentricity bins (whose centers are marked by the filled circles in Fig. 2B; see Materials and Methods).

The eccentricity map (Fig. 2C) allowed us to preview the potential population fMRI responses to the different-sized rings. This map predicts that the S- and L-rings (whose eccentricities are marked by the vertical blue and red solid lines, respectively, in Fig. 2C; the group average values, r_S = 2.78° and r_L = 2.90°, were used) produce profiles of activity that are broad across-eccentricity but that are nonetheless slightly offset with respect to one another (Fig. 2C, blue and red bell-shape curves with dotted lines at center, plotted on the right-hand vertical axis). These spatial profiles of predicted cortical responses were then convolved with the V1 HIRFs (Fig. 2D), which were estimated from the retinotopy-mapping scan runs (see Materials and Methods), to produce matrices of noise-free fMRI responses to the L-ring and S-ring stimuli (Fig. 2E, L-ring and S-ring panels, respectively; see Materials and Methods). By subtracting the matrix predicting responses to the S-ring from that predicting responses to the L-ring, we obtained the matrix predicting differential fMRI responses to the L- and S-rings (Fig. 2E, L-S panel). The predicted differential responses peaked in time at 3.3–5.5 s after stimulus onset due to hemodynamic delay, and in space at two flanking banks of eccentricity bins (Fig. 2E, blue and red pixels in L-S panel), representing the foveal and the peripheral sides of the rings.

Definition of trial-related matrices of population responses

With eccentricity-tuning curves defined for individual voxels, we expressed the V1 population responses of individual observers performing the ring-size classification task in a matrix with two dimensions: one defined by voxels' preferred eccentricities and the other defined by time frames at which fMRI measurements were acquired (Fig. 3A). To increase SNR at the cost of resolution and to be able to merge data across individuals, fMRI responses of neighboring voxels were summed with windows of weights centered at discrete-step eccentricity values (Fig. 3B; see Materials and Methods), as done similarly in previous fMRI studies (Brefczynski-Lewis et al., 2009; Kolster et al., 2010; Park et al., 2013). This smoothing procedure resulted in a 21 (the number of eccentricity bins)-by-150 (the number of time frames per scan run) matrix of responses for each scan run (Fig. 3C). As a final step of preparatory analysis of fMRI measurements, we dissected the response matrix for each scan run into “trial-related” matrices of responses with 21 rows representing the eccentricity bins and six columns representing the time frames defined relative to stimulus onset (Fig. 3C, example shown by the matrix demarcated by the dotted box). Then, with these trial-related matrices (Fig. 3D), we searched for cortical signatures of stimulus and choice by examining whether responses in each of those 126 (=21 eccentricity bins × 6 time frames) individual spatiotemporal cells covary with stimuli shown or choices made over trials.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Definition of trial-related matrices of population responses. A, Eccentricity-sorted individual voxels' time-series of fMRI measurements during a single scan run from Observer S08. The horizontal and vertical axes specify the time bin of measurement and the peak eccentricity of voxels, respectively, with image intensity corresponding to level of fMRI activity. The letters at the top represent the stimulus shown and the choice made by the observer in a given trial (stimulus|choice). B, Kernels used for spatial smoothing over eccentricity. The horizontal axis specifies the preferred eccentricity of a target eccentricity bin in cortical scale. The vertical axis specifies the center of a given voxel's eccentricity-tuning curve in visuotopic scale. The image intensities correspond to the weights of the smoothing kernels, whose widths were constant over the eccentricity in cortical scale. C, Responses at eccentricity bins. The format is identical to the one in A, except that the vertical axis specifying the preferred eccentricity is scaled in cortical distance. The blue, magenta, and red dots represent the spatiotemporal locations of S-, M-, and L-rings, respectively, presented over trials. D, An example matrix of trial-related population responses. Each trial-related matrix spanned 13.2 s (2.2 s per frame) in time and ∼5.5 degrees in space. The responses within the dashed black box in C are replotted here, with time axis magnified. The shaded rectangle and the bars within it represent the same events depicted in Fig. 1A. The dashed vertical and horizontal lines indicate the stimulus onset and the eccentricity of the M-ring stimulus, respectively.

Neither stimuli nor choices significantly correlated with raw responses

For each of the spatiotemporal cells of the trial-related matrix, we computed SPs (Tolhurst et al., 1983; Newsome et al., 1989) and CPs (Celebrini and Newsome, 1994; Britten et al., 1996) by assessing how well the trial-to-trial distributions of the raw responses predicted the stimulus actually presented and the choices that were made by individual observers. We sorted the individual trials into the six possible classes jointly defined by a stimulus shown and a choice made in a given trial: S|S, S|L, M|S, M|L, L|S, or L|L (Fig. 4A, stimulus|choice). The SPs were estimated by comparing the distributions of the raw responses belonging to two stimulus-contrast pairs (S-ring vs L-ring; Fig. 4A, horizontal brackets) of these classes, wherein the choice factor was held constant, S|S versus L|S (SP_choice=S) and S|L vs L|L (SP_choice=L). By varying the location of the discrimination criterion over those distributions (Fig. 4B, top panel), we constructed an ROC curve (Fig. 4C, green curve) and computed an SP by summing the area under the ROC curve. We defined the grand SP by taking the average of the two SPs associated with different choices, SP_choice=S and SP_choice=L (as indicated by the operations in Fig. 4A, bottom). The CPs were estimated similarly, first computing the three individual CPs for the three choice-contrast pairs (Fig. 4A, vertical brackets) of the distributions (S|S vs S|L, CP_stimulus=S; M|S vs M|L, CP_stimulus=M; and L|S vs L|L, CP_stimulus=L) and then averaging those three CPs. As a reminder, our definition of CPs is different from that used in single-cell studies (see Materials and Methods).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

Computation of SPs and CPs. A, Classification of trials and definition of stimulus- and choice-contrast pairs associated with SPs and CPs. A trial-related matrix for each trial was classified according to the stimulus|choice class. SPs and CPs were computed by averaging the stimulus-contrast (horizontal brackets) and choice-contrast (vertical brackets) pairs of the stimulus|choice classes, as indicated (Materials and Methods). B, Example distributions of responses at a representative spatiotemporal bin (black squares in D) from Observer S08. Top, Contrast of the histograms of raw fMRI responses between the L-ring|S-choice trials (open) and the S-ring|S-choice trials (filled). Bottom, Contrast of the histograms of raw fMRI responses between the M-ring|L-choice trials (open) and the M-ring|S-choice trials (filled). The dashed vertical line indicates a classification criterion that is slid to generate ROC curves. C, Example ROC curves. The horizontal and vertical axes specify the false alarm and hit rates, respectively (see Materials and Methods for definitions of α and β). The green and orange curves were derived from the top and bottom distributions, respectively, in B. D, Across-observer averages of SP, CP, and CP_stimulus=M computed for raw fMRI responses. The format for axes is identical to that in Fig. 3D. Hue and saturation represent stimulus or choice probability values, as indicated.

We estimated the SPs and CPs exhaustively over the entire trial-related matrix of the raw responses (Fig. 4D), but none of those values reached statistical significance (minimum TFCE-corrected p = 0.41, 0.87, and 0.60 among 126 spatiotemporal bins, respectively, for SP, CP, and CP_stimulus=M). Only the overall pattern of the across-observer averages of SPs (Fig. 4D, SP panel) exhibited a somewhat systematic distribution, which appeared similar to the pattern of the model-predicted BOLD differential responses (Fig. 2E, L-S panel).

We wondered that these weak probability values in the raw responses might have been caused by the interference from large nonspecific fluctuations in background cortical activity. Recent optical imaging and fMRI studies, wherein large-size population neural activities were monitored simultaneously in early visual cortex, observed large-scale cofluctuations over an entire population of neurons under observation regardless of whether or not individual neurons' stimulus preferences match incoming visual input (Sharon and Grinvald, 2002; Fiser et al., 2004; Chen et al., 2006; Jack et al., 2006; Donner et al., 2008; Sirotin and Das, 2009; Sirotin et al., 2012). In line with these findings, the raw responses in our study waxed and waned in synchrony over the entire array of eccentricity bins, which is readily appreciated by visual inspection of the sample matrix of the raw population responses (Fig. 5A, top). The presence of these so-called “untuned responses” was supported by significant widespread correlations among the eccentricity bins (Pearson's r = 0.62 (mean) ± 0.17 (SD across observers), the mean correlation of nonoverlapping eccentricity bin pairs, which are demarcated with the dotted boundary in Figure 5B). These significant positive correlations were not confined to the pairs of nearby eccentricity bins (Fig. 5C) but were also found between ones representing the directly stimulated visual region and the ones representing either the foveal or peripheral regions (e.g., Pearson's r = 0.54 ± 0.17, 0.67 ± 0.13; the mean ± SD correlations of the foveal and peripheral pairs marked by the gray boxes in Fig. 5B, respectively) and even between the ones representing the foveal and peripheral regions (e.g., Pearson's r = 0.57 ± 0.16 for the pair marked by the black box in Fig. 5B).

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Filtering out a nonspecific component from raw responses. A, Definition of TRs via averaging and subtracting operations on RRs. Top, Image in Fig. 3C is replotted. Middle, Averages of RRs over the eccentricity bins at individual time frames. Bottom, TRs were obtained by subtracting those averages from the RRs. B, A matrix of population average correlations in RRs among eccentricity bins. The dotted line boundary indicates the bin pairs whose fMRI measurements are not blended via spatial smoothing. The squares represent the values of correlations for the three possible pairs from the time-series of RRs at the three eccentricity bins, representing 1.26°, 2.84°, and 5.65°, respectively. C, Correlations in RRs between all the eccentricity bins and the seed (2.84°) bin. The dashed vertical line indicates the preferred eccentricity of the seed bin, the visuotopic representation of which covers the locations of the ring stimuli. The gray thin lines indicate the correlations from individual observers, and the black thick line indicates their population average.

Having confirmed the nonspecific nature of the moment-to-moment background fluctuations, we filtered out those correlated responses throughout the entire region of V1 under observation by averaging the raw responses across the entire set of eccentricity bins (Fig. 5A, middle) and subtracting that average from the raw responses at each time frame (Fig. 5A, bottom). These averaging and subtracting operations were validated by the additive nature of “tuned” and “unturned” responses (Bianciardi et al., 2009; Cardoso et al., 2012; Schölvinck et al., 2012) and have been routinely used in previous studies using optical imaging (Shtoyerman et al., 2000; Sharon and Grinvald, 2002; Benucci et al., 2009) and fMRI (Fox et al., 2006; Larsson et al., 2006; Donner et al., 2008, 2013; Pestilli et al., 2011; Schölvinck et al., 2012). Hereafter we will refer to the unfiltered raw responses as RRs and the filtered responses as tuned responses (TRs).

Dissociated signatures of stimulus and choice in tuned responses

The subtraction of the untuned component from the RRs revealed clear signatures of stimulus and choice. To compute the SPs and CPs for the TRs, we followed the same procedure used for the RRs. Unlike the RRs, the TRs exhibited significant SPs and CPs, respectively, at different sets of spatiotemporal cells of the trial-related matrix [significant cells (corrected p < 0.05) are marked in Fig. 6A (*); corrected for multiple comparisons across the 126 spatiotemporal cells using the TFCE method (Smith and Nichols, 2009)].

• Download figure
• Open in new tab
• Download powerpoint

Figure 6.

Stimulus and choice probabilities in tuned responses. A, Across-observer averages of SP, CP, and CP_stimulus=M in tuned responses. The format is identical to that in Fig. 4D. The white asterisks indicate the significant bins (TFCE-corrected p < 0.05; Materials and Methods). B, SP values at 5.5 s after stimulus onset from individual observers. The SPs are plotted against the eccentricity bins. Gray circles represent individual observers, and the blue- and red-filled circles represent the SPs averaged across observers at eccentricity bins, at which they were significant, as indicated by the white asterisks in the SP panel in A. C, CP_stimulus=M values at 1.1 s after stimulus onset from individual observers. The axis format and symbols are identical to those in B, except for the empty blue and red circles, which represent the CPs averaged across observers at eccentricity bins, at which they were significant, as indicated by the white asterisks in the CP_stimulus=M panel in A. D, Significant positive correlation between the model-predicted differential responses (Fig. 2E, L-S panel) and the observed SPs (SP panel in A). Gray circles represent individual spatiotemporal cells, and the colored circles represent the cells wherein either significant SPs (as indicated by the corresponding filled circles in B) or significant CPs (as indicated by the corresponding open circles in C) were found. E, No correlation between the CP_stimulus=Ms (CP_stimulus=M panel in A) and the SPs (SP panel in A). The representation of spatiotemporal cells by the symbols is the same as in D. The filled and open symbols are located far away from one another, illustrating the spatiotemporal dissociation between the significant SPs and CPs.

The SPs in the TRs (Fig. 6A, SP panel) were signed properly and clustered systematically both in space and time. At the time frames matched to the typical hemodynamic delay (3.3 s and 5.5 s) from stimulus onset, the responses to the S-ring were greater than those to the L-ring within the cortical subregion representing the side of the rings nearer to the fovea (Fig. 6A, blue pixels with SP < 0.5 in the SP panel), and the opposite was true within the cortical subregion representing the peripheral side of the rings (Fig. 6A, red pixels with SP > 0.5 in the SP panel). This emergence of the sinusoidal-shape spatial profile of SPs centered around the stimulation site at the time points a few seconds delayed from stimulus onset (Fig. 6B) is exactly what was previewed by our model prediction of differential cortical responses based on the eccentricity-tuning curves (Fig. 2E): the cortical sites that generate the largest differential responses to the ring stimuli with subtle differences are those with eccentricity preferences slightly deviated from the eccentricity of the stimuli. This strong resemblance between the spatiotemporal maps of the SPs and the model prediction of differential responses, as evidenced by the high cell-to-cell correlation between them (Fig. 6D; Pearson's r = 0.78, p < 10⁻⁹, across 126 spatiotemporal cells), assures that our fMRI measurements, once corrected for nonspecific background fluctuations, are reliable enough to delineate the cortical sites that encode the fine stimulus differences with high fidelity on a trial-to-trial basis.

Having characterized the V1 signature of stimulus by specifying which cortical sites carry that signature and when that signature is formed in relation to the stimulus onset, we set out to characterize the signature of choice in the same manner with the aim of examining whether the two signatures originate from the same neural population at the same time. This examination puts to a critical test the deterministic view of sensory neurons' role in perceptual decision, which posits that perceptual judgments on otherwise identical stimuli are caused by trial-to-trial fluctuations in responses of the neuronal ensemble that participates in encoding sensory features of relevance to a given perceptual task (Newsome et al., 1989; Salzman et al., 1990; Celebrini and Newsome, 1994; Britten et al., 1996; Shadlen et al., 1996). Hence, we reasoned, as have previous single-cell studies (Celebrini and Newsome, 1994; Britten et al., 1996; Parker et al., 2002; Romo et al., 2002; Uka and DeAngelis, 2004; Purushothaman and Bradley, 2005; Gu et al., 2007; Gu et al., 2008; Law and Gold, 2008; Ghose and Harrison, 2009; Law and Gold, 2009; Price and Born, 2010; Smith et al., 2011; Liu et al., 2013), that, if the causal view is correct, significant CPs should be found in the vicinity of the spatiotemporal cells at which the significant SPs were identified.

The observed pattern of CPs, indeed, was inconsistent with this prediction in four important ways. First, we failed to observe significant CPs at any of those seven cells that housed significant SPs (Fig. 6A, SP panel, *; Fig. 6B, blue or red filled circles) in the trial-related matrix of the TRs (CP = 0.51 ± 0.06, 0.49 ± 0.05, 0.49 ± 0.05, 0.50 ± 0.05, 0.51 ± 0.06, 0.51 ± 0.06, and 0.50 ± 0.05; CP_stimulus=M = 0.51 ± 0.06, 0.52 ± 0.08, 0.51 ± 0.08, 0.51 ± 0.11, 0.52 ± 0.10, 0.51 ± 0.07, and 0.51 ± 0.06; from foveal to periphery, respectively; mean ± SD across observers; Fig. 6A, CP and CP_stimulus=M panels). Second, instead, the significant CPs were found at the cells wherein the insignificant SPs were found. The cells with the significant CPs were quite advanced in time and far away from the site with direct stimulation in space (Fig. 6A, CP and CP_stimulus=M panels, *; TFCE-corrected p < 0.05). Only 1.1 s after the stimulus onset (fMRI activity at which probably reflects neural activity occurring before the stimulus onset), the responses at the cortical site representing a region very close to the fovea (1.51°; Fig. 6A, CP and CP_stimulus=M panels, blue pixels with *; Fig. 6C, blue open circle) were greater in the S-choice trials than in the L-choice trials, whereas the responses at the cortical site representing the far periphery (5.65°; Fig. 6A, CP_stimulus=M panel, red pixel with *; Fig. 6C, red open circle) were greater in the L-choice trials than in the S-choice trials. Third, in search of any hints of the meaningful relationship between the SPs and the CPs, we also considered the possibility that, despite the mismatch in statistical significance, the SPs and the CPs might have been weakly correlated with one another. However, the correlation analysis, which was conducted over the entire ensemble of cells constituting the trial-related matrix of the TRs, showed that the spatiotemporal distribution of CPs was not correlated (Fig. 6E; Pearson's r = −0.03, p = 0.74, for CP_stimulus=Ms) or even anticorrelated (Pearson's r = −0.38, p < 10⁻⁹ for CPs) with that of SPs. Fourth and finally, we further checked the possible involvement of the stimulus-encoding cortical sites in representing choice-associated information by comparing the spatiotemporal pattern of the CPs with that of the model prediction of differential responses to the stimuli (Fig. 2E, L-S panel). Again, we failed to observe any significant correlations between the CPs and the model predictions (Pearson's r = −0.15 and 0.02, p = 0.09 and 0.84, for CPs and CP_stimulus=Ms, respectively).

Decoding stimulus and choice information from raw responses with population read-out weights

So far, reliable signatures of stimulus or choice were available only in the TRs derived by removing nonspecific background fluctuations from the RRs in which those TRs were embedded. Does this imply that the large-scale, moment-to-moment fluctuations in “untuned” activity impose a fundamental limitation on V1's capacity to carry stimulus- or choice-related information? That implication is not necessarily correct. Instead, the failure to find reliable signatures in the RRs could reflect the limitation of our “local coding” strategy, which evaluated the stimulus- or choice-related variability in neural responses confined to local cortical sites separately. Indeed, if a decision stage in the brain relies on population coding to interpret sensory signals within V1 (Paradiso, 1988; Pouget et al., 2000), fluctuations in untuned activity, a substantial fraction of which is shared by an entire population of encoding neurons, can be efficiently canceled at the decision stage regardless how large those fluctuations are.

To test this hypothesis, we revisited the RRs, this time decoding stimulus signals and choice signals from the RRs over the entire extent of the eccentricity matrix and computing the stimulus probabilities and choice probabilities by comparing the trial-to-trial distributions of those decoded signals at the population level. We will refer to these probabilities as population SPs and population CPs to distinguish them from the probabilities estimated at the individual cells of the trial-related matrix. For population decoding, we developed three different read-out weight profiles, implementing the three major decoding schemes proposed by previous studies (Gold and Shadlen, 2001; Jazayeri and Movshon, 2006; Graf et al., 2011; Berens et al., 2012; Haefner et al., 2013). The simplest form of read-out weights was a uniform read-out, in which the eccentricity bins are divided into either the “fovea” pool or the “periphery” pool, with uniform weights assigned to the bins within each pool (Fig. 7A). In the remaining two weight profiles, the eccentricity bins had nonuniform weights that were derived from the eccentricity-tuning curves according to two different task-optimal decoding schemes. In one scheme, the read-out weights were proportional to stimulus discriminability of given cortical sites (Fig. 7B), which are similar to the profile of model-predicted differential responses at the time frame with hemodynamic peak. The other scheme determined read-out weights by evaluating the contributions of given cortical sites to probabilistic inference of differences between the S-ring and L-ring stimuli (Fig. 7C), estimated by log-likelihood ratios between tuning responses to those two stimuli (for detailed definitions of the three weight profiles, see Materials and Methods).

• Download figure
• Open in new tab
• Download powerpoint

Figure 7.

Population decoding of stimulus and choice information in raw responses. A–C, Weight profiles defined by three different population decoding schemes. Individual symbols represent arbitrary-unit weight values assigned to eccentricity bins. D, Time courses of across-observer averages of population SPs (green line and symbols) and population CP_stimulus=Ms (orange line and symbols). The circles, triangles, and squares represent the uniform (A), discriminability (B), and log-likelihood ratio (C) weights, respectively. The salient open and filled symbols represent the probability values significant (uncorrected) at p < 0.05 and p < 0.005, respectively. Error bars indicate SEM across observers. E, Comparison of population and individual SP values for the RRs. The blue open circles represent the SP values from the 10 foveal bins, which were adjusted for preference by (1 − SP). The red open circles represent the SP values from the 10 peripheral bins. The eccentricity bin corresponding to the M-ring (2.84°) is not shown. The pale green line indicates the population SPs with the discriminability weight. F, Comparison of population and individual CP_stimulus=M values for the RRs. Blue open circles represent the CP values from the 10 foveal bins, which were adjusted for preference by (1 − CP). Red open circles represent the CP values from the 10 peripheral bins. The pale orange line indicates the population CPs with the log-likelihood ratio weight.

All three decoding schemes resulted in similar outcomes, each revealing clear-cut signatures of stimulus and choice in the RRs at the time points where the significant SPs and CPs were found for the TRs (Fig. 7D). The population SPs were significant at 3.3 s and 5.5 s after stimulus onset (Fig. 7D, green-filled markers; uncorrected p < 0.005). These signatures were seen using all the three read-out schemes, but they were most conspicuous in the “discriminability” read-out (Fig. 7D, green-filled triangles). In contrast, the population CPs were significant only 1.1 s after stimulus onset and were strongest when derived using the log-likelihood ratio scheme (Fig. 7D, yellow-filled squares; uncorrected p < 0.005). This unmistakable dissociation between the population SPs and CPs estimated in the RRs, both in time and in profile shape, neatly dovetails with the results from the local SPs and CPs estimated in the TRs, further corroborating our conclusion that V1 carries stimulus and choice signatures that are embodied in different neural ensembles at different points in time.

The advantage of the population coding strategy over the local coding strategy was substantial in RRs: the best population probabilities (Fig. 7E,F, filled markers) surpassed all of the individual probabilities estimated at the local cells of the trial-related matrix of the RRs (Fig. 7E,F, open circles; for the 10 bins located in the foveal bank, their individual probabilities were adjusted for preference by taking 1-SP or 1-CP, so that they can be directly compared with the population SPs and CPs). This analysis verifies that the RRs, despite including a substantial untuned component, retain sufficient information for supporting perceptual judgments at a subsequent decision stage.

Given the advantageous effect of population coding in the RR signals (Fig. 7E,F), why does population coding not do better when applied to TRs (Fig. 8A,B)? Why, in other words, are population SPs and population CPs no larger than the best SPs and CPs exhibited by local cells of the trial-related matrix? One obvious possibility is that the beneficial effect of pooling signals from neurons with similar preferences is limited when those neurons' responses are highly correlated across trials (Averbeck et al., 2006). To check that possibility in the case of our TRs, we calculated pairwise temporal correlations among those TRs and found that the responses from nearby eccentricity bins are indeed highly correlated even after removal of global fluctuations (Fig. 8C). These correlations might reflect moment-to-moment cofluctuations among neurons with similar stimulus preferences, but they might have arisen because of our method for combining voxel signals (Fig. 3B) and/or the spatially correlated nature of the fMRI signal.

• Download figure
• Open in new tab
• Download powerpoint

Figure 8.

Population versus individual probabilities in tuned responses. A, Comparison of population and individual SP values for the TRs. The format is identical to that in Fig. 7E. The pale green line indicates the population SPs with the discriminability weight applied on the TRs. This line is identical to the line in Fig. 7E because those weights have removed the global fluctuations that distinguish TRs from RRs. B, Comparison of population and individual CP_stimulus=M values for the TRs. The format is identical to that in Fig. 7F, and again the population values are identical to those in Fig. 7F for the same reason mentioned above. The black X indicates the average (0.52) of the individual CP values at 1.1 s after stimulus onset. C, A matrix of across-observer average correlations in TRs between the eccentricity bins.

Eye movements as a potential origin of choice signature in V1?

Our primary goal was to examine the “causal” hypothesis regarding the role of V1 activity in trial-to-trial variability of perceptual choice. Although our results are inconsistent with the causal hypothesis, we were puzzled about why the choice-related cortical activity, which does not match stimulus-related responses either in timing or in neural origin, appeared in V1. One possible explanation for that puzzle attributes the seemingly errant activity to eye movements. So, accordingly, we tested eye movement-related hypotheses as the origin of V1 choice signature by conducting an eye-tracking experiment.

Although the observers in the fMRI experiment were explicitly instructed to maintain their gaze on the fixation dot throughout an entire scan run, their eyes may well have moved unintentionally (Ratliff and Riggs, 1950; Martinez-Conde et al., 2004). And it is known that tiny movements, such as drifts, tremors, and microsaccades can affect V1 neural activity (Gur et al., 1997; Martinez-Conde et al., 2000; Snodderly et al., 2001; Tse et al., 2010). Thus, we first considered these involuntary fixational (“miniature”) eye movements as a possible origin for the observed CPs in V1. Given the advanced temporal locus of the significant CPs (0–2.2 s after the onset of the ring stimulus; Fig. 6A, CP and CP_stimulus=M panels) and the hemodynamic delay of fMRI signal in the current study (4.4–6.6 s), which can be estimated from the locus of the significant SPs (Fig. 6A, SP panel), we were particularly interested in whether there were any differential eye movements associated with perceptual choices at the temporal bin spanning 4.4–2.2 s before the ring stimulus onset (Fig. 9, shaded rectangular areas), when no stimulus was presented other than the small fixation dot. This absence of a ring stimulus at the moment of the CPs in our study makes fixational eye movements unlikely to be the cause of the CPs because microsaccades, which occur with the greatest amplitude among the major fixational eye movement types, cause no changes in V1 neural activity in the absence of a stimulus other than a fixation mark (Martinez-Conde et al., 2000, 2002). However, because the impact of various fixational eye movements, including microsaccades, on fMRI measurements in V1 in the absence of stimulation has never been measured directly and because fixational eye movements might alter perception possibly via attention (Hafed et al., 2011) or gain modulation (Hafed, 2013), we explored the possibility that fixational eye movements, which were not monitored during the fMRI experiment, might have generated the choice signature in V1. This possibility can be tested behaviorally because one necessary (but not sufficient) condition for eye movements to generate the CPs observed in the fMRI experiment is that eye movements occurring at ∼4.4–2.2 s before stimulus onset should predict observers' choices yet to be made after stimulus presentation. We searched for eye movements that satisfy this necessary condition by repeating the same experiment on a new batch of observers outside the scanner, but now with their eye movements being tracked throughout the entire experiment. The stimuli, procedure, and number of observers (23) in this eye-tracking experiment were otherwise identical to those in the original fMRI experiment (see Materials and Methods). As expected, the SC and resulting performance of this new batch of observers (0.023 ± 0.006 and 81.6 ± 5.1%, respectively; mean ± SD across observers) were comparable with those who participated in the fMRI experiment.

• Download figure
• Open in new tab
• Download powerpoint

Figure 9.

Results of the eye-tracking experiment. A, Accuracy and precision of gaze position measurements in the visually guided saccade task. The horizontal and vertical gaze measurements obtained under three different fixation positions (cross-shape polygons; different symbols) are plotted against each other for individual observers (small gray symbols). The black symbols with error bars represent the group means with 0.5*SDs pooled over observers for each fixation mark. B–F, Eye movements during the ring-size discrimination task. The mean and variability statistics for five different aspects of eye movements are plotted against the time points relative to the stimulus onset, which matched the time axis of the trial-related fMRI matrix after being adjusted for hemodynamic delay. The dotted and dashed vertical lines indicate the onset of the “ready” cue and ring stimulus, respectively. The shaded rectangular areas represent the time bin at which the significant CPs were found in the fMRI experiment. B, Eye blinks. Top, Across-observer average (black line) time course of percentage blink rate with SEM (shaded area). Bottom, Plot of the differences in blink rate between the M-ring|S-choice and M-ring|L-choice trials, with error bars indicating SD/2 across observers. C, Microsaccades. Top and bottom, Time course of microsaccade frequency and the ratio of rightward microssacades, respectively, at each 2.2 s bin. Different symbols represent the trial types. The dashed curve indicates the average of the S- and L-ring trials and serves as a baseline. Error bars indicate SD/2 across observers. D, Changes in pupil size. Top, Time course of pupil size changes around its mean. Bottom, CPs computed for the entire trials (diamonds) and for M-ring trials (circles). Error bars indicate SD/2 across observers. E, Gaze positions. The format is identical to that in D. The vertical black bar represents the width of the fixation mark. F, Changes in vergence angle. Filled symbols in the bottom panel represent the statistically significant CPs (uncorrected p < 0.05).

Before conducting the eye-tracking experiment, we assessed the accuracy and reliability (precision) of eye movement measurements obtained with our video-based eye-tracker (for its model and specifications, see Materials and Methods) by computing the mean and variance, respectively, of visually guided saccades made to three different fixation targets (Fig. 9A, cross-shape polygons). Here, the fixation targets were spatially separated by only 0.12°, closely matched to the average radius difference between the small and large rings, which was quite small but big enough to produce differential fMRI responses (SPs) in V1. The eye position measurements in the horizontal axis were both accurate, as indicated by the small amounts of deviations of the mean positions from the true position (0.02 ± 0.11°; mean ± SD across observers and conditions), and precise, as indicated by the small SDs across trials (0.23 ± 0.09°; mean ± SD across observers), thus providing a spatial resolution that can reliably distinguish eye position differences up to 0.1° (see Materials and Methods). In contrast, the measurements along the vertical direction were quite noisy, as indicated by the large SDs across trials (0.50 ± 0.44°; mean ± SD across observers; compare the horizontal and vertical error bars in Fig. 9A), thus unable to detect reliably the eye position difference of 0.1°. This relatively poor resolution along the vertical axis was expected because vertical eye position measurements in video-based eye-trackers are notoriously unreliable because the eyelid often occludes the pupil. The incidence of pupil occlusion varies substantially across individuals and over time during prolonged fixation due to fluctuations in cognitive states, such as arousal or fatigue. For these reasons, we only used eye movement data along the horizontal axis to detect microsaccades (Fig. 9C) and to estimate gaze positions (Fig. 9E,F).

First, we checked whether choices correlated with the frequency of eye blinks, which are known to affect fMRI activity in human V1 in the absence (Bristow et al., 2005) and in the presence (Tse et al., 2010) of retinal stimulation. Although the overall blink frequency (Fig. 9B, top, black line) decreased around the time of stimulus onset (Fig. 9B, dashed vertical lines) and increased afterward, its time course did not differ at any time bins between the two choices either for the M-ring trials only (Fig. 9B, bottom; paired t test across observers, uncorrected 0.11 < p < 0.99) or for the entire set of trials (0.11 < p < 0.65). Next, we checked whether choices correlated with the frequency or direction of microsaccades because a microsaccade may affect the response gain of neurons representing visual regions around its target (Hamker et al., 2008; Hafed, 2013), or the direction of a microsaccade may interact with covert spatial attentional shifts (Hafed and Clark, 2002; Engbert and Kliegl, 2003; Hafed et al., 2011). Consistent with a phenomenon known as microsaccadic inhibition (Rolfs et al., 2008; Hafed and Ignashchenkova, 2013), the overall frequency (Fig. 9C, top, dashed line) decreased after onset of the ready signal (Fig. 9C, dotted vertical lines) and recovered to the baseline level after the choice period. However, neither the frequency (Fig. 9C, top) nor the direction (Fig. 9C, bottom) differed at any time bins between the two groups of choice-sorted M-ring trials (paired t test across observers, uncorrected 0.20<p < 0.97 and 0.21 < p < 0.96, respectively).

Next, we checked whether choices correlated with pupil diameter because changes in pupil size are known to accompany changes in arousal (Hess and Polt, 1960; Bradshaw, 1967; Henson and Emuh, 2010), changes in perceptual interpretation (Einhäuser et al., 2008, 2010), and task-related factors (Hess and Polt, 1964; Kahneman and Beatty, 1966; Nassar et al., 2012). These kinds of uncontrolled changes in pupil size in turn would produce variations in the retinal image (Campbell and Gubisch, 1966) that could induce changes in V1 activity. As suggested by its kin relationship with task structure, the overall pupil size (Fig. 9D, top, black line) fluctuated substantially around the stimulus onset in a manner similar to the blink rate (Fig. 9B, top) and the microsaccade frequency (Fig. 9C, top). However, the pupil size failed to predict the choices made by observers, as indicated by the absence of significant CP at any time bins (Fig. 9D, bottom), including the one spanning 4.4–2.2 s before stimulus onset, either for the M-ring trials (Student's t test across observers, uncorrected 0.52<p < 0.85) or for the entire trials (0.17 < p < 0.99).

Next, we checked whether choices correlate with gaze position, which may affect V1 activity via gain modulation (Trotter and Celebrini, 1999; Rosenbluth and Allman, 2002; Sharma et al., 2003; Merriam et al., 2013). Unlike the previous measurements (blink rate, microsaccade frequency, and pupil size), the overall gaze position (Fig. 9E, top, black line trace) remained virtually stationary, showing only negligible amounts of fluctuation inside the fixation mark (Fig. 9E, right, thick black bar). In agreement with the previous measurements, however, the gaze position did not carry choice information at all (Fig. 9E, bottom; Student's t test across observers, uncorrected 0.13<p < 0.80 and 0.39<p < 0.85; for CP and CP_stimulus=M, respectively).

Last, we checked whether choices correlated with changes in vergence angle because vergence eye movements are known to affect the perceived size of an object (Mon-Williams et al., 1997; Sperandio et al., 2013), which in turn could affect V1 activity (Murray et al., 2006). The vergence angle (Fig. 9F, top, black line trace) was not biased toward the either near or far side before and during the stimulus presentation. However, the significant correlations were found at the two consecutive time bins after stimulus onset (Fig. 9F, bottom, black diamonds; CP = 0.46 ± 0.06 and 0.46 ± 0.07, respectively; mean ± SD across observers; Student's t test, uncorrected p = 0.01 and 0.04, respectively), but not at the time bin (4.4–2.2 s before stimulus onset) associated with the significant CPs found in the V1 fMRI activity (p = 0.63). This temporal mismatch disqualifies vergence angle as a potential origin for the choice signature in V1. Instead, the delayed changes in vergence angle are likely to reflect the vergence control system's automatic reaction to the perceived size of a ring stimulus. This interpretation is supported by the fact that divergence was greater after the ring stimulus was judged to be large than when it was judged to be small (vergence angle difference at 0–2.2 s = −0.05 ± 0.12°; mean ± SD across observers).

In conclusion, our eye movement measurements were accurate and reliable enough to reveal the previously known subtle changes associated with task structure. However, none of the five aspects of eye movements in the time bin at which the significant CPs were found in the fMRI experiment was related to choice behavior. Hence, there is no reason to attribute the choice-related V1 activity found in our fMRI experiment to eye movements. In the following section, we consider other possible reasons for that activity.