Abstract
The study of conscious visual perception invariably necessitates some means of report. Report can be either subjective, i.e., an introspective evaluation of conscious experience, or objective, i.e., a forced-choice discrimination regarding different stimulus states. However, the link between report type and fMRI-BOLD signals has remained unknown. Here we used continuous flash suppression to render target images invisible, and observed a long-lasting dissociation between subjective report of visibility and human subjects' forced-choice localization of targets (“blindsight”). Our results show a robust dissociation between brain regions and type of report. We find subjective visibility effects in high-order visual areas even under equal objective performance. No significant BOLD difference was found between correct and incorrect trials in these areas when subjective report was constant. On the other hand, objective performance was linked to the accuracy of multivariate pattern classification mainly in early visual areas. Together, our data support the notion that subjective and objective reports tap cortical signals of different location and amplitude within the visual cortex.
Introduction
While our subjective experience of a visual percept is direct and immediate, its study necessitates some means of report (Malach, 2007). Such reports may be of two major types: subjective, such as rating the visibility of a target, and objective, such as a forced-choice discrimination regarding stimulus category. However, these two measures may not always agree. A striking example of such dissociation is “blindsight,” extensively documented in cortically lesioned patients (Poppel et al., 1973; Weiskrantz et al., 1974; Cowey, 2004). Blindsight patients deny any subjective visual experience while successfully performing objective visual tasks (Sahraie et al., 1997; Stoerig and Cowey, 1997; De Gelder et al., 2008). Similar, blindsight-like dissociations (also termed implicit perception) have been documented in healthy individuals (Meeres and Graves, 1990; Kolb and Braun, 1995; Lau and Passingham, 2006; Schwiedrzik et al., 2011). Methodologically, such dissociations impose serious difficulties on the search for the neuronal correlates of subjective visual experience. Consider a typical threshold experiment that relies solely on objective reports. One may incorrectly interpret objective target detection in such an experiment as indicative of a true subjective visual experience. Alternatively, relying solely on subjective reports could lead to the incorrect conclusion that, when subjects adopt a high criterion for reporting, subjective experience was lacking, when in reality a trace of subjective experience was present, but not reported. One way to resolve this methodological conundrum is by simultaneously collecting both subjective and objective reports. Indeed, a few EEG studies have used such a combined reporting strategy (Del Cul et al., 2007; Lamy et al., 2009). However, EEG recordings suffer from a poor spatial resolution, which limits their ability to precisely delineate the cortical loci linked to subjective vision.
To obtain a better anatomical localization, one needs to rely on BOLD fMRI. In an elegant fMRI experiment, Lau and Passingham (2006) adopted such a parallel subjective and objective reporting strategy during a metacontrast masking paradigm. However, in the Lau and Passingham study, both target and mask were simple shapes, thus unlikely to induce differential activations in high-order visual areas. Note that such differential “signatures” are critical to uncover the link between subjective visibility and differential fMRI activity. To examine the potential link of human visual areas to subjective perception as well as to subliminal performance, we designed an experimental paradigm that, on the one hand, provides both subjective and objective reporting, and, on the other hand, uses target and mask configurations that lead to differentiable responses to target and mask throughout the visual hierarchy of cortical areas. Our paradigm was based on the observation that the recently introduced method of continuous flash suppression (CFS) (Tsuchiya and Koch, 2005), was particularly effective in generating blindsight-like performance in normal observers. Using CFS, we found a significant link between differences in subjective perception (under equal objective performance) and BOLD activation in high-order visual areas. Differential objective performance (under equal subjective reports), on the other hand, was mostly associated with differential activity patterns in lower-order visual areas.
Materials and Methods
Subjects.
Twenty subjects with normal visual acuity participated in this study. Subjects gave written informed consent and were paid for their participation. Two subjects were excluded because of excessive head movements during fMRI scans. Of the remaining 18 subjects (9 female, average age: 28 years, range 20–42 years), 17 were right handed. Eye dominance was assessed by means of the hole-in-card test as a modified version of the original ABC test (Miles, 1930).
Apparatus and experimental protocol.
Stimuli were video-projected in a dual-projector setup (Thompson et al., 2008) onto a screen behind the head coil at a viewing distance of 100 cm using two identical projectors (LCD EMP74, Epson) with a refresh rate of 60 Hz, and two comparable PCs (2.4 GHz, 3 GB RAM, NVidia GeForce 8500GT) running Windows XP (Microsoft) and Matlab 2008a (The MathWorks). For dichoptic stimulation, linear polarizing filters (TechSpec linear polarizing film, Edmund Optics) were mounted in front of each projector's lens. The screen maintained the polarization of the light projected through it. Subjects viewed the screen with linear polarizers (orientation of polarization 45° and 135°) mounted in a MRI-compatible plastic frame. All subjects confirmed that there was no detectable “ghosting” (or cross talk) at full image contrast, i.e., the leaking of an image to one eye, when it is intended exclusively for the other eye. Stimuli were presented using the Cogent Toolbox (John Romaya, Vision Lab, UCL; http://www.vislab.ucl.ac.uk/). For CFS stimulation, one PC generated the target stimuli, while the other PC presented the continuous flash stimuli. The two PCs communicated via the serial port with latencies of <1 ms. Care was taken that target stimuli were never presented before or after the sequence of flash stimuli (maximal delay of 1 refresh frame = 17 ms).
Twelve different grayscale images of tools and 12 grayscale images of faces (250 × 250 pixels, 4.2° × 4.2° visual angle) served as target stimuli and were presented to the subject's nondominant eye in one of the four quadrants (top left, top right, bottom left, bottom right; 2.1° eccentricity) on each trial. All stimuli were shown to the subjects before the experiment. A Gaussian kernel with a radius of 6 pixels was used to blur the edges of the stimuli to facilitate suppression. Image contrast was modulated by a Gaussian envelope with an SD of 65 pixels. Mondrian-like images (7.4°) served as continuous flash stimuli and were presented to the subject's dominant eye at 10 Hz. On each flash a new stimulus was used (i.e., no immediate repetitions). Mondrian masks were generated by randomly positioning 1000 black, white, and medium gray squares of different size (ranging from 0.25° to 1.7°). A red fixation cross (0.5° visual angle) at the center of the screen and a black and white square frame (0.5°) were displayed throughout all trials. In each trial, subjects had to detect the presence of a target stimulus and report its location (Fig. 1A). The same target image was presented 10 times for 300 ms, each time followed by a blank period of 300 ms. During the first 100 ms of each target presentation, the contrast of the target stimulus was gradually ramped up to avoid an abrupt onset and thus facilitate perceptual suppression. Each CFS trial of 6 s was followed after 4 s by two response screens of 2 s duration each. The first response screen instructed subjects to make a forced-choice guess (4AFC) indicating via button press the quadrant in which a target image was presented. Next, subjects were asked to give a subjective visibility rating on a three-point scale (1 = nothing detected, 2 = image parts, 3 = whole image); they were instructed to give a rating of “1” if they did not perceive more than the flashing Mondrian masks alone. Subjects were required to perform both tasks simultaneously, i.e., differentiate the target from the Mondrians and note where it is located. The next trial started after a blank screen of variable duration (range: 1.5–4.5 s). Each scan session of ∼8.5 min duration consisted of 24 target-present trials with threshold-contrast Mondrians, and 4 target-present trials with low-contrast Mondrians. In 11 subjects, two target-absent trials (catch trials) with threshold-contrast Mondrian masks were added per session. Subjects were not informed about the number of target-present and target-absent trials. All twenty-four tool and face images were used at least once within a session, and the order of trials was randomized. All subjects performed five scan sessions.
Contrast of stimuli.
Tool and face stimuli were presented at 80% contrast. To adjust the contrast of Mondrian stimuli, we used a staircase procedure to determine target detection thresholds individually for each subject (Levitt, 1971). While the staircase procedure and the main experiment were performed under identical viewing conditions inside the scanner, there was a difference of trial structure including timing parameters (staircase experiment: 1.8 s stimulation, 1 s blank, left/right 2AFC; main experiment: 6 s stimulation, 4 s blank, quadrant 4AFC, visibility rating). Subjects performed the staircase experiment during the MPRAGE anatomical scan (∼9 min); two staircases (∼50 trials each) were completed during this period, one for faces and one for tools. The higher Mondrian contrast was used during the main experiment for both stimulus categories. The difference between Mondrian contrasts obtained for face and tool stimuli was not significant (faces: 32 ± 4%, tools: 27 ± 5%; t(17) = 1.26, p = 0.226; two-sided paired t test). The average Mondrian contrast used in the main experiment was 33 ± 5%. Finally, we tested whether the target stimuli remained fully visible in the low-contrast Mondrian condition (5% contrast), which was the case for all subjects.
Acquisition and processing of fMRI data.
Functional images were acquired by T2*-weighted gradient-echo echo-planar imaging (35 slices, flip angle = 75°, TR = 2000 ms, TE = 30 ms, voxel size 3 × 3 × 4 mm) on a 3T MRI scanner (Tim Trio, Siemens). Functional images were resliced into an iso-voxel resolution of 3 × 3 × 3 mm. We recorded five experimental sessions of 258 volumes each and a localizer session of 228 volumes. The first two volumes were skipped to account for T1 saturation effects. Anatomical images were acquired using a T1-weighted MPRAGE sequence (176 slices, flip angle = 9°, TR = 2300 ms, TE = 2.98 ms, FOV 256, voxel size 1.0 × 1.0 × 1.1 mm). We used BrainVoyager QX 2.2.0 (Brain Innovation) for image preprocessing [slice scan time correction, 3D motion correction, high-pass filter with 2 cycles/experiment, normalization to Talairach stereotactic space (Talairach and Tournoux, 1988), spatial smoothing with an isotropic Gaussian kernel of 6 and 8 mm full-width-half-maximum (FWHM) for single-subject and group analyses, respectively] and estimation of statistical maps using a general linear model (GLM) approach (Friston et al., 1994) with six rigid-body realignment parameters as nuisance covariates. In the main experiment, the “invisible” (i.e., “nothing” response in visibility rating) and “visible” (i.e., “stimulus parts” and “whole stimulus” responses) CFS conditions were modeled as epochs of 6 s duration. Response screens were modeled separately as epochs of 4 s duration. In the localizer experiment, all conditions were modeled as epochs of 9 s duration.
Cortical surfaces were reconstructed from the anatomical images using BrainVoyager QX 2.2.0. The procedure included segmentation of the white matter using a grow-region function, the smooth covering of a sphere around the segmented region, and the expansion of the reconstructed white matter into the gray matter. The sulci were smoothed using a cortical inflation procedure. The surface was cut along the calcarine sulcus and unfolded into the flattened format. The obtained statistical maps were superimposed on the unfolded cortex, and the Talairach coordinates were determined for the center of each region of interest (ROI). All group maps are based on random-effects models (Friston et al., 1999). To correct for multiple comparisons in the whole-brain statistical maps, we implemented a randomization technique to estimate a corrected cluster-level confidence for the entire volume (at α = 0.05, 1000 iterations) using the BrainVoyager Cluster-level Statistical Threshold Estimator Plugin (Forman et al., 1995; Goebel et al., 2006). The method uses a nonparametric Monte Carlo simulation that calculates the likelihood of obtaining a cluster of randomly generated voxels across the entire volume at the given individual voxel probability threshold. In-house software and the NeuroElf toolbox (Jochen Weber, http://neuroelf.net/) were used for batch processing of fMRI data and calculation of parameter estimates.
Following Lamy et al. (2009), we aimed at estimating chance-free parameter estimates in trials reported as “invisible” but with correct 4AFC performance, according to the following rationale: BOLD activity in invisible correct trials represents a mixture of (1) neural responses to invisible trials that were responded to correctly due to sufficient perceptual processing and (2) neural responses to invisible trials that were responded to correctly by chance. As described by Lamy et al. (2009, pp. 1444–1445), we therefore first calculated the actual proportion of invisible correct trials that were correctly responded to by chance, separately for each subject. Then, we estimated the BOLD activity corresponding to chance-free invisible correct trials, using brain responses in invisible incorrect trials as an index for neural activity associated with chance trials. The estimation of chance-free parameter estimates did not change the significance of results in our study, but since they provide a better estimate of activity in invisible correct trials related to available sensory evidence, they are reported in the univariate results section.
fMRI localizer experiment.
We conducted a localizer fMRI session for each subject to identify cortical regions that responded to the target and mask stimuli. We used a standard block design with 10 different blocks of stimuli: tool stimuli in one of the four visual quadrants (top left, top right, bottom left, bottom right), face stimuli in one of the four quadrants, central Mondrian masks at the subject-specific contrast as determined in the staircase procedure, and centrally presented full-contrast Mondrian masks. Image size and location were the same as in the main experiment. During each block, nine stimuli were presented (800 ms per stimulus; 200 ms interstimulus intervals). Blocks were separated by 6 s blank periods, and each condition was repeated over three blocks in counterbalanced order (total: 30 blocks, ∼7.5 min). To maintain attention, subjects had to indicate by a button press when the fixation point changed its color from red to green, which happened at variable time points twice per block.
Definition of regions of interest.
We defined several ROIs, from which we extracted parameter estimates at the single-subject level. Tool- and face-sensitive high-order visual areas as well as early visual cortex were identified on a subject-by-subject basis in two steps. First, at the group level, the contrast “tools/faces > fixation” (regardless of stimulus location) was mapped at p < 0.001 (corrected at p = 0.05 for cluster-level false-positive rate, minimal cluster size 86 mm2) and projected on an inflated and flattened cortical surface. Based on this map (shown in Fig. 2A), search spaces around the centers of three different peak activations in the left and right hemisphere were defined: lateral occipital area (LO) and posterior fusiform gyrus (pFS) belonging to the lateral occipital complex (LOC) (Malach et al., 1995; Malach et al., 2002), and intraparietal sulcus (IPS). It should be noted that high-order visual areas show a strong bias to central visual field stimulation (Levy et al., 2001), leading to reduced responses to our off-centered targets during the localizer experiment when compared to the Mondrian stimuli, which overlapped central fields. Thus, the contrasts “tools/faces > Mondrians,” “tools > Mondrians,” and “faces > Mondrians” yielded less reliable activations and were therefore not used for ROI definition. To verify the target-selective activation during the main experiment, we tested for a differential BOLD response to target stimuli in visible trials (i.e., target plus Mondrian stimuli) versus Mondrian stimuli alone in the main experiment. All high-order visual areas showed a significant target-selective response (pFS: t(10) = 4.30, p = 0.002; LO: t(10) = 2.81, p = 0.018; IPS: t(10) = 2.68, p = 0.023; two-sided paired t tests). For early visual cortex, we mapped the contrast “Mondrians > tools/faces” (regardless of stimulus location) at the group level at p < 0.001 (corrected at p = 0.05 for cluster-level false-positive rate, minimal cluster size 90 mm2), as in our previous study (Hesselmann and Malach, 2011). Next, for each subject's corresponding first-level contrasts, all voxels within the search spaces were selected that passed a more lenient threshold (p < 0.001 to p < 0.005, uncorrected). Table 1 summarizes location and size of all defined ROIs. Parameter estimates were averaged across the voxels within the ROIs and subjected to repeated-measures ANOVAs and subsequent planned paired t tests to test for differences between experimental conditions.
Talairach coordinates for the center of ROIs and their sizes (in number of functional 3 × 3 × 3 mm voxels)
Multivariate fMRI data analysis.
The support-vector-machine (SVM) method of multivariate pattern analysis (MVPA) was used to test whether BOLD response patterns in previously selected regions of interest could be used to differentiate between different perceptual states (Cox and Savoy, 2003; Mur et al., 2009). We tested the following perceptual states using binary SVM classification: visible tools versus visible faces; invisible tools versus invisible faces; targets inside versus targets outside a retinotopically defined region of interest in early visual cortex for visible trials, invisible correct trials and invisible incorrect trials; invisible correct versus invisible incorrect trials, regardless of target location. For classification, we used a linear SVM with standard cost parameter (c = 1) in the implementation of the LIBSVM 3.0 library by Chih-Chung Chang and Chih-Jen Lin (http://www.csie.ntu.edu.tw/∼cjlin/libsvm) running under Matlab R2009b (The MathWorks). Pattern classification was based on single-trial β values from unsmoothed 3 × 3 × 3 mm voxels in Talairach space. All voxels from a given ROI were used. Classification performance was assessed using a leave-one-trial-out method (Sterzer et al., 2008; Misaki et al., 2010). The dataset was randomly subdivided into a training set of N pattern vectors (vector length = number of voxels), and a test set of two pattern vectors, one from each condition. Before training, the β values in the training set were scaled (min 0, max 1); scaling was performed across samples, and the same scaling parameters were applied to the test set. The SVM classifier was iteratively trained on the training datasets (N) and then tested on two independent test datasets. For each perceptual state the available numbers of trials were as follows: visible face trials: 34.7 ± 2.9; correct invisible face trials: 10.6 ± 1.2; incorrect invisible face trials: 14.1 ± 2.3; visible tool trials: 18.2 ± 2.2; correct invisible tool trials: 16.2 ± 1.6; incorrect invisible tool trials: 24.9 ± 2.1 (average trials ± SEM). The training and testing procedure was repeated 100 times. Care was taken by means of random selection that the training sets were balanced across conditions, meaning that they consisted of the same number of trials for each condition. The resulting classification accuracies for all test data assignments were averaged. For statistical analysis at the group level, the average accuracies from each participant (N = 18) were then submitted to one-sample, one-sided t tests to test for significant positive deviations from chance level (50%).
Results
Behavioral results
On each trial, subjects (n = 18) had to make a forced-choice decision (4AFC) on the location of a target stimulus (either a face or a tool) and rate its visibility. Subjective visibility was rated on a three-point scale (“nothing,” “parts,” and “whole”), and objective forced-choice target localization could either be correct or incorrect. To render stimuli invisible in ∼50% of trials, we used a variant of the CFS paradigm (Tsuchiya and Koch, 2005; Tsuchiya et al., 2006) with individually adjusted contrasts of Mondrian masks (Fig. 1A). As shown in Figure 1B, subjects rated targets as invisible in 54.9 ± 3.7% of trials, and reported seeing stimulus parts in 34.7 ± 4.0%, and whole stimuli in 10.4 ± 2.9% (average rating frequency ± SEM). The 4AFC performance levels for these three perceptual states were 43.0 ± 3.3%, 91.5 ± 3.1%, and 91.2 ± 3.4%, respectively.
Paradigm and behavioral results. A, Schematic diagram of the CFS paradigm. Target stimuli (tools and faces) were presented to subjects' nondominant eye (ND) using polarizing filters and glasses, while Mondrian masks were flashed to the dominant eye (D) at 10 Hz. Targets and Mondrian stimuli were presented intermittently (300 ms on, 300 ms off) for 6 s. Target stimuli could appear in one of the quadrants relative to fixation. After each trial, subjects were instructed to make a forced-choice guess (4AFC) regarding in which quadrant they had detected the target stimulus, and rate the target's visibility on a three-point scale (1 = nothing detected, 2 = parts, 3 = whole). B, Results of visibility ratings in target-present trials with threshold-contrast Mondrian masks. Subjects reported that they had not detected the target stimulus in 55% of trials. Reports of target parts and fully visible targets were less frequent (35 and 10%). Insets represent the illustration of the three possible perceptual states, which were grouped into “invisible” and “visible” trials for all further analysis. Error bars represent ±SEM. C, Correct 4AFC performance in “invisible” trials across all subjects. The black dashed line represents chance level (25%). The red dashed line represents the average 4AFC performance under invisibility (43.0 ± 3.3%).
For all further analysis, trials were grouped into visible (“whole” and “parts” ratings) and invisible trials (“nothing” ratings). The frequency of invisible trials ranged between 19.7% and 81.7% across subjects. Figure 1C shows the range of correct 4AFC performance in invisible trials (22.4–72.4%); the average 4AFC performance was significantly above chance level (t(17) = 5.51, p < 0.001; two-sided paired t test against 25%), thus indicating a dissociation between subjective and objective report. Visibility of targets was not significantly modulated by their location in one of four quadrants (F(3,51) = 2.91, p = 0.062, e = 0.737; repeated-measures ANOVA, Greenhouse–Geisser corrected). Analysis of image category revealed that tools were more frequently reported as invisible than faces (68.9 ± 3.6% vs 40.9 ± 4.7%; t(17) = 7.02, p < 0.001), and that 4AFC performance was slightly higher in invisible face trials than in invisible tool trials (51.7 ± 5.5% vs 40.5 ± 3.5%; t(17) = 2.20, p = 0.042). When targets were presented together with low-contrast Mondrians, subjects reported seeing the targets in most trials (nothing: 4.6 ± 1.8%, parts: 11.1 ± 3.5%, whole: 84.3 ± 4.8%), and 4AFC performance was at 97.0 ± 5.7%. In “catch trials” with Mondrian masks but without targets, subjects gave “nothing” ratings in virtually all trials (96.4 ± 1.5%).
Univariate fMRI results
To obtain a sensitive measure for the BOLD activation levels under the different perceptual conditions, we performed an ROI analysis. The regions in early and high-order visual cortex were defined on a subject-by-subject level based on a localizer experiment (Fig. 2A, Table 1). Since no differences were found between activity in the left and right hemisphere, we collapsed the ROI data across hemispheres (“laterality”: F(1,17) = 0.03, p = 0.867; “laterality × ROI”: F(3,51) = 1.06, p = 0.365, e = 0.776; 2 × 4 repeated-measures ANOVA, Greenhouse–Geisser corrected).
Statistical parametric maps. A, Multisubject map of the localizer experiment presented on an unfolded cortical surface. The statistical group map was obtained by mapping the contrast “tools/faces > rest” at p < 0.001 (corrected at p = 0.05 for cluster-level false-positive rate, minimal cluster size 86 mm2). Regions of interest were identified individually for each subject in the ventral stream, and in the dorsal stream. Early visual cortex was identified based on the contrast “Mondrians > tools/faces” (not shown). B, Multisubject map of the visibility effect in the main experiment. The statistical group map was obtained by mapping the contrast “visible trials > invisible correct trials” at p < 0.001 (corrected at p = 0.05 for cluster-level false-positive rate, minimal cluster size 81 mm2). Both maps were projected on flattened hemispheres of a single subject. A, Anterior; P, posterior; LH, left hemisphere; RH, right hemisphere.
Figure 3A shows that we found a significant visibility effect in all high-order visual areas (pFS: t(17) = 3.75, p = 0.002; LO: t(17) = 5.17, p < 0.001; IPS: t(17) = 3.25, p = 0.005; two-sided paired t tests), but no difference between invisible correct and invisible incorrect trials (pFS: t(17) = −1.20, p = 0.246; LO: t(17) = −0.42, p = 0.680; IPS: t(17) = 0.66, p = 0.518). Testing the visibility effect selectively for trials with comparable 4AFC performance (visible trials vs invisible correct trials), yielded highly similar results (pFS: t(17) = 4.58, p < 0.001; LO: t(17) = 4.31, p < 0.001; IPS: t(17) = 2.32, p = 0.033). Early visual cortex showed the opposite pattern, i.e., no effect of subjective visibility report (t(17) = −0.37, p = 0.716), but slightly larger parameter estimates (i.e., estimated BOLD response amplitudes) in invisible incorrect than in invisible correct trials (t(17) = −2.20, p = 0.042). For the subset of 11 subjects whose scan sessions also included target-absent trials (“catch trials”), the difference between BOLD activity in invisible trials and trials in which only Mondrian masks were presented was not significant (early visual: t(10) = −0.94, p = 0.369; pFS: t(10) = 1.39, p = 0.194; LO: t(10) = −0.23, p = 0.823; IPS: t(10) = 2.08, p = 0.064).
Region-of-interest analysis. A, Univariate analysis. Red bars represent parameter estimates for visible trials; green bars: parameter estimates for invisible correct trials (corrected for chance); blue bars: parameter for invisible incorrect trials. B, Multivariate analysis. Performance of SVM classification for pairwise classification of face and tool presentations. Filled squares represent average prediction accuracies for visible trials, empty squares represent average prediction accuracies for invisible trials. Dotted lines denote chance level (50%). Error bars represent ±SEM (*p < 0.05; **p < 0.01; ***p < 0.001).
To directly test for a relationship between BOLD activity levels and forced-choice performance in invisible trials across subjects, we correlated the difference between parameter estimates in invisible correct and invisible incorrect trials with 4AFC performance under invisibility, but found no significant correlation in any of the ROIs (all p > 0.79). In criterion models, the difference in objective performance between individuals is explained by criterion shifts. Thus, individuals with a more conservative (higher) subjective decision criterion will tend to report subjective invisibility even for high sensory signals, leading to their superior 4AFC performance during the subjective invisibility condition. A straightforward consequence of such criterion models is that crossing the subjective visibility threshold for high 4AFC performers will require particularly high sensory signals, i.e., they should show higher sensory activation during the subjective visibility condition compared to poor 4AFC performers. To test this effect, we correlated the 4AFC performance with the absolute BOLD activation during the subjective visibility condition, as well as the relative BOLD activation comparing visible and invisible incorrect conditions. The results of this analysis showed no significant correlation in both absolute and relative tests (all p > 0.16).
To check to what extent the previously described between-category variability in stimulus visibility as well as the within-category variability (i.e., certain stimulus exemplars were more visible than others) influenced BOLD activity levels, we conducted two control analyses. First, we tested for an effect of visibility separately for both stimulus categories. Tool stimuli showed visibility effects in all high-order visual ROIs, while face stimuli showed effects in pFS and LO, but failed to show an effect of visibility in IPS (Fig. 4A). Second, we tested whether the parameter estimates in the different ROIs were modulated by two factors: “category” and “visibility” (high/low, based on a median split, Fig. 4B). Neither factor reached significance (all p > 0.14). Together, these results show that despite considerable between- and within-category variability in behavioral performance, BOLD activity levels in the selected ROIs were only modulated to a small degree by category and exemplars. As a further exploratory result, we found that invisible correct trials yielded smaller BOLD signals than invisible incorrect trials (Fig. 4A). In early visual cortex, pFS and LO, this effect showed as a weak trend for face stimuli, and was significant for tool stimuli (early visual: t(14) = −2.89, p = 0.012; pFS: t(14) = −2.57, p = 0.022; LO: t(14) = −2.15, p = 0.049; two-sided paired t tests). More research adding additional task manipulations and categories of stimuli is needed to clarify the source of this intriguing effect.
Analysis of stimulus category and exemplars. A, Analysis of the visibility effect, separately for faces and tools (N = 15). Three subjects were excluded because they did not have sufficient trials (at least two) in one of the conditions. Red bars represent parameter estimates for visible trials; green bars: parameter estimates for invisible correct trials (corrected for chance); blue bars: parameter for invisible incorrect trials. Error bars represent ±SEM. B, Analysis of within-category variability (N = 16). Two subjects were excluded because they did not have sufficient trials (at least two) in one of the conditions. In both stimulus categories, we grouped exemplars into two subsets of high visibility and low visibility based on the median (“low visibility” faces: 48 ± 5.9%; “high visibility” faces: 69 ± 5.0%; “low visibility” tools: 20 ± 3.7%; “high visibility” tools: 42 ± 5.0%). None of the regions showed a significant effect (repeated-measures ANOVA with factors “visibility” and “category”). As an exploratory result, early visual cortex showed a small but significant effect of visibility for face stimuli (t(15) = 2.18, p = 0.045, two-sided paired t test). *p < 0.05; **p < 0.01; ***p < 0.001.
Finally, we performed a whole-brain random-effect analysis of the visibility effect by mapping the contrast “visible trials > invisible correct trials.” As shown in Figure 2B, we found significant clusters of BOLD activity in areas pFS, LO and IPS/IPL in both hemispheres, in agreement with the results of the ROI analysis. Additional clusters were found in bilateral inferior frontal gyrus. The right parietal cluster appeared to be larger and extended into the superior parietal lobe, while the left parietal cluster had a more inferior center of mass. Because of the different contrasts used, the LOC, including areas LO and pFS, had a more anterior focus than the ROIs that were defined based on the localizer experiment, but overall there was good correspondence. The contrast “invisible correct trials > invisible incorrect trials,” keeping subjective visibility constant, yielded no significant clusters, even at a more lenient threshold (p < 0.01, uncorrected).
Multivariate fMRI results
To test whether our fMRI data and implementation of multivariate pattern analysis, MVPA (Kamitani and Tong, 2005; Haynes and Rees, 2006) could be used to successfully replicate earlier findings of category selectivity under visibility and invisibility in a CFS paradigm (Sterzer et al., 2008), we first aimed at decoding ROI response patterns related to face and tool stimuli, on a trial-by-trial basis. To that end, trials (visible/invisible face trials and visible/invisible tool trials) were divided into training and test datasets and submitted to SVM classification with leave-one-trial-out cross-validation. Figure 3B shows that average prediction accuracies for visible trials were significantly above chance level in all ROIs (early visual: 58.1 ± 3.2%, t(16) = 2.59, p = 0.010; pFS: 62.6 ± 2.6%, t(16) = 4.76, p < 0.001; LO: 59.4 ± 2.9%, t(16) = 3.19, p = 0.003; IPS: 57.9 ± 2.8%, t(16) = 2.83, p = 0.006; one-sided t tests against 50%; one subject was excluded because he produced no visible tool trials). For invisible trials, only pFS (which very likely overlapped partly with the fusiform face area) showed a significant prediction accuracy above chance level (early visual: 50.2 ± 2.4%, t(16) = 0.461, p = 0.680; pFS: 54.5 ± 2.3%, t(16) = 1.94, p = 0.035; LO: 51.8 ± 2.4%, t(16) = 0.75, p = 0.232; IPS: 47.9 ± 2.1%, t(16) = −0.98, p = 0.829; one subject was excluded because he produced only one invisible face trial). Limiting the SVM classification to the 100 most responsive voxels in each ROI (based on the localizer experiment) yielded similar results.
Next, we conducted a retinotopic analysis of BOLD responses in early visual cortex using SVM decoding. We tested separately for visible, invisible correct, and invisible incorrect trials whether the location of the target stimulus could be predicted with accuracy above chance. To transform the test of four locations into binary tests, we classified each location (e.g., top left) against the remaining locations taken together (e.g., top right, bottom left, bottom right); for each visibility condition, the resulting prediction accuracies were averaged across these four tests. Figure 5A shows that in visible trials, the target stimulus location could be decoded with an accuracy significantly above chance level (65.9 ± 3.1%, t(17) = 5.21, p < 0.001; one-sided t test), which was not the case for invisible trials (invisible correct: 52.6 ± 2.1%, t(17) = 1.23, p = 0.117; invisible incorrect: 47.7 ± 2.7%, t(17) = −0.84, p = 0.794).
Retinotopic multivariate analysis of voxels in early visual cortex. A, Results of SVM classification for classification of target stimulus location. Red squares represent average prediction accuracies for visible trials, green squares represent invisible trials with correct 4AFC response, and blue squares represent invisible trials with incorrect 4AFC response. The dotted line denotes chance level (50%). Error bars represent ±SEM (*p < 0.10; ***p < 0.001). B, Correlation between the accuracy difference (invisible correct minus invisible incorrect trials) and 4AFC performance under invisibility. Open squares represent data from single subjects. The solid line represents the fitted linear regression.
In analogy to the univariate fMRI analysis reported above, we then tested for a relationship between SVM prediction accuracies and forced-choice performance in invisible trials across subjects; we correlated the difference between prediction accuracies in invisible correct and invisible incorrect trials with 4AFC performance under invisibility. As shown in Figure 5B, the correlation turned out to be highly significant, using parametric (Pearson's r = 0.61, p = 0.007) and nonparametric (Spearman's r = 0.62, p = 0.007) coefficients. A partial correlation analysis with individual ROI size as covariate produced similar results. Importantly, however, the same correlation analysis with subjective performance (percentage “visible” trials) yielded no significant result (Pearson's r = 0.17, p = 0.507). The effect was found only in early visual cortex, arguing against a more global source for this effect, such as a differential number of correct trials across individuals.
Finally, we aimed to separate invisible correct BOLD responses from invisible incorrect responses using SVM classification in all ROIs, regardless of the target's location. All trials (invisible correct trials and invisible incorrect trials) were divided into training and test datasets and submitted to SVM classification. Prediction accuracies were not different from chance level (early visual: 53.0 ± 2.2%, t(17) = 1.37, p = 0.095; pFS: 53.5 ± 2.4%, t(17) = 1.47, p = 0.080; LO: 51.1 ± 2.2%, t(17) = 0.47, p = 0.322; IPS: 50.9 ± 2.2%, t(17) = 0.39, p = 0.351; one-sided t tests). To test for a relationship between prediction accuracies and 4AFC performance in invisible trials across subjects, we calculated linear correlation coefficients for all ROIs. Figure 6 shows that correlations for early visual cortex, pFS and IPS were not significantly different from zero (early visual: r = −0.24, p = 0.341; pFS: r = 0.01, p = 0.967; IPS: r = 0.17, p = 0.510; Pearson's r), while LO showed a significant relationship between SVM accuracies and behavioral performance, using parametric (Pearson's r = 0.59, p = 0.009) and nonparametric (Spearman's r = 0.55, p = 0.019) coefficients. Similar results were obtained when using a partial correlation analysis with individual ROI size as covariate, but correlation between SVM accuracies and subjective performance (percentage “visible” trials) yielded no significant result (Pearson's r = 0.03, p = 0.891). When correlation coefficients were calculated separately for voxels from the left and right hemisphere, none of the high-order visual areas yielded significant results (all p > 0.15).
Multivariate analysis of BOLD responses related to behavioral performance under invisibility. Prediction accuracies of SVM classification (invisible correct vs invisible incorrect trials) are plotted against 4AFC performance under invisibility. Open squares represent data from single subjects. The solid line represents the fitted linear regression (with Pearson's correlation coefficient r and associated p value).
Discussion
Using a variant of the recently introduced CFS paradigm (Tsuchiya and Koch, 2005; Sterzer et al., 2009) with individually adjusted mask contrasts, we were able to generate a dissociation between subjective (visibility rating) and objective (4AFC discrimination) behavioral reports in normal observers, i.e., a blindsight-like phenomenon (Weiskrantz et al., 1974; Meeres and Graves, 1990; Cowey, 2004). Our results, based on univariate ROI analysis (amplitude of BOLD activation) and statistical parametric mapping, showed that high-order visual areas manifest a widespread and significant activation associated with subjective visibility reports. Thus, our findings are in agreement with a number of previous studies using fMRI (Tong et al., 1998; Grill-Spector et al., 2000) and local field potentials (Fisch et al., 2009), which have shown a tight link between subjective reports of visibility and brain activity in high-order visual cortex. However, here we extend these previous results by showing that they can occur even when the objective performance of the subjects is comparable in visible and invisible trials. Importantly, the physical properties of visual stimulation were also kept identical during the visible and invisible conditions, so that the observed differences in BOLD activation could not be attributed to changes in the optic stimulation. Examining BOLD signals in early visual cortex failed to reveal a significant difference, supporting the notion that the activations related to subjective visibility originated within high-order visual areas and were not inherited from low-level changes. However, it is important to qualify this conclusion by noting that, unlike the target selective activations in high-order areas, early visual cortex showed strong activation to the mask stimuli in isolation (salient Mondrians flashed at 10 Hz), which may have obscured additional target-related activations superimposed on this massive mask-related activity.
In contrast to subjective visibility, changes in objective performance under identical subjective visibility conditions (correct and incorrect trials during subjective invisibility) failed to show a significant difference in overall BOLD activation in high-order visual ROIs; early visual cortex showed a small differential activation, but paradoxically, it was in the opposite direction (incorrect > correct). Such “negative” going effects could potentially be used by the visual system to guide the correct responses under invisibility. However, the opposite sign underlying objective performance provides additional evidence to the notion that objective and subjective reports are mediated by qualitatively different signals, with the subjective report consistently linked to positive “ignition”-like cortical activations (Dehaene et al., 2006; Fisch et al., 2009). A whole-brain analysis comparing invisible correct and invisible incorrect trials yielded no significant clusters, even at more lenient statistical thresholds. Importantly, BOLD activity levels did not correlate with 4AFC performance in any of the visual ROIs. Finally, we failed to find increased target activation during visible conditions in high 4AFC performers as would be expected if this performance was due to a more conservative (higher) criterion for subjective visibility in these individuals.
By contrast, using the more sensitive multivariate method of SVM classification (Cox and Savoy, 2003; Kamitani and Tong, 2005; Haynes and Rees, 2006), we found a highly significant correlation between subjects' objective localization performance and the classifiers' prediction accuracies for retinotopic BOLD activity in early visual cortex. A significant correlation was also found between subjects' performance and prediction accuracies related to correct and incorrect invisible trials in the object-related area LO, but only when voxels from the left and right hemisphere were analyzed together. These results indicate that subtle changes in BOLD activation, not detectable in the average signal from a given ROI, provided sufficient information to allow for correct target localization, and hence correct 4AFC responses, in invisible trials.
Overall, our findings demonstrate that the signals that drive subjective experience and those that drive objective performance are substantially different, both in signal amplitude and in their anatomical organization. These results are in agreement with a model—which is based on perceptual decision-making models (Smith and Ratcliff, 2004; Heekeren et al., 2008)—proposing that during nonconscious processing, sensory evidence is accumulated until a threshold for conscious stimulus reportability is reached (Dehaene, 2008). However, our results cannot resolve the source of the weak signals driving the correct localization performance, i.e., whether they originated from early visual cortex, bypassing high-order visual areas altogether, or whether they depended also on the LO activity patterns. The fact that the most striking illustration of blindsight has been observed in patients with V1 lesions (Cowey, 2004; Stoerig, 2006) suggests that V1 activation is not critical in driving motor responses related to objective report, but it is not clear whether the situation of patients can be extended to healthy individuals, since brain adaptation and long-term plasticity processes may have modified cortical processing in the patient population.
The blindsight phenomenon has an important advantage in allowing a better match between many nonperceptual parameters, when shifting between the visible and invisible subjective states. Thus, when contrasting the visible correct versus invisible correct conditions, both visual stimuli and the correct motor responses were equated. This allows for a better isolation of neuronal correlates of the subjective state; our results (Fig. 2B) highlight high-order visual areas as well as frontal regions. The frontal activation is intriguing since it may reflect a global spread of activity, hypothesized to play a critical role in perceptual awareness (Dehaene and Changeux, 2011). However, it is important to caution that even under such exceptionally balanced conditions, one cannot rule out subtle nonperceptual differences associated with the changes in subjective state. For example, it could be that target visibility was linked to higher arousal, greater semantic processing or some other task-related aspects; all these secondary processes could potentially contribute to the observed BOLD modulations (Malach, 2007). However, at least for high-order visual areas, the preferential target activation appeared to be task invariant (Fig. 2A,B), further supporting their link to visual rather than task-specific aspects.
It could be argued that the observed difference between the objective and subjective reports was not related to subjective state, but rather to the stimulus parameter needed to give these reports (location vs presence).While we cannot completely rule out this possibility, it should be noted that in all classic blindsight studies, such a discrepancy between objective and subjective reports was the common practice. Another point to note is that the subjective task was a simple detection task, not necessitating object judgment or recognition for its successful performance. Thus, a possible strategy by which subjects could successfully detect the presence of a target could be based on comparing the visual stimuli across different quadrants in the visual field, a task quite similar to identifying target location. Some supporting evidence for that possibility can be derived from the observation presented in Figure 5A, where it can be seen that there was a significant improvement in the classifier performance in retinotopic cortex for visible correct compared to invisible correct trials.
A number of previous studies have examined behavioral blindsight-like phenomena in normal observers. Meeres and Graves (1990) reported the localization of tachistoscopically presented unseen visual stimuli above chance level; Kolb and Braun (1995) have used a binocular rivalry and motion paradigm under situations in which subjects' confidence levels were completely dissociated from their objective performance (but see Morgan et al., 1997). Of particular relevance to the present work are studies that looked at brain signals during blindsight-like performance. Lamy et al. (2009) have measured EEG responses during a backward masking experiment in which subjects were asked to localize a target consisting of tilted line segments (figure–ground detection task). Their results are compatible with the present findings in showing a clear enhancement of postsensory EEG signals associated with subjective visibility under equal objective performance levels. However, they find a significant enhancement of evoked EEG signals for the objective correct 4AFC performance during invisible conditions, which in our study was found only using a multivariate classifier approach. It could be that the evoked EEG signals may be more sensitive during blindsight-like perceptual states compared to BOLD activation. Alternatively, it should be noted that the target stimuli (tools/faces vs line segments) and stimulation paradigms used were drastically different. Thus, differences in nonconscious processing between CFS and backward masking have been reported (Almeida et al., 2008). In conclusion, any of the mentioned differences in stimulation may change the signals associated with subjective and objective reports and thus may explain the differences in our results.
The present results have important methodological implications for studies attempting to relate brain activity to conscious perceptual processes. Note that in our paradigm we were able to analyze trials in which the sensory stimulation as well as the objective forced-choice performance of the subjects were constant, yet the BOLD activation in high-order visual areas was significantly modulated. Thus, relying solely on objective performance may miss important cognitive processes reflected in subjective report and brain activations (Dehaene and Naccache, 2001; Dehaene, 2008). Arguably, trials with different subjective and constant objective performance are special cases, and measuring the overall objective performance will still show substantial correlation with the subjective report (Del Cul et al., 2007), yet the unknown influence of subjective state may limit the interpretability of studies based purely on forced choice objective response. As a note of caution regarding the interpretation of our results, it is important to mention that we used a paradigm with a delayed response; thus, memory-related and perceptual processes might have overlapped (Courtney et al., 1998). However, in our paradigm, the memory-related aspects of the subjects' task were very easy (target's presence and location).
Together, our results highlight the tight link between subjective, introspective states and BOLD activity in high-order human visual areas. Thus, they emphasize the importance of adding introspective estimates of conscious perception in brain imaging studies. A simple approach could be to obtain both objective and subjective reports, which will together provide a more complete and informative assessment of subjects' perceptual states.
Footnotes
G.H. was supported by a Minerva fellowship (Max Planck Society) and a grant from the Deutsche Forschungsgemeinschaft (HE 6244/1-1). R.M. was supported by the Israel Science Foundation, The Hellen and Martin Kimmel Award for Innovative Investigation, Bikura, Mark Scher's estate grants, and the European 7th Framework Programme (VERE). We thank E. Okon for invaluable technical assistance.
The authors declare no conflict of interest.
- Correspondence should be addressed to Dr. Guido Hesselmann, Department of Neurobiology, Weizmann Institute of Science, 76100 Rehovot, Israel. g.hesselmann{at}gmail.com
