Abstract
When two dissimilar stimuli are presented to the eyes, perception alternates between multiple interpretations, a phenomenon dubbed binocular rivalry. Numerous recent imaging studies have attempted to unveil neural substrates underlying multistable perception. However, these studies had a conceptual constraint: access to observers' perceptual state relied on their introspection and active report. Here, we investigated to what extent neural correlates of binocular rivalry in healthy humans are confounded by this subjective measure and by action. We used the optokinetic nystagmus and pupil size to objectively and continuously map perceptual alternations for binocular-rivalry stimuli. Combining these two measures with fMRI allowed us to assess the neural correlates of binocular rivalry time locked to the perceptual alternations in the absence of active report. When observers were asked to actively report their percept, our objective measures matched the report. In this active condition, objective measures and subjective reporting revealed that occipital, parietal, and frontal areas underlie the processing of binocular rivalry, replicating earlier findings. Furthermore, objective measures provided additional statistical power due to their continuous nature. Importantly, when observers passively experienced rivalry without reporting perceptual alternations, a different picture emerged: differential neural activity in frontal areas was absent, whereas activation in occipital and parietal regions persisted. Our results question the popular view of a driving role of frontal areas in the initiation of perceptual alternations during binocular rivalry. Instead, we conclude that frontal areas are associated with active report and introspection rather than with rivalry per se.
Introduction
Our brain continuously infers a unique perception from ambiguous information. Rivalry makes this ambiguity explicit, with awareness alternating between multiple interpretations of a constant stimulus (Dutour, 1760; Necker, 1832). Specifically, “binocular rivalry” occurs when dissimilar images are presented to the eyes (Wheatstone, 1838).
A pioneering study on neural foundations of binocular rivalry tagged one stimulus by having it flicker at a fixed frequency (Lansing, 1964). Whenever this stimulus dominated perception, the flicker frequency was seen prominently in the electroencephalogram (EEG). EEG activity signaling a preceding reversal first occurs at occipital electrodes (120 ms after stimulus onset) and later at parietal (250 ms) and frontal (410 ms) sites (Kornmeier and Bach, 2005).
Using fMRI, rivalry-related activity was located throughout the visual hierarchy: in the lateral geniculate nucleus (LGN; Haynes et al., 2005; Wunderlich et al., 2005), striate cortex (V1; Polonsky et al., 2000; Tong, 2003; Lee et al., 2005; Meng et al., 2005), and higher regions (Tong et al., 1998). Pattern recognition techniques allow decoding the perceptual state from V1 and early extra-striate (V2, V3) activity (Haynes and Rees, 2005). With the exception of LGN (Lehky and Maunsell, 1996), the ubiquity of rivalry representations also holds in nonhuman primates: neurons responsive to the dominant percept (rather than to the stimulus) are found throughout the visual hierarchy (Logothetis and Schall, 1989; Leopold and Logothetis, 1996), with increasing numbers toward higher areas (Sheinberg and Logothetis, 1997).
Although visual areas represent physical and rivalry switches, BOLD activity in human parietal and frontal regions seems exclusive to rivalry switches (Lumer et al., 1998) and further evidence has been mounting that frontoparietal networks play a causal role in triggering perceptual switches (Leopold and Logothetis, 1999; Sterzer and Kleinschmidt, 2007; Sterzer et al., 2009). Knapen et al. (2011) have recently challenged this notion, finding that in frontoparietal regions, perceptual switches only elicited a different neural activity from abrupt physical stimulus changes. If instead transitions mimicked the wave-like pattern of actual rivalry transitions, activity for rivalry and physical switches was indistinguishable. As possible explanations, the investigators suggested that frontoparietal networks react to changes in perceptual experience, attentional load, or task demands associated therewith. Here, we extend this suggestion. We note that the report of one's perception modulates rivalry (Beets et al., 2010) and, in addition to the pure motor act, requires continuous self-monitoring of one's own perceptual state. Such self-monitoring is typically attributed to frontal areas, as evidenced by lesion (Lezak, 1983) and imaging data (Gusnard et al., 2001). Therefore, we hypothesize that the frontal activation typically seen in rivalry is attributable to increased self-monitoring around perceptual switches. Under this hypothesis, frontal activation should be largely reduced when no active report is required. To link neural activity continuously to an observers' perceptual state in the absence of report, we adapted a recent paradigm (Naber et al., 2011) for fMRI. Contrasting BOLD activity for passive experience to active report allows for the first time dissociating effects of rivalry per se from self-monitoring perceptual changes.
Materials and Methods
Observers.
Twenty observers (9 male, 11 female, age: 19–30 years) participated in the experiment. All were healthy and had no history of neurological or psychiatric diseases. Further inclusion criteria were right-handedness with an index exceeding +70 according to the Edinburgh Inventory of Handedness (Oldfield, 1971), a visual acuity index >0.9 without visual aids according to the Freiburg Visual Acuity and Contrast Test (Bach, 1996), and normal color vision according to the Farnsworth Test (Farnsworth, 1943). Except for one (S.F.), all observers were naive to the purpose of the experiment. They gave informed written consent before the study. The study conformed to the Declaration of Helsinki and was approved by the local ethics committee of the medical faculty of the Philipps-University Marburg.
Stimuli.
We adapted a previously developed paradigm that allows decoding the observers' perception in binocular rivalry without relying on their active report (Naber et al., 2011) for the use in fMRI. The paradigm exploits the optokinetic nystagmus (OKN) and the pupil reflex as objective measures of the currently dominant percept. These “reflexes” follow the perceived rather than the physical stimulus (perceived motion direction and luminance, respectively) and have been widely used in human (Enoksson, 1963; Lorber et al., 1965; Brenner et al., 1969; Fox et al., 1975; Fahle et al., 2011; Naber et al., 2011) and animal studies (Logothetis and Schall, 1990; Fries et al., 1997) of rivalry, although not in the context of human imaging.
We used two types of binocular-rivalry stimuli, static and dynamic gratings. For the dynamic stimulus, the currently dominant percept was decoded from the direction of the OKN slow phase; for the static stimulus, the currently dominant percept was decoded from pupil size. Using two stimulus sets and two distinct measures allows for some generalization of our design across binocular-rivalry types.
The OKN stimulus set consisted of two large sinusoidal gratings (height 22.2 degrees, width 30 degrees of visual angle, spatial frequency: 0.27 cycles per degree) drifting in opposite lateral directions (i.e., left and right) at an absolute speed of 22.3 deg/s (Fig. 1A, left). Because the gratings had dissimilar colors (i.e., red and green) and opposite movement directions, all observers experienced binocular rivalry as percepts alternated between a green grating moving to the left and a red grating moving to the right (and vice versa). In addition, the moving gratings induced an OKN with a slow phase directed to either the left or the right, depending on the currently dominant stimulus. Using pilot data, the luminance of the gratings had been adjusted such that both percepts were dominant for approximately equal periods and the luminance settings were kept constant across all observers. Indeed, there was no consistent bias for color (average dominance of green grating: 49.7 ± 0.6%; t(19) = −0.06, p = 0.96) and approximately half of the individuals (8/20) had a preference for green, with all individual preferences falling in the range between 43% and 58% (with 50% preference implying equal amount of dominance for both gratings). Similarly, there was no preference for direction (average dominance of rightward moving grating: 51.4 ± 0.9%; t(19) = 0.83, p = 0.42) with a preference for the rightward percept in 12/20 individuals (range: 41–57%).
The pupil stimulus set consisted of two sinusoidal gratings with a spatial frequency of 1.2 cycles per degree in a circular aperture of 14.7 degree diameter on a black background. Gratings had dissimilar colors (red and green as above), were stationary, and were oriented in opposite directions (60 degrees clockwise or counterclockwise, respectively; Fig. 1B, left). To induce a smaller pupil whenever one (the green) grating was dominant, the luminance of the red grating was chosen to be substantially smaller than the green grating. Despite the difference in luminance, there was no consistent bias toward either percept (average dominance of high-luminance grating: 50.5 ± 1.3%; t(19) = 0.36, p = 0.72); exactly half of the observers (10/20) had slight preference for the high-luminance grating, the other half for the low-luminance grating, with individual preferences ranging between 43% and 63% for the high-luminance percept.
All stimuli were presented via MRI-compatible goggles (VisualSystem; Nordic NeuroLab) attached to the MR head coil using a custom-made mounting system.
Experimental design.
The experiment consisted of 32 trials lasting 60.9 s each (corresponding to 42 functional whole brain scans). To avoid fatigue, the experiment was split into two sessions of 16 trials. Sessions were interleaved with a 15 min break during which structural scans were acquired. Each session used one rivalry type (OKN/pupil). For each rivalry type, there were four conditions: a rivalry stimulus (rivalry trials) or a matched physically changing stimulus (replay trials), and observers either actively reported visual perception (active-report trials) or just passively experienced ongoing rivalry dynamics (passive-viewing trials). Together with the two rivalry types, the experimental design therefore was a three-way repeated measures within-subjects design. Each of the three factors had two levels (TYPE: OKN/pupil; RIVAL: rivalry/replay; REPORT: active report/passive viewing), resulting in a total of eight different conditions; each condition was performed four times during the experiment.
In rivalry trials, we presented different stimuli to each eye. In replay trials, we presented the same stimuli to both eyes, but simulated rivalry by physically changing both gratings. The temporal dynamics of these physical alternations were based on observers' button presses of the preceding rivalry trial with active report. In this way, we mimicked the time course of binocular rivalry without actually exposing the observer to an ambiguous situation.
In active-report trials, observers had to indicate perceptual dominance of one or the other percept by holding down the right-hand index or middle finger on a MRI-compatible response box (i.e., perceptual alternations were characterized by switching from one to the other button). Observers were explicitly instructed to always indicate one percept as dominant (i.e., most dominant), even when both percepts were partially visible (piecemeal rivalry). In passive-viewing trials, observers just passively experienced perceptual alternations without reporting their percept.
In the present study, the button presses served two functions: (1) to verify the validity and robustness of the objective measures and (2) to ensure that observers needed to self-monitor their own perceptual state more for active-report trials than for passive-viewing trials. Therefore, we decided to use a straightforward instruction that all observers could readily comply with inside of the fMRI setting. As a consequence, our button-press data only contains information about the dominant percept, but not about any degree of piecemeal rivalry, which is likely to occur to some degree given the size of stimuli (O'Shea et al., 1997). Whereas in the debriefing, most observers indeed mentioned periods of piecemeal rivalry, each observer reported a unique percept (by holding exactly one button) at least 95% of the time. This indicates that all observers were able to comply with the instruction. Importantly, all main comparisons (between passive viewing and active report) were based on the objective measures (OKN/pupil). Because, in both cases, rivalry-replay contrasts were considered (see section ‘Comparing active report with passive viewing’), any effect of piecemeal rivalry would affect both conditions (active-report/passive-viewing) alike. It seems unlikely that the objective measures themselves, being continuous measures of rivalry dynamics, were affected by the exact instruction regarding button presses.
To allow matching the temporal statistics of the simulated switches in replay trials closely to the endogenous switches in awareness in rivalry trials, the trial sequence was as follows: active report rivalry, active report replay, passive viewing rivalry, passive viewing replay. This sequence was repeated four times for each rivalry type. Both replay conditions simulated switches based on the timing of the button presses in the preceding active-report rivalry trial. The order of rivalry types (OKN/pupil) was balanced across observers.
OKN velocity and pupil size as objective measures of perceptual state.
To obtain OKN or pupil size as objective measures of perceptual alternations for the respective rivalry type, videos of observers' right eye were recorded at a rate of 60 Hz using the camera integrated into the goggles. Videos had a resolution of 640 × 480 pixels, with 1 pixel corresponding to approximately 0.04 degrees of visual angle (because absolute position information was not needed in the present experiment, we refrained from performing a calibration of the eye-tracker to display coordinates per individual). Pupil size and location was extracted using an open-source implementation of the starburst algorithm (Li et al., 2005) in MATLAB R2009b (MathWorks).
OKN.
Because the head was fixed relative to the camera and only eye velocity (rather than absolute position) was of interest in the present context, the position of the pupil provided a sufficiently robust estimate of gaze direction to analyze the horizontal component of the OKN. Analysis focused on the OKN′s slow phase (the OKN component in the direction of the motion direction of the stimulus), whereas the OKN′s fast phase (the component quickly reorienting the eye opposite to the stimulus direction) was removed. Fast phases were detected as phases in which the horizontal speed changed by >300 pixels/s in 500 ms or the acceleration exceeded 2500 pixels/s2 and verified by visual inspection, as were blink onsets. After removal of blinks and fast phases, the resulting signal was interpolated linearly and high-frequency noise was suppressed by averaging with a 500-ms-wide sliding average.
As the OKN changed direction according to the dominant percept, the OKN slow-phase velocity moved back and forth from positive (rightward movement) to negative (leftward movement) velocities (Fig. 1A, right). The time points when horizontal eye velocity crossed zero were defined times of perceptual alternations, provided the preceding maximum (minimum) was larger (smaller) than 12 pixels/s (−12 pixels/s) and the temporally adjacent zero-crossing was at least 400 ms in temporal distance.
Pupil size.
To make data commensurate across observers, pupil size (area) in each trial was normalized to z-score (zero mean and unit variance). Blinks were detected as for the OKN analysis and pupil size was interpolated linearly across blinks. High-frequency noise was suppressed by averaging with a 167-ms-wide (10 samples) sliding average. From this low-pass-filtered signal, the low-frequency trend, computed as sliding average over 4.167 s (250 samples), was subtracted. Zero-crossing points of these z-normalized and detrended pupil traces that were preceded by a sufficiently high maximum/minimum (±0.3 SD) and at least 400 ms apart from each other were detected as timings of observers' perceptual alternations during binocular rivalry.
MR image acquisition.
Time courses of the observers' brain activity were acquired using a 3-tesla MR scanner (Siemens) with a 12 channel head matrix receive coil at the Department of Psychiatry and Psychotherapy, Philipps-University Marburg. A T2*-weighted gradient-echo echo-planar-imaging sequence (EPI) was used to provide functional images sensitive to the blood oxygen level dependent (BOLD) contrast (parallel imaging factor of 2 [GRAPPA], 30 slices, TR = 1450 ms, TE = 25 ms, matrix size 64 × 64 voxels, voxel size 3 × 3×4 mm, gap size = 0.6 mm, flip angle 90°). Slices of the functional whole brain scans were acquired in descending order and tilted relative to the intercommissural (AC-PC) plane by 20° to reduce EPI artifacts (Weiskopf et al., 2006; Weiskopf et al., 2007). A high-resolution anatomical image was acquired using a T1-weighted magnetization-prepared rapid gradient-echo (3d MP-RAGE) sequence in sagittal plane (176 slices, TR = 1900 ms, TE = 2.26 ms, matrix size 256 × 256 voxels, voxel size 1 × 1×1 mm, flip angle 9°) to allow for subsequent coregistration with the functional images. During the experiment, the observers' head was placed in a helmet made of foam within the MR head coil to minimize scanner noise and head movements.
Processing of functional MR imaging data.
Functional image preprocessing and analysis were conducted using the SPM8 software package (Statistical Parametric Mapping, Wellcome Trust Center for Neuroimaging, London, UK; http://www.fil.ion.ucl.ac.uk) and MATLAB R2009b. The first four scans were discarded from the analysis. Functional images were realigned to the mean image of each individual to control for small head movements during the experiment. Using slice timing correction, we further controlled for within-volume acquisition time differences. The corrected functional images were coregistered with the high-resolution anatomical image and then spatially normalized into the Montreal Neurological Institute (MNI) standard space using the unified segmentation-normalization approach (Ashburner and Friston, 2005). Finally, normalized functional images were spatially smoothed with an isotropic Gaussian smoothing kernel at a full-width at half-maximum of 6 mm.
Neural activity in each voxel of every observer was identified by means of a general linear model (GLM; Friston et al., 1995; Worsley and Friston, 1995). We used an event-related approach to define the timings of perceptual alternations during binocular rivalry.
Statistical power of continuous measures.
Because OKN speed and pupil size do not only serve as objective measures of perceptual switches, but also provide information on a continuous scale, we were first interested in determining to what extent such continuous measures may enhance the statistical power of a GLM. We therefore defined three GLMs incorporating distinct information from the subjective or objective measures during active-report trials as regressors in the design matrix. In addition, the six motion parameters of both sessions obtained during realignment of the functional scans were entered into each GLM as nuisance regressors to control for movement-related artifacts. A high-pass filter with a cutoff period at 512 s was used to remove low-frequency noise, whereas the experimental variability remained unaffected.
For the first GLM (“discrete subjective” analysis), the time points when observers pressed a button were used as discrete onsets in our model. In the second GLM (“discrete objective” analysis), discrete onsets were defined by the time points for which OKN or pupil size indicated a shift from one percept to the other as defined above. In the third GLM (“continuous objective” analysis), all information on OKN slow phase velocity or pupil size traces was used to form continuous regressors. For this purpose, traces were subsampled to match the repetition time of the functional scans (TR = 1.45 s). To image neural activity as a function of alternations in awareness, only the absolute values of the traces were used and absolute traces were then flipped (subtraction of the maximum and subsequent multiplication with −1). This yielded traces where former zero-crossing points (i.e., timings of perceptual alternations) appeared as regressor peaks in the data curves regardless of the direction of the switch (similar to onsets in an event-related design). To use the continuous regressors in the GLM, flipped traces were convolved with SPM's canonical hemodynamic response function. Fitting all three GLMs to the same functional data allowed us to investigate how well subjective and objective measures explained the neural mechanisms that underlie binocular rivalry in the brain. To do so, we ran linear contrast of brain activation associated with the perception of a rivalry stimulus compared with a replay stimulus (i.e., rivalry > replay) for each of the three GLMs in every observer. The individual contrast images then entered three distinct random-effects group analyses (one-sample t tests) to assess the rivalry-related neural pattern for each analysis separately. To address benefits of the continuous measures directly, we additionally ran post hoc paired t tests to reveal whether differences between the subjective discrete and objective discrete analysis and between the subjective discrete and objective continuous analysis were statistically significant.
Comparing active report with passive viewing.
In a subsequent analysis, we then investigated the effects of active report and introspection compared with passive viewing. By applying the continuous objective analysis based on OKN speed or pupil size traces of passive-viewing trials, we addressed brain activity when observers passively experience rivalry. To reveal effects of active report on the neural pattern, we compared rivalry-related activation during active-report trials to passive-viewing trials for each voxel. To do so, we defined a GLM for each observer based on the continuous objective analysis for active-report and passive-viewing trials. Similar to the previous GLMs, the six motion parameters of each session were entered as nuisance regressors and a high-pass filter (cutoff period at 512 s) was used. From this GLM, we calculated linear contrasts revealing the neural regions of rivalry-related activation during active-report and passive-viewing trials separately. These contrasts were calculated for each rivalry type individually (i.e., restricted to either OKN or pupil trials). This yielded four contrast images for each observer which then entered a random-effects group analysis (two-way within-subject ANOVA). For each rivalry type separately, we defined a linear contrast on the group level that revealed neural regions where rivalry-related activation was higher during active-report compared with passive-viewing trials (“active report rivalry > active report replay” > “passive viewing rivalry > passive viewing replay”). To pinpoint the neural loci that show higher rivalry-related activation for active report than for passive viewing, a global null conjunction analysis was conducted on the resultant two activation patterns (OKN ∩ pupil size). Using the conjunction analysis, we tested whether the obtained results were independent of the exact rivalry type.
For all group-level analyses, anatomical localization and characterization of the neural loci was achieved using the anatomical automatic labeling toolbox within SPM8 (Tzourio-Mazoyer et al., 2002).
Results
We robustly induced binocular rivalry inside the MR scanner by presenting two gratings with distinct motion directions (Fig. 1A, left) or luminance (Fig. 1B, left). Because we were interested in the effects of active report on neural activity patterns, we used two objective, continuous measures (OKN and pupil size) of the dominant percept. We first verified behaviorally that these measures were reliable indicators of the perceptual state and tested how report and introspection influenced switching behavior; second, we replicated earlier fMRI results based on active report alone; third, we tested the statistical power of using OKN or pupil size as regressors in GLMs for fMRI analysis; fourth, we tested whether passive viewing yielded different neural activation patterns compared with active report.
Binocular rivalry and its objective measures. A, Top, Left, OKN stimulus set consisting of two distinctly colored (red/green) sinusoidal gratings that drifted in opposite lateral directions (i.e., left/right). Bottom, Left, Idealized expected OKN patterns: direction of slow phase follows the perceived direction of motion. Right, Raw horizontal gaze speed (gray), smoothed OKN slow phase speed (black), and button presses (purple vertical lines) during a typical active report rivalry OKN trial. B, Top, Left, Pupil stimulus set consisting of two stationary sinusoidal gratings that differed in orientation (i.e., ±60°), color (red/green) and luminance. Bottom, Left, Example images from eye-tracking video with pupil detection, pupil size adapts to the perceived luminance. Right, Detrended pupil size (black) and button presses (purple vertical lines) during an exemplary active report rivalry pupil trial. C, D, Average OKN speed (mean and SEM over observer median traces; C) and average pupil size (D) across observers temporally aligned to the report of a perceptual transition. The horizontal black bars indicate time periods during which traces representing a perceptual switch from percept A to percept B differed significantly from traces representing a perceptual switch from percept B to percept A. The significance criterion was adjusted to an expected false discovery rate of 0.05 (Benjamini and Hochberg, 1995) to correct for multiple comparisons. E, Median dominance durations (i.e., the time period a single percept was visible until alternated for the competing percept) for OKN and pupil trials and for the different report conditions separately. sb - w, subjective with active report; ob - w, objective with active report; ob - wo, objective with passive viewing). Dominance durations indicated by both objective indicators were shorter than those indicated by subjective report. In addition, the switch rate tended to be increased when observers had to report perceptual alternations.
OKN and pupil size are reliable measures of subjective perception
First, we tested for the trials in which observers actively reported their perception whether OKN and pupil-size measures were consistent with subjective report. We found that the speed of the OKN′s slow phase (Fig. 1A, right) and pupil size (Fig. 1B, right) changed as a function of the perceptual state: OKN′s slow phase was directed to the left when observers reported a leftward moving percept to be dominant and to the right when the rightward moving percept was dominant (Fig. 1C). Differences in OKN direction were significant between the two percepts (p < 0.05, t test, FDR corrected, black line in Fig. 1C) at each time point, from −2167 ms to −583 ms before the report of the transition and from −317 ms before to 750 ms after. The crossover point for OKN is seen at −450 ms before the report. This indicates that OKN is consistent with button press data, but may respond somewhat quicker to the perceptual switch than the manual response. A similar pattern was found not only in the average data on the group level, but also for each individual. Similarly, for the stationary gratings differing in luminance, differences in pupil size were significant at each time point from −1767 ms to −767 ms before and from −200 ms to 817 ms around the report (Fig. 1D) and, again, this pattern was consistently observed in each individual.
In addition to testing for significant differences of the average traces, we measured how well OKN and pupil size matched each observer's report. Provided the transition times defined by zero-crossings, OKN and pupil make a prediction of the observer's current perceptual state at any point in time. As straightforward measure for the quality of this prediction, we determined the fraction of time for which the prediction was correct with respect to the button-press data. By testing a range of possible latencies between the objective measure and the button press from 0 to 2 s and using the optimal latency for each individual, we found correct predictions for 82 ± 2% of the time for OKN and 59 ± 1% for pupil size. These numbers increased to 88 ± 2% and 83 ± 1% in replay trials, possibly indicating a better time locking due to the sharper transition in replay compared with more piecemealed transitions in rivalry. All of these numbers were significantly different (all p < 0.001, all t ≥ 7.78) from chance (50%). This verifies that OKN and pupil size are reliable indicators of subjective perceptual experience.
Based on the optimal latencies, we can estimate the lag between objective indicators and manual response to a perceptual transition during genuine rivalry. The transitions in the objective measures preceded the report by approximately 400–500 ms (OKN: 0.52 ± 0.07 s; pupil: 0.42 ± 0.12 s). For rivalry, the precise timing of the perceptual switch is obviously unknown. However, the replay conditions allow an estimate of the typical latencies between physical switch and either measure. The lag between the actual physical switch and the transition in the objective measure (OKN: 0.46 ± 0.03 s, pupil: 0.30 ± 0.11 s) is shorter than the lag between physical switch and report (OKN: 0.60 ± 0.02 s, pupil: 0.61 ± 0.10 s). The smaller lag between objective measures and button presses in replay compared with rivalry is possibly a consequence of the sharper transition in replay, where piecemealing is not simulated. However, even if piecemeal rivalry yields an additional delay, the interindividual variability is on the order of 0.1 s and thus is substantially smaller than the latency between transition and report. Although this does not exclude that longer periods of piecemeal rivalry are experienced subjectively, it indicates consistency as to how the observers comply with instructions. In any case, the objective measures are insensitive against the exact instruction and mode of report and thus against putative effects of piecemeal rivalry. Even if piecemeal rivalry had a substantial effect, it would affect active-report and passive-viewing alike and thus would not confound any of the comparisons between these two conditions.
Active report speeds up rivalry
As a next step, we tested whether active report had an effect on the dynamics of perceptual switching on a behavioral level. When comparing median dominance durations inferred from active report (button presses) with those obtained from the objective measures, we already found longer dominance durations in active-report trials when using button-press information (button press in OKN trials: 1.78 ± 0.16 s; OKN: 1.57 ± 0.14 s; z = −3.14, p = 0.002, Wilcoxon test; button press in pupil trials: 1.74 ± 0.15 s; pupil size: 1.48 ± 0.04 s; z = −2.39, p = 0.017; Fig. 1E), which is consistent with earlier findings (Fahle et al., 2011; Naber et al., 2011). This difference probably reflects that very short periods of dominance may affect the objective measures, even if they do not last long enough to reach the threshold for report. This interpretation is supported by shorter latencies for objective measures compared with the button presses, as computed above. Importantly, when comparing median dominance durations revealed by objective measures for active report with those for passive viewing, rivalry slowed down for passive-viewing trials (OKN: 1.58 ± 0.17 s, pupil: 1.57 ± 0.05 s). Although this effect failed to reach significance for the OKN stimulus set (z = −0.26, p = 0.79), it was significant for the pupil stimulus set (z = −2.91, p = 0.004). This shows that the mere act of actively reporting one's subjective perception influences rivalry dynamics, at least for the stationary gratings.
Neural activity during active report: replication of previous findings
When using only the button-press information, our active-report trials closely mimicked previous paradigms of binocular rivalry in fMRI. Therefore, we first tested whether our data replicated previous findings on frontoparietal networks in rivalry (Lumer et al., 1998). The random-effects group analysis (one-sample t test) on the linear contrast between rivalry and replay transitions during active report revealed that a large network of cortical and subcortical regions underlies the processing of ambiguous visual input (Fig. 2A). This rivalry-related activation pattern was strongly lateralized to the right hemisphere and consisted primarily of occipital, parietal, and frontal brain regions (Table 1). We found this pattern consistently across individuals. The patterns were qualitatively similar for both rivalry types and for the same statistical threshold as above (cluster-extent threshold of p < 0.01, FWE corrected, voxel level threshold of p < 0.001, uncorrected), we found no significant differences between the neural patterns when restricting the analysis to either OKN or pupil trials only. Using a more liberal cluster-extent threshold of p < 0.05 (uncorrected), we did find differences, but only in occipital regions (V1). This difference is not surprising given that, for the different rivalry types, different low-level features (orientation, motion direction) are involved in the rivalry competition. Therefore, our active-report trials replicate previous findings: independent of rivalry type, rivalry seems to be mediated by occipital, parietal, and frontal regions predominantly in the right hemisphere of the brain.
Binocular rivalry versus replay for active report trials. A, “Discrete subjective”: activation pattern shows neural loci that were more activated time-locked to the moment of a subjective report (i.e., button press) during rivalry trials compared with replay trials. R, right hemisphere; L, left hemisphere. B, “Discrete objective”: rivalry-related activation pattern (rivalry > replay) time locked to the moment of an objective alternation (i.e., zero-crossing of OKN or pupil size traces). C, “Continuous objective”: activation pattern indicates neural loci as revealed by the complete OKN and pupil-size traces. D, Pattern indicates neural loci where statistical power was significantly increased when using the “continuous objective” analysis compared with the “discrete subjective” analysis. Results in A–D are thresholded at a conservative cluster-extent threshold of p < 0.01, FWE corrected (voxel level threshold p < 0.001, uncorrected). All active report trials are included in the analyses regardless of whether OKN or pupil stimulus set was presented. Activation patterns were rendered onto the surface of a standard anatomical template image.
Coordinates and z-scores for transient rivalry-related activations (rivalry > replay) time locked to observers' button presses
OKN and pupil size as valid and powerful indicators of binocular rivalry's neural mechanisms during active report
As next step, we still restricted our analysis to the active-report trials, but compared the neural patterns obtained by the button-press data with those obtained from the objective measures. This allowed us to test the applicability of the objective measures to identify the neural correlates of switches in perception. Unlike button presses, OKN speed and pupil size do not only provide the discrete events of switch times, but their continuous value at any point in time may also relate to strength of the dominance of the current percept. We therefore also exploited this continuous nature of the objective measures in addition to defining discrete transition points (zero-crossings of OKN speed or pupil size). This results in three possible analyses, which we refer to as “discrete subjective” (button presses), “discrete objective” (zero-crossings of the objective measures), and “continuous objective” (time course of the continuous values of the objective measures), respectively. We tested if either the discrete objective analysis or the continuous objective analysis increased the statistical power of the GLM compared with the discrete subjective analysis.
All three analyses revealed significant neural activity associated with perceptual alternations during rivalry in occipital, parietal, and frontal regions in the right hemisphere (Fig. 2A–C). Therefore, using information from the objective measures yielded qualitatively similar results compared with the discrete subjective analysis. Specifically, activation patterns revealed no significant differences between the discrete subjective and discrete objective analysis given a cluster-extent threshold of p < 0.01, FWE corrected (voxel level threshold p < 0.001, uncorrected). This suggests that both discrete analyses were equally sensitive in mapping the neural network underlying binocular rivalry. Remarkably, the continuous objective analysis reached higher statistical power in the GLM than the discrete subjective analysis. These benefits were significant given a cluster-extent threshold of p < 0.01, FWE corrected (voxel level threshold p < 0.001, uncorrected). Importantly, this higher statistical power was not an artifact of the continuous analysis having more data points, because all regressors had the same length (continuous data were subsampled to the TR of the MR scanner). Neural loci showing higher power for the continuous objective analysis were most prominently located in superior occipital, superior parietal, and middle frontal brain areas (Fig. 2D). This benefit in statistical power was specifically restricted to the rivalry-related sites, because we found significant increase of t values only for regions within the relevant network. Therefore, the increase of t values was not caused by a global gain unrelated to rivalry, but by the more sensitive nature of the continuous measures in mapping the neural mechanism underlying binocular rivalry.
Report and introspection partially cause the typically observed neural-activation patterns
So far, we have shown that OKN and pupil size are valid and sensitive measures of the neural mechanisms underlying binocular rivalry when active report was required. Extending beyond this confirmation of earlier findings, however, these measures also allowed us to measure the neural correlates of rivalry when no active report was required and thus to dissociate the effects of rivalry switching per se from the effects of active report.
Comparing rivalry-related neural loci (i.e., rivalry > replay) for active-report and passive-viewing trials based on the continuous objective analysis at first glance revealed qualitatively similar activation patterns (Fig. 3A,B). Similar to the active-report condition, the neural network revealed by the rivalry-versus-replay contrast was strongly lateralized to the right hemisphere during passive-viewing trials. Activation within this network was primarily located in the inferior occipital gyrus and the parietal lobe, as well as in the right inferior orbitofrontal lobe. However, despite the similarities, there were also clear differences in the activation patterns. The intensity of the rivalry-related neural network was reduced when observers passively experienced binocular rivalry compared with when they reported perceptual alternations. Specifically, neural activation in the middle frontal gyrus was not only diminished but completely vanished during passive-viewing trials. This pattern was consistent for OKN and pupil trials separately: for the same statistical threshold as above (cluster-extent threshold of p < 0.01, FWE corrected; voxel level threshold of p < 0.001, uncorrected), we found significant differences between the rivalry types only in occipital regions. Specifically, either V1 or V4 showed higher rivalry-related activation when the analysis was restricted to OKN or pupil trials, respectively. Again, this is consistent with early visual areas' involvement in rivalry being related to the specific features of the stimuli used. In contrast, the absence of differential frontal activation generalizes across the rivalry types tested.
Influence of active report on the neural network underlying binocular rivalry. A, B, Activation patterns (based on the continuous objective analysis) show neural loci that were identified to be more activated time locked to a perceptual transition during rivalry compared with replay when observers actively reported their percept (A) or just passively experienced rivalry dynamics (B). Results in A and B are thresholded at a cluster-extent threshold of p < 0.01, FWE corrected (voxel level threshold p < 0.001, uncorrected). C, Pattern indicates neural loci where rivalry-related activation (i.e., rivalry > replay) was significantly higher during active report trials compared with passive viewing trials, regardless of the objective measure OKN speed or pupil size (i.e., global null conjunction analysis). Results in C are thresholded at p < 0.001, uncorrected.
To determine whether the absence of activity in middle frontal regions during passive-viewing trials is an effect of the chosen significance threshold, we explicitly tested significance for each voxel individually. We calculated linear contrast images that reveal neural regions of rivalry-related activation for each report and indicator condition separately. These four contrast images were then entered into a random-effects group analysis (two-way within-subject ANOVA). On the group level, we conducted a global null conjunction analysis to pinpoint the neural loci where rivalry-related activation was higher when observers reported their perception compared with when they passively experienced rivalry regardless of the rivalry type. The conjunction analysis revealed that neural loci showing significantly higher rivalry-related activation during active-report trials (p < 0.001, uncorrected) were most prominently located bilaterally in the middle frontal gyrus, where clusters were most extended (Fig. 3C, Table 2). Furthermore, regions bilaterally within the supplementary motor areas, the insula and the middle occipital lobe, were more activated when observers reported their perception. The conjunction analysis additionally revealed increased rivalry-related activity in the superior and inferior parietal lobule in the left hemisphere due to report. This shows that the difference in neural activation between active-report and passive-viewing trials is specific to certain, predominantly frontal, regions and not due to a general upregulation of the rivalry-related network when active report is required. Therefore, part of the neural activation pattern typically observed for rivalry is a consequence of active report and possibly the associated requirement for introspection about one's own perception.
Coordinates and z-scores show neural loci for which the continuous objective analysis revealed higher rivalry-related activations (rivalry > replay) when observers actively reported their perception compared with when they passively experienced rivalry
Discussion
We investigated the neural loci underlying perceptual alternations during binocular rivalry. Although previous brain-imaging studies relied on button presses and thus introspection as measures of perceptual alternations, we adapted an objective approach for fMRI. Using continuous and objective measures of perceptual alternations, we dissociated activity related to rivalry alternations from those merely related to active report or introspection.
Percept-related action modulates rivalry (Wohlschlager, 2000; Maruya et al., 2007; Beets et al., 2010). Specifically, similar to the speed-up by attention (Paffen et al., 2006; Alais et al., 2010), report can accelerate rivalry. One possible cause for this report-related speed-up is therefore increased attention when introspection is required for report. Several studies on the relation of attention and awareness circumvented report of the perceptual state: Lee et al. (2007) related the traveling-wave dynamics of transitions to cortical activity, whereas Watanabe et al. (2011) and Yuval-Greenberg and Heeger (2013) used binocular-suppression paradigms. Unlike our study, however, they induced perceptual transitions exogenously or used stimuli of vastly different dominance. No rivalry imaging study has yet addressed the effect of report as such.
We confirmed OKN and pupil as reliable indicators of rivalry dynamics. Their continuous nature also increases the statistical power of the rivalry-replay contrast in the GLM, providing a more sensitive way to map rivalry's neural processes than conventional analyses based on button presses. Our large stimulus size makes it likely that the continuous measures capture the transition process through periods of piecemeal rivalry, possibly reflecting neural adaptation and reciprocal inhibition between percepts (van Loon et al., 2013), whereas button presses do not. Importantly, however, regardless of whether button presses or reflexes were used, all analyses for active-report trials revealed similar activation patterns. These patterns were largely consistent with earlier fMRI studies (Lumer et al., 1998): the neural network underlying perceptual switches that are actively reported is strongly lateralized to the right hemisphere and mainly consists of frontal, parietal, and occipital regions.
To verify generalization, we compared pupil and OKN data and all imaging results were indistinguishable between the two, except in occipital regions. These differences likely result from distinct stimulus features, consistent with data on rivalry across areas when very dissimilar stimuli compete (Tong et al., 1998).
Our objective measures allow fMRI analyses time locked to perceptual transitions without report. Already the standard approach (rivalry-replay contrasts) appears to discount effects of action by mimicking the motor act in replay. However, whenever report is required, different effects of attention to the stimulus in replay compared with rivalry, different requirements for performing the task (Knapen et al., 2011) or for self-monitoring, and the direct feedback of action on perceptual representations (Wohlschlager, 2000; Beets et al., 2010) cannot be dissociated from the endogenously triggered perceptual switch. Indeed, we find clear differences between active report and passive viewing: rivalry-related activity is reduced or absent when no report is required. Importantly, we find differences between the rivalry-replay contrasts of the two report conditions, which excludes a pure motor-related effect (within each report condition, motor requirements for rivalry and replay are identical). Direct comparison shows that differences between the active-report and passive-viewing conditions are not a matter of significance threshold.
The reduced BOLD signal for the passive-viewing condition compared with the active-report condition is most pronounced bilaterally in the middle frontal gyrus. This casts doubt on a driving role of frontal areas in initiating perceptual alternations (Lumer et al., 1998; Leopold and Logothetis, 1999; Sterzer and Kleinschmidt, 2007; Sterzer et al., 2009). Instead, activity in frontal areas is associated with active report and thus introspection rather than with rivalry per se. This interpretation accords with a key role of those regions in introspection and self-monitoring (Lezak, 1983; Gusnard et al., 2001; Goldberg et al., 2006). Therefore, frontal areas are unlikely to induce changes in perceptual awareness; instead, they serve a monitoring role required for reporting the change.
Additional regions show reduced rivalry-related activity during passive viewing: (1) bilaterally in the supplementary motor area, involved in initiating hand movements (Grefkes et al., 2008) such as button presses; (2) bilateral insula, involved in consciousness, self-awareness, and cognitive functions linked to emotions, including perception and motor-control (Craig, 2009); (3) neural loci within the inferior parietal lobule, responsible for the organization and interpretation of motor action (Rozzi et al., 2008), and (4) the superior parietal lobule, involved in sensorimotor integration by maintaining an internal representation of one's own state (Wolpert et al., 1998). Together, these data suggest that neural patterns typically observed when rivalry is measured through the observers' active report, in particular in frontal regions, have a substantial relation to introspection, self-related processing, and action.
Analogous to our study, a recent MEG study questioned the notion of a widespread network underlying binocular rivalry (Kamphuisen et al., 2008): in contrast to earlier EEG/MEG studies (Tononi et al., 1998; Srinivasan et al., 1999; Cosmelli et al., 2004), the increase in neural activity associated with perceptual dominance of a tagged stimulus does not necessitate a pan-cortical network for rivalry, but can be modeled by a limited set of occipital sources. Two studies specifically challenged the causal role of frontal areas: De Graaf et al. (2011) showed that transcranial magnetic stimulation to frontal areas modulates the rate of perceptual switches under volitional control, but not of spontaneous switches, and therefore frontal areas are responsible for voluntary top-down modulation, but not for initiating transitions. Knapen et al. (2011) showed that rivalry transitions and exogenous changes during replay elicit indistinguishable BOLD activity in frontoparietal cortex, when replay simulates the spatiotemporal pattern of rivalry transitions faithfully. They suggested four hypotheses for the role of frontal areas: (1) changes in visual experience differ in salience between rivalry and abrupt physical changes, (2) task demands increase more for rivalry than for abrupt physical changes, (3) continuous reorienting of spatial attention when following the wave-like transitions in rivalry, and (4) task switching between monitoring the end of a dominance period and awaiting the next.
Our data narrow down these possibilities selectively to hypotheses 2 and 4. Unless salience or reorienting are assumed to be task modulated, our data argue against hypothesis 1 and 3: frontal activity is absent for passive viewing, although rivalry and replay still differ in salience and reorienting demands. In our view, reporting triggers highest demands around the transitions, which are—consistent with hypothesis 2—larger for rivalry transitions. Whether this increase is generic or a specific switch between monitoring modes (hypothesis 4) may be addressed in future research by combining Knapen et al.'s with our approach. Together, there is strong evidence against frontal regions causing perceptual transitions. Instead, frontal regions subserve a monitoring role that is modulated by the task at hand.
Using dynamic causal modeling (Friston et al., 2003), Weilnhammer et al. (2013) found modulations of top-down connectivity from frontal regions to visual cortex while viewing a bistable figure. Contrary to our conclusions, this argues for a driving role of the frontoparietal network. However, the investigators did not study binocular rivalry and relied on report. Investigating effective connectivity in our paradigm remains an issue for future research.
Two other fMRI studies addressed rivalry-related activation without report. Lumer and Rees (1999) measured functional correlations between cortical regions when observers passively experienced rivalry. Activity in extrastriate visual areas, but not in V1, was correlated with other extrastriate areas and with parietal and frontal regions. This seemed to suggest frontal and parietal areas as essential loci for rivalry and conscious vision. However, Lumer and Rees could not relate activity to events of perceptual switches. Despite viewing passively, it is likely that participants upregulated and downregulated the degree to which they monitored their own perceptual state. Periods of increased self-monitoring would then be associated with increased frontoparietal activity. Such endogenous upregulation of self-monitoring would also enhance activity in visual areas. This simultaneous modulation, even if unrelated to perceptual-transition events, could yield the observed correlations. This interpretation is supported by the role of early visual areas in representing ambiguous stimuli (Polonsky et al., 2000; Tong, 2003; Lee et al., 2005; Meng et al., 2005), which suggests that activity related to rivalry transitions should then also correlate with V1, contrary to Lumer and Rees' observations. Wilcke et al. (2009) had observers passively watch rivalry stimuli and used binocular fusion (rather than permanent suppression) as a control, excluding task-difficulty differences between replay and rivalry (O'Shea and Corballis, 2005). They identified frontal, parietal, occipital, and limbic areas for the processing of ambiguous stimuli without report, consistent with a separate active-report experiment. However, their design precluded time-locked analysis of switching for passive viewing, rendering any conclusion about a driving (versus monitoring) role of frontal regions impossible.
In conclusion, for the first time, we have used a design that combines an analysis time locked (i.e., event-related) to the perceptual switch with passive viewing in the same condition in fMRI. Although we cannot fully exclude that, for some forms of rivalry, frontal areas do play a role in rivalry switching, based on the present data, their role in monitoring changes in awareness is at least substantially larger than in inducing such changes.
Footnotes
This work was supported by the von Behring Röntgen Stiftung (Grant 57-0032 to J.S.), the Else Kröner-Fresenius Stiftung (Grant 2012_A219 to A.J.), the Dutch NWO Rubicon (Grant 446-11-013), the Marie Curie Cofund Action (to M.N.), and the German Research Foundation (DFG Grant EI 852-3 to W.E.). We thank Alexander Platzner for technical support and Sören Krach and Frieder Paulus for fruitful discussions.
The authors declare no competing financial interests.
- Correspondence should be addressed to Stefan Frässle, Philipps-University Marburg, Section BrainImaging, Department of Psychiatry and Psychotherapy, Rudolf-Bultmann-Straβe 8, 35037 Marburg, Germany. fraessle{at}med.uni-marburg.de