Research ReportNonlinear SSVEP responses are sensitive to the perceptual binding of visual hemifields during conventional ‘eye’ rivalry and interocular ‘percept’ rivalry
Introduction
Binocular rivalry arises when two incongruent images are presented one to each eye of the observer. The two images compete for conscious perception, resulting in alternating episodes of perceptual dominance, usually lasting about 2 s, during which only one image is visible to the observer. The other image is suppressed and not perceived. The source of competition in binocular rivalry has been a subject of debate since its discovery. One notion is that rivalry occurs between eyes (Lee and Blake, 2004) at the first stages of binocular interaction between monocular pathways, presumably early in the visual system (Blake and Logothetis, 2002). Another view is that rivalry is competition between two percepts presumably higher up in visual processing (Blake and Logothetis, 2002) and even in the frontal lobes (Tononi et al., 1998, Srinivasan et al., 1999, Srinivasan and Petrovic, 2006). This view supports the idea that binocular rivalry is a useful paradigm to investigate conscious perception in EEG (Lansing, 1964, Cobb and Morton, 1967, Brown and Norcia, 1997, Koernbach et al., 1999, Roeber U Schroger, 2004, Srinivasan, 2004), MEG (Tononi et al., 1998, Srinivasan et al., 1999, Srinivasan and Petrovic, 2006), neurophysiological (Leopold and Logothetis, 1996, Logothetis et al., 1996, Fries et al., 2002), or fMRI studies (Lumer et al., 1998, Tong et al., 1998, Polonsky et al., 2000).
In these physiological experiments on binocular rivalry, the two incongruent images presented one to each eye always formed a single coherent percept. Such stimuli could give rise to competition between the eyes and competition between percepts. The physiological evidence obtained in studies using rival unitary monocular images appears to support both interpretations. Leopold and Logothetis (1996) recorded single cell activity in V1, V2, V4, and MT while monkeys reported grating orientation during binocular rivalry. While most of the recorded cells in V1 and V2 always responded to their preferred grating orientation, activity in only 18% of the cells was modulated by the perceived dominance/nondominance of their preferred stimulus. A higher percentage of cells in V4 (34%) and MT (43%) were modulated by perceptual dominance. In a follow-up study, Sheinberg and Logothetis (1997) recorded from cells in IT, where almost 90% of recorded cells were modulated by conscious perception. Human MEG and EEG studies using whole head recording have also demonstrated robust increases in power and phase-locking of frontal lobe responses to flicker presented to one eye during conscious perception of the flicker (Srinivasan et al., 1999, Srinivasan, 2004, Srinivasan and Petrovic, 2006). While these monkey and human studies suggest that the degree of modulation increases as one moves up the visual pathway, this implies neither that rivalry takes place higher up in the visual system, nor between percepts. For one thing, at least some cells early in the visual system do appear to modulate their firing rate with conscious perception (Leopold and Logothetis, 1996), and greater selectivity in higher stages of the visual system may just reflect the consequences of the activity of these cells. The sensitivity of early visual areas to conscious perception has also been reported in local field potentials (Fries et al., 1997) and fMRI studies (Polonsky et al., 2000).
Psychophysical studies have demonstrated the existence of percept rivalry by manipulating the timing of inputs to each eye. Binocular rivalry has been observed between two images that were flickered one to each eye, but never presented simultaneously to the observer (O'Shea and Crassini, 1984, Srinivasan and Petrovic, 2006). The images were presented in alternation, with a brief dark period between the images. When the monocular flicker frequency was < 4 Hz, the observers primarily experienced a unitary flicker that alternated between the two images. When the flicker frequency was > 4 Hz, the observers primarily experienced two single-image flickers that exhibit conventional rivalry with episodes of perceptual dominance of each single-image flicker. These observations are consistent with earlier psychophysical studies of the Gestalt flicker frequency (van de Grind, et al., 1973). The rivalry observed in this study is between two percepts of single image flicker rather than between the individual images presented to the two eyes.
Another technique used to demonstrate percept rivalry is to rapidly swap the stimuli between the eyes. One study swapped stimuli between the eyes every 333 ms while flickering the images at 18 Hz (Logothetis et al. 1996). They found that observers' experiences were similar to that of conventional rivalry, with episodes of dominance lasting about 2 s — much longer than the stimuli swap rate. Lee and Blake (1999) extended this study by using stimuli swap rates ranging from 1.4 Hz (stimulus exchanged every 722 ms) to 6 Hz (167 ms) and instructed the observer to categorize their perceptual experience. A “slow, irregular change,” longer than the stimulus swap rate, signified percept rivalry; “fast, rapid changes” occurring several times per second suggested eye rivalry. They showed that percept rivalry is seen under limited conditions — at a swap rate of about 300–400 ms, a spatial frequency of about 6–7 cpd, and when stimuli were abruptly and not gradually swapped. With other stimulus parameters, eye rivalry was mostly reported. In a further extension of this experimental method, it has also been shown that both eye and percept rivalry can co-exist (Bonneh et al., 2001) supporting the notion of competition at multiple levels depending on stimulus parameters in these displays. Recently developed neural models (Wilson, 2003) incorporate a hierarchy of competition between eyes early in the visual system and competition between percepts later in the visual system to explain these data.
Physiological effects specific to competition between percepts in binocular rivalry have not yet been identified, primarily due to the use of rivalry between unitary monocular images in all physiological experiments. The phenomenon of rivalry between two percepts formed by interocular grouping of complementary image fragments, first reported by Diaz-Caneja in 1928 (see translation by Alais et al., 2000) and subsequently investigated by Wade (1973) and Kovacs et al. (1996), have also provided evidence of percept rivalry. In the study by Kovacs et al. (1996) complementary random fragments of two images were distributed between the two eyes. The observers were able to experience rivalry between two coherent images formed by grouping relevant elements from both eyes. In this case, perceptual competition seems to be the source of rivalry, since the two rival percepts are each constructed from inputs to both eyes. In the present paper, we further investigated this phenomenon using psychophysical and physiological methods to identify neural correlates of percept rivalry. The results demonstrate that the competing percepts are always perceptually bound during both conventional ‘eye’ rivalry and interocular ‘percept’ rivalry.
We first carried out a psychophysical experiment (Experiment 1) to contrast the percepts reported for each of the exemplar displays shown in Fig. 1. In order to make use of these displays for a subsequent EEG experiment (Experiment 2), the grating presented in each hemifield of each eye was flickered at a distinct rate as exemplified in Fig. 2 for display A. In each of displays A–D when the two images in the upper row are presented one to each eye, the observer may perceive one of the monocular images presented to each eye, corresponding to conventional ‘eye’ rivalry between the eyes (MO rivalry). In this case, when the image presented to one eye is perceived, the image presented to the other eye is not perceived. Alternatively, interocular ‘percept’ rivalry can take place between two percepts constructed by interocular grouping of complementary hemifields of each monocular image (IO rivalry). As shown in Fig. 1, in this case, one hemifield from each eye is perceived, while the other pairing of hemifields is not perceived.
In our displays, observers can potentially group relevant hemifields of an image based on certain common characteristics (eye, color, and orientation). The results of experiment 1 indicated that only for displays of the type shown in Fig. 1A there is roughly equal probability of MO rivalry and IO rivalry; for displays B–D, MO rivalry is mostly reported. We used only displays of type shown in Fig. 1A in Experiment 2, and recorded steady-state visual evoked potentials (SSVEPs) at each flicker frequency corresponding to each visual field of each eye. We contrasted the modulations of the SSVEPs specific to each hemifield of each eye between episodes of dominance and suppression, to identify correlates of conscious perception specific to the grating presented in each hemifield of each eye during the two types of rivalry.
We then made use of the nonlinear properties of SSVEPs to investigate the binding of the hemifields into a coherent percept. SSVEPs have long been reported to show combination responses at nf1 ± mf2 when two flickers are presented at frequencies f1 and f2, where m and n are positive integers (Zemon and Ratliff, 1984, Regan and Regan, 1988). In our experiment, we detected the first order combination responses at the sums of the flicker frequencies (i.e., f1 + f2). These nonlinear SSVEPs clearly reflect neural populations responding to the pair of stimuli each flickering at a different frequency, and thus potentially relate to binding the corresponding images into a coherent percept. We examined whether these combination responses are sensitive to the type of rivalry the subject experienced (IO rivalry versus MO rivalry), to examine the role of binding of percepts in each type of rivalry.
Section snippets
Experiment 1: psychophysical results
The purpose of Experiment 1 was to contrast the observers' reports for the 4 types of displays shown in Figs. 1A–D in 6 subjects. The examples shown in Fig. 1 are a subset of the displays, with a fixed red grating in the left visual field of the left eye, out of a total of 16 different displays shown to each subject in order to counterbalance color and orientation. Each display was also shown 4 times to each subject to counterbalance flicker frequencies, as shown in Fig. 2 for an example
Grouping of visual hemifields by eye, orientation, and color cues
We enabled competition between four percepts by using stimuli conducive to both conventional ‘eye’ rivalry between two percepts corresponding to images presented to each eye (MO rivalry) and interocular ‘percept’ rivalry between two percepts constructed by interocular grouping of complementary hemifields, one from each eye (IO rivalry). We manipulated the color and orientation of gratings presented in each hemifield of each eye as exemplified in Fig. 1. Psychophysical results in Fig. 3 suggest
Experiment 1: psychophysics
We conducted a psychophysical experiment to determine whether any of the stimulus configurations shown in Fig. 1 is equally favorable to MO and IO rivalry, thus allowing contrasts between the two types of rivalry in an EEG experiment. Behavioral data were obtained from six right-handed adults (3 female and 3 male) aged 21–29. Informed consent was obtained from each observer.
Acknowledgments
This research was supported by a grant from the NIH R01-MH68004. The authors wish to thank Prof. Charles Chubb and Prof. Emily Grossman of the Department of Cognitive Sciences at UC Irvine for many useful comments. We also want to thank one of the reviewers for guiding us to the relevant history of research into the phenomena of interocular grouping.
References (33)
- et al.
A transition between eye and object rivalry determined by stimulus coherence
Vis. Res.
(2001) - et al.
A method for investigating binocular rivalry in real-time with the steady-state VEP
Vis. Res.
(1997) - et al.
Rival ideas about binocular rivalry
Vis. Res.
(1999) - et al.
A fresh look at interocular grouping during binocular rivalry
Vis. Res.
(2004) - et al.
A frequency domain technique for characterizing nonlinearities in biological systems
J. Theor. Biol.
(1988) - et al.
Binocular rivalry and visual awareness in human extrastriate cortex
Neuron
(1998) Spatial sampling of head electrical fields: the geodesic sensor net
Electroencephalogr. Clin. Neurophysiol.
(1993)- et al.
On binocular alternation
Perception
(2000) - et al.
Visual competition
Nat. Rev. Neurosci.
(2002) The psychophysics toolbox
Spat. Vis.
(1997)
Cerebral potentials evoked by pattern reversal and their suppression in visual rivalry
Nature
Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency
Cereb. Cortex
Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry
Proc. Natl. Acad. Sci. U. S. A.
Oscillatory neuronal synchronization in primary visual cortex as a correlate of stimulus selection
J. Neurosci.
Effects of consciousness on human brain waves following binocular rivalry
NeuroReport
When the brain changes its mind: interocular grouping during binocular rivalry
Proc. Natl. Acad. Sci. U. S. A.
Cited by (52)
Steady state evoked potentials indicate changes in nonlinear neural mechanisms of vision in sight recovery individuals
2021, CortexCitation Excerpt :In fact, intermodulation frequency responses seemed to vary with stimulus complexity. Binding of visual stimuli such as two gratings, one presented to each eye (Sutoyo & Srinivasan, 2009), or binding of simple shapes to form illusory percepts (Aissani et al., 2011; Gundlach & Müller, 2013) have been found to be associated with higher amplitudes of intermodulation frequency responses, particularly at the sum term. Binding of two halves of human faces into a single facial representation, has been observed to produce intermodulation frequency responses as well (Boremanse et al., 2013, 2014).
From intermodulation components to visual perception and cognition-a review
2019, NeuroImageCitation Excerpt :These reports reveal intriguing neural interactions that occur during rivalry. Sutoyo and Srinivasan presented four half-circle gratings-one to each eye/hemifield combination, each of which differed according to its colour (either red or green) and orientation (one of two orthogonal diagonal orientations) ((Sutoyo and Srinivasan, 2009); Fig. 7A). These combinations allowed percepts to be formed either by the combination of two hemifields within the same eye or by the combination of two complementary hemifields-one from each eye.