Elsevier

Brain Research

Volume 1251, 28 January 2009, Pages 245-255
Brain Research

Research Report
Nonlinear SSVEP responses are sensitive to the perceptual binding of visual hemifields during conventional ‘eye’ rivalry and interocular ‘percept’ rivalry

https://doi.org/10.1016/j.brainres.2008.09.086Get rights and content

Abstract

We conducted behavioral and EEG experiments to identify physiological correlates of perceptual binding during two types of binocular rivalry: (1) conventional ‘eye’ rivalry where perception alternates between the two monocular images presented one to each eye and (2) interocular ‘percept’ rivalry, where perception alternates between percepts formed by grouping complementary hemifields one from each eye. We employed ‘frequency-tagging’ by flickering a grating in each hemifield of each eye at different frequencies to elicit SSVEP responses specific to each hemifield of each eye. When the gratings in complementary visual fields of the two eyes were congruent in color and orientation, robust interocular ‘percept’ rivalry was observed with roughly equal probability to conventional ‘eye’ rivalry. The SSVEPs evoked by the flickering gratings were enhanced by conscious perception at both posterior and frontal electrodes only during conventional ‘eye’ rivalry and not during interocular ‘percept’ rivalry, suggesting that dominance of one eye is the basis of most previous reports of SSVEP modulation by conscious perception. We also observed nonlinear SSVEP responses at the sums of our four fundamental frequencies. These combination responses were only produced by flicker in complementary visual hemifields — in the same eye or across eyes, but never by incongruent flickering gratings that occupy the same visual field across eyes, suggesting that they are related to the binding of the visual hemifields (monocular or interocular) into a coherent percept. These combination responses were modulated by the type of rivalry experienced by the observer, but not by the specific conscious perception. Neural processes related to perceptual binding of both rival percepts take place during binocular rivalry even when only one percept is consciously perceived. This suggests that conventional ‘eye’ and interocular ‘percept’ rivalry both involve competition between percepts.

Introduction

Binocular rivalry arises when two incongruent images are presented one to each eye of the observer. The two images compete for conscious perception, resulting in alternating episodes of perceptual dominance, usually lasting about 2 s, during which only one image is visible to the observer. The other image is suppressed and not perceived. The source of competition in binocular rivalry has been a subject of debate since its discovery. One notion is that rivalry occurs between eyes (Lee and Blake, 2004) at the first stages of binocular interaction between monocular pathways, presumably early in the visual system (Blake and Logothetis, 2002). Another view is that rivalry is competition between two percepts presumably higher up in visual processing (Blake and Logothetis, 2002) and even in the frontal lobes (Tononi et al., 1998, Srinivasan et al., 1999, Srinivasan and Petrovic, 2006). This view supports the idea that binocular rivalry is a useful paradigm to investigate conscious perception in EEG (Lansing, 1964, Cobb and Morton, 1967, Brown and Norcia, 1997, Koernbach et al., 1999, Roeber U Schroger, 2004, Srinivasan, 2004), MEG (Tononi et al., 1998, Srinivasan et al., 1999, Srinivasan and Petrovic, 2006), neurophysiological (Leopold and Logothetis, 1996, Logothetis et al., 1996, Fries et al., 2002), or fMRI studies (Lumer et al., 1998, Tong et al., 1998, Polonsky et al., 2000).

In these physiological experiments on binocular rivalry, the two incongruent images presented one to each eye always formed a single coherent percept. Such stimuli could give rise to competition between the eyes and competition between percepts. The physiological evidence obtained in studies using rival unitary monocular images appears to support both interpretations. Leopold and Logothetis (1996) recorded single cell activity in V1, V2, V4, and MT while monkeys reported grating orientation during binocular rivalry. While most of the recorded cells in V1 and V2 always responded to their preferred grating orientation, activity in only 18% of the cells was modulated by the perceived dominance/nondominance of their preferred stimulus. A higher percentage of cells in V4 (34%) and MT (43%) were modulated by perceptual dominance. In a follow-up study, Sheinberg and Logothetis (1997) recorded from cells in IT, where almost 90% of recorded cells were modulated by conscious perception. Human MEG and EEG studies using whole head recording have also demonstrated robust increases in power and phase-locking of frontal lobe responses to flicker presented to one eye during conscious perception of the flicker (Srinivasan et al., 1999, Srinivasan, 2004, Srinivasan and Petrovic, 2006). While these monkey and human studies suggest that the degree of modulation increases as one moves up the visual pathway, this implies neither that rivalry takes place higher up in the visual system, nor between percepts. For one thing, at least some cells early in the visual system do appear to modulate their firing rate with conscious perception (Leopold and Logothetis, 1996), and greater selectivity in higher stages of the visual system may just reflect the consequences of the activity of these cells. The sensitivity of early visual areas to conscious perception has also been reported in local field potentials (Fries et al., 1997) and fMRI studies (Polonsky et al., 2000).

Psychophysical studies have demonstrated the existence of percept rivalry by manipulating the timing of inputs to each eye. Binocular rivalry has been observed between two images that were flickered one to each eye, but never presented simultaneously to the observer (O'Shea and Crassini, 1984, Srinivasan and Petrovic, 2006). The images were presented in alternation, with a brief dark period between the images. When the monocular flicker frequency was < 4 Hz, the observers primarily experienced a unitary flicker that alternated between the two images. When the flicker frequency was > 4 Hz, the observers primarily experienced two single-image flickers that exhibit conventional rivalry with episodes of perceptual dominance of each single-image flicker. These observations are consistent with earlier psychophysical studies of the Gestalt flicker frequency (van de Grind, et al., 1973). The rivalry observed in this study is between two percepts of single image flicker rather than between the individual images presented to the two eyes.

Another technique used to demonstrate percept rivalry is to rapidly swap the stimuli between the eyes. One study swapped stimuli between the eyes every 333 ms while flickering the images at 18 Hz (Logothetis et al. 1996). They found that observers' experiences were similar to that of conventional rivalry, with episodes of dominance lasting about 2 s — much longer than the stimuli swap rate. Lee and Blake (1999) extended this study by using stimuli swap rates ranging from 1.4 Hz (stimulus exchanged every 722 ms) to 6 Hz (167 ms) and instructed the observer to categorize their perceptual experience. A “slow, irregular change,” longer than the stimulus swap rate, signified percept rivalry; “fast, rapid changes” occurring several times per second suggested eye rivalry. They showed that percept rivalry is seen under limited conditions — at a swap rate of about 300–400 ms, a spatial frequency of about 6–7 cpd, and when stimuli were abruptly and not gradually swapped. With other stimulus parameters, eye rivalry was mostly reported. In a further extension of this experimental method, it has also been shown that both eye and percept rivalry can co-exist (Bonneh et al., 2001) supporting the notion of competition at multiple levels depending on stimulus parameters in these displays. Recently developed neural models (Wilson, 2003) incorporate a hierarchy of competition between eyes early in the visual system and competition between percepts later in the visual system to explain these data.

Physiological effects specific to competition between percepts in binocular rivalry have not yet been identified, primarily due to the use of rivalry between unitary monocular images in all physiological experiments. The phenomenon of rivalry between two percepts formed by interocular grouping of complementary image fragments, first reported by Diaz-Caneja in 1928 (see translation by Alais et al., 2000) and subsequently investigated by Wade (1973) and Kovacs et al. (1996), have also provided evidence of percept rivalry. In the study by Kovacs et al. (1996) complementary random fragments of two images were distributed between the two eyes. The observers were able to experience rivalry between two coherent images formed by grouping relevant elements from both eyes. In this case, perceptual competition seems to be the source of rivalry, since the two rival percepts are each constructed from inputs to both eyes. In the present paper, we further investigated this phenomenon using psychophysical and physiological methods to identify neural correlates of percept rivalry. The results demonstrate that the competing percepts are always perceptually bound during both conventional ‘eye’ rivalry and interocular ‘percept’ rivalry.

We first carried out a psychophysical experiment (Experiment 1) to contrast the percepts reported for each of the exemplar displays shown in Fig. 1. In order to make use of these displays for a subsequent EEG experiment (Experiment 2), the grating presented in each hemifield of each eye was flickered at a distinct rate as exemplified in Fig. 2 for display A. In each of displays A–D when the two images in the upper row are presented one to each eye, the observer may perceive one of the monocular images presented to each eye, corresponding to conventional ‘eye’ rivalry between the eyes (MO rivalry). In this case, when the image presented to one eye is perceived, the image presented to the other eye is not perceived. Alternatively, interocular ‘percept’ rivalry can take place between two percepts constructed by interocular grouping of complementary hemifields of each monocular image (IO rivalry). As shown in Fig. 1, in this case, one hemifield from each eye is perceived, while the other pairing of hemifields is not perceived.

In our displays, observers can potentially group relevant hemifields of an image based on certain common characteristics (eye, color, and orientation). The results of experiment 1 indicated that only for displays of the type shown in Fig. 1A there is roughly equal probability of MO rivalry and IO rivalry; for displays B–D, MO rivalry is mostly reported. We used only displays of type shown in Fig. 1A in Experiment 2, and recorded steady-state visual evoked potentials (SSVEPs) at each flicker frequency corresponding to each visual field of each eye. We contrasted the modulations of the SSVEPs specific to each hemifield of each eye between episodes of dominance and suppression, to identify correlates of conscious perception specific to the grating presented in each hemifield of each eye during the two types of rivalry.

We then made use of the nonlinear properties of SSVEPs to investigate the binding of the hemifields into a coherent percept. SSVEPs have long been reported to show combination responses at nf1 ± mf2 when two flickers are presented at frequencies f1 and f2, where m and n are positive integers (Zemon and Ratliff, 1984, Regan and Regan, 1988). In our experiment, we detected the first order combination responses at the sums of the flicker frequencies (i.e., f1 + f2). These nonlinear SSVEPs clearly reflect neural populations responding to the pair of stimuli each flickering at a different frequency, and thus potentially relate to binding the corresponding images into a coherent percept. We examined whether these combination responses are sensitive to the type of rivalry the subject experienced (IO rivalry versus MO rivalry), to examine the role of binding of percepts in each type of rivalry.

Section snippets

Experiment 1: psychophysical results

The purpose of Experiment 1 was to contrast the observers' reports for the 4 types of displays shown in Figs. 1A–D in 6 subjects. The examples shown in Fig. 1 are a subset of the displays, with a fixed red grating in the left visual field of the left eye, out of a total of 16 different displays shown to each subject in order to counterbalance color and orientation. Each display was also shown 4 times to each subject to counterbalance flicker frequencies, as shown in Fig. 2 for an example

Grouping of visual hemifields by eye, orientation, and color cues

We enabled competition between four percepts by using stimuli conducive to both conventional ‘eye’ rivalry between two percepts corresponding to images presented to each eye (MO rivalry) and interocular ‘percept’ rivalry between two percepts constructed by interocular grouping of complementary hemifields, one from each eye (IO rivalry). We manipulated the color and orientation of gratings presented in each hemifield of each eye as exemplified in Fig. 1. Psychophysical results in Fig. 3 suggest

Experiment 1: psychophysics

We conducted a psychophysical experiment to determine whether any of the stimulus configurations shown in Fig. 1 is equally favorable to MO and IO rivalry, thus allowing contrasts between the two types of rivalry in an EEG experiment. Behavioral data were obtained from six right-handed adults (3 female and 3 male) aged 21–29. Informed consent was obtained from each observer.

Acknowledgments

This research was supported by a grant from the NIH R01-MH68004. The authors wish to thank Prof. Charles Chubb and Prof. Emily Grossman of the Department of Cognitive Sciences at UC Irvine for many useful comments. We also want to thank one of the reviewers for guiding us to the relevant history of research into the phenomena of interocular grouping.

References (33)

  • CobbW.A. et al.

    Cerebral potentials evoked by pattern reversal and their suppression in visual rivalry

    Nature

    (1967)
  • DingJ. et al.

    Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency

    Cereb. Cortex

    (2006)
  • FriesP. et al.

    Synchronization of oscillatory responses in visual cortex correlates with perception in interocular rivalry

    Proc. Natl. Acad. Sci. U. S. A.

    (1997)
  • FriesP. et al.

    Oscillatory neuronal synchronization in primary visual cortex as a correlate of stimulus selection

    J. Neurosci.

    (2002)
  • KoernbachC. et al.

    Effects of consciousness on human brain waves following binocular rivalry

    NeuroReport

    (1999)
  • KovacsI. et al.

    When the brain changes its mind: interocular grouping during binocular rivalry

    Proc. Natl. Acad. Sci. U. S. A.

    (1996)
  • Cited by (52)

    • Steady state evoked potentials indicate changes in nonlinear neural mechanisms of vision in sight recovery individuals

      2021, Cortex
      Citation Excerpt :

      In fact, intermodulation frequency responses seemed to vary with stimulus complexity. Binding of visual stimuli such as two gratings, one presented to each eye (Sutoyo & Srinivasan, 2009), or binding of simple shapes to form illusory percepts (Aissani et al., 2011; Gundlach & Müller, 2013) have been found to be associated with higher amplitudes of intermodulation frequency responses, particularly at the sum term. Binding of two halves of human faces into a single facial representation, has been observed to produce intermodulation frequency responses as well (Boremanse et al., 2013, 2014).

    • From intermodulation components to visual perception and cognition-a review

      2019, NeuroImage
      Citation Excerpt :

      These reports reveal intriguing neural interactions that occur during rivalry. Sutoyo and Srinivasan presented four half-circle gratings-one to each eye/hemifield combination, each of which differed according to its colour (either red or green) and orientation (one of two orthogonal diagonal orientations) ((Sutoyo and Srinivasan, 2009); Fig. 7A). These combinations allowed percepts to be formed either by the combination of two hemifields within the same eye or by the combination of two complementary hemifields-one from each eye.

    View all citing articles on Scopus
    View full text