Abstract
Continuously moving objects sometimes appear to spontaneously reverse their motion direction. The mechanisms underlying this bistable phenomenon (the “continuous wagon wheel illusion”) are heavily debated, but one interpretation suggests that motion information is perceived in discrete episodes at a rate between 10 and 15 Hz. Here, we asked observers to report the perceived direction of a continuously rotating wheel while 32-channel electroencephalogram (EEG) was recorded. We then separated periods of perceived true from illusory (reversed) motion and compared the EEG power spectrum under these two perceptually distinct yet physically identical conditions. The only reliable difference was observed ∼13 Hz over centroparietal electrodes, independent of the temporal frequency of the wheel. Thus, it is likely to reflect internal processes rather than purely stimulus-driven activity. EEG power (13 Hz) decreased before the onset of illusory motion and increased before transitions back to real motion. Using this relationship, it was possible to predict above chance, on a trial-by-trial basis, the direction of the upcoming perceptual transition. These data are compatible with the idea that motion perception occurs in snapshots <100 ms in duration.
Introduction
The precise temporal organization of our perceptual experience is a major unanswered question (Crick and Koch, 2003; Eagleman and Churchland, 2005): do we perceive the world as a continuous flow of information or in discrete episodes or “snapshots” as in a regular movie (Stroud, 1956; Shallice, 1964; Harter, 1967; VanRullen and Koch, 2003)? The wagon wheel illusion may provide some insights into this debate (Andrews and Purves, 2005). This phenomenon occurs, for example, when the sampling rate of a video camera is too slow compared with the temporal frequency of a moving object (such as a rotating wagon wheel): on film, the object will appear to move in the opposite direction. Under some conditions, the same illusion can also be experienced by humans observing continuously moving objects in the real world (Schouten, 1967). One explanation of this curious phenomenon could be that the visual system processes motion information just as a video camera in a rapid sequence of discrete snapshots (Purves et al., 1996). Recent experiments estimated the rate of these postulated snapshots between 10 and 15 Hz (Simpson et al., 2005; VanRullen et al., 2005). Other interpretations that assume continuous processing, however, have been proposed for this illusion (Schouten, 1967; Pakarian and Yasamy, 2003; Kline et al., 2004), which remains a debated topic (Andrews and Purves, 2005; Andrews et al., 2005; Holcombe et al., 2005). Here, we investigate the electrophysiological correlates of this illusion in the human brain: if illusory motion perception is triggered by discrete processing (∼10–15 Hz), then the power spectrum of the electroencephalogram (EEG) during the illusion may reflect the temporal frequency of this discrete subsampling. Indeed, we found that only one EEG spectral component (∼13 Hz) was affected by the continuous wagon wheel illusion.
Materials and Methods
Stimulus. The stimuli were rotating “wheels” consisting of 16 cycles of a sinusoidally modulated luminance pattern (i.e., 16 “spokes”) at 100% contrast. The wheel occupied 12.5° of visual angle, and a central gray disk of 0.6° provided for a fixation point. These wheels were displayed on a computer monitor with a refresh rate of 160 Hz, fast enough to avoid contamination by temporal framing artifacts, as determined from our previous investigations (VanRullen et al., 2005). For all intent and purposes, this series of images on a monitor approximated “continuous” motion. We used wheels rotating at a temporal frequency (or spoke alternation rate) of 10 and 7.5 Hz. Previous research (Simpson et al., 2005; VanRullen et al., 2005) demonstrated that the continuous wagon wheel illusion can be observed in both cases and is optimal for the former temporal frequency (10 Hz).
Subjects. Twelve observers (five females; 23–30 years of age) participated in this study. Two of them were authors of this study. The others were graduate and undergraduate students who were naive to the purpose of the experiment. All subjects signed informed consent and had normal or corrected-to-normal vision. Three of the subjects were left-handed; all subjects used the right hand for providing manual reports. Those who had never experienced the illusion were first accustomed to it for a few minutes (<10) before starting the experiment.
Experimental procedures. A constantly rotating wheel was presented for 2 min, during which observers had to report the perceived direction of rotation by continuously pressing the corresponding arrow on a keyboard (the left arrow with the right index finger for counterclockwise motion, the right arrow with the right middle finger for clockwise motion). Subjects were asked to perform 20 such 2 min trials with a wheel rotating at 10 Hz and 10 such trials with a wheel rotating at 7.5 Hz. These two types of trials were interleaved within the same experimental sessions. Subjects were free to rest between trials and to decide when to start the next trial. For any given subject, the actual rotation direction of the wheels was kept constant throughout the experiment, and this direction was counterbalanced across subjects. Some subjects reported mild discomfort induced by the constant stimulation, and the experiment was aborted for these subjects; but a minimum of 12 min of data were collected for each subject in each condition.
EEG recordings. A 32-channel EEG (Neuroscan, El Paso, TX) was recorded continuously (sampling rate, 1000 Hz) for the length of each trial. The EEG recording was synchronized with the display computer by means of pulses sent at the beginning and end of each trial and at the onset of each stimulus cycle during the trial (i.e., at 10 or 7.5 Hz, depending on the stimulus condition). The electrode layout was modified from the 10–20 system with an additional row of occipital electrodes and a linked-ears reference. A hardware notch filter was applied at 50 Hz (European electrical standards) to discard ambient electrical noise.
Power spectrum analysis. Based on the recorded subjects' responses, the EEG data for each 2 min trial was divided into a number of variable-length periods, during which actual motion was experienced, and a comparable number (plus or minus one) of periods during which illusory motion was perceived. This was done after shifting the EEG time frame by 250 ms with respect to the time frame of the behavioral report, to take into account the subjects' reaction time (Luce, 1986). For each subject and stimulus condition, the extent of the periods of perceived actual motion (generally longer than the illusory motion periods) was reduced so that the distributions of period durations for perceived actual and illusory motion were comparable (so the two datasets could be directly compared). The procedure for data reduction involved determining the distribution of illusory period durations and for each of these periods extracting a comparable amount of data from the middle of a “real motion” period of sufficient duration. The resulting periods for each perceptual condition were then concatenated (the potential spectral artifacts induced by this concatenation were minimized using custom-designed Matlab code and manual rejection for any remaining artifacts), and the signals were bandpass filtered between 2 and 100 Hz. The power spectrum was then calculated for each perceptual condition and electrode (using Welch's averaged, modified periodogram method). Differences between conditions were estimated using t scores (mean difference divided by the SE across subjects).
EEG dynamics and receiver operator characteristic analysis. After we determined that the 13 Hz spectral component of the EEG was the main correlate of the illusion (see Results), we selected the three electrodes in which this effect was most pronounced (electrodes C4, P4, and PO4 in the 10/20 nomenclature) and calculated the average 13 Hz power (±0.25 Hz) for these electrodes at each time point of the entire experiment (Fourier analysis with a fixed-length sliding Hanning window). These signals were then divided in 3 s epochs straddling each perceptual transition (i.e., the moment at which a subject's report changed on the keyboard), and these epochs were averaged with respect to the time of the perceptual reversal. This was done separately for transitions to and from illusory motion. We also used these signals (before averaging) to perform the following receiver operator characteristic (ROC) analysis: for each perceptual transition, the slope of the signal (that is, the change in the absolute amount of power, ∼13 Hz) was estimated over the period (–2000 to 150 ms) before the transition occurred. A threshold was set and slopes that passed the threshold were classified as transitions from illusory motion, whereas slopes that did not pass the threshold were classified as transitions to illusory motion. A correctly classified transition from illusory motion was counted as a “hit,” and an incorrectly classified transition to illusory motion was counted as a “false alarm.” The procedure was repeated for all applicable values of the threshold, and the proportion of hits was plotted as a function of the false alarm rate. The area under the resulting ROC curve indicated (for each subject and stimulus condition) the ability of our 13 Hz EEG power signal to predict, on a trial-by-trial basis, the direction of the upcoming subjective perceptual reversal. A variant of this procedure was also applied, in which the critical time window used for determining the slope of the 13 Hz signal was allowed to vary for each subject, with the constraint that the window should be 1000 ms in length and entirely contained within the interval (–2000 to 150 ms). Finally, a control analysis was performed (for both variants of the ROC analysis) by first randomly shuffling the directions of the perceptual transitions and then applying the corresponding ROC analysis (this was repeated 100 times, and the results were averaged over repetitions). This control provided an independent estimate of the actual “chance level” for our ROC results.
Results
We collected large samples of EEG data (up to 60 min overall per subject) while our 12 human observers fixated a constantly rotating wheel. According to the subjects' reports, we separated these data into periods of perceived actual motion and periods of experienced illusory motion (as in previous reports, the illusory percept occurred ∼25–30% of the total viewing time) and compared the power spectrum of the EEG in these two conditions. Because the physical stimulus was identical in both cases, and only the subjective percept differed, we thus hoped to isolate the EEG correlates of the continuous wagon wheel illusion.
The resulting power spectra in the two perceptual conditions were very similar (Fig. 1A,B), with a characteristically decreasing profile and a local peak at 10 Hz in the α band, for some of the electrodes. Nevertheless, a statistical comparison revealed one primary difference between these conditions (Fig. 1C,D): ∼13 Hz the EEG power was stronger during periods of experienced real motion compared with periods of perceived illusory motion. This difference at 13 Hz was significant for several electrodes, even after a Bonferroni correction for multiple comparisons across electrodes was applied (t(23) > 3.3; p < 0.05). There were no other significant differences between the two perceptual conditions, even in more classical frequency bands such as the theta (4–7 Hz), α (8–12 Hz), β (15–25 Hz), or gamma bands (30–70 Hz), in which such correlates may have been expected.
Next, we examined whether the 13 Hz EEG component may have been directly driven or “entrained” by the periodic visual stimulus or whether it could be considered a reflection of internal processes operating independently of the visual stimulation. To this end, we contrasted the results obtained for the two stimulus conditions (i.e., with wheels rotating at 7.5 vs 10 Hz). For simplicity and to increase statistical power, we first computed the global power spectrum over the entire scalp (i.e., averaged across all 32 electrodes) and again subtracted the two (global) power spectra corresponding to perceived real and perceived illusory motion. This comparison (Fig. 2) again yielded a single peak at ∼13 Hz, in which the difference was highly significant (t(23) = 5.3; p = 0.00001), confirming our previous observation. Importantly, when the same difference was calculated separately for the two stimulus conditions (wheels rotating at 7.5 or 10 Hz), the results were very similar (Fig. 2A), and the 13 Hz component was the only jointly significant effect (Fig. 2B) (t(11) = 4.5, p < 0.0005 for the wheels at 10 Hz; t(11) = 3.2, p < 0.005 for the wheels at 7.5 Hz). The convergence of results derived from entirely independent sets of trials is a further indication that the 13 Hz difference is a statistically solid phenomenon. Furthermore, this convergence implies that the present EEG correlate of illusory motion does not vary with the temporal frequency of the stimulus and is thus likely to reflect an internal generator of the illusion rather than a mere byproduct of the periodic visual stimulation.
EEG correlates of illusory motion perception. The power spectrum of the EEG was calculated independently for all periods during which real motion (A) and illusory motion (B) was experienced. All curves are averaged over 12 subjects. Each curve corresponds to one of the 32 scalp electrodes. Signals display a characteristically decreasing power spectrum, with a local peak at 10 Hz (in the α frequency range). The strong decrease of EEG power at 50 Hz is a consequence of the hardware notch filter. The signals around this value were not analyzed further and are hidden by hatched rectangles in all remaining plots. C, The subtraction of the two power spectra (real minus illusory motion; expressed as a percentage of the absolute power obtained in the illusory motion condition) reveals a strong difference, ∼13 Hz, observed for several electrodes simultaneously. D, This difference is highly significant, as shown by the t values estimated over the group of subjects (and here, collapsed over the 2 stimulus conditions). The dashed red lines mark the p < 0.05 significance threshold, corrected for multiple comparisons across electrodes (Bonferroni's correction). More than one dozen electrodes exceed this threshold in the vicinity of 13 Hz, whereas no other significant correlate of the illusion is observed in other frequency bands.
The spatial distribution of these effects on the scalp is illustrated in Figure 3, which again reveals that differences in the subjectively perceived direction of motion (i.e., real vs illusory) were mostly reflected in the 13 Hz spectral component rather than in more classical frequency bands. The topography of the 13 Hz spectrum difference highlighted a group of centroparietal electrodes, with a right hemisphere bias, an observation also confirmed by the corresponding significance scalp maps. Furthermore, the topographies obtained for the two stimulus conditions (i.e., wheels rotating at 7.5 and 10 Hz) were remarkably similar, although they were derived from entirely independent sets of trials: this again suggests that the underlying effect is highly reliable.
The power difference at 13 Hz is independent of the temporal frequency of the stimulus.A, The difference in global power spectrum (averaged over all subjects and electrodes) between the periods of real and illusory motion percepts is computed for all stimulus conditions together and also separately for the wheels rotating at 7.5 and 10 Hz. The peak difference at 13 Hz is common to both curves. B. The t values confirm that the difference at 13 Hz is highly significant for both stimulus conditions. The dashed red lines mark the p < 0.05 significance threshold.
In our procedure, subjects reported illusory motion by pressing one key and real motion by pressing another (half of the subjects pressed the left arrow key to report illusory motion, while the other half pressed the right arrow key). Given the lateralization of the topography obtained in Figure 3 and the proximity of motor cortex to our most significant effects, one might argue that what we are recording is in fact simply a correlate of pressing one or the other arrow key. This would imply, however, that the effect should happen in opposite directions for the two groups of subjects, because illusory and real motion corresponded to opposite sets of keys for those two groups. To address this issue, we analyzed the power spectrum difference between the two perceptual conditions separately for each group (over the same, three most significant electrodes; as determined from Fig. 3). For both groups, there was a single significant spectral component peaking at ∼13 Hz (t test; p < 0.00001 for subjects who reported the illusion with the left arrow key and p < 0.0001 for the other group). Importantly, this component was of the same sign for both groups (higher 13 Hz amplitude during real motion perception), indicating that there was no confound as a result of the key press (supplemental Fig. 5, available at www.jneurosci.org as supplemental material). For completeness, we also verified that the data obtained from left-handed and right-handed subjects (three vs nine subjects, respectively) was compatible. Again, both groups revealed a single significant component at ∼13 Hz (t test; p < 5.10–3 and p < 5.10–7 for left- and right-handed subjects, respectively), with a peak of the same sign (supplemental Fig. 6, available at www.jneurosci.org as supplemental material).
Scalp distribution of the EEG correlates of illusory motion perception (average over 12 subjects). For six different temporal frequencies (±0.25 Hz) of the EEG, the relative power difference between periods of perceived real and perceived illusory motion is plotted as a color code (scale to the left) at the corresponding electrode location. The bottom row illustrates the statistical significance of this difference in a similar manner (scale to the right). At 13 Hz, where these effects are dominant, the scalp topography of the effects is comparable for the two stimulus conditions (wheels rotating at 7.5 or 10 Hz).
Variations in 13 Hz power predict upcoming transitions. A, Within the 2 s leading up to a perceptual reversal (i.e., a change of the subject's response), the EEG power at 13 Hz (calculated over the 3 most significant electrodes; determined from Fig. 3) is a good indicator of the type of transition: it decreases before transitions from real to illusory motion and increases before transitions from illusory to real motion. The curves are averaged over all 12 subjects, and the shaded area represents SEM. B, Using the slope of this signal (calculated from –2000 to –150 ms before each perceptual reversal), we attempted to predict the direction of the upcoming subjective transitions on a trial-by-trial basis. The histogram represents the distribution of areas under the resulting ROC curves for all subjects and stimulus conditions, reflecting the accuracy of the prediction. The vertical arrows point to the average area (55.7%), which was significantly higher than chance (p = 0.002). C, We could predict the direction of subjective reversals more accurately when we allowed the time window used for estimating the slope of the 13 Hz signal to vary between different subjects (keeping the window length equal to 1000 ms and entirely contained within the interval –2000 to 150 ms). This accounted for the possibility that the time course of 13 Hz EEG power may differ across subjects. In this case, the average area was 60.9%, significantly higher than chance (p < 10–7).
Our final analysis explored the temporal dynamics of the relationship between changes in the absolute amount of power at ∼13 Hz in the EEG and the perceived direction of motion. As shown in Figure 4A, we found that 13 Hz power (calculated over the three electrodes displaying the most significant effect; as determined from Fig. 3) tended to decrease during the 2 s preceding a perceptual transition from real to illusory motion (as determined from the subjects' keyboard responses) and to increase during the 2 s preceding the opposite perceptual transition, from illusory to real motion. In other words, the slope of the 13 Hz power signal within the last 2 s before a subject experienced a perceptual reversal appeared to indicate the direction of this upcoming transition. We tested whether this relationship could be sufficiently consistent to predict the direction of perceptual transitions on a trial-by-trial basis by performing an ROC analysis using the slope of the 13 Hz EEG power over the period –2000 to 150 ms before each perceptual reversal as the prediction variable. The area under the resulting ROC curve (Fig. 4B) was on average 55.7% (±1.6%), which was significantly above chance level (50%; t(23) = 3.2; p = 0.002). This was also true when chance level was estimated by shuffling the direction of the perceptual transitions corresponding to each trial and then performing the ROC analysis on this shuffled data. The shuffling procedure was repeated 100 times, and the resulting chance level (averaged over repetitions) was estimated at 50.9%, still significantly below (paired t test; t(23) > 2.5; p < 0.01) our ROC predictions.
In a variant of this analysis, we acknowledged that different subjects may use different strategies and/or rely on different temporal dynamics of the 13 Hz power to determine their response. Thus, we allowed the time window used for determining the slope of the 13 Hz power to vary across subjects; the window duration was fixed at 1000 ms, and only the window onset was permitted to vary between 2000 and 1150 ms before the transition. The obtained predictions were better in this case (Fig. 4C), with an average area under ROC curve of 60.9% (±1.3%), which was significantly higher than 50% (t(23) > 7.3; p < 10–7) and also significantly higher than the chance level determined from the shuffling procedure applied with the same variable window method (57.2%; paired t test; t(23) > 2.5; p < 0.01).
Discussion
The only component of the EEG power spectrum that was affected during the continuous wagon wheel illusion was found at ∼13 Hz. Furthermore, changes in the amount of power in the 13 Hz band of the EEG predicted above chance, on a trial-by-trial basis, the direction of perceived motion, although the physical stimulus never changed on the retina. This is compatible with the notion that some neuronal process with a periodicity in the 13 Hz regime may trigger the perceptual switch. Although the accuracy of these predictions is only marginal, it is important to remember that they rely on single-trial EEG data, which in general are deemed inherently too noisy to be analyzed (Picton et al., 2000). The level of prediction obtained is, in fact, comparable with results from single-cell experiments in monkeys, in which perceived motion direction (during constant physical stimulation) is also predicted with accuracy between 55 and 60% (Britten et al., 1996). We hope that our reasonably successful endeavor may thus motivate additional studies of the relationship between single-trial EEG activity and conscious visual perception.
An important outcome of our results is the absence of any correlate of the continuous wagon wheel illusion in the “classical” frequency bands of the EEG. In particular, the gamma frequency band (30–70 Hz) has often been found to reflect conscious visual perception (Tallon-Baudry et al., 1997; Rodriguez et al., 1999; Tallon-Baudry and Bertrand, 1999), and thus correlates of illusory motion perception might have been expected to turn up in this band. Why was this not the case here? The continuous wagon wheel illusion may very well engage different neuronal dynamics than in those previous experiments, but it is also worth noting that the aforementioned studies have simply discarded (or high-pass filtered) spectral components <15 or 20 Hz on the assumption that they mostly reflect cortical inactivity (Pfurtscheller et al., 1996). A reexamination of the relationship between low-frequency (i.e., <20 Hz) and high-frequency (>20 Hz) components of the power spectrum may thus be called for in future studies (von Stein and Sarnthein, 2000; Varela et al., 2001).
Previous work has explored the spectral correlates of perceptual reversals during binocular rivalry, another type of bistable phenomenon. Both human EEG experiments (Kobayashi et al., 1996; Doesburg et al., 2005) and electrophysiological investigations in the primary visual cortex of mammals (Fries et al., 1997; Gail et al., 2004), however, have so far failed to come to an agreement. Certain studies register mainly gamma-band correlates of perceptual switching (Fries et al., 1997; Revonsuo et al., 1997; Doesburg et al., 2005), whereas for others, the critical spectral components are ∼10 Hz (Kobayashi et al., 1996; Gail et al., 2004). It is still an open question whether one or several spectral correlates exist for the different types of bistable phenomena (including Necker cubes, figure-ground ambiguous stimuli, binocular rivalry, and others).
The fact that 13 Hz power decreases during the continuous wagon wheel illusion does not necessarily imply that the underlying neural sources are less active in this perceptual condition than during real motion. Alpha-band activity, which may include this spectral component, is usually most clearly visible when the cortex is at rest (Adrian and Yamagiwa, 1935). There is a considerable body of literature showing event-related desynchronization (ERD) in lower-frequency bands (5–25 Hz) as a result of performing various visual tasks (Pfurtscheller et al., 1994, 1996; Pfurtscheller and Lopes da Silva, 1999; Klimesch et al., 2000). The amplitude of this ERD correlates with task complexity (Van Winsum et al., 1984; Dujardin et al., 1995) and attention (Dujardin et al., 1993; Foxe et al., 1998; Worden et al., 2000). Interestingly, these EEG or MEG power decreases in the 5–25 Hz frequency range have been linked to task-dependent increases in the functional magnetic resonance imaging blood oxygen level-dependent hemodynamic response (Singh et al., 2002). Thus, our decreased 13 Hz power during the illusion may, in fact, correspond to increased activation of the underlying neuronal sources.
What, then, could be the sources of the present effect? Inferring this from EEG data recorded at the scalp is a necessarily speculative endeavor, which must only be done with caution. The topographical analysis (Fig. 3) hints at an involvement of right parietal regions in the illusion. These regions have been implicated numerous times in visual tasks involving attention (Buchel et al., 1998; Coull and Frith, 1998; Rees and Lavie, 2001; Corbetta and Shulman, 2002), which would be compatible with the known involvement of attention during the continuous wagon wheel illusion (VanRullen et al., 2005). The same regions also display increased activity at the time of perception switches during bistable stimulation (Lumer et al., 1998). An important observation is that patients with right parietal lesions, although they can discriminate continuous motion and also perceive flicker normally, tend to show bilateral deficits in the perception of apparent motion (Battelli et al., 2001) and in the discrimination of the direction (onset vs offset) of temporal events (Battelli et al., 2003). These temporal discrimination abilities may be critical for perceiving reversed motion in our illusion; this line of reasoning would thus bring us to predict that such patients may never experience the continuous wagon wheel illusion.
What are the theoretical implications of our results with respect to the temporal organization of visual perception? The present report is purely descriptive: we show that 13 Hz EEG power correlates with the illusion, but this does not inform us as to how this correlation comes about, and in particular what neural mechanisms could underlie. Note also that the mere existence of this correlation does not allow us to infer with certainty the direction of any causal link between 13 Hz EEG power and the illusion (in other words, reversed motion perception may be causing the 13 Hz power decrease rather than the opposite). We previously proposed a Fourier motion energy model based on discrete subsampling ∼15 Hz that could accurately predict the quantitative psychophysical properties of this illusion (VanRullen et al., 2005). It is easy to draw a direct link between this model and our current results, but at this stage, such a link can only be speculative. The present results by themselves cannot resolve the debated question of discrete versus continuous motion processing; yet they constrain the range of models of motion perception that can explain the illusion. Obviously, models based on discrete subsampling at 10–15 Hz (Purves et al., 1996; Simpson et al., 2005; VanRullen et al., 2005) are in a good position to satisfy these new constraints. Other explanations, however, such as those based on the properties of Reichardt motion detectors (Reichardt, 1961; van Santen and Sperling, 1985) and neuronal adaptation may still account for the illusion (Schouten, 1967; Kline et al., 2004), but for this they would have to be modified to accommodate not only for the temporal frequency dependency of this illusion, as shown previously (Simpson et al., 2005; Van-Rullen et al., 2005) and discussed by Andrews and Purves (2005), Andrews et al. (2005), and Holcombe et al. (2005), but now also for the very specific electrophysiological correlates at 13 Hz demonstrated here.
Footnotes
This work was supported by the Centre National de la Recherche Scientifique and the National Science Foundation–Engineering Research Center. We thank the two anonymous referees for constructive comments on this manuscript.
Correspondence should be addressed to Rufin VanRullen, Centre de Recherche Cerveau et Cognition, Centre National de la Recherche Scientifique, Faculte de Medecine Rangueil, 31062 Toulouse Cedex 9, France. E-mail: rufin{at}klab.caltech.edu.
DOI:10.1523/JNEUROSCI.4654-05.2006
Copyright © 2006 Society for Neuroscience 0270-6474/06/260502-06$15.00/0
