Abstract
Vision is known to be shaped by context, defined by environmental and bodily signals. In the Taylor illusion, the size of an afterimage projected on one's hand changes according to proprioceptive signals conveying hand position. Here, we assessed whether the Taylor illusion does not just depend on the physical hand position, but also on bodily self-consciousness as quantified through illusory hand ownership. Relying on the somatic rubber hand illusion, we manipulated hand ownership, such that participants embodied a rubber hand placed next to their own hand. We found that an afterimage projected on the participant's hand drifted depending on illusory ownership between the participants' two hands, showing an implication of self-representation during the Taylor illusion. Oscillatory power analysis of electroencephalographic signals showed that illusory hand ownership was stronger in participants with stronger α suppression over left sensorimotor cortex, whereas the Taylor illusion correlated with higher β/γ power over frontotemporal regions. Higher γ connectivity between left sensorimotor and inferior parietal cortex was also found during illusory hand ownership. These data show that afterimage drifts in the Taylor illusion do not only depend on the physical hand position but also on subjective ownership, which itself is based on the synchrony of somatosensory signals from the two hands. The effect of ownership on afterimage drifts is associated with β/γ power and γ connectivity between frontoparietal regions and the visual cortex. Together, our results suggest that visual percepts are not only influenced by bodily context but are self-grounded, mapped on a self-referential frame.
SIGNIFICANCE STATEMENT Vision is influenced by the body: in the Taylor illusion, the size of an afterimage projected on one's hand changes according to tactile and proprioceptive signals conveying hand position. Here, we report a new phenomenon revealing that the perception of afterimages depends not only on bodily signals, but also on the sense of self. Relying on the rubber hand illusion, we manipulated hand ownership, so that participants embodied a rubber hand placed next to their own hand. We found that visual afterimages projected on the participant's hand drifted laterally, only when the rubber hand was embodied. Electroencephalography revealed spectral dissociations between somatic and visual effects, and higher γ connectivity along the dorsal visual pathways when the rubber hand was embodied.
Introduction
By pushing the visual system to its limits, optical illusions provide invaluable tools to study perception, and lead researchers to constantly refine their theoretical models of the way we represent the world (Eagleman, 2001). Most famously, the role of spatial cues for size perception was first empirically shown by the Ebbinghaus-Titchener illusion (Ebbinghaus, 1902), in which the perceived size of a circle varies according to the size of surrounding flankers, and by the Ponzo illusion (Ponzo, 1910), in which linear perspective brought by converging railway tracks changes the perceived length of two identical lines. Both illusions suggest that vision is contextual by essence and involves the integration of different parts of space into unified percepts, a property found to be reflected as early as primary visual cortex (Schwarzkopf et al., 2011).
Beyond visual and spatial contexts, visual size perception is also known to depend on bodily context, as an afterimage induced by a brief flash of light on one's hand expands/shrinks as one moves the hand toward/away from the face (Taylor illusion) (Gregory, 1959). Because active and passive movements have been reported to produce equivalent illusory size changes (Carey and Allan, 1996), the Taylor illusion is held to primarily stem from afferent proprioceptive rather than efferent motor signals. By highlighting the impact of bodily (proprioceptive) signals on spatial vision, this illusion thus implies that visual percepts exceed the mere product of inferential processes made on retinal inputs and actually relate to signals from nonvisual modalities including proprioception (for review, see Faivre et al., 2015).
When integrated with visual and tactile signals, proprioceptive signals not only encode the position of limbs in space, but also give rise to the sense of ownership, that is the sensation that body parts belong to us. In the somatic rubber hand illusion (sRHI) (Botvinick and Cohen, 1998; Ehrsson et al., 2005; Pozeg et al., 2014), changes in hand ownership and perceived hand location are induced by the following procedure: when touching a fake hand with their right hand while their left hand is being touched in synchrony by an experimenter, participants typically feel to touch their own hand (i.e., self-touch illusion), and feel their touched hand as displaced toward the fake hand's position (i.e., proprioceptive drift). This illusion and its numerous variants have been instrumental to define the mechanisms responsible for body ownership based on correlated multisensory signals (Blanke, 2012).
Here, by combining the sRHI paradigm with the Taylor illusion, we assessed whether changes in illusory self-touch altered the representation of visual position (sense where an object is located in space), or put differently, whether afterimage drifts in the Taylor illusion occur in case the subjective position of an illusory owned hand changes while the physical hand position defined by proprioception remains constant. After an afterimage was elicited by a flash of light, an experimenter induced the sRHI for 15 s by stroking a right rubber hand with the participants' left index finger while simultaneously stroking the corresponding part of the participants' right hand (Fig. 1). Participants were then asked to locate the position of the afterimage in their field of view. To test whether the position of the afterimage was influenced by the self-touch illusion, we compared location ratings after synchronous versus asynchronous stroking, as the latter condition does not induce illusory self-touch and hand position recalibration. We assessed whether the visual position in the Taylor illusion is modulated based on the integration of proprioceptive and tactile signals during the sRHI. Finally, we combined behavioral measures with EEG and determined the neural mechanisms of these effects, expecting the sRHI to involve α band suppression over somatosensory regions (Kanayama et al., 2007, 2009; Lenggenhager et al., 2011; Evans et al., 2013) and the afterimage drifts to be associated with changes in functional connectivity between somatosensory regions involved in the sRHI and visual cortex (Kanayama et al., 2007, 2009).
Experimental setup and procedure. Top, After a circular array of LEDs flashed on the participant's right hand, an experimenter induced the rubber hand illusion by stroking a right rubber hand with the participant's left index finger while simultaneously stroking the corresponding part of the participants' right hand (synchronous condition, as depicted here), or asynchronously stroking a noncorresponding part of the hand (asynchronous condition, which served as baseline). Bottom, Time course of a trial. After a sound of 1 s duration indicating the beginning of the trial, a fixation LED turned on for 1 s, followed by a flash of 150 ms. Participants then closed their eyes, and oriented their head so that they faced a ruler placed in front of them. After 4 s, during which the afterimage stabilized, the experimenter induced the rubber hand illusion during 15 s. At the end of each trial, participants opened their eyes and rated the position of the afterimage in their field of view using a ruler placed in front of them (afterimage drift effect), and indicated how much they felt like they were touching their own hand (illusory self-touch).
Materials and Methods
Overview.
In the sRHI, an experimenter moves the left index finger of a blindfolded participant to stroke a right rubber hand (that is positioned on the right side of the participant), while simultaneously stroking the corresponding part of the participant's right hand (that is positioned on the left side of the participant with crossed hands; Fig. 1). When the stroking is temporally and spatially matched on both hands (hereafter, synchronous condition), and even though participants actually touch the fake hand, they feel like they touch their own hand (i.e., illusory self-touch), and erroneously locate their hand toward the fake hand (i.e., proprioceptive drift) (Ehrsson et al., 2005; Pozeg et al., 2014). Importantly, the illusion vanishes in case the stroking on the real and fake hands is temporally and spatially incongruent (hereafter, asynchronous condition), which offers a baseline condition in which no illusion occurs under similar physical stimulation. Here, we sought to estimate the influence of hand ownership on visual perception, and tested whether the perceived position of a visual stimulus differed following synchronous versus asynchronous stroking, or in other words, whether a proprioceptive drift was accompanied by an afterimage drift. In a dark room, a bright light was flashed on the participant's right hand, so that a negative afterimage was formed on the retina. After the flash, participants closed their eyes, and the experimenter induced the sRHI during 15 s (Fig. 1). Participants then opened their eyes, located the position of the afterimage in their field of view, and indicated how much they felt they were touching their own hand during the induction phase. In Experiment 1, head movements were monitored using a motion tracking device. Experiment 2 consisted in a replication of Experiment 1, including measures of eye movements and brain activity using EOG and EEG, respectively. The next sections describe in more detail the devices and procedures we used.
Participants.
Sixty eight participants (Experiment 1: 12 including 5 females, mean age = 21.0 years, SD = 3.4 years; Experiment 2: 23 including 11 females, mean age = 22.4 years, SD = 3.3 years; Experiment 3: 12 including 5 females, mean age = 25.3 years, SD = 5.4 years; Experiment 4: 9 including 6 females, mean age = 23.0 years, SD = 3.6 years; Experiment 5: 12 including 6 females, mean age = 24.2 years, SD = 3.1 years) from the student population at Ecole Polytechnique Fédérale de Lausanne took part in this study, in exchange for monetary compensation (20 CHF per hour). All participants were right-handed, had normal or corrected-to-normal vision, and had no psychiatric or neurological history. They were naive to the purpose of the study and gave informed consent, in accordance with institutional guidelines and the Declaration of Helsinki. One participant was excluded from Experiment 1, as her location ratings had a null variance. Three participants were excluded from Experiment 2: one due to a technical failure and two others due to excessive artifacts in the EEG signal.
Apparatus.
Afterimages were induced by a custom-made circular array of white LEDs attached to the dorsum of the right hand (hereafter image inducer: 60 mm diameter, 130 lm intensity, 8000 K correlated color temperature; Super Bright LEDs). A red LED was placed in the center of the circular array and served as a fixation point. The device was controlled by an Arduino Leonardo board, connected to MATLAB via the serial port. The position (horizontal, vertical, and depth axes) and orientation (pitch, roll, and yaw axes) of the head were tracked throughout the experiment with a motion tracking system with a temporal resolution of 20 frames per second (V120 Trio, Optitrack). While participants had their eyes closed, ocular movements were measured with EOG using bipolar reference. Eye blinks and vertical saccades were measured with two electrodes placed above and below the midline of the right eye. Horizontal saccades were measured with two electrodes placed on the outer canthus of each eye (i.e., bitemporal electrode configuration), so that the measured voltage was linearly related to the horizontal eye position within a range of 30° in each direction (Eggert, 2007). A 64-channel electroencephalograph (Biosemi) with 1024 Hz sampling rate was used to record brain activity.
Procedure.
Participants sat behind a desk in a dark and shielded room, their right hand aligned with the mid-sagittal body plane, palm facing down (Fig. 1). They wore a plastic glove on the right hand to match the tactile sensation of a right rubber hand placed 25 cm on the right, wearing the same glove. Each trial started with an auditory tone (1 s duration sine wave, 800 Hz), during which participants were asked to fixate on a red LED at the center of the afterimage inducer. Upon the red fixation LED turning off, the afterimage inducer flashed for 150 ms, and participants subsequently closed their eyes. After a stabilization period of 4 s, a tone was played (0.1 s duration sine wave, 650 Hz), and the experimenter started stroking the rubber hand with the participant's left index finger while at the same time stroking the participant's right hand (rubber hand induction phase). The tactile stimulation of both hands consisted of regular taps and strokes, either temporally and spatially matched in the synchronous condition or unmatched in the asynchronous condition. It was performed on crossed hands as it was shown to increase the self-touch illusion (Pozeg et al., 2014), for a duration of 15 s based on previous findings showing that self-touch sensations occur after ∼10 s of stroking (Ehrsson et al., 2005). The induction was performed under dim-light conditions, which allowed the experimenter to tap accurately while monitoring that participants did not move their hand. A tone (0.1 s duration sine wave, 650 Hz) indicated the end of the induction phase, upon which participants were asked to open their eyes, and provide two subjective reports using a numerical pad with their left hand. First, they had to locate the position of the afterimage on a scale placed 60 cm in front of them, graded between 0 and 40 inches with a resolution of 1 inch (the 20 inches mark was placed at the center of their field of view, aligned with the body midline). Second, they had to rate on a 1–10 scale how much they felt they were touching their own hand: 1, “not at all”; 10, “completely.” They were told to respond 99 on both questions in case they noted anything wrong during the trial (e.g., afterimage not present at the end of the induction, eyes kept open during the induction, etc.). Accordingly, 0.8% of trials in Experiment 1, 1.8% in Experiment 2, 2.3% in Experiment 3, 1.0% in Experiment 4, and 2.1% in Experiment 5 were removed from the analysis. The experiment consisted of 40 trials in the synchronous condition and 40 trials in the asynchronous condition, pseudo-randomized with a maximum of 3 consecutive trials of the same condition. The condition of each trial was indicated to the experimenter by an auditory cue played through headphones, unbeknownst to the participant. A short break was enforced after each block of 20 trials. One trial in each condition was performed beforehand as training. Each experiment lasted ∼1 h.
Behavioral statistical analysis.
Behavioral results: location and self-touch ratings were standardized, so that the mean and SD per participant and per block were equal to 0 and 1, respectively. This Z score transformation allowed us to capture differences between the synchronous and asynchronous conditions despite the important interindividual variability in the raw ratings. Trials with scores >2.5 SD or <2.5 SD from the mean were excluded (corresponding to 1.2% and 0.24%, respectively, of trials for the location and self-touch ratings in Experiment 1, 1.3% and 0.1% in Experiment 2, 8.0% and 0.0% in Experiment 3, 1.7% and 0.3% in Experiment 4, and 1.4% and 0.0% in Experiment 5), as well as trials in which the self-touch rating was above average in the asynchronous condition (i.e., when participants felt they were touching their own hands despite asynchronous stroking, corresponding to 4.9% and 5.0%, respectively, of trials in Experiments 1 and 2). Statistical significance of the location effect was assessed using permutation tests. For each participant, a null distribution of the median difference in location ratings after synchronous versus asynchronous stroking was created by shuffling the condition labels (synchronous/asynchronous) for each participant over 10,000 iterations while keeping the actual number of trials per participant and condition. Null distributions were then averaged across participants, and p values were estimated by counting the proportion of shuffled samples exceeding the observed average difference in location ratings (2-sided tests). All analyses were performed with R (2014, RRID:SCR_001905).
EEG acquisition and preprocessing.
In Experiment 2, continuous EEG was acquired at 1024 Hz with a 64-channel Biosemi ActiveTwo system referenced to the common mode sense-driven right leg ground (CMS-DRL). Signal preprocessing was performed using custom MATLAB (The MathWorks) scripts using functions from the EEGLAB (version 13.2.1, RRID:SCR_007292) (Delorme and Makeig, 2004), Adjust (RRID:SCR_009526) (Mognon et al., 2011), and Sasica toolboxes (Chaumon et al., 2015). The signal was first downsampled to 512 Hz and bandpass filtered between 1 and 45 Hz (Hamming windowed sinc finite impulse response filter). Following visual inspection, three participants had one continuous artifact-contaminated electrode removed (T7, FT8, and T8, respectively), corresponding to 0.25% of total data. On each trial, the last 12 s of stroking were broken into six nonoverlapping epochs of 2 s. Another epoch of 2 s before the afterimage induction (i.e., during fixation) served as baseline. The signal from each electrode was centered to zero, average-referenced, and epochs with amplitude changes of ±200 μV DC offset were rejected (corresponding to 3.4% of epochs, SD = 5.6%). Independent component analysis (Makeig et al., 1996) was applied to individual datasets, followed by a semiautomatic detection of artifactual components based on measures of autocorrelation, correlation with vertical and horizontal EOG electrodes, focal channel topography, and generic discontinuity (Chaumon et al., 2015). Automatic detection was validated by visually inspecting the first 15 component scalp map and power spectra. After artifact rejection, epochs with amplitude changes of ±100 μV DC offset were excluded (corresponding to 1.4% of epochs, SD = 3.5% so an average of 519 epochs, SD = 71, per subject were analyzed), and the 3 artifact-contaminated electrodes were interpolated using spherical splines (Perrin et al., 1989).
Statistical analyses of EEG data.
All analyses were performed using custom MATLAB scripts using functions from the Fieldtrip toolboxes (RRID:SCR_004849) (Oostenveld et al., 2011). Frequency analysis in canonical bands of interest was performed on each epoch using discrete prolate spheroidal sequences (Slepian sequences) as tapers, with spectral smoothing through multitapering (α: 8–12 Hz, smoothing ±2 Hz; low β: 13–21 Hz, smoothing ±4 Hz; high β: 22–30 Hz, smoothing ±4 Hz; low γ: 31–45 Hz, smoothing ± 7 Hz). Power values were log transformed to dB units, averaged across epochs, and normalized by subtracting the spectral activity found in baseline epochs (i.e., 2 s periods during which participants stared at the fixation LED before the afterimage induction). Statistical significance within each frequency band was assessed through cluster-based permutation statistics across participants (Maris and Oostenveld, 2007). Specifically, a two-tailed paired t test was performed for each electrode, and a cluster statistic was defined as the sum of the t values of all spatially adjacent electrodes exceeding a critical value corresponding to an α level of 0.05. Then, we compared this cluster statistic with the maximum cluster statistics of 1000 random permutations, with a threshold Monte Carlo p value of 0.05. The same logic was applied for correlation analyses: a linear regression was performed for each electrode, and the sum of the corresponding t values within a cluster was compared with the maximum cluster statistics of 1000 linear regressions on random permutations, with a threshold Monte Carlo p value of 0.05.
Source localization.
We localized the sources of oscillatory power in the α and low β bands, as well as the sources of phase synchrony in the low γ band using the Brainstorm toolbox (RRID:SCR_001,761) (Tadel et al., 2011). Specifically, we generated a forward model with OpenMEEG (RRID:SCR_002510) on the cortical surface of a template MNI brain (colin27 atlas) with a 1 mm resolution (Collins et al., 1998; Tzourio-Mazoyer et al., 2002). Cortical sources in the synchronous and asynchronous conditions were then estimated with an unconstrained inverse model of EEG sources using the standard weighted minimum-norm current estimate (Baillet et al., 2001) and mapped to a distributed source model consisting of 15,000 current dipoles. The reliability of the difference in absolute dipole current values was assessed by computing t values between the synchronous and asynchronous conditions, and searching for regions containing at least 20 adjacent vertices whose t value exceeded an uncorrected p value of 0.05 (arbitrary cutoff). This approach was complemented by a cluster-based permutation test approach (Maris and Oostenveld, 2007) across the 15,000 vertices of the cortical model, in which we compared the cluster statistics of each cluster with the maximum cluster statistics of 500 random permutations, with a threshold Monte Carlo p value of 0.05. The anatomical description provided in the main text is based on the parcellation proposed by Destrieux et al. (2010).
Functional connectivity.
The phase synchrony of low γ power was analyzed in the source space, following the method by Lachaux et al. (1999) using the Brainstorm toolbox (Tadel et al., 2011). Phase-locking values between each vertex of a seed, including the left precentral, paracentral, and postcentral cortices, and other brain vertices were computed, to reflect the consistency of phase lag between two vertices across a 2 s window. Phase-locking values were averaged across seed vertices, trials, and conditions and compared at the group level using two-sample t tests. The scope of our connectivity analysis was limited to the primary and secondary visual cortex, and the inferior parietal cortex along the dorsal visual stream. Regions of interest were defined according to the parcellation proposed by Destrieux et al. (2010).
Electro-oculography.
Saccades were defined by an abrupt change in voltage over a duration of at least 10 ms (corresponding to a velocity of at least 40°/s), followed by a plateau during at least 60 ms. A calibration phase was used before each recording to convert voltage into visual angle. Preprocessing was done with the EOGUI MATLAB toolbox (Eogui, software to analyze EOG recordings). Only saccades occurring during the induction phase (average number of 731 saccades per participant, SD = 478), or between the end of the induction and the first behavioral report (average number of 218 saccades per participant and condition, SD = 140) were analyzed. Robust regressions between head/eye movement amplitudes and behavioral responses were performed using R and the robustbase package, which present the advantage of being less affected by outliers (Rousseeuw et al., 2009).
Results
Experiment 1
In this first experiment, we tested the influence of illusory self-touch on the perception of visual position using both objective behavioral measures and subjective reports. First, we found that the sRHI was effective because illusory self-touch was stronger after synchronous (mean Z score = 0.87, SD = 0.12) versus asynchronous stroking (mean Z score = −0.91, SD = 0.11, permutation test: p < 0.001; Fig. 2; Table 1). We then tested our main hypothesis (i.e., that illusory self-touch was accompanied by a change in the perceived position of the afterimage: afterimage drift effect). We found a rightward drift in location ratings after synchronous (mean Z score = 0.16, SD = 0.15) versus asynchronous stroking (mean Z score = −0.13, SD = 0.36; permutation test: p = 0.002), confirming that the afterimage drifted toward the subjectively owned hand.
Behavioral results. Standardized location and self-touch ratings in the synchronous (blue) and asynchronous conditions (red) of Experiments 1 and 2. Error bars indicate 95% CIs. In Experiment 2, the correlation between the afterimage location drift and the magnitude of illusory self-touch is shown.
Raw and standardized values (SD) of self-touch ratings (1–10 scale) and afterimage location ratings (in inches)
Because the Taylor illusion may also occur in the presence of head movements while the hands remain still (Gregory, 1959), one possibility is that the afterimage drifts we observed were due to participants moving their heads (despite our instructions) and thus regardless of the modulation of illusory self-touch. To assess the influence of head movement on the visual afterimage drift, we first compared the amplitude of head movements in the synchronous versus asynchronous conditions. The amplitudes of horizontal translation and yaw angle were averaged separately over two periods of interest: the induction phase (15 s), and the period between the end of induction phase and the first behavioral response (variable duration depending on reaction times). No differences regarding head translation and rotation between synchronous and asynchronous stroking were found (all p values >0.1; average horizontal position during synchronous stroking = −1.7 cm, SD = 9.8 cm; average horizontal position during asynchronous stroking = −0.9 cm, SD = 10.5 cm; see Figure 3). To check whether more subtle differences in head movements could explain the behavioral effects aforementioned, we further performed regressions between the magnitude of afterimage drifts and the amplitude of head movements in the synchronous versus asynchronous conditions. Regressions were performed both across trials separately for each participant, and on average amplitude values across participants. No significant correlation was found in any of our analyses (all p values >0.29), suggesting again that visual afterimage drifts were not explained by head movements.
Head tracking results. Each graph represents the average time course of head horizontal translation and yaw rotation across trials, in the synchronous (blue) and asynchronous (red) conditions of Experiments 1 and 2. From left to right, Vertical dashed lines indicate the time at which the light flashed (1 s), the beginning of the rubber hand induction phase (6 s), and the end of the rubber hand induction phase (21 s). The raw data were standardized, so that an increase in z score represents rightward translations and rightward rotations in the yaw plane. Participants shifted and turned their head rightward after the flash and before the rubber hand induction, as instructed. Gray areas around the means represent 95% CIs. Black dots represent samples for which a significant difference between conditions was found (p < 0.05).
In Experiment 2, we aimed at replicating our behavioral effects in a new sample of participants and controlled that visual afterimage drifts were not explained by eye or head movements. We also recorded 64-channel EEG signals during the sRHI induction to investigate the electrophysiological signature of illusory self-touch, the afterimage drift, and the relation between both effects.
Experiment 2
Behavioral results
As in Experiment 1, participants had a stronger feeling of self-touch in the synchronous (mean Z score = 0.95, SD = 0.08) versus asynchronous condition (mean Z score = −0.96, SD = 0.07; permutation test: p < 0.001), confirming that the sRHI was effectively induced (Fig. 2). Here, the rightward drift in the afterimage location during synchronous (mean Z score = 0.10, SD = 0.24) versus asynchronous stroking (mean Z score = −0.04, SD = 0.22) did not reach significance (one-tailed permutation test: p = 0.054). We note that the effect size here seems smaller than what we found in Experiment 1, although this difference between Experiment 1 and 2 did not reach significance (Z score difference: 0.11 vs 0.29, two-sample t test: p = 0.3). Although the effect did not reach significance, the effect of hand ownership on vision was corroborated by a positive correlation between the amplitude of the afterimage drift and the self-touch ratings in the synchronous versus asynchronous condition (adjusted R2 = 0.20, permutation test comparing the observed slope vs 10,000 surrogate slopes calculated on shuffled samples: p = 0.0025; Fig. 2). No such correlation was found in Experiment 1, potentially due to a lower variance in the afterimage drift effect. We note that the measures we performed did not allow us to directly correlate the amplitude of proprioceptive and afterimage drifts. Two reasons led us to measure the self-touch illusion rather than proprioceptive drift as a signature of the sRHI. First, proprioceptive drift is a rather unspecific signature of limb ownership, as it is induced by both synchronous and asynchronous stroking, as opposed to changes of limb ownership, which are specifically induced by synchronous stroking (Rohde et al., 2011). Second, illusory ownership, but not proprioceptive drift, is increased by crossed posture (Pozeg et al., 2014), a property we used to boost our behavioral effect. Thus, even though proprioceptive drift and the feeling of ownership may be related, different mechanisms are likely to be involved, so that conclusions about feelings of ownership cannot be drawn from measures of proprioceptive drift alone. Nevertheless, it will be interesting to assess how afterimage drifts relate to illusory proprioceptive drifts, in link with has been done in the classical Taylor illusion.
As in Experiment 1, no significant difference was found between head movements in the synchronous and asynchronous conditions, and no correlation was found between the amplitude of the afterimage drift and translational or rotational head movements (all p values >0.07), ruling out potential confounds in terms of overt movements. Furthermore, to assess the influence of eye movements on afterimage drifts, we compared the amplitude of vertical and horizontal saccades in the synchronous versus asynchronous conditions during the induction phase and during the period preceding the first behavioral response. No significant difference was found (all p values >0.3). Finally, we performed regressions between saccadic amplitude and the afterimage drift amplitude in the synchronous versus asynchronous conditions on average values across participants, and found no significant correlation (all p values >0.17), suggesting that afterimage drifts were not explained by oculomotor behavior. Previous work has shown that, beyond proprioceptive inputs, the classical Taylor illusion is largely explained by eye movements, and more specifically, eye vergence, because moving the hand toward the face was shown to increase eye vergence (i.e., ocular adduction) in correlation with the afterimage's perceived size (Sperandio et al., 2013). Eye vergence occurs for movements in the depth axis and is therefore unlikely to explain the horizontal drifts that we observed in the present study.
Electrophysiological results
Relying on EEG, we sought to describe the interplay between vision and the self at the neural level, by measuring the neural correlates of the illusory self-touch and the afterimage drift effect. We analyzed EEG data according to two main hypotheses regarding oscillatory power and functional connectivity. Concerning oscillatory power, we expected illusory self-touch to involve α suppression over sensorimotor areas, as reported before for related, but distinct, ownership manipulations (Lenggenhager et al., 2011; Evans et al., 2013; Serino et al., 2015). Conversely, we expected afterimage drifts to stem from visual striate and extrastriate cortices, considering that afterimages are known to involve retinal, but also visual cortical neurons, as they undergo perceptual filling-in (Shimojo et al., 2001; van Lier et al., 2009), are subject to the Ebbinghaus-Titchener illusion (Sperandio et al., 2012b), and are modulated by attention (Suzuki and Grabowecky, 2003; van Boxtel et al., 2010). To further assess the link between spectral EEG and behavior, we complemented this contrastive analysis by quantifying how changes in oscillatory power correlated with the magnitude of the illusory self-touch and afterimage drift effects. Concerning functional connectivity, we measured how sensorimotor regions related to illusory self-touch and visual brain areas related to the visual afterimage were connected during synchronous versus asynchronous stroking. We focused on the low γ band, in line with the prominent role of γ power for cross-modal effects in vision and multisensory processing (e.g., Kanayama et al., 2007, 2009; for review, see Engel et al., 2012).
Frequency analysis
We computed the topographies of oscillatory power in the α (8–12 Hz), low β (13–21 Hz), high β (22–30 Hz), and low γ bands (31–45 Hz) in the synchronous versus asynchronous stimulation at the level of the whole scalp. In the α band, we found a cluster of 20 electrodes contralateral to the hand receiving the stroking (over left frontocentral regions) with smaller oscillatory power in the synchronous versus asynchronous condition (p = 0.038; average power in the synchronous condition: 1.01 dB, SD = 1.43; average power in the asynchronous condition: 1.23 dB, SD = 1.51; paired t test: t(18) = −3.63, p = 0.002; Fig. 4a, top). Source localization analysis suggested that these differences measured at the scalp level stemmed from three main regions, including the posterior cingulate cortex bilaterally, and a large cluster extending from left premotor cortex to inferior parietal cortex (Fig. 4b; Table 2; these results did not survive correction for multiple comparisons and should be therefore considered with caution). These results are in line with previous studies relying on the visual version of the rubber hand illusion, reporting α suppression during synchronous versus asynchronous stroking, predominantly over the somatosensory cortex contralateral to the stroked hand (Lloyd et al., 2006; Schaefer et al., 2006). In the low β band, a cluster of 29 electrodes over bilateral frontoparietal scalp regions also revealed oscillatory suppression in the synchronous versus asynchronous condition (p = 0.012; average power in the synchronous condition: −0.89 dB, SD = 0.85; average power in the asynchronous condition: −0.76 dB, SD = 0.86; paired t test: t(18) = −4.11, p < 0.001; Fig. 4a, bottom). No other cluster was below the threshold Monte Carlo p value of 0.05 in any of the other frequency bands.
Oscillatory power and correlation analyses in the α (8–12 Hz, top row) and low β (13–21 Hz, bottom row) bands. a, Topographies of t values following a paired Student's t test comparing oscillatory power in the synchronous versus asynchronous condition. *Electrodes showing a significant difference (p < 0.05 based on cluster-based permutation statistics across participants). b, Source localization corresponding to the topographies shown in a.
Source localization
To ensure that these differences were not due to eye movements, we computed the correlation between the differences in oscillatory power and saccadic amplitude during synchronous versus asynchronous stroking. Significant correlations were found between the amplitude of horizontal saccades and oscillatory power in the low γ band within a cluster of frontocentral electrodes, suggesting that horizontal saccades may have contributed to differences in oscillatory power during synchronous versus asynchronous stroking in the low γ band, but not in the α and low-β band, which are of interest here. Together, these results show that synchronous versus asynchronous stroking induced α suppression predominantly in frontoparietal cortex contralateral to the hand receiving the stroking, and β suppression in bilateral frontoparietal cortex.
Correlation analysis between EEG and behavioral data
Next, we assessed whether changes in oscillatory power during synchronous versus asynchronous stroking correlated with the illusory self-touch and afterimage drift effects. To avoid spurious effects due to potential outliers, we relied on robust regressions with iteratively reweighted least squares, and corrected for multiple comparisons with a cluster-based permutation test approach (see Materials and Methods). We found that changes in oscillatory power in the high β band were positively correlated with illusory self-touch in two posterior parietal electrodes (Table 3), reflecting that illusory self-touch was stronger in participants with higher β power during synchronous versus asynchronous stroking (Fig. 5). In addition, we found that α band power correlated with differential self-touch at C3, contralateral to the hand receiving the stroking (adjusted R2 = 0.39, p = 0.003). Although this effect did not resist the correction for multiple comparisons, this electrode position fits well our results on α suppression, as well as previous results linking the contralateral sensorimotor cortex to illusory self-touch (Dieguez et al., 2009), the rubber hand illusion (Evans et al., 2013) or illusory self-identification (Lenggenhager et al., 2011). Together, these results extend our previous results and suggest that illusory self-touch is more pronounced in participants showing stronger α/β suppression contralateral to the stroked hand.
Correlations between oscillatory power and self-touch and afterimage drift effectsa
Correlation analysis between the difference in oscillatory power in the high β (22–30 Hz, left), and low γ bands (31–45 Hz, right) bands, and behavioral ratings. Top row, Illusory self-touch as measured by comparing self-touch ratings in the synchronous versus asynchronous condition. Bottom row, Afterimage location drift, as measured by comparing location ratings in the synchronous versus asynchronous condition. t values are shown. *Electrodes showing a significant correlation (p < 0.05).
Regarding the magnitude of afterimage drifts, we found significant correlations with changes in oscillatory power in the high β band over bifrontal scalp regions (extending to left temporal regions), as well as in the low γ band over frontotemporal scalp region (Table 3). Whereas there was no difference in oscillatory power in this frequency range at the group level, these correlations reveal that the magnitude of afterimage drift was bigger in participants showing higher β/γ power in the synchronous versus the asynchronous condition. Importantly, there was no spatial overlap between this effect and the correlation between γ power and horizontal saccades described above. Together, these results suggest a spectral dissociation, with illusory self-touch predominantly reflected in the α and low β bands over frontoparietal sites, and the afterimage drift in the high β and low γ bands over frontotemporal sites. Qualitatively similar results were found when considering a broader γ band (30–80 Hz), although effect sizes were of smaller amplitude compared with the low γ band (for further evidence regarding the role of neural activity in the low γ band during bodily illusions, see Lenggenhager et al., 2011).
Connectivity analysis
Following previous results (Kanayama et al., 2007, 2009), we measured γ connectivity during synchronous versus asynchronous stroking conditions between the left sensorimotor cortex involved in hand ownership (seed region), and visual regions arguably involved in the afterimage drift effect, namely, the inferior parietal cortex and primary/secondary visual cortices (Sperandio et al., 2012a). When comparing average phase locking values within regions of interest (see Materials and Methods), we found an increase in connectivity between the seed region (left somatosensory cortex) and the left inferior parietal cortex during synchronous versus asynchronous stroking (t(18) = 2.8, p = 0.01), but not left primary and secondary visual cortices (p = 0.9). An increase in connectivity between C3 and left dorsovisual pathways was also found when connectivity was computed at the scalp level (data not shown). This finding is particularly relevant considering the role of the dorsal visual pathways to program and control elementary movements, as opposed to the ventral visual pathways mainly involved in object discrimination (Goodale and Milner, 1992). No significant effect was found in the visual regions of interest located in the right hemisphere (all p values >0.6). Whole-brain analysis revealed connectivity differences between left and right somatosensory cortices, as well as bilateral visual areas, which, however, did not survive corrections for multiple comparisons using cluster-based permutation tests (Fig. 6).
Difference in functional connectivity between synchronous and asynchronous stroking in the low γ band (31–45 Hz), considering a seed in the left somatosensory cortex. Regions of interest corresponding to the left inferior parietal cortex, and left primary and secondary cortex are outlined in white. Uncorrected t values are shown.
Additional experiments
To further explore the properties of afterimage drifts, we performed three additional experiments. First, in Experiment 3, we assessed whether tactile-proprioceptive synchrony responsible for illusory self-touch could induce afterimage drifts, even in case the illusory owned hand was not displaced to the right. To this end, we set the participant's right hand to be perfectly aligned with the body-midline instead of being aligned with the right shoulder, and we placed the rubber hand 25 cm in front of the participant's hand, rather than 25 cm shifted to the right. As a result, the only displacement between the real hand and the illusory owned hand occurred along a sagittal axis, orthogonal to the rightward afterimage drift we found in Experiment 1. With such settings, participants again felt that they were touching their own hand in the synchronous compared with asynchronous condition, with a misalignment along the sagittal axis instead of the coronal one (mean Z score in the synchronous condition = 0.80, SD = 0.25; mean Z score in the asynchronous condition = −0.76, SD = 0.29, p < 0.001). Although the self-touch illusion occurred along the sagittal axis, a rightward afterimage drift was found when comparing visual location ratings during synchronous and asynchronous stroking (mean Z score in the synchronous condition = 0.19, SD = 0.17; mean Z score in the asynchronous condition = 0.01, SD = 0.28, p = 0.02). These results suggest that tactile-proprioceptive synchrony is the key factor both for illusory self-touch and afterimage drifts. They also show that the physical positions of the participant's right and left hands (determined by proprioceptive cues) are not crucial because rightwards afterimage drifts occur when the right hand is aligned with the right shoulder or with the body midline. Next, to assess whether afterimage drifts occur in case the afterimage is not aligned with the position of the stroked hand, we replicated Experiment 3, this time flashing the light 25 cm to the right of the participant's right hand, positioned at the body midline. Because the afterimage was not induced on the participant's hand itself, we could assess in Experiment 4 whether afterimage drifts induced by tactile-proprioceptive synchrony would only occur when the position of hand and afterimage were aligned. Here again, by comparing ratings in the synchronous versus asynchronous conditions, we found that synchronous tactile-proprioceptive stimulation made participants feel they were touching their own hand (mean Z score in the synchronous condition = 0.94, SD = 0.05; mean Z score in the asynchronous condition = −0.87, SD = 0.13, p < 0.001); moreover, participants reported the afterimage to be displaced to the right (mean Z score in the synchronous condition = 0.12, SD = 0.10; mean Z score in the asynchronous condition = −0.12, SD = 0.26, p = 0.004). This suggests that afterimage drifts do not require the afterimage to be coregistered on the physical hand before the RHI induction. This differs from the classical Taylor illusion, in which a specific registration of proprioceptive and visual signals is necessary for the afterimage drift to occur (Carey and Allan, 1996).
Interpreting the results from Experiments 1–4, it appears that the direction of the afterimage drifts was in the direction of the participant's hand being stroked because, in all four experiments, the hand performing the stroking was the left one while the hand receiving the stroking was the right one (independent of proprioceptive signals and the hand's position in external spatial coordinates). To test whether the body side of the stroked hand was indeed the driving factor for the direction of afterimage drifts, we devised Experiment 5, in which the experimenter used the participant's right hand to stroke a left rubber hand while stroking the participant's left hand (all other parameters were similar to Experiment 3). As in the other experiments, participants felt that they were touching their own hand in the synchronous compared with asynchronous condition (mean Z score in the synchronous condition = 0.96, SD = 0.07; mean Z score in the asynchronous condition = −0.96, SD = 0.10, p < 0.001), revealing that the self-touch illusion also occurred for the left hand. Crucially, a trend toward a leftward afterimage drift was found when comparing visual location ratings during synchronous and asynchronous stroking (mean Z score in the synchronous condition = −0.09, SD = 0.26; mean Z score in the asynchronous condition = 0.01, SD = 0.16, p = 0.07). Together, our experiments show afterimage drift toward the right hemifield when the left hand is stroking a rubber hand while the right hand is receiving the same strokes synchronously, and drift toward the left hemifield when the right hand is stroking a rubber hand while the left hand receives synchronous strokes. Thus, the laterality of the body (with respect to the hand stroking and the hand receiving the stroking) determines the direction of the afterimage drift, independent of proprioceptive signals from the stroked hand.
Discussion
There is now good evidence that vision is shaped by nonvisual bodily inputs (for review, see Faivre et al., 2015), including vestibular (Salomon et al., 2015), proprioceptive (Salomon et al., 2013b), tactile (Lunghi et al., 2010; Faivre et al., 2016; Salomon et al., 2015), motor (Maruya et al., 2007; Salomon et al., 2013a), cardiac signals (Salomon et al., 2016a), and full-body stimuli, including body size (van der Hoort et al., 2011, 2014; Banakou et al., 2013). By showing in five independent experiments that the position of an afterimage depends on the synchrony of tactile-proprioceptive stimulation, we report that the Taylor illusion does not just depend on bodily inputs, but also on the sense of body ownership. In Experiment 1, we found a drift in the location of an afterimage after the induction of illusory self-touch (i.e., afterimage drift effect), revealing that visual position is not only influenced by proprioceptive signals conveying hand position (i.e., Taylor illusion), but also by integrated somatosensory signals from both hands during the sRHI. With three additional experiments, we showed that tactile synchrony between both hands is sufficient for afterimage drifts to occur, that a spatial alignment between the afterimage and physical hand position is not necessary, and that the direction of the afterimage drift is independent of proprioceptive signals from the stroked hand. Finally, we controlled that this change in visual position perception was not caused by overt head or eye movements.
The classical Taylor illusion has been interpreted as a consequence of the resolution of multisensory conflicts. Following retinal and cortical encoding steps within retinotopic reference frames, the afterimage is mapped onto the arm-centered spatial reference frame corresponding to the participant's hand (Boussaoud and Bremmer, 1999). When participants move their hand, the position of the afterimage is updated to “explain away” the multisensory mismatch and therefore minimizing prediction error. This mechanism can be operationalized through successive neural events forming the “activate-predict-confirm perceptive cycle” (Enns and Lleras, 2008). Within this framework, bottom-up signals from activated visual neurons first generate a prediction about the afterimage position, which is subsequently compared in the predict phase with information from higher-level multisensory areas, including illusory hand position. This comparison between visual and proprioceptive coordinates results in a mismatch between the encoded location of the light flashed on the participant's hand and the updated hand position. Prediction error is minimized incrementally, by a remapping of the visual position taking into account the hand position inferred from proprioceptive cues. The “confirm” phase of the cycle corresponds to the final conscious percept of an afterimage, once a resonance of congruent bottom-up and top-down inputs occurs (Lamme and Roelfsema, 2000). However, this model applies only partly to the present illusion because Experiments 3 and 4 showed that tactile synchrony between the two hands of the participants is sufficient to induce afterimage drifts, independently of the positions of the participant's stroked hand and the rubber hand. A similar model holds for the classical Taylor illusion, the only difference being that, in that case, the afterimage drift does not depend on illusory arm position signals but on overt bodily changes, namely, the participants' hand position. According to this view, the afterimage drift is caused by top-down signals, the byproduct of prediction error minimization rather than a default in bottom-up visual processing. Our finding may relate to a variant of the Taylor illusion in which the visual afterimage of the participants' hands fades out upon actively or passively moving the hands (“crumble effect,” see Davies, 1973). In this context, it was shown that relocating the hands during the brief blind period between the flash and the afterimage appearance weakens the crumble effect and decreases the feeling of ownership over the hand's afterimage (Hogendoorn et al., 2009). According to the latter authors, displacing the hands before the afterimage appearance decreases the crumble effect because proprioceptive signals conveying hand movements and visual signals conveying the image of a disowned hand are no longer inconsistent. Experiment 4 speaks against these findings, as an afterimage drift occurred even when the light was flashed 25 cm to the right of the rubber hand, instead of being flashed on the rubber hand itself. This suggests that the present variant of the Taylor illusion also occurs when visual and proprioceptive signals have distinct somatotopic and external spatial coordinates, although it may be argued that alignment within peripersonal hand space is sufficient (i.e., Makin et al., 2008; Blanke et al., 2015). Experiment 5 suggested that afterimage drifts in the opposite direction occur in case the right hand is the one stroking the rubber hand and the left hand is the one receiving the stroking by the experimenter. Thus, the direction of afterimage drifts depends on the laterality of the stroked hand, compatible with higher-order bilateral bodily signals (independent of proprioceptive signals). In the following, we discuss the neural correlates of illusory self-touch, of afterimage perception, and how both interact.
Illusory self-touch and α power over sensorimotor cortex
Previous work showed that decreased oscillatory power in the α band reflects increased activation in sensorimotor and premotor cortices (Oakes et al., 2004), related to tactile stimulation (Pfurtscheller, 1981). α band suppression was also found during illusory body ownership in the visual rubber hand illusion (Evans et al., 2013; Kanayama et al., 2007, 2009), full body illusion (Lenggenhager et al., 2011), and illusory finger ownership (Dieguez et al., 2009). Beyond body ownership, oscillatory α power was involved in other visual and multisensory illusions, modulating excitability of sensory cortical neurons and regulating corticocortical connectivity (Lange et al., 2014). In the present study, we compared oscillatory power during synchronous versus asynchronous stroking and found a decrease in frontoparietal α power during the illusion condition corresponding to sensorimotor regions contralateral to the hand receiving the stroking, and a decrease in low β power over bilateral frontoparietal sites. Beyond these changes as a function of synchrony, illusory self-touch at the group-level correlated with oscillatory suppression over sensorimotor regions contralateral to the hand receiving the stroking, which cannot directly stem from the manipulation of synchrony and further support the role of sensorimotor cortex for illusory self-touch. Contrary to the visual RHI where only one hand is involved, here the left hand was stroking while the right hand was being stroked. Reminiscent of our study and of previous neuroimaging results on the sRHI reporting bilateral activations in the premotor cortices and intraparietal sulci (Ehrsson et al., 2005), a related finger illusion (Dieguez et al., 2009) was recently shown to involve activity within primary sensorimotor cortex corresponding to both the stroked and stroking fingers (Martuzzi et al., 2015).
Changes in afterimage position and β-γ power over frontotemporal regions
Analyzing the changes in oscillatory power with the change of afterimage position during synchronous versus asynchronous stroking, we found that afterimage drifts were more pronounced in participants showing higher β and γ power over bilateral frontotemporal regions. Control analyses ruled out eye movements as a cause of this effect because there was no difference in saccadic amplitude between the synchronous and asynchronous conditions, no correlation between saccadic amplitude and afterimage drifts, and no correlation between saccadic amplitude and β-γ power in frontotemporal regions. γ power has been involved in cross-modal interactions and ownership illusions (Kanayama et al., 2007, 2009), and the “temporal correlation hypothesis” (Singer and Gray, 1995) proposed that oscillatory brain activity may be a key mechanism not only for unisensory processing (e.g., featural binding, attention guiding), but also for cross-modal processing (Engel et al., 2012). Increased γ power was found in multisensory illusions, such as the sound-induced flash illusion (Bhattacharya et al., 2002; Mishra et al., 2007; Balz et al., 2016), which is in line with the present data in suggesting that oscillatory activity in the γ band reflects perceptual changes resulting from cross-modal interactions. The finding that a different set of electrodes reflected changes in the magnitude of illusory self-touch (sensorimotor regions) versus the afterimage drift (frontotemporal regions) suggests that both phenomena rely on different spectral mechanisms in anatomically different systems (Fig. 5).
Increased γ connectivity between sensorimotor and visual cortices
Neural synchrony in the γ band enhances visuomotor coupling (Womelsdorf et al., 2006) and enables visual awareness (Melloni et al., 2007), by supporting integration and by joining distributed sets of neurons into coherent ensembles during visual (Rodriguez et al., 1999) or multimodal processing (Roelfsema et al., 1997). The role of γ synchrony in relation to multimodal processing related to the self is corroborated by previous studies relying on the rubber hand illusion, showing that illusory limb ownership coincided with an increase of γ connectivity over large parts of cortex (Kanayama et al., 2007, 2009). Here, conducting analyses at the scalp and source levels, we found that the sRHI involved modulations of γ connectivity between somatosensory and occipital sites, including left early visual areas and the left dorsal visual pathways. Thus, in addition to its role for limb ownership, and in line with the role of γ synchrony for top-down processing and predictive mechanisms (Engel et al., 2001), γ connectivity could instantiate the activate-predict-confirm perceptive cycle (Enns and Lleras, 2008) and enable the remapping of visual representations in the presence of conflicting somatosensory inputs during the sRHI.
In conclusion, by combining the sRHI with the Taylor illusion, we reveal that visual perception depends on the sense of ownership, which itself is based on the synchrony of somatosensory signals from the two hands. This suggests that vision is not only influenced by contextual (Eagleman, 2001) and bodily signals (Faivre et al., 2015) but is also grounded in the sense of self, corresponding here to hand ownership. Beyond vision, testing the impact of bodily self-consciousness on other sensory modalities and cognitive domains will help assessing how the sense of self serves as a scaffold for perceptual and conceptual mental capacities.
Footnotes
This work was supported by Swiss National Science Foundation 51AU40_125759 and the Bertarelli Foundation. N.F. is an Ecole Polytechnique Fédérale de Lausanne Fellow cofunded by Marie Skłodowska-Curie and was supported by the European Union Human Brain Project. R.S. was supported by the National Center of Competence in Research (SYNAPSY: The Synaptic Bases of Mental Diseases), financed by Swiss National Science Foundation 51AU40_125759.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Nathan Faivre, Laboratory of Cognitive Neuroscience, Campus Biotech H4, Chemin des Mines 9, 1202 Genève, Switzerland. nathanfaivre{at}gmail.com
