Abstract
The brain constructs a flexible representation of the body from multisensory information. Previous work on monkeys suggests that the posterior parietal cortex (PPC) and ventral premotor cortex (PMv) represent the position of the upper limbs based on visual and proprioceptive information. Human experiments on the rubber hand illusion implicate similar regions, but since such experiments rely on additional visuo-tactile interactions, they cannot isolate visuo-proprioceptive integration. Here, we independently manipulated the position (palm or back facing) of passive human participants' unseen arm and of a photorealistic virtual 3D arm. Functional magnetic resonance imaging (fMRI) revealed that matching visual and proprioceptive information about arm position engaged the PPC, PMv, and the body-selective extrastriate body area (EBA); activity in the PMv moreover reflected interindividual differences in congruent arm ownership. Further, the PPC, PMv, and EBA increased their coupling with the primary visual cortex during congruent visuo-proprioceptive position information. These results suggest that human PPC, PMv, and EBA evaluate visual and proprioceptive position information and, under sufficient cross-modal congruence, integrate it into a multisensory representation of the upper limb in space.
SIGNIFICANCE STATEMENT The position of our limbs in space constantly changes, yet the brain manages to represent limb position accurately by combining information from vision and proprioception. Electrophysiological recordings in monkeys have revealed neurons in the posterior parietal and premotor cortices that seem to implement and update such a multisensory limb representation, but this has been difficult to demonstrate in humans. Our fMRI experiment shows that human posterior parietal, premotor, and body-selective visual brain areas respond preferentially to a virtual arm seen in a position corresponding to one's unseen hidden arm, while increasing their communication with regions conveying visual information. These brain areas thus likely integrate visual and proprioceptive information into a flexible multisensory body representation.
- extrastriate body area
- multisensory integration
- posterior parietal cortex
- premotor cortex
- proprioception
- vision
Introduction
The brain uses information from multiple sensory modalities to construct a flexible representation of the body, including its current structure and position (Dijkerman and de Haan, 2007; Makin et al., 2008; Tsakiris, 2010; Blanke, 2012; Ehrsson, 2012). For such a body representation, vision and proprioception continually provide estimates of (changing) limb position, which is crucial to guide actions (Wolpert et al., 1998; Graziano and Botvinick, 2002; Holmes and Spence, 2004). Behavioral experiments using prisms or mirrors have shown that visuo-proprioceptive limb position information is integrated based on the relative reliability of the unisensory modalities, with vision usually “dominating” proprioception due to its higher spatial acuity (van Beers et al., 1999; Holmes and Spence, 2005). This is also suggested by the rubber hand illusion (RHI) (Botvinick and Cohen, 1998), in which visual position information from a fake hand “overrides” proprioception after visuo-tactile costimulation of the fake hand and the real hand. Notably, the RHI only works if the fake hand is in a position similar to the real hand (Pavani et al., 2000; Ehrsson et al., 2004; Lloyd, 2007; Makin et al., 2008), which implies that the brain only integrates visuo-proprioceptive information under sufficient cross-modal congruence.
The potential neuronal basis of such a visuo-proprioceptive limb position representation has been illuminated by electrophysiological recordings in monkeys. These seminal experiments have revealed neurons with multimodal receptive fields in the PPC (Sakata et al., 1973; Rushworth et al., 1997; Graziano et al., 2000) and PMv (Graziano et al., 1994, 1997; Graziano, 1999), which encode seen and felt limb position and the activity of which is additionally modulated by the congruence of visuo-proprioceptive limb position information. A similar mechanism is tentatively suggested in the human brain by the fact that the PPC and PMv are activated by the RHI (Ehrsson et al., 2004; Gentile et al., 2013). However, the visuo-tactile stimulation necessary for the RHI to occur precludes an interpretation of these activations as indicating visuo-proprioceptive integration per se. Another candidate brain region for visuo-proprioceptive processing is the body-selective extrastriate body area (EBA) (Downing et al., 2001). The EBA is also activated by the RHI (Limanowski and Blankenburg, 2015a,2015b) and, interestingly, by movements (Astafiev et al., 2004; cf. Gallivan et al., 2016), which may suggest that the EBA receives not only visual but also proprioceptive information. However, the contribution of the EBA to multisensory body representation remains unclear (Peelen and Downing, 2007; Downing and Peelen, 2011).
In the present fMRI experiment, we sought direct evidence for visuo-proprioceptive integration in the human brain. We presented a photorealistic virtual 3D arm in a position congruent or incongruent to the participant's real unseen arm while changing the real arm's position between visual presentations, thus independently manipulating visual and proprioceptive arm position information within experimental runs. Congruent versus incongruent visuo-proprioceptive arm position information produced increased activation levels in the PPC and PMv, at locations corresponding to the monkey recording sites, and in the EBA. These areas were also activated by passive arm position changes without vision. Activity in the PMv moreover reflected interindividual differences in experienced virtual arm ownership. Finally, a connectivity analysis revealed that the same brain areas relatively increased their coupling with the primary visual cortex when the seen arm position information was congruent with the felt one. These results suggest that human PPC, PMv, and EBA evaluate visual and proprioceptive information about limb position and, in the case of sufficient cross-modal congruence, integrate it into a multisensory representation of limb position.
Materials and Methods
Participants and experimental design.
Twenty healthy, right-handed volunteers (7 male, mean age = 26 years, range = 21–39) participated in the experiment, which was approved by the ethics committee of the Freie Universität Berlin. One participant was excluded from data analysis because her hand could not be fully rotated to either end position during scanning, so no perceived match with the virtual arm's position could be reached.
During the experiment, participants lay inside the fMRI scanner with their right hand placed across their chest in a custom foam-padded apparatus. This apparatus could be rotated about 180° back and forth by the experimenter pulling on two nylon strings from outside of the scanner so that either the palm or the back of the hand was facing the participant (Fig. 1A). Digital stereoscopic goggles (VisuaSTIM, 800 × 600 pixels, 30° eye field) and the Blender 3D graphics software package (http://www.blender.org) were used to present participants a photorealistic virtual arm in 3D in a plausible posture with respect to their real arm (i.e., an anatomically plausible configuration and location in space), with either the palm or the back of the virtual hand facing the participant. Therefore, depending on the position of the participant's real arm (which was occluded from view by the goggles), the visible arm's position could be congruent or incongruent. It should be noted that, because the participant's head could only be moderately tilted inside the scanner head coil, the virtual arm was presented at a slightly more elevated location than the unseen real arm; however, this moderate external location discrepancy was matched across congruent and incongruent arm position conditions. Further, as a control object for an implicit functional localization of the body-selective EBA (arm vs object vision), we presented a virtual object, which was a stretched cylinder that matched the virtual arm's spatial location, color, and approximate size, but did not appear to be body part shaped and moreover gave no information about a potential (mis)match with the real arm's position.
A, Experimental setup and stimuli. Participants placed their right hand in a foam-padded apparatus (schematically shown) that allowed passive rotation of the hand from outside of the scanner into a “palm facing” or “back facing” position. A photorealistic virtual right arm (palm facing or back facing) or an object (used as a control condition for implicitly localizing the EBA) was presented in 3D using stereoscopic goggles. B, Partial experimental sequence. Visual presentations were separated by a 12 s fixation-only period, during which the real arm was passively rotated (1 s). Depending on the current position of the real arm, the seen and felt arm positions could thus be congruent or incongruent (or “neutral” in the case of the control object). C, Behavioral visuospatial attention control experiment using the same setup. Participants (n = 5) maintained fixation on the white dot while directing their endogenous attention to a Gabor patch presented randomly in one of nine possible orientations every 890 ms and pressing a button whenever the grating lines were vertical (target marked red for display purposes).
Visual stimuli were presented for 8 s, separated by a 12 s interstimulus interval (ISI) while participants constantly fixated a white dot in the middle of the visual display (Fig. 1B). After 4 s through each ISI, the experimenter swiftly rotated the participant's hand from palm to back facing or vice versa. The complete rotation took 1 s and was timed by auditory cues presented to the experimenter via headphones. To avoid surprise, the rotation was announced to the participant by the fixation dot turning red for 0.5 s before rotation onset. For each participant, 2 random stimulus sequences were used for 2 consecutive runs each; each visual stimulus type was presented 6 times per run in randomized order, resulting in 6 min run length. The runs ended with the participant's hand in the position opposite to the initial one (i.e., palm facing if the run started with back facing, and vice versa), so each participant completed four runs that were balanced in terms of initial hand position (order randomized across participants) and stimulus presentations across hand positions.
Participants completed a practice session before the scanning session to get familiar with the setup. After the experiment, they completed a brief questionnaire intended to assess the possibility that they engaged in mental rotation or could not differentiate between congruent and incongruent positions and to assess the degree to which they might have experienced an “ownership illusion” while viewing the congruent arm. The questionnaire comprised the following questions, each to be rated on a 7-point scale ranging from −3 (“do not agree at all”) to 3 (“fully agree”): “I was able to distinguish whether the virtual 3D-arm's position and my real arm‘s position were congruent (corresponding) or incongruent (not corresponding)” (Q1); “It felt as if the 3D arm was my own arm,” asked separately for congruent arm (Q2), incongruent arm (Q3), and object (Q4); and “While I was looking at the 3D arm or object, I imagined rotating my real arm into its position” (Q5). As a proxy of experienced ownership of a visuo-proprioceptively compatible (i.e., the congruent arm) versus noncompatible (i.e., the incongruent arm, but also the object) visual stimulus, we calculated a relative ownership index by (congruent arm ownership rating) − (incongruent arm ownership rating + object ownership rating)/2 (cf. Ehrsson et al., 2004). Mean ratings and rating differences were assessed for statistical significance using two-tailed t tests with Bonferroni-corrected α levels of 0.05/5 = 0.01.
To address potential differences in cognitive demands between congruent and incongruent position conditions, we further conducted a behavioral control experiment on five of our participants. We adopted an endogenous visuospatial attention task from Zimmer and Macaluso (2007; cf. Gentile et al., 2013) using the same apparatus, stereoscopic goggles, and design parameters as in the fMRI session (presentation and rotation length and order; only the ISI was shortened to 6 s); each participant completed two experimental runs, again starting with different hand positions. This time, we also presented a Gabor patch in the visual periphery above fixation, the orientation of which was changed randomly every 890 ms between 1 of 9 orientations (−45°, −33.75°, −22.5°, 11.25°, 0°, 11.25°, 22.5°, 33.75°, 45°) during presentation of the 3D arm or object (Fig. 1C). Participants were instructed, as in the fMRI experiment, to maintain fixation on the white fixation dot (monitored via online eye tracking) and had to respond via left-hand button press whenever the orientation of the grating lines was vertical (i.e., 0°). Differences in mean percentage of correct responses and mean reaction times between congruent and incongruent conditions, and between arm (congruent + incongruent) versus object vision were assessed for statistical significance using two-tailed t tests with Bonferroni-corrected α levels of 0.05/2 = 0.025.
fMRI data analysis.
The fMRI data were recorded using a 3 T scanner (Tim Trio; Siemens) equipped with a 12-channel head coil. T2*-weighted images were acquired using a gradient echoplanar imaging sequence (3 × 3 × 3 mm3 voxels, 20% gap, matrix size = 64 × 64, TR = 2000 ms, TE = 30 ms, flip angle = 70°). For each participant, we recorded 752 functional image volumes in total, a GRE field map (TE1 = 10.00 ms, TE2 = 12.46 ms), and a T1-weighted structural image (3D MPRAGE, voxel size = 1 × 1 × 1 mm3, FOV = 256 × 256 mm2, 176 slices, TR = 1900 ms, TE = 2.52 ms, flip angle = 9°). FMRI data were preprocessed and analyzed using SPM8 (www.fil.ion.ucl.ac.uk/spm/).
Artifacts at the slice level were corrected via interpolation using the SPM ArtRepair toolbox (Mazaika et al., 2009). Functional images were then realigned and unwarped using field maps, corrected for slice acquisition time differences, normalized to MNI space using DARTEL, and resliced to 2 mm voxel size, spatially smoothed with an 8 mm full width at half maximum Gaussian kernel, detrended (Macey et al., 2004), and volumes featuring excessive scan-to-scan movement were corrected via interpolation (ArtRepair toolbox). A general linear model (GLM, 300 s high-pass filter) was fitted to each participant with regressors modeling the stimulus presentations and arm rotations. The first five principal components accounting for the most variance in the cerebrospinal fluid or white matter signal time course each (Behzadi et al., 2007) were added alongside the realignment parameters as regressors of no interest. First-level contrast images were entered into a group-level repeated-measures ANOVA with the factors real arm position (palm facing, back facing) and visual stimulus (congruent arm, incongruent arm, object). One-sample t tests were used to assess the effect of arm rotation and the effect of initial hand position (to rule out any influence of different starting positions on our main effects, we calculated the respective interactions on the first level, contrasting the congruent vs incongruent conditions across runs with different starting positions). Likewise, blood oxygenation level dependent (BOLD) signal correlations with subjective arm ownership were assessed using a one-sample t test with the ownership index (see above) as a covariate. As a control analysis to test for potential timing differences between congruent and incongruent conditions, we repeated the main GLM analysis including time and dispersion derivatives of the first-level regressors. We then tested for differences between these derivatives of the different conditions and also compared the results of the congruent versus incongruent contrast with the main analysis.
We analyzed the connectivity (i.e., changes in the statistical dependencies of BOLD signal time series between a seed region of interest and voxels in the whole brain under presentation of congruent versus incongruent arm positions) of brain regions revealed by the standard GLM analysis via psychophysiological interactions (PPIs; Friston et al., 1997). For each participant, the experimental runs with their noise regressors were concatenated into a single data sequence. Region-specific BOLD time series were then extracted as the first eigenvariate of all significant voxels within a 4 mm radius sphere centered on each participant's local maximum as obtained from the individually thresholded contrast congruent versus incongruent arm positions; the mean coordinates and SDs of these seed region peaks were as follows: left aSPL: x = −10.2 ± 3.7, y = −48.6 ± 3.7, z = 65.6 ± 6.8; left pSPL: x = −13.7 ± 4.1, y = −67.8 ± 4.9, z = 51.2 ± 7.5; left IPL: x = −59.9 ± 3.5, y = −26.7 ± 5.2, z = 32.7 ± 6.3; left EBA: x = −51.3 ± 4.5, y = −72.3 ± 3.9, z = 2.3 ± 4.3; left PMv: x = −53.6 ± 5.5, y = 6.5 ± 2.7, z = 22.4 ± 4.5. We then calculated the interaction between the congruent versus incongruent arm conditions (the “psychological variable”) and the extracted seed region BOLD signal time course to reveal voxels across the whole brain, in which activity would be more strongly correlated with the seed region's activity under congruent versus incongruent arm positions. These resulting first-level contrast images were evaluated on the group level using a repeated-measures ANOVA with the factor seed region location.
Activations obtained from group-level contrasts were assessed for statistical significance applying a threshold of p < 0.05, false discovery rate corrected for multiple comparisons on the cluster level with an initial voxelwise threshold of p < 0.001. Within the PMv, which we a priori expected to be involved in visuo-proprioceptive processing based on previous work (Graziano, 1999; Ehrsson et al., 2004; Gentile et al., 2013), we applied peak-level familywise error small volume correction within a 10 mm radius spherical ROI centered on coordinates reported in a recent RHI fMRI study by Gentile et al. (2013). The resulting statistical parametric maps (SPMs) are projected onto the mean normalized structural image or rendered on SPM's brain template at p < 0.001, uncorrected, showing all activated voxels (cluster extent threshold k = 0). All reported coordinates are in MNI space; the SPM Anatomy toolbox (Eickhoff et al., 2005) was used for anatomical reference.
Results
Contrasting congruent versus incongruent seen and felt arm positions revealed significant (p < 0.05, corrected for multiple comparisons; Fig. 2A, Table 1) activation differences in the left PPC, comprising clusters in the anterior and posterior superior parietal lobe [aSPL, Brodmann's area 5 (BA5); pSPL, BA 7], the left PMv, the left inferior parietal lobe (IPL, centered on the supramarginal gyrus, SMG), and in a region within left lateral occipital cortex (LOC) corresponding to the EBA (see below; we henceforth use the label “EBA” for these activations). The observed effects in these regions were consistent across different arm positions and independent of the initial real hand starting position (no significant interaction effects of real hand position or starting position). Further notable activation differences were located in the left intraparietal sulcus (IPS; x = −34, y = −44, z = 52, t = 3.59) and in the bilateral dorsal premotor cortex (PMd; left: x = −26, y = −12, z = 54, t = 3.52; right: x = 20, y = −10, z = 50, t = 3.73), but these did not survive corrected threshold. In a control analysis including the time and dispersion derivatives of the different conditions, we found activation differences to congruent versus incongruent positions in the same brain areas (p < 0.05, corrected in the PMv and IPL; p = 0.063, corrected in the aSPL, pSPL, and EBA; no significant differences between the time or dispersion derivatives of the congruent versus incongruent conditions). Therefore, the main results are unlikely to have been effected by timing differences between conditions. The contrast of incongruent versus congruent arm positions revealed significant activation differences only in the anterior cingulate cortex (x = 8, y = 40, z = 34, t = 4.50, p < 0.05, corrected).
A, Congruent versus incongruent seen and felt arm positions produced significant activation differences in the left PPC (aSPL, pSPL, and IPL), the left PMv, and the left EBA (p < 0.05, corrected; Table 1). B, Interindividual differences in the subjective feeling of arm ownership were reflected by left PMv activity under congruent versus incongruent seen and felt arm positions (p < 0.05, corrected, the plot shows the correlation between mean-centered parameter estimates and the ownership index).
Significant activation differences obtained from contrasting congruent versus incongruent seen and felt arm positions
The evaluation of the questionnaire ratings showed that participants were well able to differentiate between congruent and incongruent arm positions (Q1: mean = 2.89, SD = 0.32, Wilcoxon's signed-rank test because these data did not pass the Jarque–Bera test for normality, z = 4.19, p = 4.8e-19) and did not engage in mental rotation (Q5: two-tailed t test, M = −0.42 ± 2.32, t(18) = 0.79, p = 0.44). The reported feeling of ownership was significantly higher for the arm seen in congruent positions (Q2: M = −0.32 ± 1.80) than for the arm in incongruent positions (Q3: M = −1.74 ± 1.05; Q2 vs Q3: t(18) = 5.30, p = 0.00004) and the object (Q4: M = −1.95 ± 2.32; Q2 vs Q4: t(18) = 4.75, p = 0.00016; no difference between incongruent arm and object, Q3 vs Q4: t(18) = 0.89, p = 0.38). Crucially, the arm ownership index correlated with left PMv activity obtained from the contrast congruent versus incongruent arm positions (x = −58, y = 6, z = 22, t = 4.37, p < 0.05, corrected within PMv ROI; Fig. 2B); we found a corresponding correlation when using contrast images from the contrast congruent arm versus incongruent arm and object presentation (x = −58, y = 8, z = 24, t = 4.10, p < 0.05, corrected within PMv ROI). Therefore, PMv activity reflected interindividual differences in experienced congruent arm ownership.
As an implicit localizer for the visually body-selective EBA (following the original definition by Downing et al., 2001), we contrasted vision of the arm (congruent and incongruent positions together) versus vision of the object, which revealed significant activations in the visual cortex spanning to bilateral LOC peaks close to previously reported EBA coordinates (left, x = −46, y = −82, z = −2, t = 9.34; right, x = 46, y = −78, z = 0, t = 12.12; cf. Downing et al., 2001; Astafiev et al., 2004). We next tested whether these body-selective voxels were also involved in distinguishing between congruent and incongruent arm positions; that is, we looked for voxels that were significantly activated by the contrast congruent versus incongruent position and also by the contrast arm versus object vision (“null conjunction” of the two contrasts, each thresholded at p < 0.001, uncorrected), which indeed revealed significant activity only in the left EBA (x = −48, y = −74, z = 6, t = 3.90, p < 0.05, corrected; Fig. 3B). Crucially, the left EBA responded parametrically to the “own-body-relatedness” of visual information: most strongly to vision of an arm in a congruent position, less for vision of an incongruently positioned arm, and least for vision of the control object. The right EBA, in contrast, showed no such preference for vision of congruently versus incongruently positioned arms despite strong body selectivity (see plot of the parameter estimates in Fig. 3C). Further, although no other regions were significantly activated by the above conjunction, we found that at a more liberal threshold (p < 0.05, uncorrected), the left SPL and PMv, along with the left IPS and bilateral PMd, also responded more strongly to congruent arm versus object vision.
A, Surface renders (left hemisphere) of the effects of arm versus object vision and of passive arm rotation. Note that passive arm rotation also activated the EBA. B, Top, Conjunction of the contrasts congruent versus incongruent arm positions (main analysis, see Fig. 2A) and arm versus object vision showing significant (p < 0.05, corrected) activation of the left EBA. Bottom, Contrast congruent versus incongruent arm positions masked with the effect of rotation. C, Plot of the parameter estimates (with SEs) of the main effect of visual stimulus showing parametric responses of the left EBA for congruent > incongruent arm presentations > object presentations; the right EBA showed strong body selectivity, but no preference for congruent versus incongruent arm positions (two-tailed t tests, ***p < 0.001, **p < 0.01, these effects were independent of hand position, all palm vs back facing differences p > 0.1). D, Detail of the left lateral occipital cortex (cf. C) showing partial activation overlaps in the EBA (significant activations obtained from the contrasts arm versus object vision, blue, congruent versus incongruent arm positions, red, and passive arm rotation, green).
Passive rotation of the participant's arm produced significant activations in the bilateral primary and secondary somatosensory and motor cortices, the insulae, the cerebellum, and also the EBA (Fig. 3A). Masking the congruent versus incongruent position contrast with the effect of rotation revealed activation overlaps in the left EBA, IPL, and SPL (p < 0.001, uncorrected, Fig. 3B) and at a more liberal threshold (p = 0.005) also in the left PMv; no significant effects of the direction of rotation (palm to back rotation or vice versa) were found in these areas. There was an overlap of activations from all three comparisons in the left EBA (x = −50, y = −72, z = 6, t = 3.59, p < 0.001, uncorrected; null conjunction of the contrasts congruent versus incongruent positions and arm vs object vision, masked with the effect of rotation, each contrast thresholded at p < 0.001). However, the EBA activation by arm rotation (x = −52, y = −70, z = 8, t = 6.52, p < 0.05, corrected) was located slightly more anterior and superior than that by arm versus object vision (see above). Figure 3D shows a color-coded detail of the respective activations and their overlaps.
The PPI connectivity analysis revealed that, under congruent versus incongruent visuo-proprioceptive arm position information, the left aSPL, pSPL, PMv, IPL, and EBA significantly increased their functional coupling with the primary visual cortex (V1, BA 17, p < 0.05, corrected; these activations were partly spanning to inferior, middle, and superior occipital gyri/pIPS). A null conjunction across the PPIs from the five different seed regions (each contrast thresholded at p < 0.001, uncorrected) revealed significant activation in the right V1 (BA 17, p < 0.05, corrected) and a statistical trend in the left V1 (t = 3.96, p = 0.072, corrected). These activations fell within occipital areas activated by arm vision per se (contrast arm vision vs fixation calculated in the main GLM, which also produced slightly stronger activations in right than left V1, global peak: x = 12, y = −84, z = −4). See Figure 4 and Table 2 for details.
Results of the PPI connectivity analysis. Seed regions in the left aSPL, pSPL, IPL, PMv, and EBA (location schematically shown in blue) significantly increased their coupling with the primary visual cortex (V1) under congruent versus incongruent seen and felt arm positions (p < 0.05, corrected; Table 2). A null conjunction across the PPIs from all seed regions revealed activation overlaps in the right V1 (p < 0.05, corrected, and a statistical trend in left V1, p = 0.072). All V1 activations fell within areas activated by visual presentation of the arm (render shown for comparison).
Significant increases in the functional coupling of brain regions responsive to visuo-proprioceptive congruence (as identified in the main analysis) under congruent versus incongruent arm positions
Finally, in a small behavioral control experiment, we used an endogenous visuospatial attention task to test for potential cognitive differences between stimulus conditions in five participants (Fig. 1C). There were no significant differences between congruent versus incongruent arm positions in neither the mean reaction time (congruent arm: 625 ± 45 ms, incongruent arm: 608 ± 23 ms, two-tailed t test, t = 1.20, p = 0.30) nor the accuracy (congruent arm: 70 ± 10%, incongruent arm: 73 ± 16%, t = 0.59, p = 0.59) during the task (during object vision, reaction time was slightly faster, 593 ± 45 ms, and accuracy lower, 62 ± 15%, than during arm vision, but these differences were not significant). Therefore, participants seemed to deploy attentional resources equivalently in the congruent and incongruent arm position conditions.
Discussion
The left PPC, PMv, and EBA showed significantly higher activation levels when the position of a visible right virtual arm was congruent (vs incongruent) with the position of one's real unseen arm, while increasing their coupling with V1; most of these regions also encoded unseen passive changes in real arm position. These results imply a role of these regions in multisensory limb position representation.
The effects in the PPC comprised the SPL and IPL. The anterior SPL cluster was located in BA 5, corresponding to previous monkey recording sites (Sakata et al., 1973; cf. Hyvärinen, 1982; Graziano et al., 2000). Sakata et al. (1973) showed that monkey BA 5 neurons encode proprioceptive limb position. However, some SPL regions also receive visual input (Rizzolatti et al., 1997). Graziano et al. (2000) demonstrated that BA 5 neurons of the monkey were responsive, not only to the position of the monkey's hidden (contralateral) arm, but also to the position of a visible fake arm. Crucially, the response of some of these neurons was further modulated by matching fake and real arm positions. Here, we observed the same modulation of human BA 5 activity by visuo-proprioceptive congruence.
The second SPL cluster was located anterior to the parieto-occipital sulcus (POS) and medially to the pIPS. The pIPS processes the visual space surrounding the hand (Makin et al., 2007); the more medial pSPL may correspond to the monkey parietal reach region (PRR) (Rizzolatti et al., 1997; Batista et al., 1999). In this putative human PRR homolog, activity increases during planning of pointing movements (Connolly et al., 2003) and movements from novel versus repeated starting locations (Pellijeff et al., 2006). Similarly, hand orientation before and during reach-to-grasp movements is encoded in a region around the human dorsal POS (Monaco et al., 2011; cf. Gallivan et al., 2016), closely resembling monkey V6A, a visual area activated by reaching and grasping (Galletti et al., 1997; Fattori et al., 2009). Therefore, the human medial SPL may in fact contain multiple reach-related regions (cf. Filimon et al., 2009).
The activation in the IPL was focused on the SMG, which, like the PMv, has been linked to visuo-tactile integration during the RHI under congruent hand positions (Gentile et al., 2013) and may correspond to monkey area 7b (cf. Hyvärinen, 1982, Rushworth et al., 2001). Joint area 5/7b (SPL/IPL) lesions in monkeys produce impaired reaching from different versus same starting positions without vision (Rushworth et al., 1997), suggesting that these areas encode changes in (unseen) arm position for guiding movements. Similar deficits in limb position representation can be observed in humans with SPL or IPL lesions (Wolpert et al., 1998; Buxbaum et al., 2007).
The PPC (particularly the SPL) seems to maintain an internal estimate of limb state, which is continually updated in the light of changing input, for example, during actions and their preparation (Wolpert et al., 1998; Zimmermann et al., 2012; Barany et al., 2014). Our results demonstrate that regions in the human PPC perform the same visuo-proprioceptive integration as monkey areas 5 and 7b, which likely form the basis of such a limb state estimate.
We further found that the left PMv was sensitive to visuo-proprioceptive congruence. Work in monkeys has shown that some PMv neurons have visual (Graziano et al., 1994) or visuo-tactile (Graziano et al., 1997) receptive fields that are “anchored” to specific body parts (e.g., a hand-centered receptive field shifts along with hand, but not with eye movements). Graziano (1999) showed that monkey PMv neurons encode the position of the monkey's contralateral hidden arm and also the position of a visible fake arm and concluded that, like PPC neurons, PMv neurons integrate visual and proprioceptive information to represent the upper limb for action. Our results suggest the same. Moreover, here PMv activity reflected interindividual differences in arm ownership (which was impossible to assess in monkeys; Graziano et al., 2000). Our results provide further evidence for two main results of Ehrsson et al.'s (2004) seminal RHI fMRI study, in which the left PMv reflected arm ownership and responded most strongly when the fake hand was touched synchronously with the real hand and lying in an anatomically plausible position. Crucially, our design allows to trace these effects back to visuo-proprioceptive integration without any additional effects of visuo-tactile stimulation. Here, reported ownership was significantly stronger for the congruent virtual arm, but somewhat lower than usual in RHI experiments (Ehrsson et al., 2004; Gentile et al., 2013). This fits with models of the RHI proposing that, under visuo-proprioceptive congruence, synchronous visuo-tactile stimulation enhances illusory limb ownership (Makin et al., 2008; Tsakiris, 2010).
In sum, our results show that the human PPC and PMv perform similar multisensory integration as their monkey homologs, supporting their proposed importance for human body representation (Ehrsson et al., 2004; Makin et al., 2008; Brozzoli et al., 2012; Gentile et al., 2013). The PMv might instantiate “higher-level” functions such as representing the (fake) hand for action on nearby objects (Graziano, 1999; Graziano et al., 2000; Ehrsson et al., 2004), whereas multisensory integration may occur at an “earlier stage” in the PPC (Gentile et al., 2013; Limanowski and Blankenburg, 2015a). However, a potential limitation of fMRI experiments, including our study, is their relatively low spatiotemporal resolution compared with electrophysiological recordings in monkeys. Future work using, for example, electroencephalography could further illuminate the temporal evolution of activity in the reported areas and thus specify their role in multisensory integration.
We found further effects of visuo-proprioceptive congruence in the left EBA, which responded more strongly to arm versus object vision and was moreover activated by passive arm rotation without vision. EBA activity has been associated with the RHI (Gentile et al., 2013; Wold et al., 2014; Limanowski and Blankenburg, 2015a,2015b), but the visuo-tactile stimulation necessary to induce the RHI complicates the interpretation of these results (cf. Peelen and Downing, 2007). Astafiev et al. (2004; cf. Gallivan et al., 2016) have reported left EBA activation during pointing with the right arm without vision and concluded that proprioceptive input generated by movements could update the body representation in EBA dynamically (but cf. Zimmermann et al., 2012, who propose that action goal representations in EBA are static and visual). This has been debated (Downing and Peelen, 2011) because the EBA was initially defined as a unimodal visual area and perceived motion or motor imagery might have activated nearby motion-sensitive areas. Here, participants had no task, did not engage in imagery (although, due to offline ratings, we cannot exclude this with certainty), and movements were passive. Our results thus suggest that the EBA is sensitive to visual and proprioceptive arm information and their congruence in the absence of action, visual or tactile motion, and motor imagery. That we found such effects only in the EBA contralateral to the right hand fits the results of previous studies (Astafiev et al., 2004; Ehrsson et al., 2004; Limanowski and Blankenburg, 2015a). Interestingly, slightly different portions of the EBA were activated by arm versus object vision (posterior) and passive arm rotation (anterior), which might support proposals of multiple functionally distinct body representations (Weiner and Grill-Spector, 2011) or functional gradients (Lingnau and Downing, 2015) in the EBA.
Finally, our connectivity analysis revealed that, under visuo-proprioceptive congruence, regions that were significantly responsive to such congruence (PPC, PMv, and EBA) increased their coupling with V1 regions encoding vision of the arm (slightly more strongly with right V1, perhaps reflecting the visually “richer” hand located in the left visual hemifield). An increase in coupling between two brain regions may indicate that a certain context facilitates their interaction, which in turn may imply functional integration (Friston et al., 1997). Positional information integration is usually biased in favor of vision, presumably due to its higher spatial acuity, as demonstrated by behavioral experiments introducing visuo-proprioceptive conflicts (van Beers et al., 1999; Pavani et al., 2000; Holmes and Spence, 2005). However, in the RHI, visual hand location estimates only dominate proprioception if the hands are sufficiently aligned (Ehrsson et al., 2004; Lloyd, 2007; Makin et al., 2008; Zeller et al., 2015). Therefore, in our experiment, visuo-proprioceptive congruence may have enhanced interactions between multisensory and visual areas, potentially reflecting an (increased) influence of visual information on multisensory limb representation.
It should be noted that our experiment was designed for the differential congruent versus incongruent contrast and featured no proper baseline, so the contrasts rotation or arm vision versus fixation should be interpreted with some caution. Further, although our behavioral control experiment and related work (Zimmer and Macaluso, 2007; Gentile et al., 2013) suggest that low-level multisensory integration processes might be largely unaffected by cognitive mechanisms, we cannot fully exclude potential cognitive differences between conditions. Future work should examine the generalizability of the reported effects across different contexts, for example, under explicit perceptual or active tasks.
In conclusion, our results suggest that the human PPC, PMv, and EBA encode the position of the contralateral upper limb based on combined visual and proprioceptive estimates. These findings strongly support the proposed fundamental contribution of these areas to own body representation.
Footnotes
We thank Simon Ludwig and Jan Herding for assistance during the experiment.
The authors declare no competing financial interests.
- Correspondence should be addressed to Jakub Limanowski, Freie Universität Berlin, FB Erziehungswissenschaft und Psychologie, Habelschwerdter Allee 45, 14195 Berlin. jakub.limanowski{at}fu-berlin.de
