Abstract
Perceptual judgments of relative depth from binocular disparity are systematically distorted in humans, despite in principle having access to reliable 3D information. Interestingly, these distortions vanish at a natural grasping distance, as if perceived stereo depth is contingent on a specific reference distance for depth-disparity scaling that corresponds to the length of our arm. Here we show that the brain's representation of the arm indeed powerfully modulates depth perception, and that this internal calibration can be quickly updated. We used a classic visuomotor adaptation task in which subjects execute reaching movements with the visual feedback of their reaching finger displaced farther in depth, as if they had a longer arm. After adaptation, 3D perception changed dramatically, and became accurate at the “new” natural grasping distance, the updated disparity scaling reference distance. We further tested whether the rapid adaptive changes were restricted to the visual modality or were characteristic of sensory systems in general. Remarkably, we found an improvement in tactile discrimination consistent with a magnified internal image of the arm. This suggests that the brain integrates sensory signals with information about arm length, and quickly adapts to an artificially updated body structure. These adaptive processes are most likely a relic of the mechanisms needed to optimally correct for changes in size and shape of the body during ontogenesis.
Introduction
Stereopsis is a crucial mechanism for visual depth perception. However, stereopsis suffers from an intrinsic ambiguity: the same binocular disparity can be generated by an infinite family of different objects (Wallach and Zuckerman, 1963; Cumming et al., 1991; Johnston, 1991; Rogers and Bradshaw, 1993). For example, based on disparity information alone, a marble ball that you hold between your fingers and a basketball at a distance of two meters appear virtually identical. In theory, the brain can resolve this ambiguity from an estimate of fixation distance provided by ocular convergence. However, the brain fails to properly calibrate relative depth, because perceived relative depth is overestimated for objects that are closer than a few tens of centimeters from the observer and underestimated for objects that are further than a few tens of centimeters (Norman et al., 1996).
Despite this systematic failure of depth constancy, objects located at a natural grasping distance (i.e., a few tens of centimeters from the observer) are perceived veridically (Richards, 2009). This raises the intriguing hypothesis that stereoscopic depth perception is calibrated or fine-tuned to be optimal at a distance (reference distance for depth-disparity scaling) where relative depth estimation is essential for handling, exploring, and manipulating objects (Fig. 1a, blue; zG). This calibration must somehow link proprioceptive information, about the natural position of our hand, to ocular convergence, which specifies the distance of the object held. In this view, if our body changes, as during development, the brain must adjust to adapt the perceptual scaling of binocular disparities to the new arm length.
Scaling of stereo depth. a, When an observer looks at a series of identical objects located at different viewing distances, perceived relative depth from binocular disparities is only veridical (green triangle) at the natural grasping distance (blue arm, zG), the reference distance for depth-disparity scaling. Perceived relative depth of objects at other distances is systematically distorted: closer objects are overestimated (yellow triangles) and farther objects are underestimated (blue triangles). We predict that an artificially induced change in reach extent (red arm) could shift the mapping between binocular disparities and relative depth perception along the depth dimension so that perceived depth is veridical at the new grasping distance (zG + ΔzG). b, Predicted relation between fixation distance (zf) and scaling distance (zS) after Normal Reach (blue line) and Extended Reach (red line) visuomotor adaptation. The intersection between these lines and the unity line indicates the fixation distance at which perceived relative depth is veridical. c, Predicted perceived relative depth after Extended Reach adaptation (red lines) based on the perceived relative depth obtained after the Normal Reach adaptation (blue lines). d, Predicted depth shifts depend both on fixation distance and simulated depth.
Here we provide a crucial test of the hypothesis that reaching and grasping play a role in calibrating relative depth perception from stereopsis. Without subjects being aware, we altered the relationship between fixation distance and proprioception of the arm (using motion tracking of the arm in a virtual reality environment), so as to instantaneously “shift” the extent of reach (Fig. 2b). After training with this altered sensorimotor contingency, subjects were tested with a perceptual task of relative depth perception that did not involve reaching. Remarkably, after only a brief session of the visuomotor adaptation, the perceptual scaling of binocular disparities changed dramatically, as if to precisely adjust for the new perceived “length” of the arm (Fig. 1a, red; zG + ΔzG).
These results reveal dynamic processes that quickly tailor the decoding of binocular disparity information to the updated reach extent. During visuomotor adaptation, the same muscle configurations produced longer apparent reaches; thus, the internal image of the arm may have been updated. Consistent with this hypothesis, we also found changes in tactile sensitivity on the arm following the visuomotor adaptation, as if the skin of the adapted observer were stretched over an apparently elongated arm. These powerful and rapid effects on visual and tactile perceptual processes suggest a high degree of plasticity of adult perceptual systems, molded by instantaneous sensorimotor contingencies.
Materials and Methods
Subjects
Forty-one naive subjects (20 males and 21 females), all with normal or corrected-to-normal vision, participated in the experiments [visual depth estimation (VDE) task, n = 16; manual depth estimation (MDE) task, n = 15; tactile discrimination task, n = 10]. Experiments were undertaken with the understanding and written consent of each subject, with the approval of the Comitato Etico per la Sperimentazione con l'Essere Vivente of the University of Trento, and in compliance with national legislation and the Code of Ethical Principles for Medical Research Involving Human Subjects of the World Medical Association (Declaration of Helsinki).
Apparatus and stimuli
Subjects were seated in a dark room in front of a high-quality, front-silvered 400 × 300 mm mirror. The mirror was slanted at 45° relative to the subjects' sagittal body midline and reflected the image displayed on a ViewSonic 9613, 19 in. CRT monitor placed directly to the left of the mirror. For consistent vergence and accommodative information, the position of the monitor, attached to a linear positioning stage (Velmex), was adjusted on a trial-by-trial basis to equal the distance from the subjects' eyes to the virtual object. To present visual stimuli in 3D, we used a frame interlacing technique in conjunction with liquid crystal FE-1 goggles (Cambridge Research Systems) synchronized to the monitor's frame rate. Head, index, and thumb movements were acquired on-line with submillimeter resolution by using an Optotrak 3020 Certus motion capture system with two position sensors (Northern Digital). Head movements updated the subjects' viewpoint to present the correct geometrical projection of the stimulus in real time. The position of the tip of each finger was calculated during the system calibration phase with respect to three infrared-emitting diodes attached on each distal phalanx. A custom C++ program was used for stimulus presentation and response recording.
High-contrast random-dot visual stimuli were rendered in stereo simulating one (visuomotor interaction sessions) or three (VDE and MDE tasks) vertically oriented rods with a dot density of 30%. The radius of the rods was 5 mm and the height was 40 mm. In the visuomotor interaction sessions, the rod was positioned at a random distance in the range 420–570 mm along the line of sight. In the VDE and MDE tasks, the three-rod configuration, in which one rod was positioned midway and in front of two flanking rods, was presented with depth-extents (Δz) of 30, 40, 50 mm at viewing distances of 420, 495, and 570 mm along the line of sight. Visual stimuli were presented for 2.5 s.
For tactile discrimination assessment, subjects laid the dorsal side of the right arm on a horizontal surface and an aesthesiometer was used to apply one or two spatially separated tactile contacts on the ventral side of the forearm.
Procedure
Visuomotor interaction sessions.
Subject's right hand started the movements from a fixed, out-of-view position shifted relative to the body by ∼250 mm from the sagittal plane and 150 mm from the coronal plane. The position of the tip of their index finger represented by a red dot was provided constantly from the moment the finger entered in the visual field. The task was to reach-to-point a virtual rod positioned at a random distance along the line of sight. In the Normal Reach visuomotor sessions, the visual feedback of the index finger corresponded to the actual finger position (Fig. 2a). In the Extended Reach visuomotor sessions, reach was apparently extended by displacing the feedback dot 150 mm in depth (Fig. 2b), thus altering the sensorimotor contingency between visual and proprioceptive information. In total, subjects performed 60 reaching movements in each interaction session. On postexperimental questioning, all subjects reported being unaware of the displacement of the visual feedback.
VDE task.
Subjects used a computer mouse to adjust the width (Δx) of a three-rod configuration to match its perceived relative depth-extent (Δz), as shown in Figure 2c. Each combination of depth-extent and viewing distance was presented five times. This procedure was performed before (pre-test) and after (post-test) the visuomotor interaction session for a total of 90 estimates/subject. Subjects were randomly assigned either to the Extended Reach or to the Normal Reach visuomotor session.
MDE task.
Subjects rested their hand on a platform that was shifted relative to the body by ∼200 mm from the sagittal plane and 300 mm from the coronal plane. When the visual stimulus appeared, subjects spanned their index and thumb fingers to estimate the perceived relative depth-extent (Δz) of the three-rod configuration, as shown in Figure 2d. The fingers were brought again together when the stimulus disappeared. Each combination of depth-extent and viewing distance was presented five times. This procedure was performed before (pre-test) and after (post-test) the visuomotor interaction session for a total of 90 estimates/subject. Subjects were randomly assigned either to the Extended Reach or to the Normal Reach visuomotor session.
Two-alternative forced choice tactile discrimination task.
Two consecutive tactile stimuli were applied on the ventral side of the forearm in an orthogonal or parallel orientation with respect to the proximo-distal axis of the arm. In a random order, one of the stimuli consisted of one tactile contact and the other of two spatially separated tactile contacts. Subjects had to respond which (first/second) of the stimuli consisted of two spatially separated tactile contacts. The spatial separation of the two tactile contacts stimuli was changed according to a staircase procedure following a 1-up-2-down rule. Two randomly interleaved staircases were used: one for the orthogonal and one for the parallel orientation. Each staircase was terminated after 10 reversals to settle the threshold on a steady value. A cumulative Gaussian function was fitted to estimate the discrimination threshold. This procedure was performed for all subjects after the Extended Reach and the Normal reach visuomotor session in a counterbalanced fashion.
Data analysis
Statistical analyses were performed using linear mixed-effects models with subjects as random effects (Pinheiro and Bates, 2000). Two-tailed p values were obtained using Markov chain Monte Carlo simulations (10,000 samples). The dependency of the observed depth shift was analyzed using standard multiple linear regression.
Results
We hypothesized that the brain updates the internal representation of the arm's morphological structure as a result of visuomotor adaptation, and that this modification causes a change in stereoscopic depth perception and tactile discrimination. Our experimental approach was to present subjects with a brief period of visuomotor adaptation by having them reach to a target with an apparently elongated arm in a virtual reality environment. We then tested their stereoscopic depth perception and tactile discrimination using perceptual tasks that did not require reaching.
Predicted changes in stereoscopic depth perception: quantitative model
Before describing the results, a simple model is needed to establish some quantitative predictions in both sensory domains. First, it is known that binocular disparities are normally incorrectly scaled in determining relative depth (Wallach and Zuckerman, 1963; Johnston, 1991; Norman et al., 1996). We assume this is due to an incorrect estimate of fixation distance (absolute depth) that varies linearly with the true fixation distance. This linear function only takes on a veridical value at what we define as the disparity scaling reference distance that corresponds to the natural grasping distance (zG). Second, we hypothesize that a change in apparent arm length causes a corresponding change of the natural grasping distance and hence of the disparity scaling reference distance, resulting in a predictable shift of the scaling function. Because the new scaling function specifies the predicted scaling of binocular disparities after visuomotor adaptation, it ultimately predicts perceived stereo-depth. Third, we hypothesize that the elongation of the arm produces a uniform expansion of the spatial representation of the forearm surface, so that the represented locations of two points on the skin surface are pushed farther apart after adaptation, by an amount proportional to the arm elongation. We thus predict an increased sensitivity to a two-point tactile stimulation by the same proportion.
After the visuomotor adaptation with the apparently elongated arm, we examined stereoscopic depth perception using the following perceptual task. Subjects judged the relative depth Δz of a triangle specified by a three-rod configuration, in which one central rod is in front of two flanking rods (Fig. 2c,d). The relative disparity (δ) between the central rod and one of the flanking rods is related to their depth separation Δz through the following approximate equation (Howard and Rogers, 1995):
where zf is the fixation distance and IOD the interocular distance.
Visuomotor adaptation sessions and depth estimation tasks. a, b, Reach-to-point movements were made toward a visual target (red object) and a motion tracker recorded the finger position, which was presented to the subject as a bright red dot (visual feedback). a, During Normal Reach visuomotor adaptation the visual feedback was spatially coincident with the actual position of the finger. b, To artificially extend the reach, the visual feedback was presented 150 mm further away than the actual finger position (Extended Reach). Identical visual stimulations were thus coupled with different arm's postures (gray arms). The red arm shows the hypothetical rescaling of the arm due to inconsistent visual feedback. c, VDE task. d, MDE task. In both tasks observers judged perceived relative depth of a three-rod configuration with depths of 30, 40, 50 mm at viewing distances of 420, 495, and 570 mm.
Accurate perception of relative depth (Δz) requires a correct estimate of the fixation distance zf for the scaling of the relative disparity δ. In absence of other visual information, the vergence angle, encoded by the efferent signals related to the eyes' rotation, and ocular accommodation specify zf. However, if the brain fails to accurately estimate zf, then it scales binocular disparities by the wrong scaling distance zS. This leads to a wrong relative depth estimate:
If zf is underestimated (zS < zf) or overestimated (zS > zf), it follows that the depth of an object will be underestimated (Δẑ < Δz) or overestimated as well (Δẑ > Δz).
As described above, scaling of disparity is only veridical at a specific viewing distance zG, the disparity scaling reference distance, which usually coincides with a position in space where objects can be grasped comfortably (Fig. 1a). The relative depth of objects closer than zG is overestimated whereas that of objects further than zG is underestimated (Wallach and Zuckerman, 1963; Johnston, 1991; Norman et al., 1996; Richards, 2009). Therefore, if the natural scaling distance zSN is a linear function of the fixation distance zf (which is a reasonable approximation for a small range of fixation distances), then it will intercept veridicality at the point zG = zf (Fig. 1b, blue line; zG = 450 mm) and have a slope less than one (k < 1):
Suppose that through visuomotor adaptation reach extent is increased by ΔzG, so that a natural grasp can take place at zG + ΔzG (Fig. 1a, red). If the visual system fully adapts the scaling of binocular disparities to accommodate the new natural grasping distance, then the new scaling distance function zSA is veridical at zG + ΔzG (Fig. 1b, red line; 600 mm), the updated disparity scaling reference distance:
assuming that the slope k has not changed. A modification of the link between eye and hand control is supported by the literature that presupposes a shared eye-hand coordinate system (Nemire and Bridgeman, 1987; Bekkering et al., 1995; de Graaf et al., 1995; Nanayakkara and Shadmehr, 2003; Crawford et al., 2004; Cotti et al., 2007; Pélisson et al., 2010).
Equation 4 shows that is possible to predict scaling of disparities after adaptation, given that the scaling function zSN before adaptation is known. It also allows a prediction of the amount of change in perceived relative depth from before (ΔẑSN) to after (ΔẑSA) visuomotor adaptation. ΔẑSN and ΔẑSA are determined by substituting zSN and zSA for zS in Eq. 2. Therefore, the shift ΔẑSA−ΔẑSN of perceived depth can be predicted by:
We therefore expect a depth shift that increases with simulated depth Δz and decreases with fixation distance zf, and depends on the interaction of the two (Fig. 1c,d).
Visuomotor adaptation: altering the contingency between vision and proprioception
To study the effects of a changed reach extent we had participants engage in a brief visuomotor adaptation session. During all visuomotor sessions subjects reached with their right index finger for a virtual rod, positioned at different viewing distances. Importantly, in the Normal Reach and Extended Reach sessions (Fig. 2) the visual stimulations were virtually identical. The only difference was the position of the participant's fingertip (and thus the arm posture) in relation to the feedback dot. Thus, any perceptual differences arising from the two types of visuomotor sessions could not be due to differences in simple visual adaptation. In addition, in a pre-experiment we established that the Extended Reach visuomotor adaptation procedure induced the characteristic motor aftereffect (von Helmholtz, 1867; Held and Freedman, 1963; Köhler, 1964; Kornheiser, 1976). That is, subjects tended to make shorter reaches after the Extended Reach sessions than after the Normal Reach sessions.
Perceived relative depth from binocular disparities depends on the updated extent of reach
Before (pre-test) and after (post-test) the visuomotor adaptation sessions, we examined how perceived relative depth varied with fixation distance, using two different methods (Fig. 2c,d). In both cases, the subjects viewed three virtual rods configured so that two of the rods were offset in depth from the third. In the VDE task (Fig. 2c), subjects used a computer mouse to adjust the width of the triangular rod configuration (Δx) to match its perceived relative depth (Δz). In the MDE task (Fig. 2d), subjects used the sensed separation between their index finger and thumb of their unseen hand to match the perceived relative depth of the rod configuration. These two very different methods (VDE and MDE) were used to test the generality of the findings.
The predictions of relative depth estimates in the Extended Reach condition for both the VDE and MDE tasks were made on the basis of the VDE-task data, because we assumed that in the VDE task the same scaling distance zS determines both the perception of width (Δx̂) and the perception of depth (Δẑ), as reported previously (van Damme and Brenner, 1997). Whereas perceived stereoscopic depth Δẑ depends on the scaling of binocular disparities (Eq. 2), perceived width Δx̂ depends on the scaling of the visual angle α subtended by the two back rods. If this angle is small, then 
Because the VDE task consisted in adjusting the physical width Δx so as to match the perceived width (Δx̂) to the perceived depth (Δẑ), the scaling distance zS is the solution of the equation Δx̂ = Δẑ, obtained from Equations 2 and 6:
Equation 7 was used to transform the settings Δx of the VDE task into values of scaled distance. In the Normal Reach condition a linear fit was applied to these values, which determined an estimate of the parameters k and zG of Equation 3 (Fig. 3a, blue line). The slope of this function was <1, as expected (Johnston, 1991), and the correct scaling distance zG was ∼420 mm. We predicted that visuomotor adaptation with an extended reach would modify the scaling function by an amount related to the shift in reach extent (ΔzG = 150 mm), according to equation 4 (Fig. 3a, red dashed line; 68% confidence interval). The scaling distances calculated with Equation 7 in the Extended Reach condition (Fig. 3a, red data points) were remarkably close to that predicted by the model: after visuomotor adaptation veridical scaling was observed at zG + ΔzG (570 mm).
Visuomotor adaptation effects on stereo depth. a, Relation between scaling distance and fixation distance after Normal Reach (blue line) and Extended Reach (red line) visuomotor adaptation. The red dashed line represents the predictions with 68% confidence interval. The gray line represents the unity line. b, Perceived relative depth ±SEM as a function of fixation distance in the VDE (n = 16) and MDE (n = 15) tasks for the different simulated depths (Δz). Shaded bands represent predicted 68% confidence intervals. Values above and below the horizontal gray lines indicate overestimation and underestimation, respectively. Perceived relative depth was larger after the Extended Reach (red lines) than after Normal Reach (blue lines) visuomotor adaptation. c, Predicted versus observed shifts in relative depth in the VDE (top) and MDE (bottom) tasks. Colors denote different relative depths (green: Δz = 30 mm, yellow: Δz = 40 mm, magenta: Δz = 50 mm), whereas frame thicknesses denote different fixation distances (none: zf = 420 mm, medium: zf = 495 mm, thick: zf = 570 mm).
Perceived relative depth in the VDE task had to be computed from the adjustments Δx in the VDE task. This was achieved by substituting Equation 7 in Equation 2, leading to Δẑ = Δx2/Δz. In the MDE task the separation between index and thumb was considered to be a direct measure of Δẑ. For both tasks, we predicted the perceived relative depth in the Extended Reach condition by adding the shift of Equation 5 to the perceived relative depth in the Normal Reach condition (Fig. 3b, red shaded bands; 68% confidence interval). The predicted shift was exclusively based on the linear fit parameters (k, zG) of the VDE task data in the Normal Reach condition. ΔzG was fixed at 150 mm, the depth offset of the feedback dot during visuomotor adaptation. Thus, perceived relative depth in the Extended Reach condition was predicted from the data of the Normal Reach condition without additional free parameters. For both tasks the results agreed well with the predictions, for all viewing conditions (Fig. 3b, solid red lines).
In both the VDE and MDE tasks (Fig. 3b), perceived relative depth decreased with fixation distance in all conditions (VDE: t = −12.72, p = 0.0001; MDE: t = −8.094, p = 0.0001) and it increased as a function of the simulated stereoscopic depth of the objects (VDE: t = 35.63, p = 0.0001; MDE: t = 28.89, p = 0.0001). Critically, we found an interaction between pre/post-testing and Normal Reach/Extended Reach visual feedback conditions (VDE: t = 4.96, p = 0.0001; MDE: t = 8.778, p = 0.0001). Perceived depth extent did not differ between pre-tests (VDE: t = 1.025, p = 0.21; MDE: t = 0.694, p = 0.3112). However, perceived depth increased in the post-test Extended Reach conditions with respect to the post-test Normal Reach conditions, for both the VDE and MDE tasks (VDE: t = 1.985, p = 0.0126; MDE: t = 2.11, p = 0.0014). Only 60 reaching movements (10 min) with a displaced visual feedback produced dramatic changes in perceived relative depth (Fig. 3b, red lines vs blue lines).
The predicted shift in Equation 5 is a nonlinear function of fixation distance (zf) and simulated depth (Δz), which defines a prediction surface entirely based on the data from the Normal Reach condition (Fig. 1d, meshed surface). The observed shifts closely match this surface, confirming the prediction that observed shifts depend on both fixation distance (zf: F = 19.622, p = 0.0005) and simulated depth (Δz: F = 136.985, p = 0.0001), and on the interaction of the two (F = 6.297, p = 0.025). A direct comparison between the predicted depth shifts (Eq. 5) and the observed depth shifts is shown in Figure 3c. Note that if no shift had occurred all the data points should cluster on the x-axis at y = 0.
In summary, we found that perceived relative depth from binocular disparities rapidly changes to conform to the newly learned reach extent. It was previously shown that perceived depth from binocular disparities changes following prolonged viewing of a stereoscopic stimulus, the phenomenon of depth adaptation (Blakemore and Julesz, 1971), or as a consequence of wearing glasses that causes a change in accommodation and convergence of the eyes (Wallach et al., 1972). However, in our study the visual stimulation was identical between the Normal and Extended Reach visuomotor sessions; thus we can rule out any explanation based on depth adaptation or based on a calibration driven by other visual cues that could represent veridical distances during the adaptation period (Wallach et al., 1972).
Sensitivity to tactile stimulation depends on the updated extent of reach
Given the strong effects on binocular disparity driven by updated reach extent, we wondered whether related alterations could occur in the tactile domain. The existence of phenomena such as the vivid experience of a phantom limb in amputees (Ramachandran and Hirstein, 1998) and its progressive fading (telescoping) suggests that how we experience our body shell is not uniquely constrained by the physical presence of the body parts. Perceived morphology and position of body parts can also be modified in healthy humans (Lackner, 1988; Ehrsson et al., 2005; Tsakiris and Haggard, 2005). For example, dummy rubber hands (Botvinick and Cohen, 1998) as well as scaled dummy bodies (van der Hoort et al., 2011) or virtual arms (Sanchez-Vives et al., 2010; Kilteni et al., 2012) can be easily sensed as your own. Together, these findings indicate that the subjective perception of our body is highly malleable even in adults, and is constantly molded by new intra-and intermodal contingencies.
Dynamic restructuring as a result of body growth exists in the tactile domain, because the ability to discriminate two separated points of contact on the skin surface is similar in children and adults (Thibault et al., 1994; Stevens and Patterson, 1995), despite dramatic changes in body size. The sensitivity of two-point discrimination could be attributed to peripheral mechanisms that would be expected to change only slowly, such as the density of touch receptors in the skin. However, tactile perception can undergo more dynamic plasticity, even during adulthood (Tegenthoff et al., 2005; Schaefer et al., 2007). For example, tactile resolution changes after direct vision of a magnified arm (Kennett et al., 2001; Taylor-Clarke et al., 2002, 2004), by vibrating the biceps tendon to induce the percept of an elongated body part (de Vignemont et al., 2005), or by physically extending the arm with a mechanical grabber (Cardinali et al., 2009). We thus predicted that the virtual elongation of reach in our experiment would also induce dynamic plasticity in tactile perception.
To examine this hypothesis, we asked participants to discriminate between two somatosensory stimuli consisting of either one or two spatially separated tactile contacts (two-alternative forced choice tactile discrimination task). The stimuli were applied on the ventral side of the forearm in an orthogonal or parallel orientation with respect to the proximo-distal axis of the arm. Using a staircase procedure, we measured the discrimination threshold, the smallest gap between two points on the skin that is experienced as two spatially distinct tactile sensations. This method identifies the granularity of tactile acuity. If the internal image of the entire arm's length is scaled after interaction with an apparently extended reach, the granularity of tactile acuity on the skin surface might become finer, such that the spatial acuity on the apparently elongated arm would match that of the original.
During visuomotor adaptation participants viewed the visual target at a certain distance (zf). However, to reach for that target they had to move their hand to a distance less than zf (zp), due to the offset between the actual position of their index finger and the visual feedback of their finger. This sort of intersensory discrepancy usually results in an altered proprioceptive information about the hand location, as if the hand were closer to the position of the virtual hand (Botvinick and Cohen, 1998; Sanchez-Vives et al., 2010). We hypothesized that not only the hand is perceived to be at a farther distance, but that the internal image of the entire arm's length is scaled. The possibility that a more general body scaling occurs cannot be a priori excluded. However, to avoid unsupported assumptions we restricted our predictions to the scaling of the arm only.
The amount of scaling can be predicted by a simple geometric model. When reaching to a target, the distance to the target is specified by the angle between the upper and the lower arm (Fig. 4a). The segment from the tip of the index finger to the elbow (lp) and the segment from the elbow to the shoulder (up) represent two sides of a scalene triangle, whereas the third side (dp) connects the shoulder to the tip of the finger that is positioned at zp. The segment that connects the shoulder to the visual feedback of the finger (df) is part of the scaled triangle. Because the proportion between the scaled segments lf and uf and the angle between them is the same as the proportion and the angle between lp and up, the two triangles are geometrically similar, i.e., all their angles are congruent and their corresponding sides are proportional. We can take advantage of this similarity to calculate the ratio between lf and lp:
The two sides df and dp depend on the visual target position (zf) and the distance between the shoulder and the center of the head (s):
Because on each trial the visual target was placed at a random position uniformly distributed along the line of sight, and the distance from the shoulder to the center of the head varied between subjects, we performed a Monte Carlo estimation of the ratio between lf and lp, which corresponded to 1.28 (SD = 0.003). It follows that on average the segment between the tip of the index finger and the elbow should be scaled by 28% after visuomotor adaptation in which the visual feedback was displaced by 150 mm (Extended Reach condition).
Effect of visuomotor adaptation on tactile perception. a, Sketch of the relation between arm's posture (gray solid lines) and its scaled internal image (gray dashed lines). To reach for the target (red circle) during Extended Reach visuomotor adaptation the hand is moved at distance zp, but it is seen at distance zf. The triangle formed by the upper and lower arm segments (up and lp) and the segment between the shoulder and the finger tip (dp) is a similar triangle to the scaled triangle formed by uf, lf, and df. The distance between the center of the head and the shoulder is denoted by s. b, Mean discrimination thresholds ±SEM after the Extended Reach (red bars) and Normal Reach (blue bars) visuomotor adaptation for parallel and orthogonal orientations with respect to arm's proximo-distal axis (n = 10). c, Per subject threshold change between Extended and Normal Reach sessions. A negative change represents an improvement in tactile acuity. d, Predicted versus observed thresholds after Extended Reach visuomotor adaptation. Squares and diamonds represent thresholds for parallel and orthogonal orientations with respect to the arm's proximo-distal axis, respectively. The red unity line indicates changes in threshold that are predicted by the scaling of the internal image of the arm's length.
We indeed found that the tactile discrimination threshold on the forearm decreased (i.e., tactile acuity increased) after the Extended Reach session (t = −3.956, p = 0.0014), as shown in Figure 4b,c. Two spatially separated contact points that were usually perceived as one after the Normal Reach session were more readily perceived as two after the Extended Reach session.
The discrimination between two somatosensory stimuli consisting of either one or two spatially separated tactile contacts is a signal-to-noise problem. Hence, whether or not the scaling is actually reflected in improved tactile acuity is indicative of the stage of processing at which the scaling occurs. The positions of two nearby spatial stimuli can be represented by overlapping response profiles across a population of detectors. If the scaling occurs at an early stage, the amount of overlap of the two populations would remain unchanged, that is, the width of the profiles would be scaled by the same amount as the distance between their peaks. If, however, the noise limiting discriminability were generated centrally, after the scaling operation, the distance between profiles peaks would change without affecting the width of the profiles. We can distinguish between an early (prescaling) and a late (postscaling) effect of noise on discriminability in the processing hierarchy by computing the best-fitting line through the origin for the data presented in Figure 4d. According to the prescaling hypothesis, the slope should be equal to 1/(1 − 0.28) ≈ 1.39 (blue line). Instead, according to the postscaling hypothesis, the slope should be equal to one (red line). Indeed, the computed slope was equal to 1.007 (95% CI, 0.893–1.121). The changes in tactile discrimination threshold were coherent with the predictions based on a scaling of the internal image of the arm's length that occurs before the processing reaches the site of noise limiting the discrimination. We can thus conclude that the improvement in tactile sensitivity is the result of an updated spatial representation of the forearm skin caused by visuomotor adaptation.
Discussion
We found that experimentally induced misalignment between vision of a single dot and proprioception causes a sizable change in how binocular disparities are interpreted. When subjects were exposed to a sensorimotor misalignment that was consistent with a longer reach extent, their subsequent perceived relative depth judgments of 3D visual stimuli were veridical at a larger distance than without any sensorimotor misalignment. These results provide the first direct evidence that the perceptual scaling of binocular disparities can be flexibly adjusted to a new actor-environment relation. Evidently, the primary aim of this adaptive process is to allow for optimal processing of binocular disparities at a reference distance where correct depth estimation is absolutely necessary, i.e., where objects are grasped and manipulated (Richards, 2009). Alternatively, one could consider that depth estimates are most accurate at reaching distances precisely because success (or failure) to reach accurately to targets could provide critical feedback needed to calibrate the depth estimation from binocular disparity. The deviations from veridicality found at distances closer and farther than grasping distance (Wallach and Zuckerman, 1963; Cumming et al., 1991; Johnston, 1991; Norman et al., 1996; Richards, 2009) suggest that binocular disparities are only partially compensated as a function of fixation distance.
These findings differ dramatically from those of previous studies. Visuomotor adaptation studies have shown that motor functions can adjust to new visuomotor mappings (von Helmholtz, 1867; Stratton, 1897; Held and Freedman, 1963; Köhler, 1964; Kornheiser, 1976; Bedford, 1989; Ghahramani et al., 1996; Redding and Wallace, 1996), and also that proprioceptive perception, such as the perceived position of the hand, can be altered (Harris, 1963; Hay and Pick, 1966; Welch et al., 1979; Cressman and Henriques, 2009, 2010). Furthermore, several studies reported shifts in the visual straight ahead after adaptation to left-right translations of visuomotor feedback of the hand (Wallach et al., 1963; Hay and Pick, 1966; Uhlarik and Canon, 1971; Welch et al., 1974; Redding and Wallace, 1998; Hatada et al., 2006). Instead, we show that visuomotor adaptation produces rapid and quantitatively precise changes in a crucial visual function—the perception of the 3D structure of distal objects—measured independently of the motor contingency. Moreover, the design of our visuomotor adaptation paradigm excluded low-level visual adaptation effects, or depth-contrast effects from comparing the perceived depth of the object with the body or surrounding environment.
Concurrently, we show that the sensorimotor misalignment elicits marked changes in the tactile domain by decreasing two-point discrimination thresholds on the adapted forearm. The plasticity was not the result of an unusual or aberrant sensory stimulation: the sensory inputs in our experiment—visual and proprioceptive—did not differ substantively between visuomotor adaptation conditions. Nonetheless, the spatial relationship between the visual and proprioceptive inputs differed between conditions, and thus must have driven also the rapid somatosensory plasticity.
Together, these results raise the intriguing possibility that the modulations of visual depth perception and touch stem from a common origin. The visuomotor adaptation induced by the misalignment between vision and proprioception might have altered the internal image of the arm (de Vignemont et al., 2005; Cardinali et al., 2009) to preserve coherence with the updated reach extent that must have driven the recalibration of both visual and tactile sensory systems. The ease with which these recalibrations occur reveals highly adaptable processes, but belies the complexity of the underlying neural mechanisms, most of which remain to be discovered.
It is interesting to consider our findings in light of ontogenetic mechanisms. These results provide evidence that, even in adulthood, sensory systems are not fixed structures with immutable functions. It is of course widely appreciated that sensory systems can be significantly modified by the input they receive during development (Warren and Whang, 1987; Clifton et al., 1988; Kording et al., 2007). For example, to achieve optimal motor behavior for reaching and grasping, visual and somatic senses must be continually adjusted as different body parts grow at different pace. One might expect such adjustments to proceed very slowly and to be largely restricted to immature organisms. For instance, processing of binocular disparities is believed to stabilize in humans within a few months after birth (Braddick et al., 1980); but we have instead found strong sensory plasticity that can be evoked within minutes in adults.
Our findings suggest a direct link between the morphological structure of the body and the tuning of the visual and tactile sensory systems. We modified both perceived visual depth and the grain of tactile perception through an altered visual feedback of the arm, the primary body segment with which humans interact with the environment. This suggests that knowledge about body morphology is essential for sensing the size and shape of objects as well as for optimally guiding body movements. Moreover, our results raise the possibility that similar rapid recalibration mechanisms may also operate during human ontogenesis, perhaps leading to longer-term, consolidated structural plasticity.
Footnotes
We thank C. Nicolini for help in programming, and C. Campagnoli and A. Monni for help in data collection.
- Correspondence should be addressed to Dr Robert Volcic, Center for Neuroscience and Cognitive Systems@UniTn, Istituto Italiano di Tecnologia Corso Bettini 31, 38068 Rovereto (TN), Italy. robert.volcic{at}iit.it
