Spatial target information for movement planning appears to be coded in a gaze-centered reference frame. In touch, however, location is initially coded with reference to the skin. Therefore, the tactile spatial location must be derived by integrating skin location and posture. It has been suggested that this recoding is impaired when the limb is placed in the opposite hemispace, for example, by limb crossing. Here, human participants reached toward visual and tactile targets located at uncrossed and crossed feet in a sensorimotor decision task. We characterized stimulus recoding by analyzing the timing and spatial profile of hand reaches. For tactile targets at crossed feet, skin-based information implicates the incorrect side, and only recoded information points to the correct location. Participants initiated straight reaches and redirected the hand toward a target presented in midflight. Trajectories to visual targets were unaffected by foot crossing. In contrast, trajectories to tactile targets were redirected later with crossed than uncrossed feet. Reaches to crossed feet usually continued straight until they were directed toward the correct tactile target and were not biased toward the skin-based target location. Occasional, far deflections toward the incorrect target were most likely when this target was implicated by trial history. These results are inconsistent with the suggestion that spatial transformations in touch are impaired by limb crossing, but are consistent with tactile location being recoded rapidly and efficiently, followed by integration of skin-based and external information to specify the reach target. This process may be implemented in a bounded integrator framework.
SIGNIFICANCE STATEMENT How do you touch yourself, for instance, to scratch an itch? The place you need to reach is defined by a sensation on the skin, but our bodies are flexible, so this skin location could be anywhere in 3D space. The movement toward the tactile sensation must therefore be specified by merging skin location and body posture. By investigating human hand reach trajectories toward tactile stimuli on the feet, we provide experimental evidence that this transformation process is quick and efficient, and that its output is integrated with the original skin location in a fashion consistent with bounded integrator decision-making frameworks.
It is commonly assumed that the brain uses an eye or gaze-centered reference frame to code the spatial location of movement targets (Henriques et al., 1998; Batista et al., 1999; Medendorp et al., 2003). Whereas a gaze-centered code is native to the visual system, and therefore visually perceived targets (Medendorp et al., 2011), it is less obvious which choice of reference frame the brain might use when a target is defined tactually (Heed et al., 2015). Tactile location is first represented relative to the skin surface in the primary somatosensory cortex's homunculus (Penfield and Boldrey, 1937). However, both saccade and hand reaches to tactile events involve gaze-centered coding (Buchholz et al., 2011, Mueller and Fiehler, 2014a,b; Buchholz et al., 2013), suggesting that action targets in vision and touch are ultimately coded in a common reference frame. Derivation of the recoded location for touch requires the integration of skin location with postural information, a process referred to as tactile remapping (Driver and Spence, 1998).
The original, skin-based touch coordinate is retained after transformation (Heed and Röder, 2010; Buchholz et al., 2013); therefore, conflict can arise when spatial information in the original and the transformed reference frames are incongruent. For example, when the limbs are crossed over the body midline, the right limb lies in left space. Such situations can result in marked performance deficits, evident in increased errors and prolonged processing time (Yamamoto and Kitazawa, 2001; Shore et al., 2002; Heed and Azañón, 2014). These “crossing effects” have frequently been attributed to the tactile remapping process. However, reference frame transformations are abundant in sensorimotor processing and concurrent representation of information in different reference frames appears to be a common coding principle of the brain that usually does not lead to noticeable processing deficits (Andersen et al., 1993; Snyder, 2000; Pouget et al., 2002; Schlack et al., 2005; Pesaran et al., 2006; Chen et al., 2013; Makin et al., 2013). An alternative explanation of crossing effects is, therefore, that tactile localization comprises two distinct stages (Badde et al., 2014b, 2015a,b; Heed et al., 2015): (1) touch location is remapped from the anatomical into an external reference frame and (2) information from the two reference frames is integrated to derive an optimal touch location estimate. In this framework, coordinate transformation is fast and efficient for all postures and performance impairments in crossed postures are due to the integration of conflicting information available in different reference frames.
To contrast these two accounts, we characterized the timing and spatial profile of hand reaches toward visual and tactile targets located at uncrossed and crossed feet in a sensorimotor decision task. Reach trajectories are subject to voluntary motor control (Todorov and Jordan, 2002; Scott, 2004) and reflect cognitive processing in spatial trajectory biases and in the timing of trajectory redirection (Song and Nakayama, 2009; Gallivan and Chapman, 2014). In the context of decision making, cognitive biases toward one of several choice alternatives translate into spatial biases in reach trajectories toward target locations associated with the chosen alternative (Spivey et al., 2005; Chapman et al., 2010a). If crossing effects in tactile localization are due to delayed coordinate transformation, reaches toward tactile locations should initially depend exclusively on anatomical coordinates until the transformation has been performed (Azañón and Soto-Faraco, 2008). Accordingly, when tactile stimuli are presented to crossed feet, reaches should regularly be deflected toward the incorrect foot and be corrected later during the reach. This is because the tactile location coded in an anatomical reference frame points toward the touched limb's body side, not its current position in space. In contrast, if deficits are due to integration of information in different reference frames, then reaches should not regularly be deflected toward the anatomical target location and should instead be delayed by integration of incongruent information, but guided by the external coordinate once the decision has been made.
Materials and Methods
In the main experiment, participants made hand reaches to visual and tactile target stimuli located at uncrossed and crossed feet.
Fourteen right-handed students (Oldfield, 1971), eight of them female, age 25.1 years (range: 20–39 years), took part in Experiment 1. Participants in this and the following experiments all had normal or corrected-to-normal vision and did not report any tactile impairments. They gave written informed consent and received course credit or monetary compensation. The experiments were approved by the ethics committee of the German Psychological Society (DGPs).
Participants sat on the floor and leaned against a metal bar to support an upright sitting position (Fig. 1A, top row). The bar's height was individually aligned with the lower episternum. A vibrotactile stimulator (bone-conductor BC 461-012; Oticon) was attached to each big toe. Red light-emitting diodes (LEDs) were attached just above each tactile stimulator. A central fixation LED was placed on the floor ∼10 cm in front of the participant's feet aligned with the starting position and the body midline. To avoid the impression of apparent motion in visual target conditions, the fixation LED was yellow and was always illuminated. Tactile stimuli consisted of suprathreshold 200 Hz vibrations of 30 ms duration. For visual stimulation, the red target LEDs were illuminated for 30 ms. Legs and feet were placed such that stimulus locations on both feet were freely accessible by reaches. Location of the stimulators in space was identical across conditions, with a distance of ∼30 cm between stimulators. Ear plugs, as well as white noise presented through head phones, masked the sound elicited by the tactile stimulators. Experimental protocols were controlled via MATLAB version 7.14 (The Mathworks). Stimuli were controlled by custom-made hardware and triggered with precision of <1 ms through National Instruments PC cards.
A piece of felt on the metal bar, facing away from the participant and aligned with the body midline, marked the reach hand's start position. Reach trajectories were recorded with a sampling rate of 100 Hz using a camera-based motion tracker (Visualeyez II VZ4000v PTI; Phoenix Technologies). Markers for motion recording were attached to the nail of each index finger and near the tactile and visual stimuli on the feet. An additional marker (the “trigger” marker) was placed next to the participant, positioned ∼15 cm in front of the hand's start position. During reaches, hand and trigger marker positions were read out and compared online, so that stimulus presentation could be elicited when the hand marker passed the trigger marker in the depth dimension.
Participants made reaches with the right and left hands toward tactile and visual target stimuli located at uncrossed and crossed feet. A trial started when the reaching hand had been still at the start position for 500 ms. After an additional interval of 1000–1400 ms, a tone instructed a straight index finger movement toward the fixation LED. The target was presented inflight when the finger passed the trigger marker. Participants had to change course and steer the hand toward the target as quickly as possible and remain still at the target for 500 ms. A tone then indicated that the hand should return to the start position for the next trial. The task was practiced extensively before running the experiment so that movements were smooth, continuous, and started out straight at the beginning of the movement.
There were four experimental factors. The factors foot posture (uncrossed vs crossed) and reach hand (left vs right) varied blockwise in randomized order. The factors target modality (visual vs tactile) and target location (left vs right foot) were varied pseudorandomly within blocks. However, because we did not have any hypotheses about differences in trajectory characteristics for reaches to the left and right foot, we pooled over the two target limbs (see below) and omitted factor target location in all analyses. Each of the 16 factor combinations was presented 40 times, distributed across four blocks of 160 trials. Each block was preceded by 12 practice trials that were not analyzed. Participants had to maintain fixation continuously. They rested and stretched out their limbs after each block.
Movement offset and duration.
Reach trajectories were analyzed in MATLAB. Movement offset was identified using a recursive algorithm (Teasdale et al., 1993) based on 3D tangential velocity. Movement duration was defined as the time between target presentation and movement offset.
Functional representation of movement trajectories.
Participants made decisions between targets on the left and right foot, so further analysis focused on the hand trajectories' right–left component (that is, the x-coordinate) over time. For trajectory averaging, individual trajectories were extended to the duration of a participant's slowest movement by filling all sampling points between the movement's last true sampling point and the last sampling point of the longest trajectory with the movement's spatial end point. Trajectories were fitted with sixth-order B-splines using the functional data analysis toolbox MATLAB implementation “FDAfuns” (Ramsay and Silverman, 2005; Ramsay et al., 2009). The fitting criterion lambda that trades off data fit against trajectory smoothness was determined based on the degrees of freedom of the spline fit by accounting for 20% of data points of each trajectory (Hastie and Tibshirani, 1990). The resulting functional trajectory representations, as well as their first two derivatives, were evaluated with 1 ms resolution for further analysis. Acceleration (i.e., the second derivative) was low-pass filtered with a second-order dual-pass Butterworth filter with a cutoff frequency of 50 Hz after function evaluation.
Identification of trajectory turn points.
Because movement duration differed across experimental conditions, it was not possible to normalize reach trajectories over time for trial averaging. Instead, we identified, for each individual trial, the point at which the turn toward the correct target was initiated by decelerating from its current direction, evident in a local extremum in velocity (implying a zero-crossing in acceleration) before the hand turned toward the correct side (Soechting and Lacquaniti, 1983; Day and Lyon, 2000; Aivar et al., 2008; Oostwoud Wijdenes et al., 2014). For brevity, we refer to this point as “turn point”. We interpreted the time of this landmark as an indicator that external spatial information had become available for reach planning (Day and Lyon, 2000; Overvliet et al., 2011; Friedman et al., 2013) and its spatial location in the left–right dimension as an indicator of a potential reach bias toward one of the targets. To analyze this bias, trajectories were normalized with respect to start and end point to eliminate spatial variance unrelated to the target decision. Trajectories to the left side were flipped so that reaches to the left and right could be pooled. Therefore, turn points were analyzed in normalized and flipped reach coordinates.
We excluded trials when at least one of five criteria was met: (1) the reach ended at the wrong target foot (1.6%); (2) movement duration was slower than the condition's mean ± 3 SDs (1.1%); (3) the trajectory's end point deviated more than ± 3 SDs from a condition's mean (0.6%); (4) the hand decelerated or was already directed toward the target foot at the time of stimulation [i.e., before the movement goal had been specified (4.6%)]; or (5) the trajectory comprised a continuous sequence of >20% sampling points with missing data (0.8%). The mean number of trials remaining per condition and subject was 36.4 (SD = 4.0) of 40 possible trials.
We used R version 3.2.2 (R Core Team, 2015) and the package lme4 (Bates et al., 2015) to perform linear mixed-effects analyses. Factors were specified with effect coding. Participants were entered as random effects and, whenever possible, the random-effects structure comprised random intercepts and slopes for all predictors and interactions (Schielzeth and Forstmeier, 2009; Barr et al., 2013). If models did not converge, the maximal random-effects structure that was supported by the data was selected. We assessed significance of fixed effects with likelihood ratio tests of the model with the maximal predictor structure and a reduced model without the fixed effect of interest (Pinheiro and Bates, 2000) using the package afex (Singmann, 2015). Fixed effects were considered significant at p < 0.05. For post hoc analysis of significant interactions, we conducted Tukey tests of the estimated least square means (LSM, package: lsmeans; Lenth, 2015). Condition averages and their confidence intervals (CIs) as presented in the figures were calculated based on LSM as well unless stated otherwise. To adhere to the assumption of normality and homoscedasticity, dependent measures were box-cox transformed before setting up the model when appropriate (Box and Cox, 1964).
The time of turn points was analyzed with two related models. The first model comprised factors reach hand, foot posture, and target modality. We included movement duration as a centered covariate to control for trivial effects of trajectory duration across conditions; for example, reaches across the midline were slightly longer and thus may take longer than reaches within a hemifield. Moreover, we added centered tangential velocity at the time of stimulation as a covariate to the main model to control for differences in hand speed at the time of stimulation.
In the second model, we included target history as an additional factor to test whether sequential trial dependencies accounted for timing effects (Chapman et al., 2010b). Target history was defined as the number of maximally four consecutive previous trials that the target had been presented from one side regardless of target modality and correct execution of the reach, with left targets coded as negative and right targets as positive. For example, if in the two trials before the current trial, the target had been presented on the left foot and the target before that had been right, then we coded trial history as −2 for two consecutive left previous targets. For the mixed model, the factor was coded with a normalized linear contrast (Rosenthal et al., 2000). Model comparison revealed that adding the factor target history to the main model did not improve data fit (χ2(123) = 86.37, p > 0.99). Therefore, we restrict our report to the first model.
The spatial location of turn points was also analyzed with two related models. Both models comprised factors reach hand, foot posture, and target modality. In the second model, we included target history as a fourth factor to test whether spatial biases could be accounted for by sequential trial dependencies (Chapman et al., 2010b). We tested separately whether the occurrence of reach deflections toward the anatomical touch location for reaches to tactile targets at crossed feet could be predicted from target history using a generalized linear mixed-model analysis that comprised only the fixed effects factor target history and subjectwise intercepts as random effects.
Experiment 2 tested whether the effects observed in Experiment 1 were truly attributable to stimulus localization. The same stimuli as in Experiment 1 were presented, but served as movement triggers for stereotypical movements rather than as movement targets, eliminating the need for stimulus localization.
Ten right-handed students (Oldfield, 1971), eight of them female, age 23.9 years (range: 20–35 years), took part in Experiment 2.
The setup of this control experiment was largely identical to that of Experiment 1. However, a midline-aligned button, positioned at the distance of the trigger marker used in Experiment 1, served as the start position of the reach hand. A second button, positioned in front of the feet and aligned with the start button, marked the reach goal (Fig. 1A, bottom row). The button release of the start button was recorded as reaction time (RT).
Participants made right and left hand reaches toward the central goal button upon detection of a tactile or visual stimulus that was presented to one of the uncrossed or crossed feet. A trial started when the participant's finger had been still at the start button for 500 ms. The stimulus was presented after an additional random interval of 1000–1800 ms. As quickly as possible, participants had to release the start button and reach toward the predefined goal button. Once the goal button was depressed, a tone indicated that the hand should return to the start button for the next trial. If a movement was initiated prematurely, that is, within 100 ms after stimulation, an error tone indicated that the hand should return to the start position and the trial was repeated at a later time.
We used the same factors as in Experiment 1 for statistical analysis: foot posture (uncrossed vs crossed, varied block-wise), reach hand (left vs right, varied block-wise), and target modality (visual vs tactile, varied within blocks). Each factor combination was presented 18 times, distributed across four blocks of 72 trials.
We analyzed RT as the dependent measure. Trials were excluded if their RT was greater than the condition's mean ± 2 SDs. The mean number of trials remaining per condition and subject was 17.5 (SD = 0.1) of 18 possible trials.
The general statistical analysis approach was identical to that of Experiment 1.
In analogy to Experiment 1, the model for the RT analysis of Experiment 2 comprised factors reach hand, foot posture, and target modality.
In Experiment 1, reaches were always initiated toward the visual fixation LED. Accordingly, tactile stimulation implied a change of modality for reach planning, whereas visual stimulation did not. We conducted Experiment 3 to eliminate this confound, testing whether reach trajectories were modulated by the modality of the stimulus toward which the reach was initiated.
Ten right-handed students (Oldfield, 1971), nine of them female, age 23 years (range: 20–30 years), took part in Experiment 3.
To provide a tactile fixation location, participants now made reaches with their right hand toward locations on the left arm (see Fig. 3A). They sat at a table with their left forearm positioned in a 45 degree angle relative to the torso. The yellow fixation LED, as well as a custom-made vibrotactile stimulator consisting of a cell phone vibration motor glued into a plastic casing, were attached to the center of the left forearm. This central tactile stimulator was driven by electric current to vibrate, but was weaker and clearly distinguishable from the tactile target stimuli. Target stimuli were attached with equal distance to the left (elbow) and right (wrist) of the central “fixation” stimulators. A piece of felt on the table, positioned 50 cm away from the left arm and aligned in depth with the left arm's central fixation location, marked the right hand's reach start position. Markers for motion tracking were attached to the index finger of the right hand and next to the fixation and target stimuli on the left arm. As in Experiment 1, stimulation was triggered after the movement had been initiated. The trigger marker was placed 10 cm away from the start point in movement direction. All other aspects of the experimental setup were identical to those of Experiment 1.
Participants made right hand reaches toward tactile and visual target stimuli located on the left arm. The course of a trial was identical to that of Experiment 1, with the exception that the fixation stimulus toward which the reach had to be initiated could be either visual or tactile. As with the visual fixation stimulus in Experiment 1, visual and tactile fixation stimuli in Experiment 3 were presented for the entire duration of a trial.
The analysis comprised three factors: target modality (visual vs tactile), target location (left vs right), and fixation modality (visual vs tactile). Fixation modality varied block-wise in randomized order, whereas target modality and target location varied pseudorandomly within blocks. Each of the eight factor combinations was presented 40 times, distributed across four blocks of 80 trials. We limit our report to the statistical comparison relevant to control for the potential confound of Experiment 1; that is, reaches to tactile targets that were initially directed toward a congruent (tactile) versus toward an incongruent (visual) fixation stimulus.
Movement offset and duration.
Movement offset and duration were computed as in Experiment 1.
Functional representation of movement trajectories.
Reach trajectories were converted into functional representations, as in Experiment 1, accounting for 90% of data points per trajectory. The greater restriction during B-spline fitting was chosen because reaches in this experiment were less variable than those of Experiment 1. Because target locations were spaced closer together than in Experiment 1, identification of turn points was not possible in many trials. Instead, we averaged functional trajectory representations after normalizing across movement time by extracting the 3D finger position at 200 equally spaced time points (Gallivan and Chapman, 2014). Normalization across time was adequate, because neither movement time nor hand speed at the time of stimulation differed significantly across conditions in a mixed-effects model comprising all experimental factors (Whitwell and Goodale, 2013). Because participants made decisions between left and right targets on the left arm, further analysis focused on the normalized hand trajectories' right–left component (with the arm as the x-coordinate) over time.
We used the same trial selection criteria as in Experiment 1. The reach ended at the wrong target or the fixation (6.0%); movement duration was slower than the condition's mean ± 3 SDs (0.8%); the trajectory's end point deviated more than ± 3 SDs from a condition's mean (0.2%); the hand decelerated or was already directed toward the target location at the time of stimulation (i.e., before the movement goal had been specified; 2.3%); and the trajectory comprised a continuous sequence of >20% sampling points with missing data (0.1%). The mean number of trials remaining per condition and subject was 36.3 (SD = 3.8) of 40 possible trials.
We used MATLAB algorithms developed by Chapman and colleagues (2010a) to compare the normalized functional representations of the left-right component of trajectories across experimental conditions and subjects in a functional ANOVA (fANOVA). Therefore, we compared reach trajectories of the different conditions time point by time point. This difference in analysis between Experiments 1 and 3 is due to the fact that we analyzed turn points in Experiment 1, but could not use this approach in the control experiment. p values were corrected using the Greenhouse–Geisser approach whenever necessary due to violations of sphericity.
Time and spatial analysis.
The fANOVA for the spatial analysis of the right–left reach component over time comprised factors fixation modality and target location.
Time-related characteristics of reach trajectories
Experiment 1 assessed hand reach trajectories to visual and tactile targets located at the feet. Participants initiated a reach directed toward a visual fixation stimulus centrally located between the two feet. Target stimuli were presented in midflight and reaches had to be changed from the initial straight direction toward the new target location. Reaches were usually accurate and rarely ended at the wrong foot. Therefore, we did not analyze error trials. If additional computational effort is required when reaching to tactile as opposed to visual targets, then turn points for tactile reaches should occur later than for visual reaches. In addition, turn points should be delayed even more when tactile targets were presented on crossed rather than uncrossed feet because the conflict between anatomical and external, gaze-centered reference frames has to be resolved. The effect of foot posture should be specific to tactile targets because coordinates for reaching to visual targets are presumed to be natively eye centered.
Results of the turn point time analysis are illustrated in Figure 1, B and C. There were main effects of target modality (χ2(1) = 29.61, p < 0.001) and foot posture (χ2(1) = 13.91, p < 0.001), as well as an interaction of the two (χ2(1) = 14.08, p < 0.001). We observed no further significant effects.
Post hoc analysis showed that turns toward tactile targets were initiated later than turns toward visual targets. Turn point time differed significantly between modalities when reaching to targets located at both uncrossed (t(14.86) = 7.92, p < 0.001) and at crossed feet (t(15.25) = 8.78, p < 0.001). Turn point time was delayed with crossed feet for tactile (t(15.14) = 5.91, p < 0.001), but not for visual targets (t(15.07) = 2.36, p = 0.13). The results of the time analysis are summarized in Figure 1C and an example subject's single trajectories and turn points are illustrated in Figure 2.
We conducted two control experiments to scrutinize the results obtained in Experiment 1. First, it was crucial to show that the differences between visual and tactile conditions in our main experiment were due specifically to the process of target localization and not to more general processing differences between modalities that are unrelated to spatial localization. To dissociate these alternatives, participants made reaches to a single, prespecified target in all trials. These stereotyped reaches were cued by the same visual and tactile stimuli as in Experiment 1. The important difference between this and our main experiment was that the control experiment did not require stimulus localization; instead, stimuli merely had to be detected to initiate a stereotyped response. Therefore, if the timing differences between touch and vision observed in Experiment 1 were related specifically to tactile localization, then no difference in RT should be observed in Experiment 2. Note that the two experiments also differed in that stimuli were presented in midflight for Experiment 1, but served as response triggers in Experiment 2. This is because initiation of an inflight correction would have required target reprogramming, potentially eliciting localization processes and thus destroying the purpose of the control experiment. Therefore, we note the caveat that movement initiation probably involves additional processes compared with online movement correction (Day and Lyon, 2000).
We observed a trend toward longer RT for the left than the right hand (main effect of reach hand, χ2(1) = 3.62, p = 0.057; left hand: LSM = 235.87 ms, 95% CI [224.33, 247.41], right hand: LSM = 229.57 ms, 95% CI [219.05, 240.09]). Critically, RT did not differ significantly between modalities (main effect of target modality, χ2(1) = 2.17, p = 0.14; tactile: LSM = 231.81 ms, 95% CI [221.27, 242.36], visual: LSM = 233.62 ms, 95% CI [223.11, 244.14]). None of the interactions in the model reached significance. These results suggest that the time difference between turn points in the two modalities observed in Experiment 1 was specifically related to tactile localization.
Second, in the main experiment, participants started moving toward a visual fixation and then corrected reach direction inflight toward a tactile or visual target. Therefore, trials contained a modality switch in tactile, but not in visual, target conditions. This switch may take time and, accordingly, confound our experimental results. Our original experimental design, with targets presented at the feet and a visual fixation placed between the two limbs, prevented tactual presentation of the fixation location. In a new experiment, participants made reaches with the right hand toward the left arm. The fixation location was placed on the forearm midway between elbow and wrist and could thus be indicated visually or tactually. Results of Experiment 3 are illustrated in Figure 3. fANOVA of reach trajectories revealed a significant main effect of target location, reflecting that reaches to left and right targets differed in the last 29.5% of movement time. Crucially, the congruence of fixation and target modality did not significantly affect trajectories at any point in time, suggesting that the results of Experiment 1 were not due to modality switches in the different experimental conditions.
Spatial characteristics of reach trajectories
After we had characterized the temporal effects related to tactile remapping, we next analyzed the spatial profile of reach trajectories with the aim of specifying the mechanism underlying crossing effects in tactile localization. Time delays may be due to impairment of coordinate transformation itself or to integration of spatially conflicting information after transformation. During transformation, only the original, anatomical spatial information should be available. Therefore, if delays stemmed from the transformation process, then external, gaze-centered information should also become available after considerable delay (Azañón and Soto-Faraco, 2008). Correspondingly, reach trajectories toward crossed feet should regularly deviate toward the tactile target's anatomical location and be corrected only once the transformation has been performed. For tactile targets at crossed feet, anatomical and remapped spatial information implicate different sides of space for the movement target. In contrast, with uncrossed feet, both anatomical and external coordinates implicate the same side of space. Therefore, the spatial location of turn points for reaches to tactile targets should be consistently biased toward the incorrect side in crossed relative to uncrossed conditions. If, however, time delays stemmed from spatial integration rather than from coordinate transformation, no consistent spatial bias toward the wrong side should emerge when reaching to tactile targets at crossed feet because information available in both reference frames should be available early and at approximately the same time, but correction of the reach trajectory from a straight path toward the target should be delayed until integration of conflicting information is complete. According to both the transformation and the integration account, foot posture should not affect reaches to visual targets because the visual system natively provides external spatial coordinates, immediately providing appropriate coordinates of the reach target.
Results of the spatial turn point analysis are illustrated in Figure 4. We flipped reach trajectories directed to left-side targets (in space) to pool them with those of right target reaches for statistical analysis. A leftward deflection of trajectories displayed in Figure 4 therefore corresponds to a bias toward the anatomical target coordinate when reaching to tactile targets located at crossed feet. Linear mixed-model analysis of turn point location revealed a main effect of target modality (χ2(1) = 13.25, p < 0.001), as well as a significant interaction of target modality and foot posture (χ2(1) = 4.46, p = 0.035). Post hoc analysis of the interaction revealed that, for tactile targets, the location of turn initiation was located 7% further toward the incorrect target when the feet were crossed than when they were uncrossed (t(14.60) = −2.27, p = 0.039; Fig. 4A). This difference was exclusive to tactile stimulation (comparison of turn points for visual targets in uncrossed and crossed postures: t(14.53) = −0.87, p = 0.40). Inspection of the distribution of tactile turn points (Fig. 4B, top) revealed that they were not shifted toward the incorrect target in the majority of trials. Instead, for most trials, turn point location was indistinguishable between uncrossed and crossed feet. Therefore, the majority of trials did not exhibit a spatial bias toward the anatomical side of tactile stimulation (Fig. 4B, top, overlap between light and dark red points in x-direction; note that differences in y-direction pertain to time, not space). The large variance observed in the crossed tactile condition was caused by a subset of 17.2% (range: 4–36.5%) of reaches, for which the turn point deviated >2 SDs beyond the mean location in uncrossed trials. These trajectories initially digressed far toward the incorrect side and then turned around to the correct side (Fig. 4B, top, dashed average trajectory). These deviations were mainly observed toward the incorrect target side and thus do not reflect a general tendency for greater trajectory variability in the crossed condition. We term these reaches with deviating reach points turn-around reaches (for a similar effect in saccade trajectories, see Overvliet et al., 2011). When we excluded this subset of trials for illustration purposes, the spatial profile of reaches was visually indistinguishable across conditions (see overlap of turn points in uncrossed and crossed conditions in Fig. 4B, bottom), suggesting that most reaches had similar turn point locations independent of foot posture and target modality. We repeated the mixed-model anaylsis, but excluded all turn-around reaches; consistent with the visual impression, reaches to tactile targets did not exhibit any crossing effects. Correspondingly, turn points of reaches to tactile targets at crossed feet were distributed bimodally in space, with peaks reflecting regular and turn-around reaches across participants (data not shown). Therefore, although the spatial profile of reaches appeared consistent with the hypothesis that crossing effects result from coordinate transformation when considering averaged information, this was not the case when considering behavior on a trial-to-trial basis.
The bias toward the incorrect target was specific for tactile turn points in the crossed condition, but visual turn points were also more variable in the crossed than uncrossed condition (Fig. 4A). Crucially, for visual targets, variation was centered around zero (see Fig. 4B, top, overlap between light and dark blue points in x-direction and dashed average trajectory of outliers). Recall that target modality was varied from trial to trial, so the additional, spatially unbiased variation of reaches to visual targets may reflect uncertainty induced by limb crossing. We note that variance in visual, but not tactile, trials was comparable for uncrossed and crossed postures when we used stricter criteria to select trials for statistical analysis (a strategy that had been criticized and therefore abandoned during the review process). Therefore, the high variance in crossed, visual trials appears to be related to deviation in other characteristics of the reach such as end location and reach duration.
We next explored whether trajectory profiles conform with the assumption that crossing effects stem from integration-related processing. We reasoned that, if crossing effects are the consequence of an integration process, then other variables might be integrated in the tactile decision as well and, accordingly, affect the reach trajectories. A popular and well researched class of decision-making models are bounded integrator models. These models assume that a decision variable for each choice alternative accumulates toward a bound by integrating evidence over time (Ratcliff and Rouder, 1998; Hanes and Carpenter, 1999; Cisek et al., 2009). The alternative in which the bound is reached first determines the choice. Occasional premature decisions can be triggered when initial sensory evidence in support of a choice alternative is high and be corrected by continued evidence accumulation (Resulaj et al., 2009; Burk et al., 2014). Furthermore, choices made in prior trials may induce an additive offset to the cumulative decision variable (Gold et al., 2008). Given these previous findings, we reasoned that, if trajectories reflected an integration process, then we should observe an effect of target history in our visual and tactile sensorimotor decision task. In fact, target history effects on the spatial profile of reach trajectories have been demonstrated previously in visual reaching paradigms. Specifically, repetition of target location was shown to bias trajectories toward the repeated side, with the number of repetitions increasing the spatial bias (Chapman et al., 2010b). To explore sequential target effects in the present study, we repeated the analysis of turn points with target history as an additional predictor. As before, we observed a main effect of target modality (χ2(1) = 13.33, p < 0.001) and an interaction of target modality and foot posture (χ2(1) = 4.21, p = 0.040). There was a significant main effect of target history (χ2(1) = 8.22, p = 0.004) that did not interact with any other factor, indicating that target history influenced trajectories independently of modality and posture (Fig. 4C). However, the increased number of degrees of freedom in the model with target history as factor rendered its comparison with the original model nonsignificant (χ2(2) = 0, p > 0.99).
We therefore used a second approach to assess the potential impact of target history. Consider reaches to tactile stimuli at crossed feet within the bounded integrator framework: when the opposite target than the current one had been presented in previous trials, this would induce a bias toward the anatomically coded target in the current trial and increase the probability that the bound implicating the incorrect choice alternative was reached faster by moving the starting point of the decision variable toward that choice. If the incorrect bound were then reached, the trajectory would be deflected toward the incorrect target until the decision is eventually corrected. This framework predicts that the probability for turn-around trajectories—that is, reaches that severely deflect toward the incorrect side when the feet are crossed—should increase when the currently incorrect target had been presented in previous trials. In accordance with this prediction, the probability of a turn-around reach was predicted by its target history (χ2(1) = 7.14, p = 0.008) and the proportion of turn-around reaches was twice as high for a target history of four targets at the opposite compared with four targets at the same side as the current trial (percentage of turn-around reaches, 23.5% vs 11.7%).
In sum, the spatial pattern of reach turn points is consistent with coordinate integration, but not transformation, as the underlying cause of performance deficits in crossed postures. As a last step, we tested whether this conclusion is also warranted by reach timing when turn-around reaches are excluded. Recall that the transformation account predicts a spatial deviation of reaches due to the prolonged availability of only the anatomical target coordinate in the crossed posture. Accordingly, reaches that do not express such a spatial deviation should not be prolonged compared with reaches to uncrossed limbs. Contrary to this prediction, a model that excluded turn-around reaches revealed main effects of target modality (χ2(1) = 26.86, p < 0.001) and foot posture (χ2(1) = 12.48, p < 0.001), as well as an interaction of the two (χ2(1) = 9.59, p < 0.01), just like the original model that analyzed all trials. The transformation account does not explain why reaches to crossed limbs would be prolonged but not deviate spatially toward the incorrect side. In contrast, the integration model predicts prolonged but spatially unbiased reaches and accounts for occasional deviations toward the anatomical target side.
We sought to characterize the processes involved in tactile localization by exploiting the context of a sensorimotor decision making task. Tactile localization has been proposed to involve two separate processes: tactile remapping (i.e., coordinate transformation from anatomical into external coordinates) and subsequent integration of anatomically coded information with remapped, externally coded, gaze-centered spatial information (Badde et al., 2014a,b, 2015b; Heed et al., 2015). To clarify the contribution of these processes to localization behavior, participants executed hand reaches toward visual and tactile targets at uncrossed and crossed feet. We analyzed the timing and location of the point in reach trajectories at which the turn toward the correct target was initiated.
Our study revealed three key results. First, turn points were significantly delayed by foot crossing for tactile, but not for visual, reaches. Second, we did not observe a consistent spatial bias toward the stimulus when touch was delivered to crossed feet. Whereas, in this situation, a subset of reaches exhibited pronounced initial deflections toward the incorrect target (termed turn-around reaches), the majority of trajectories turned at a horizontal location centered around the start position, just like reaches toward uncrossed feet. Third, target repetition biased trajectories toward the repeated location and the proportion of turn-around reaches was twice as high when repetition had induced a bias toward the incorrect target than when the bias had been directed toward the correct target.
Our study capitalized on several known features of hand reaches to characterize the processes involved in a tactile localization decision. Reaches are nonballistic (Jeannerod, 1988) and can be promptly corrected inflight (Pélisson et al., 1986; Desmurget et al., 1999; Todorov and Jordan, 2002; Scott, 2004). They are also known to reflect cognitive processing in the context of motor planning and perceptual decision making (Körding and Wolpert, 2006; Trommershäuser et al., 2008; Resulaj et al., 2009; Chapman et al., 2010a; Selen et al., 2012; Friedman et al., 2013) on a single-trial level (Day and Lyon, 2000). Furthermore, motor correction paradigms reduce motor-related processing unrelated to the cognitive task of interest because reach initiation is completed before the investigated process begins (Day and Lyon, 2000).
Timeline of tactile remapping
Previous findings concerning the timing of tactile remapping have been heterogeneous, probably because of the use of divergent paradigms and research methods across studies. A spatially congruent tactile cue can speed up a decision about a subsequent visual stimulus (Azañón and Soto-Faraco, 2008). When the tactile cue led the visual stimulus by 60 ms, spatial congruence was effective in an anatomical reference frame; that is, a right hand stimulus improved the decision about a right-side visual stimulus independently of where the hand was positioned in space. When the tactile cue led by 180 ms, then spatial congruence was effective in an external reference frame; that is, a hand (whether left or right) positioned on the right side in space improved the decision for a right-side visual stimulus. This result pattern was interpreted to indicate that tactile remapping is completed between 60 and 180 ms after stimulus onset.
Consistent with this time range, several studies have reported effects of hand posture in event-related potentials (ERPs) evoked by tactile stimulation in the time range of 70–160 ms after stimulation (Heed and Röder, 2010; Rigato et al., 2013; Soto-Faraco and Azañón, 2013). However, it remains unclear whether ERP modulation reflects coordinate transformation or spatial integration of different reference frames and if the averaged signals truly reflect processing on a single-trial level. Finally, when saccades had to be directed to tactile locations at crossed hands, saccade RT for straight saccades was, on average, 284 ms (Overvliet et al., 2011). Based on estimates for motor preparatory processing in monkey neurophysiology, the investigators suggested that remapping was completed after ∼190 ms.
The current study clarifies previous work in three ways. First, by identifying the time point at which a decision becomes evident in hand reaches, we specified the timing of spatial processing on a continuous time scale. Second, the use of an inflight motor correction paradigm minimized the effects of processes unrelated to stimulus localization. Third, analysis of individual reach trajectories circumvents inferential limitations of trial averaging. Figure 1C summarizes the timing of stimulus localization revealed by the current experiments. Reaches to visual stimuli were unaffected by posture and were redirected, on average, 138 ms after visual stimulation. This latency is consistent with previous response time estimates of ∼120–160 ms for inflight correction of reaches toward a visual target (Day and Lyon, 2000; Scott, 2012). Reaches to tactile targets were redirected 158 ms after stimulation when the feet were uncrossed. Therefore, even when anatomical and external reference frames were aligned, decision times for touch were longer than those for vision. In contrast, RT did not differ between modalities in our Experiment 2, which did not require stimulus localization. Therefore, the time difference of ∼20 ms between tactile and visual choices appears to be specifically related to tactile stimulus localization. This suggests that a tactile decision involves remapping of anatomical into external coordinates also when the feet are uncrossed and that inflight correction of the hand movement is not initiated before this process is complete. When anatomical and external reference frames were incongruent, the turn toward the correct tactile target was initiated, on average, after 200 ms. Therefore, tactile targets at crossed feet were localized 42 ms later than tactile targets at uncrossed feet; that is, in the absence of reference frame conflict. These processing time estimates in the presence of conflict are shorter than previous time estimates of tactile remapping (e.g., saccade RT effects of 190 ms in Overvliet et al., 2011). They indicate that previous experiments may have overestimated the time necessary to localize tactile stimuli in space and demonstrate the feasibility of movement paradigms that involve online movement corrections for the isolation of specific cognitive processes (Day and Lyon, 2000; Gallivan and Chapman, 2014).
Tactile localization as a bounded integrator process
The spatial profile of reach trajectories renders new insight into the origin of the time delays observed in touch localization under reference frame conflict. If prolonged processing time were due to coordinate transformation, then behavior should regularly be affected by the anatomical reference frame; that is, single reaches should usually show an initial spatial bias toward the incorrect target when the feet are crossed. Although average trajectories of reaches to tactile targets were affected by limb crossing, this effect was caused by a minority of trials that were severely deflected toward the incorrect side. In contrast, most trajectories turned at a spatial location that was indistinguishable from turn points of trajectories to uncrossed feet. Therefore, spatial trajectory patterns did not support the notion that anatomical spatial information regularly guided initial reaching behavior. Instead, the current results suggest that behavior reflected the integration of anatomical and external, gaze-centered spatial information. This account could be implemented, for example, within a bounded integrator framework of decision making (Hanes and Carpenter, 1999; Cisek et al., 2009; Wolpert and Landy, 2012). According to such models, evidence for each alternative relevant for a decision is accumulated until the threshold for one choice is reached and triggers a response associated with the respective alternative (Gold and Shadlen, 2007).
Extending this framework to touch localization implies that spatial information about touch location is available in anatomical and external reference frames and that both types of information are integrated as sensory evidence toward the two possible target locations (Overvliet et al., 2011; Buchholz et al., 2012). The motor response to turn the reach toward a target is initiated once the bound of either target has been passed. With uncrossed feet, anatomical and external spatial information are congruent and both contribute to evidence accumulation for the same target foot. Although one might expect that the redundant information from the two reference frames should lead to faster integration compared with reaching to visual targets, the formation of the spatial decision took longer for tactile than for visual targets. In two control experiments, we confirmed that this additional processing time was directly related to the process of localization rather than to unspecific differences between the visual and tactile modalities and that it was unrelated to modality switch costs inherent in our paradigm. The origin of the posture-independent, localization-specific delay in tactile processing thus appears to be related to the transformation of anatomical into external, eye-centered coordinates.
When anatomical and external information were incongruent, most trajectories toward tactile targets turned later in time, but at a similar right–left location, as trajectories to stimuli at uncrossed feet. In the bounded accumulator framework, this result pattern suggests that reach adjustments were initiated only after evidence accumulation had been completed and the decision for a target had been formed. In the crossed posture, the decision was prolonged presumably because evidence accumulation proceeded more slowly due to conflicting input from different reference frames. If instead coordinate transformation had been impaired or slowed by limb crossing, anatomical spatial information would nonetheless have been available early on and should have resulted in a consistent reach bias toward the incorrect target side. This interpretation is consistent with the increased probability of turn-around reaches when target history favored the incorrect foot. To summarize, hand reaches indicating tactile decisions about targets defined in incongruent reference frames were regularly delayed, but not consistently biased in space. This result pattern suggests that anatomical spatial information did not dominate early stages of the movement toward tactile targets. Instead, the time delays induced by limb crossing likely originate from integration of incongruent spatial information. These results can be explained by assuming a bounded integrator process that integrates tactile coordinates of anatomical and external reference frames.
This work was supported by the German Research Foundation (DFG Emmy Noether Grant HE 6368/1-1 to T.H.). We thank Sarah Kolb, Julia Kozhanova, Martina Obst, Phyllis Mania, and Selina Pradel for help with data acquisition; Renate Kiesewalter, Dirk Waschatz, and Rainer Schäfer for technical assistance; and Adjmal Sarwary and Ivar Clemens for providing C-code to optimize timing accuracy for stimulus presentation.
The authors declare no competing financial interests.
- Correspondence should be addressed to Janina Brandes, Biological Psychology and Neuropsychology, University of Hamburg, Von-Melle-Park 11, 20146 Hamburg, Germany.