Abstract
Accurate motor execution is achieved by estimating future sensory states via a forward model of limb dynamics. In the current experiment, we probed the time course over which state estimation evolves during movement planning by combining a bimanual arm crossing movement with a temporal order judgment (TOJ) task. Human participants judged which of two successive vibrotactile stimuli delivered to each index finger arrived first as they were preparing to either cross or uncross their hands. TOJ error rate was found to systematically vary in a time- and direction-dependent manner. When planning to cross the hands, error rate systematically increased as the vibrotactile stimuli were delivered closer in time to the onset of the movement. By contrast, planning to uncross the hands led to a gradual reduction in error rate as movement planning progressed. In both cases, these changes occurred before the actual alteration in hand configuration. We suggest that these systematic changes in error represent an interaction between the evolving state estimation processes and decisions regarding the timing of successive events.
Introduction
To accurately reach to a target, the brain is thought to compensate for delays inherent in the relevant sensory signals by using a state estimate of the peripheral motor apparatus (Miall et al., 2007; Mulliken et al., 2008). Such an estimate can be used to predict the sensory consequences of movement and allow rapid online updating as the movement unfolds. Although the existence of these processes has been inferred in adaptation studies by systematically perturbing the system over the course of many movements (Shadmehr et al., 2010), the underlying dynamics of the state estimate during the planning of individual movements is poorly understood. In the present study, we used a cutaneous temporal order judgment (TOJ) task to probe this aspect of state estimation. In particular, we took advantage of the fact that the accuracy of temporal judgments for vibrotactile stimuli delivered to each hand is influenced by the relative positions of the two hands; when the hands are crossed, accuracy is substantially worse than when the hands are uncrossed (Yamamoto and Kitazawa, 2001a,b; Shore et al., 2002). Furthermore, movement planning biases the spatial perception of such stimuli: the brain perceives stimuli to have occurred at a location ahead of where they were actually delivered (Dassonville, 1995; Watanabe et al., 2009; Buckingham et al., 2010). We combined these two approaches and asked to what extent planning to either cross or uncross the arms influences the accuracy of decisions regarding when vibrotactile stimuli were delivered to each hand. Moreover, by systematically manipulating when the TOJ stimuli arrived relative to the ongoing movement-planning processes, we were able to directly probe the evolution of the dynamics of state estimation during individual movements.
Materials and Methods
Participants.
Eight participants (five female, three male; mean age: 25.6 years) took part in this study after providing written informed consent. All had normal or corrected-to-normal vision and none reported any neurological deficits. The experimental procedures were approved by the Committee for the Protection of Human Subjects at the University of Oregon.
TOJ task.
During a single session, the participant sat at a table and made a judgment about which of two vibrotactile stimuli, separated by 100 ms, arrived first. The stimuli were delivered by a piezoelectric device controlled by three 5 V pulses separated by 2 ms, thus producing a 1 mm displacement of a 2 mm2 contact point. The stimulators were attached with straps to the distal pad of each index finger. In half the trials, the left index finger was stimulated first and in the other half the right index finger was stimulated first.
Arm-crossing task.
In separate blocks of trials within the session, participants moved their hands from an uncrossed to crossed or a crossed to uncrossed configuration, or in a straight-ahead manner, as quickly and accurately as possible in response to an auditory cue presented at stimulus onset asynchronies (SOAs) of 0, 50, 100, 150, 200, or 250 ms before the initial vibrotactile stimulation (Fig. 1). Targets (blue 2 × 2 cm squares) centered on the midline and separated by 15 cm were attached to the tabletop for this purpose. Infrared-emitting diodes (IREDs) were attached to the fingernail of each index finger so that hand motion could be captured using an Optotrak system (100 Hz; Northern Digital). A baseline condition was also completed in which judgments were made with the hands remaining stationary in either the crossed or uncrossed configuration.
Participants responded to an auditory cue by moving from an uncrossed-to-crossed or crossed-to-uncrossed configuration, or straight ahead. The cue was presented 0–250 ms before the first vibrotactile stimulation to either the right or left index finger; 100 ms later the second vibrotactile stimulation was delivered to the other finger.
Control condition.
A control condition was completed by eight additional participants (five female, three male; mean age: 25.6 years) to examine the potential nonspecific effects of task complexity on the TOJ decision errors. The control condition was the same in all respects to the main task except that the arm movements were asymmetrical and did not cross. This was accomplished by having participants move their left hand from a starting position located 7.5 cm to the left of the midline and the right hand from a starting position located 17.5 cm to the right of the midline toward targets (blue 2 × 2 cm squares) separated by 5 cm at the midline. This resulted in movements of 5 cm for the left hand and 15 cm for the right hand. As with the main task, a baseline condition was also completed in which the hands remained stationary in the uncrossed configuration.
Data collection and analysis.
After each trial, the participants responded by saying “left” or “right” to indicate which hand they perceived to be stimulated first. Sixty trials in each of the movement blocks and 10 trials in each of the stationary blocks were completed for a total of 210 trials. The main dependent variable was error rate, defined as the percentage of trials for each condition in which the participant responded incorrectly. In addition, analysis of the movement trajectory from each hand also allowed us to determine the reaction time (RT; delay from tone to hand movement onset) and movement time (MT; period from onset to termination of movement). To ensure that the temporal decisions were uncontaminated by actual movement, only trials in which the hands started moving after the second cutaneous stimulus were included in the analysis. This criterion resulted in the removal of a small percentage of trials in the 250 and 200 ms SOA conditions (4% and 1%, respectively). For the asymmetric moving condition, a small percentage of trials (2.7%) were excluded for reaction time and movement time analysis because the motion could not be confirmed due to IRED occlusion. For each of the three conditions in the main experiment, no differences were found between trials in which the left or right hands were stimulated first, so the data were combined and submitted to a 3 (movement condition: crossed-to-uncrossed, uncrossed-to-crossed, straight-ahead) × 6 (SOA: −250, −200, −150, −100, −50, 0 ms) ANOVA to determine if error rate varied across the different combinations of conditions. Linear regression analyses were also applied to the error rate—SOA relation from each participant and the resulting slopes were compared across the three movement conditions using a one-way ANOVA. Similar analyses were performed in the control condition. Significance was set at p < 0.05.
Results
As an initial comparison, we assessed the extent to which error rate changed as a function of the context under which the decision was made. For this purpose, we compared error rates in the stationary control condition to the condition in which the first finger stimulation arrived coincident with the cue to begin movement (0 ms SOA). Planning to cross the hands led to a significant increase in error rate (t test, p = 0.038), whereas planning to uncross the hands led to a corresponding decrease in error rate (t test, p = 0.01). By contrast, planning to move the hands straight ahead led to no change in error rate (t test, p = 0.51). Thus, even though the cue-triggering movement occurred at the same time as the first finger stimulation, because participants were prepared to either cross or uncross their hands, the error rates increased or decreased, respectively. This suggests that the mental set associated with a specific future limb configuration is sufficient to bias decision-making regarding the current temporal sequence of events.
To determine whether this decision making evolves further over time, we examined the error rates in each condition across the course of the movement-planning period. The data in Figure 2A demonstrate that planning to cross the hands led to progressively higher error rates such that the accuracy of the decision for stimuli delivered just before the onset of hand motion was similar to that observed if the hands were stationary but physically crossed. By contrast, planning to uncross the hands led to a progressive reduction in error rates to a level similar to that observed when the hands actually were uncrossed. When participants planned to move the hands straight ahead, error rates remained unchanged across the movement-planning period—implying that the effects observed when the hand configurations changed were not due to planning limb movements in general, but rather to the specific details of the movement itself. ANOVA revealed a significant effect of movement condition (F(2,143) = 24.998, p < 0.001) and a movement condition × SOA interaction that approached significance (F(10,143) = 3.784, p = 0.065). Post hoc Tukey's tests demonstrated that the error rate was significantly different (p < 0.05) between the crossed-to-uncrossed and straight-ahead conditions at the −100, −50, and 0 ms SOAs; between the uncrossed-to-crossed and straight ahead conditions at the −250, −200, −150, and −100 ms SOAs; and between the crossed-to-uncrossed and uncrossed-to-crossed conditions and the −250 and 0 ms SOAs. These trends in the data were confirmed by the significant difference in the group means for the slopes derived from linear regression analyses applied to the error rate—SOA relation for each participant (F(2,23) = 10.51, p < 0.001) (Fig. 2B). The slope in the crossed-to-uncrossed condition was positive and significantly different from that in the straight-ahead condition; whereas the slope in the uncrossed-to-crossed condition was negative and significantly different from that in the straight-ahead condition.
A, Error rates for temporal order judgments plotted for the uncrossed-to-crossed (black circles), crossed-to-uncrossed (gray squares), and straight-ahead (gray diamonds) conditions for each SOA. Symbols on the right and horizontal lines represent the mean error rates in control conditions with the hands in a crossed or uncrossed stationary starting configuration. Error bars, 1 intersubject SE. B, Group means for slope values from the linear regression analyses of the error rate—SOA relation from each participant. C-UC, Crossed-to-uncrossed; UC-C, uncrossed-to-crossed; Straight, straight ahead.
In addition to error rates, we also examined the movements themselves to determine whether they were affected by the concurrently performed TOJ task. We found that crossing or uncrossing the hands led to longer reaction and movement times than moving the hands straight ahead (condition effect: RT: F(2,143) = 61.9, p < 0.001; MT: F(2,143) = 24.3, p < 0.001). However, unlike the error rate data, these movement variables did not vary as a function of the SOA (SOA effect: RT: F(5,143) = 0.276, p = 0.764; MT: F(5,143) = 0.165, p = 0.892). The arms actually crossed ∼46% into the movement on average and this too did not vary across SOAs. Given the changes observed in error rates and the average reaction (324 ms) and movement (364 ms) times in the arm-crossing conditions, this implies that the perception of the relative positions of the arms starts to change as early as 491 ms (i.e., the 0 ms SOA condition) before the moment the arms actually cross (or uncross) and is complete ∼241 ms (i.e., the 250 ms SOA condition) before this point in time.
It is possible that the systematic influence of movement planning on the error rates was directly related to the complexity of the task rather than the arm crossing per se. To examine this possibility, a different set of participants completed a control condition in which they generated arm movements that were asymmetric in amplitude but did not cross. Results from this condition showed that error rates were increased relative to the stationary control condition as in the main experiment, but there was no effect of SOA (F(5,84) = 0.516, p = 0.763) (Fig. 3). Interestingly, the error rates were larger overall when the right hand was stimulated first (F(1,84) = 8.917, p = 0.004). This hand effect was not observed in the main experiment. Given that the right hand was required to make the larger amplitude movement, this implies that some aspect of planning movements that vary in amplitude interacts with the decision-making processing associated with judgments of temporal order.
Error rates for temporal order judgments plotted for the asymmetric hand movement control condition for SOA. Different symbols represent condition in which either the left (white squares) or right (black circles) was stimulated first. Horizontal line represents the mean error rates in the stationary condition with the hands in the uncrossed starting configuration. Error bars, 1 intersubject SE.
Discussion
Everyday tasks such as reaching to grasp a coffee mug appear to be inherently simple activities that can readily be achieved by healthy individuals. To explain such seamless responses, theories of limb motor control have emphasized the predictive or anticipatory nature of the planning process (Desmurget and Grafton, 2000; Diedrichsen et al., 2010). In particular, to accurately reach to an object of interest, the brain is thought to compensate for system delays by using a forward model to predict the sensory consequences of movement (Miall et al., 2007). Such a predictive signal, when integrated with sensory signals derived from the actual movement, can be used to generate a state estimate and allow rapid online updating as the response unfolds. There is substantial evidence derived from sensorimotor adaptation studies to support the existence of these predictive signals. In general, these studies use systematic perturbations to the sensorimotor system—for example, through exposure of a hand-held manipulandum to a force field—and characterize how the system adapts over the course of a finite period of time (Shadmehr et al., 2010). When the perturbation is subsequently removed, an aftereffect is revealed that serves as a measurable proxy for changes induced in the predictive signals. Although adaptation protocols have allowed researchers to gain insight into the general features of these predictive mechanisms, a limitation of this approach is that it does not allow direct access to how these mechanisms contribute to the planning of individual movements. In particular, it is currently poorly understood over what time course predictive signals have their influence during the period leading up to movement onset. In the present study, we probed the dynamics of state estimation during movement planning by combining bimanual arm crossing and TOJ tasks. Our results indicated that judgments of temporal order were systematically influenced simply by planning limb movements that brought the hands into or away from a crossed configuration. This is consistent with the notion that the brain predicts the sensory consequences of upcoming limb movements and that these predictive signals interact with the perception of temporal order to bias subsequent decisions regarding the sequencing of successive events. Moreover, by systematically manipulating the SOA between the movement cue and the TOJ stimuli, we were able to demonstrate that the state estimation evolves over the course of the movement-planning period.
A potential alternative explanation for the current results is that the TOJ errors were driven by the complexity of the task constraints—more complex combinations of motor planning (arm crossing or uncrossing) and TOJ stimulus timing may systematically bias the temporal decision making. This seems unlikely because it would imply that making temporal decisions about TOJ stimuli that arrive immediately after the cue to uncross the arms is as complex as making a similar decision about TOJ stimuli that arrive 250 ms after the cue to cross the arms, but that intermediate cue times allow these decisions to be made more easily. Moreover, the fact that the error rates varied in the arm-crossing conditions with SOA but the underlying characteristics of the movements themselves (i.e., the reaction and movement times) did not suggests that the modulation of the error rates was not driven by the task complexity. This point is supported by the finding in the asymmetric arm movement control condition in which error rates were elevated relative to the stationary condition, but did not vary with SOA. Thus, we are confident in our conclusion that the systematic bias observed in the TOJ decisions across the SOAs in the arm-crossing conditions is due to the effects of the evolving state estimation inherent in the motor-planning process. This conclusion is supported by the findings of Yamamoto and Kitazawa (2001a), who showed that TOJ error rate increased as the arms became progressively more crossed under static conditions. This demonstrates that the error rate is an accurate measure of the actual sensory state under static conditions and, therefore, can be used as a proxy for the predicted sensory state under dynamic conditions.
The predictive signals underlying this process must carry efference copy and/or proprioceptive information related to the pending limb motion (Hwang et al., 2003; van Donkelaar et al., 2004; Buneo and Andersen, 2006; Avillac et al., 2007) and contribute to the dynamic remapping of the spatial representation of each arm (Bolognini and Maravita, 2007; Kavounoudias et al., 2008). The neural processes accounting for these interactions most likely are distributed across a number of cortical and subcortical nodes. Accumulating evidence suggests that one of these nodes is centered in the posterior parietal cortex (PPC). Brain imaging studies have demonstrated activation in the PPC during TOJ tasks (Davis et al., 2009) and that transcranial magnetic stimulation (TMS) to this region disrupts TOJ decisions (Woo et al., 2009). To determine its role in movement planning, Vesia and colleagues (2008) used TMS to disrupt the dorsal-lateral PPC during reaching movements generated with varying degrees of visual feedback. They found that when vision was restricted to the final endpoint of the movement, TMS disrupted the reach vector, implying that signals associated with initial hand position were affected. This result is consistent with the role of the dorsal-lateral PPC in the internal monitoring of movement required for predictive planning. Moreover, Sirigu and colleagues (2004) found that damage to the parietal lobe disrupts the awareness of voluntary action in a manner consistent with altered-state estimates.
Another area that appears to be directly involved in these processes is the cerebellum. Certainly, adaptation studies have demonstrated that the cerebellum plays a critical role in the forward-modeling processes required to learn new sensorimotor mappings. To more directly examine the contribution of the cerebellum to predictive planning processes, Miall and coworkers (2007) used TMS to disrupt the lateral cerebellum during the interruption of a slow lateral upper limb movement to rapidly reach toward a visual target. They found that TMS induced directional errors that were consistent with an estimated hand position that was several tens of milliseconds out of date. They suggest that this evidence is consistent with the cerebellum providing a state estimate of the hand to be used during the predictive planning of the reaching response. To our knowledge, this study represents the clearest attempt to map onto a specific brain structure the time course of predictive planning during the period leading up to movement onset.
In conclusion, we have developed a unique dual-task protocol that allows us to directly probe the temporal dynamics of state estimation during the planning process before limb movement. Our results show that the predictive processes captured in the TOJ error rates slowly evolve over the ∼300–400 ms planning period. In particular, TOJ decisions based on stimuli that are presented early in the planning process (i.e., with small SOAs) have error rates only slightly different from those that occur when the hands are stationary and in the same configuration as that at the start of the movement. By contrast, the error rates of TOJ decisions driven by stimuli presented late in the planning process (i.e., with large SOAs) are similar to those generated when the hands are stationary and in the opposite configuration to that at the start of the movement.
Footnotes
This work was supported by the Graduate School at the University of Oregon (R.H.) and the Elsass Foundation, the Nordea Denmark Foundation, the Knud Højgaard Foundation, and the Oticon Foundation (A.R.R.).
The authors declare no competing financial interests.
- Correspondence should be addressed to Paul van Donkelaar, Department of Human Physiology, University of Oregon, 122C Esslinger Hall, Eugene, OR 97403-1240. paulvd{at}uoregon.edu
This article is freely available online through the J Neurosci Open Choice option.
