Abstract
Numerous studies have shown that people are adept at learning novel object dynamics, linking applied force and motion, when performing reaching movements with hand-held objects. Here we investigated whether the control of rapid corrective arm responses, elicited in response to visual perturbations, has access to such newly acquired knowledge of object dynamics. Participants first learned to make reaching movements while grasping an object subjected to complex load forces that depended on the distance and angle of the hand from the start position. During a subsequent test phase, we examined grip and load force coordination during corrective arm movements elicited (within ∼150 ms) in response to viewed sudden lateral shifts (1.5 cm) in target or object position. We hypothesized that, if knowledge of object dynamics is incorporated in the control of the corrective responses, grip force changes would anticipate the unusual load force changes associated with the corrective arm movements so as to support grasp stability. Indeed, we found that the participants generated grip force adjustments tightly coupled, both spatially and temporally, to the load force changes associated with the arm movement corrections. We submit that recently learned novel object dynamics are effectively integrated into sensorimotor control policies that support rapid visually driven arm corrective actions during transport of hand held objects.
SIGNIFICANCE STATEMENT Previous studies have demonstrated that the motor system can learn, and make use of, internal models of object dynamics to generate feedforward motor commands. However, it is not known whether such internal models are incorporated into rapid, automatic arm movement corrections that compensate for errors that arise during movement. Here we demonstrate, for the first time, that internal models of novel object dynamics are integrated into rapid corrective arm movements made in response to visuomotor perturbations that, importantly, do not directly perturb the object.
Introduction
When transporting a grasped object with familiar dynamics, people modulate grip force in phase with, and hence in anticipation of, movement-dependent load forces, allowing them to prevent the object from slipping while at the same time avoiding unnecessarily high grip forces (Flanagan and Wing, 1993, 1995; Flanagan et al., 1993; Flanagan and Tresilian, 1994). Moreover, people quickly learn to modulate grip force in phase with the load force when transporting an object with unfamiliar movement-dependent dynamics (Flanagan and Wing, 1997; Flanagan et al., 2003; Danion et al., 2012, 2013). Thus, the brain quickly attains and makes use of internal models of the dynamics of grasped objects, as well as the dynamics of the limb, to predict the consequences of arm motor commands on the load forces acting at the object–hand interface (Kawato, 1999; Wolpert and Flanagan, 2001; Bursztyn et al., 2006; Flanagan et al., 2006; Ahmed et al., 2008; Wolpert and Flanagan, 2010; Franklin and Wolpert, 2011; Wolpert et al., 2011).
In the current study, we asked whether newly learned internal models of object dynamics are accessible for the control of rapidly elicited corrective motor responses that quickly (within 105–180 ms) update movement trajectories during visually guided reaching when the target or the viewed hand suddenly changes its position (Prablanc and Martin, 1992; Desmurget et al., 1998; Saunders and Knill, 2003, 2004; Franklin and Wolpert, 2008; Gomi, 2008; Oostwoud Wijdenes et al., 2011). These rapid responses, which can occur without awareness of the perturbation (Goodale et al., 1986; Saunders and Knill, 2003), are often termed automatic responses to distinguish them from slower, clearly volitional corrections.
To answer our question, we designed a reaching task in which the participant moved a grasped object, attached to a lightweight robotic device, to visual targets. The robot could apply a novel force-field to the object such that corrective arm movements, elicited by small displacements of either the target or cursor representing the grasped object, resulted in substantial increases or decreases in load force. Evidence that these visual perturbations give rise to rapid corrective movements associated with coupled changes in grip and load force would indicate that knowledge of the object dynamics can be incorporated in fast visuomotor responses so as to maintain grasp stability. However, if the sensorimotor system is not able to rapidly generate such coupled grip and load force changes, we might expect one of two possible outcomes. First, corrective arm movements could be delayed so as to allow time for the implementation of coordinated changes in grip and load force. Second, corrective arm movements could be generated rapidly with grasp stability preserved by maintaining an increased grip force throughout the movement. In the later situation, reactively driven grip force changes based on peripheral afferent information, which lag behind load force changes by ∼0.1 s, could help ensure grasp stability during the later part of the corrective action (Johansson and Westling, 1988b; Johansson et al., 1992a, 2b; Blakemore et al., 1998).
Materials and Methods
Subjects
Seven men and six women between 19 and 32 years of age (mean 26.5 years) participated after providing written, informed consent. All participants self-reported having normal or corrected-to-normal vision, being right handed, and being neurologically healthy. The experimental protocol was approved by the General Research Ethics Board at Queen's University in compliance with the Canadian Tri-Council Policy on Ethical Conduct for Research Involving humans. The experimental session lasted ∼1 h. Participants were compensated $10 per hour of participation.
Apparatus and stimuli
Participants performed an object transport task that required them to move an object, grasped between the thumb and index finger, from a start position to a visual target in a horizontal plane. The object had two axially centered parallel circular grip surfaces (2.5 cm in diameter) covered in sandpaper and separated by 6.4 cm. Each surface was a circular cap mounted on a six-axis force-torque transducer (Nano force/torque, ATI Industrial Automation) that measured linear forces (0.05 N resolution) and torques in three dimensions. The caps could freely rotate, on ball bearings, about the long axis of the object (running through the centers of the two grip surfaces).
Participants were instructed to hold the object such that the grip surfaces were in a horizontal plane, with the index finger and the thumb contacting the top and bottom surfaces, respectively. Thus, the grasp axis was orthogonal to the plane of movement, allowing us to dissociate movement-dependent horizontal load forces tangential to the grasp surfaces from vertical grip forces normal to the grip surfaces.
The center of the object was attached to a lightweight robotic device (Phantom 3.0 haptic interface, Sensable) through a two-joint linkage. The combination of this linkage and the freely rotating caps allowed full 3D rotation of the object. Optical encoders in the robotic device recorded the center position of the object with a spatial accuracy of 0.1 mm. Two air-sleds that moved across a horizontal glass surface with near-zero friction supported the arm. These sleds were fitted with cuffs with one positioned slightly below the elbow and one at the wrist.
A display system consisting of a 30-inch computer monitor (frame rate of 60 Hz and a spatial resolution of 1280 × 800 pixels) and a mirror was used to present visual images (Fig. 1A). The mirror was positioned halfway between the monitor and the plane of hand movement such that these images appeared in the hand movement plane. In each trial, the participant moved a circular cursor representing the position of the object (yellow, 1 cm in diameter) from a circular start position (green, 1 cm in diameter), located 30 cm in front of the participant at midline, to a circular target (blue, 1.6 cm in diameter) located, a various angles, 20 cm from the start position. A gray arc occluded the view of the object cursor while it passed beneath (Fig. 1B). The arc had an inner radius of 2.5 cm, referenced to the start position, and an outer radius of 5.5 cm and spanned from −55° to 55°.
The robotic device created a rotary position-dependent horizontal load force applied by the object to the hand (Fig. 1B, red arrows and red shaded region). The load was directed 90° counterclockwise from the vector from the start position to the object. The magnitude of the force (F) was determined by the following equation:
where K = 0.12 N/mm, R is the distance (in mm) from the start position to the object position, and θ is the angle (in degrees) of the object (Fig. 1B). That is, for θ > −5°, the magnitude of the load force was proportional to the arc length, from −5° to θ, of a circle with radius R centered at the start position. For θ ≤ −5°, no load force was applied.
Procedure
Task.
Each trial began with the presentation of the start position, occluder, and the target; the object did not apply a load on the hand at this time. The participant was first required to move the object to the start position, at which time the force field was activated. After a delay of 150 ms, the target changed from blue to red, which instructed the participant to move to the target when ready. The participant was encouraged to reach the target between 450 and 750 ms after movement onset and, at the conclusion of each trial, text was displayed indicating whether the movement was “too fast,” “too slow,” or “good.” In addition, a tone was delivered 600 ms after movement onset, and the participant was instructed to time the movements such that the cursor arrived at the target simultaneously with the tone.
Training block.
The training block was designed to give participants experience with the task and an opportunity to learn the force field applied to the hand via the object. In each trial, a target was presented at one of 10 possible locations, 20 cm from the start position at angles of −12.5°, −9.7°, −6.9°, −4.2°, −1.4°, 1.4°, 4.2°, 6.9°, 9.7°, and 12.5° (Fig. 1B). The three leftward targets fell within the no-force region and the remaining 7 fell within the force region. The load forces when the object was located at these targets ranged from 0 to 7.3 N. Each target was presented 15 times in random order for a total of 150 training trials.
Test block.
In the test block, only two targets were used: a left target at −10°, where no load force was applied by the robot when the object was positioned at the target, and a right target at 5°, where the load force applied by the robot was 4.19 N when the object was positioned at the target (Fig. 1C, black targets). The test block contained 10 different trial types: 8 that involved visual perturbations and 2 that did not. The eight perturbation trial types included all combinations of target (−10° and 5°), perturbation type (cursor and target displacements), and perturbation direction (left or right). The two unperturbed trial types served as control movements directed to the two targets. All conditions were randomized throughout the test block with each condition presented 50 times yielding 500 trials.
On perturbation trials, the position of either the target or the object cursor was displaced 1.5 cm orthogonal to the vector from the start position to either the cursor or the target, respectively, and this displacement could be either leftward or rightward (the blue and red circles in Fig. 1C show these displacements for the right 5° target). The cursor displacements occurred when the object cursor passed beneath the occlude; therefore, information related to the cursor displacement was available only once that cursor had emerged from the occluder after passing beneath. Self-reports following the experimental session indicated that none of the participants was aware of the cursor displacements. Target displacements were triggered when the cursor emerged from beneath the occluder; and, on average, the target displacement occurred when the cursor was 0.9 cm beyond the outer radius of the occluder. (This distance varied slightly depending on the speed of the movement.) All participants were aware of the target displacements. Leftward cursor displacements and rightward target displacements required a rightward movement of the grasped object to bring the cursor to the target, whereas rightward cursor displacements and leftward target displacements required a leftward object movement. All of the figures are color coded, with red representing conditions involving rightwards movement responses and blue representing conditions involving leftwards movement responses.
Analysis
Position and force signals were sampled at 1 kHz and then digitally smoothed, off-line, using a fourth-order, zero-phase lag Butterworth filter with a cutoff frequency of 14 Hz. Using the smoothed position signals, we computed the component of object velocity in the direction of the vector from the start position to the target as well as the object velocity perpendicular to that direction. Grip force was calculated as the average normal force applied by the thumb and index finger to the two force sensors. Load force was calculated using the vector sum of the tangential forces applied by the thumb and index finger. For the analysis of responses to cursor jumps, perturbed and unperturbed trials were aligned in time relative to the time at which the cursor fully emerged from beneath the occluder. For the analysis of responses to target jumps, perturbed and nonperturbed trials were aligned to the time of the target jump, which was triggered by the emergence of the cursor from beneath the occluder.
Onset latencies of corrective arm movement responses, relative to the time of the perturbation, were determined using the object velocity in the direction perpendicular to vector from the start position to the target direction (Franklin and Wolpert, 2008; Knill et al., 2011). For each perturbation type, target, and participant, we compared: (1) leftward and rightward displacement trials, (2) leftward displacement trials and control trials, and (3) rightward displacement trials and control trials, to determine the point of divergence. This involved two steps. First, for each comparison, we performed a running independent samples t test and determined the point when the p value first dropped <0.001 and remained <0.001 for 30 ms (Fig. 1G, gray curve and dashed vertical line, which compare mean perpendicular velocity profiles for right and left target jump trials involving the 5° target for a single participant). Second, we searched backward in time (from the p < 0.001 point) to the first minima in the rate of change of the p value (Fig. 1G, black curve and dashed vertical line). The time of this minima was taken as the response time. Similar approaches are standardly used in signal detection and have also been previously used to determine corrective movement response times (Nashed et al., 2012, 2014). For movements directed to the 5° target, where movement corrections gave rise to appreciable changes in load force, we used the same procedure to determine grip force and load force response times; the latter provide another measure of latencies of corrective arm movements (Danion and Sarlegna, 2007).
We excluded trials from our analysis if: (1) the angle from the start position to the handle was not within ±10° of the angle from the start to the target at 25% of the movement amplitude or (2) the cursor did not come within 0.5 cm of the target center within 400–800 ms after movement onset. This resulted in the exclusion of 17% of all trials. We used repeated-measures ANOVA to assess experimental effects with Bonferroni corrected post hoc comparisons. p values <0.05 were considered significant.
To assess whether participants successfully maintained the object in a vertical orientation during corrective responses, for all trials involving the right (5°) target and either a cursor or target perturbation (i.e., all trials with corrective responses associated with significant load force and, as we will demonstrate, grip force, changes), we computed three correlations: (1) the correlation between the normal forces recorded from the two sensors (contacted by the thumb and index finger); (2) the correlation between the tangential forces recorded from the two sensors; and (3) the correlation between the load force (computed from the tangential forces at both sensors, see above) and the commanded load force sent to the Phantom robot. Importantly, if the object remains vertical, we would expect all of these correlations to be very close to 1. Conversely, if significant tilting occurred, these correlations should markedly decrease. The average r values, based on participant means, for these three correlations were 0.999 (SE <0.001), 0.985 (SE = 0.003), and 0.988 (SE = 0.001), respectively. Finally, because the surface caps mounted on the force sensors were free to rotate around the long (i.e., vertical) axis of the object, participants could not generate torques about the long axis of the object.
Results
Training trials
Each participant first completed 150 training trials with 10 different target locations (10 targets × 15 reaches). In the first few trials, reach trajectories in the force field region were markedly deviated. In agreement with previous results showing that people can readily learn novel position-dependent force fields (Flanagan and Wing, 1997; Tong et al., 2002; Sing et al., 2009; Danion et al., 2013), our study participants quickly adapted to the force field applied to the grasped object so as to produce approximately straight-line movements to all targets (Fig. 1D, which shows average object paths toward each of the 10 different targets for a representative participant.) Likewise, participants quickly learned to modulate grip force in synchrony with load force. To assess the coupling of grip and load force during the training trials, we computed the mean correlation coefficient between grip and load force for each participant. Overall, grip and load force was highly correlated, with an average correlation, based on participant means, of 0.97 (SE = 0.005). We also evaluated the average movement time for each participant. On average, participants reached the target in 562 ms (SE = 10 ms), based on participant means.
Apparatus and experimental design. A, Drawing of the experimental setup (for details, see text). B, Configuration of the start position, occluder, and the 10 training targets. Red force vectors, within the red shaded region, indicate the position-dependent force field applied to the grasped object. C, Configuration for the experimental trials with targets at −10° and 5° and the same force field as shown in B (illustrated with fewer red arrows). Shown only for the 5° target, but also present for the −10° target, are the cursor jumps (filled colored circles) and target jumps (open colored circles). Red and blue circles represent perturbations requiring a rightward and leftward movement response, respectively. D, Average object paths toward the 10 training targets for a single participant, with trajectories aligned to the time at which the cursor emerged from the occluder. Shaded regions represent ±1 SD orthogonal to the target direction. E, F, Average object paths for each target and perturbation type for the same participant as in D. Red and blue traces represent perturbation trials requiring a rightward and leftward movement, respectively. Gray traces represent unperturbed trials (with the same unperturbed trials shown in E and F. G, Procedure for calculating arm movement response latency, relative to the time of the perturbation (t = 0), illustrated for a single participant and target jumps involving the 10° target. We first determined when the p value from a running t test comparing the average object velocity perpendicular to the vector from the start position to the target for right (red trace) and left (blue trace) target jumps dropped <0.001 (vertical gray dash-dotted line), and then backtracked to the first minima in the rate of change of the p value (vertical black dashed line) .
Arm movement corrective responses
To reach the target, participants corrected for the visual perturbations caused by sideways jumps of the target and/or the cursor by moving the grasped object either leftward or rightward (relative to the target direction), depending on perturbation type (Fig. 1E,F). To assess the kinematics of the corrective arm responses, we computed the velocity of the object perpendicular to a vector from the start position to the target (i.e., the component of velocity in line with the perturbation) (Fig. 2A–D, which shows results from a representative participant). To quantify the speed of the corrective movement, we determined, for each participant, perturbation type, target, and response direction, the absolute difference between the maximum of the average perpendicular velocity in perturbed trials minus the average perpendicular velocity in unperturbed trials, taken at the same time (Fig. 2B, dash-dotted red and blue vertical lines). A three-way repeated measures ANOVA uncovered a main effect of perturbation type (cursor, target; F(1,12) = 24.6, p < 0.001), and target (−10°, 5°; F(1,12) = 5.4, p < 0.05) but failed to show an effect of response direction (leftward, rightward). The absolute difference scores were greater for target displacements (mean ± SE, 7.4 ± 0.2 cm/s) compared with cursor displacements (6.3 ± 0.2 cm/s), and greater for movements toward the left target (7.0 ± 0.3 cm/s) compared with movements toward the right target (6.6 ± 0.2 cm/s). The ANOVA also uncovered a perturbation type by response direction interaction (F(1,12) = 5.2, p < 0.05), where the effect of perturbation type was slightly larger for leftward versus rightward responses. There were no other interactions.
Object velocity profiles for each target and perturbation type for a single participant. A–D, Average perpendicular velocity profiles, with trials aligned in time to the perturbation (t = 0), for unperturbed trials (gray) and perturbed trials requiring a leftward (blue) or rightward (red) correction. Positive indicates rightward relative to the movement direction. Dashed blue, red, and gray vertical lines indicate the times at which the blue and gray, red and gray, and blue and red velocity profiles differed, respectively (see Materials and Methods), with these times included as color-coded text. Solid blue, red, and gray vertical lines indicate the mean times the participant reached the target for each trial type. B, Dash-dotted red and blue vertical lines indicate, for perturbations requiring right and left responses, respectively, the difference between the peak average velocity in perturbation trials and the average velocity in unperturbed trials. Before averaging, profiles were aligned to perturbation onset, or the corresponding time in unperturbed trials (dash-dotted lines). The heights of the shaded regions represent ±1 SD. E–H, Corresponding plots for velocity in the same direction as the vector connecting the start position and target.
As illustrated by the representative participant shown in Figure 2, response onset latencies were short, consistent with automatic responses. Figure 2A–D (blue, red, and gray vertical dashed lines) shows the latencies obtained by comparing the leftward responses with baseline, the rightward responses with baseline, and the leftward and rightward responses, respectively. Overall, this participant responded earlier to cursor jumps compared with target jumps (see values in the figure) but exhibited similar latencies for the left and right targets and for leftward (red) and rightward (blue) responses. This pattern of results was similar to that observed at the group level, which will be described below.
The visual perturbations had relative little effect on the object velocity in the direction of the target (Fig. 2E–H), indicating that the corrective responses were largely in the direction of the required adjustment to the displacement of the target or cursor.
Grip and load force responses to visual perturbations
Figure 3 shows, for a representative participant, average grip force (Fig. 3A–D) and load force (Fig. 3E–H) traces for perturbed and unperturbed trials, with separate traces shown for each target and perturbation type. To quantify the magnitude of grip and load force changes in response to the perturbations, we computed, for each trial, the mean force over the last 50 ms before reaching the target (i.e., the final force levels observed as the object arrived at the target). For each participant, perturbation type, target, and response direction, we then computed the difference between the average mean force in perturbation trials and the average mean force in unperturbed trials. These difference scores enabled us to examine how grip and load force, in perturbation trials, changed relative to baseline.
Grip and load force profiles for each target and perturbation type for a single participant. A–D, Average grip force profiles for unperturbed trials (gray) and perturbed trials requiring a leftward (blue) or rightward (red) correction. Dashed blue, red, and gray vertical lines, shown for the right target only, indicate the response latencies obtained by comparing the leftward and baseline, rightward and baseline, and leftward and rightward responses, respectively. Solid blue, red, and gray vertical lines indicate the mean times the participant reached the target for each trial type. Before averaging, profiles were aligned to perturbation onset, or the corresponding time in unperturbed trials (t = 0). The heights of the shaded regions represent ±1 SD. E–H, Corresponding plots for load force.
A three-way repeated-measures ANOVA using the load force difference scores revealed, as expected, a main effect of response direction (F(1,12) = 2724, p < 0.001). There was also a response direction by target interaction (F(1,12) = 4261.7, p < 0.001). For the right target, perturbations requiring rightward and leftward responses resulted in large increases (1.7 ± 0.04 N) and decreases (−2.0 ± 0.04 N) in load force, whereas minimal changes in load force were observed during perturbation trials involved the left target. There was no effect of target or perturbation type and no other interactions.
A corresponding analysis for grip force yielded similar results: a main effect of response direction (F(1,12) = 198, p < 0.001) and a response direction by target interaction (F(1,12) = 181, p < 0.001). On average, for the right target, participants increased their grip force by 2.3 N (SE = 0.2 N) and decreased their grip force by 1.6 N (SE = 0.1 N), relative to baseline, during rightward and leftward movement corrections, respectively. This contrasted with an increase of 0.4 N (SE = 0.1 N) and a decrease of 0.2 N (SE = 0.1 N) for rightward and leftward corrective responses, respectively, for movements toward the left target. There was no effect of target or perturbation type and no other interactions.
Kinematic and force corrective response onset latencies
To assess and compare kinematic (i.e., perpendicular velocity of the object), grip force, and load force response latencies, we focused on movements directed to the right (5°) target, where corrective responses resulted in appreciable changes in grip and load force. Figure 4A–C (bars) shows these response latencies, averaged across participants, for each perturbation type (i.e., cursor or target) and latency measure (i.e., left response vs baseline, left response vs right response, and right response vs baseline). Figure 4C (circles) also shows the movement response latency for left (−10°) target.
Grip force and movement response latencies. A–C, Average response latencies, based on participant means, for grip force, load force, and perpendicular velocity for trials involving the right target. Colored bars represent the three latency measures calculated by comparing leftward response and unperturbed trials (blue), rightward and leftward response trials (gray), and rightward response and unperturbed trials (red). Error bars indicate ±1 SE. C, Circles represent the movement response latencies for the left target. D, Grip force response latency versus movement response latency, where the latter was computed from the average of the load force and perpendicular velocity latencies. Each circle represents a single participant, and all three latencies are based on averaging the three latency measures. Open and filled circles represent latencies for the cursor and target jump trials, respectively. Dashed line indicates the unity line. Solid line indicates the regression slope computed using all points.
To quantify response latency effects, we performed a three-way repeated-measures ANOVA with response measure (grip force, load force, perpendicular velocity), perturbation type (cursor, target), and latency measure (left response vs right response, left response vs baseline, right response vs baseline) as factors. This analysis revealed a main effect of response measure (F(2,24) = 27, p < 0.001). Post hoc comparisons showed grip force latencies (179 ± 4 ms) were slightly but significantly (p < 0.05) longer, than both load force (163 ± 3 ms) and perpendicular velocity (157 ± 3 ms) latencies, but that there was no significant difference between load force and perpendicular velocity latencies. Slightly longer latencies observed for grip force compared with load force may arise, at least in part, because of greater trial-to-trial variability in grip force amplitude compared with load force amplitude (Fig. 3). The analysis also uncovered a main effect of perturbation type (F(1,12) = 108, p < 0.001), with responses to cursor displacements (156 ± 3 ms) preceding responses to target displacements (176 ± 3 ms), on average. We measured the response to cursor displacements from the time at which the cursor fully emerged from the occluder. Importantly, the average duration, based on participant means, between when the leading edge of the cursor first appeared and when the cursor fully appeared was 22.4 ms (SE = 0.9 ms), which is very close to the response time advantage we see for cursor displacements over target displacements (20 ms). Thus, even though previous work has shown that responses to cursor displacements are quicker than responses to target displacements (Brenner and Smeets, 2003), the difference found in the current study should be treated with caution, and the important point is that responses to both types of perturbation were rapid. Finally, we found a main effect of latency measure (F(2,24) = 4.1, p < 0.05). Post hoc comparisons showed that latencies obtained by comparing leftward versus rightward responses (158 ± 3 ms) were shorter (p < 0.05) than those obtained by comparing rightward versus baseline responses (172 ± 6 ms) and marginally shorter (p = 0.08) than those obtained by comparing leftward versus baseline responses (169 ± 3 ms). However, there were no differences between latencies obtained by comparing leftward versus baseline responses and those obtained by comparing rightward versus baseline responses, indicating that participants responded equally quickly to leftward and rightward perturbations (i.e., increases and decreases in force). The ANOVA did not uncover any interactions among response measure, perturbation type, or latency measure. Finally, we performed a separate paired t test to compare movement response latencies, derived using perpendicular velocity, for the two targets and did not find a significant difference.
To assess the temporal coupling between grip force and arm movement responses, we examined the correlation, across participants and perturbation type, between grip force response latency (based on the average of the three latency measures) and the average of the perpendicular velocity and load force response latencies (each based on the average of the three latency measures), both of which measure the arm movement response. The slope and intercept of the best fit regression line (Fig. 4D) were 1.1 and 0.005 ms, respectively, and the correlation was 0.75 (p < 0.001). Thus, although the grip force response slightly lagged the arm movement response, on average these responses were tightly coupled in time.
Grip-load force coordination during corrective actions
Figure 5A, B illustrates the coordination between grip and load force in right target trials with cursor and target jumps, respectively. Separate traces are shown for perturbation requiring rightward (red) and leftward (blue) movement responses, and both figures show unperturbed trials (gray traces) for comparison. Overall, grip force was modulated in phase with load force both during the initial, uncorrected portion of the movement and during the corrective portion of the movement. We quantified the coupling between grip and load force by determining, for each participant, the mean correlation coefficient for unperturbed trials and for cursor and target jump trials requiring leftward or rightward corrections. Planned comparisons showed no significant differences between the unperturbed trials and any of the perturbation trials (p > 0.05 in all four cases). Overall, the average correlation coefficient was 0.97 (SE = 0.004).
Coordination of grip and load force. A, B, Average grip force (GF) plotted against average load force (LF) for movements to the right target involving either cursor (A) or target (B) jumps. Average force based on participant means. Blue and red traces represent trials requiring rightward and leftward movement corrections, respectively. Gray traces represent unperturbed trials. Open squares represent the average point at which the perturbation occurred. Open circles represent the average point at which participants reached the target. C, D, Grip and load forces, averaged over the 50 ms before reaching the target, as a function of the arc length of the target position relative to −5°. Each circle represented an average based on participant means. Blue and red circles represent perturbation trials requiring leftward and rightward responses, respectively. Gray circles represent unperturbed trials. Black circles represent training trials. Error bars indicate ±1 SE.
Figure 5C, D shows, for cursor and target jumps, respectively, grip (filled circles) and load (open circles) forces averaged over the last 50 ms of the movement before reaching the target, as a function of the arc length of the target position relative to −5° (i.e., the perimeter distance, from −5° to the angle of the target, of a circle, centered at the start position, with a radius equal to the distance from the start position to the target). Each circle represented an average based on participant means. The blue and red circles represent perturbation trials, with either the −5° or 10° target, requiring leftward and rightward responses, respectively; the gray circles represent unperturbed trials, with either the −5° or 10° target; and the black circles represent training trials with each of the 10 training targets. As expected given the force field applied to the object, the final load forces (i.e., when the object reached the vicinity of the target) for all experimental and training trial types fall along the same function. To a first approximation, the same is true of the final grip forces. However, the final grip forces in perturbation trials involving the right target, and especially the perturbation requiring a left response, were slightly greater than would be predicted from the training trials given the arc length. The slightly elevated final grip forces in perturbations requiring a leftward correction can also be appreciated in Figure 5A, B (blue traces). The tendency not to fully decrease grip force during corrections involving a decrease in load force is consistent with previous work on precision grip lifting where the object is lighter than expected (e.g., Johansson and Westling, 1988a; Flanagan and Beltzner, 2000), and is perhaps unsurprising given that a slightly greater grip force does not threaten task success.
Discussion
Our results show that, when reaching with a grasped object, people produce rapid corrective arm movements in response to viewed shifts in target or object position, and that the underlying control processes use recently acquired knowledge about the dynamics of the object. The use of an internal representation of object dynamics was evident from the close coupling (in time, magnitude, and direction) between the changes in load force and the grip force responses supporting grasp stability during corrective movements of the arm. That is, the grip force responses predicted the changes in load force during the corrective arm movements, which required knowledge of object dynamics. The rapid and appropriate grip force responses observed are impressive given that load force changes were linked arbitrarily to the required corrective arm movements via novel and complex object dynamics. That is, participants could not rely on well-established priors linking the visual stimulus to load force change, and instead had to make use of an internal model of the complex force field, involving forces that depended on the distance and angle of the object from the start position, learned through recent experience (i.e., during the training trials).
Our finding that adaptable internal representations of object dynamics support rapid corrective responses to visual perturbations during object transport complements previous findings indicating that rapid responses to mechanical arm perturbations during reaching relies on an adaptable internal model of the arm itself (Kurtzer et al., 2008; Cluff and Scott, 2013). These previous findings, together with the current results, highlight an important link between feedforward and feedback control mechanisms where learning of automatically triggered corrective actions that incorporate knowledge about both arm dynamics and dynamics of handheld objects appears to go hand in hand with learning of goal-oriented feedforward commands. However, learning a novel mapping between visual targets and required motor commands does not always translate into this knowledge being integrated into rapid corrective responses. Thus, people can learn to launch appropriately directed reaching movements under a visuomotor reversal (which causes a severe spatial incongruency between target direction and required arm movement direction) but still do not generate rapid, appropriate corrections in response to target jumps (Gritsenko and Kalaska, 2010).
Even though trials with visuomotor perturbations frequently occurred, participants did not simply increase grip force throughout the movement, a strategy that would prevent slip but lead to unnecessarily high grip forces (Johansson and Westling, 1988b; Cole and Johansson, 1993). In contrast, an increase in baseline grip force would be expected in the face of frequent mechanical (as opposed to visual) perturbations that directly threaten grasp stability, and recent work has shown that the magnitude of grip force is scaled to the experienced statistics of such perturbations (Hadjiosif and Smith, 2011). Neither did we observe reactively driven grip force changes triggered by the changes in load forces. That is, we did not see grip force responses that lagged behind the load force changes at latencies (∼0.1 s) compatible with such reactive responses (Johansson and Westling, 1988b; Johansson et al., 1992a). Rather, grip force was modulated in parallel with load force and predicted the magnitude and direction of load force changes throughout the corrective action.
The onset latencies of the corrective arm actions we observed were in the range of those previously reported for use of vision in automatic online control of reaching movements, as probed by small-amplitude target or cursor displacements (i.e., 120–180 ms) (Prablanc and Martin, 1992; Desmurget et al., 1999; Pisella et al., 2000; Sarlegna et al., 2003; Saunders and Knill, 2003, 2004; Franklin and Wolpert, 2008; Gritsenko et al., 2009). These rapid responses appear to constitute a component of the sensorimotor control policy implemented during reaches, which cannot only counter unexpected perturbations but also help handle errors due to sensory and/or motor noise (Harris and Wolpert, 1998; Todorov and Jordan, 2002; Hamilton et al., 2004; Scott, 2004; Todorov, 2004; van Beers, 2009; Pruszynski and Scott, 2012).
The onset latencies of the corrective arm movements we observed in response to target jumps (∼150 ms) was considerably shorter than the latency of corresponding responses (284 ms) reported in a previous study dealing with grip force adjustments during transport of an object grasped between the index finger and thumb (Danion and Sarlegna, 2007). Various experimental factors might have contributed to this difference. First, the target jumps used in the previous study were much larger (8 cm as opposed to 1.5 cm) and occurred in the principal direction of movement (i.e., backward or forward with reference to the initial target) rather than laterally as in the present study. Thus, large corrections, some of which involved a reversal of the movement direction, were required in the previous study. Moreover, in the previous study, the grasped object was attached to a rather stiff elastic cord requiring quite high load forces at the three target positions (13, 16, and 19 N, respectively). This, in turn, required grip forces in the 30–50 N range, which is approaching the maximum force that people generally can generate between the tips of the index finger and thumb (Mathiowetz et al., 1985). The large-onset latencies reported in the previous study indicate that the observed corrective movements were voluntary, involving “reprogramming” of movement trajectories, rather than rapid, more automatic responses. In addition to the long latency, this interpretation is supported by the fact that the movement time in trials with target jumps was substantially prolonged (by ∼300 ms and ∼225 ms for backward and forward jumps, respectively) compared with unperturbed trials, which was not the case with the corrections observed in the present study (Fig. 2E–H, compare gray traces with the blue and red traces).
It has recently been shown that arm movement corrections to target jumps, made while standing participants perform target-directed reaching movements, are preceded by postural adjustments in the legs (Leonard et al., 2011). Interestingly, grip force adjustments providing grasp stability during goal-directed object handling can be viewed as postural adjustments that provide stability during goal-directed movement (Wing et al., 1997). Hence, these findings, combined with the current results, indicate that the control of target-directed arm movements are supported by multiple rapidly operating neural mechanisms that serve to defend attainment of the task goal.
In conclusion, we have shown that coordinated grip force responses are linked to rapid movement responses to visuomotor perturbations applied during reaching movements. This result indicates that participants could not only learn, and make use of, knowledge of novel object dynamics for feedforward control but could also implement a sensory feedback control policy that incorporated this knowledge.
Footnotes
This work was supported by the Canadian Institutes of Health Research, the Wellcome Trust, the Human Frontiers Science Program, the Royal Society, the Swedish Research Council, and the Strategic Research Program in Neuroscience at the Karolinska Institute. We thank Martin York, Sean Hickman, and Mike Lewis for technical support.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. J. Randall Flanagan, Department of Psychology, Centre for Neuroscience Studies, Queen's University, Kingston, Ontario K7L 3N6, Canada. flanagan{at}queensu.ca
