Previous Article | Next Article 
The Journal of Neuroscience, September 1, 2002, 22(17):7721-7729
A Real-Time State Predictor in Motor Control: Study of Saccadic
Eye Movements during Unseen Reaching Movements
Gregory
Ariff,
Opher
Donchin,
Thrishantha
Nanayakkara, and
Reza
Shadmehr
Laboratory for Computational Motor Control, Department of
Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland
21205
 |
ABSTRACT |
Theoretical motor control predicts that because of delays in
sensorimotor pathways, a neural system should exist in the brain that
uses efferent copy of commands to the arm, sensory feedback, and an
internal model of the dynamics of the arm to predict the future state
of the hand (i.e., a forward model). We tested this theory under the
hypothesis that saccadic eye movements, tracking an unseen reaching
movement, would reflect the output of this state predictor. We found
that in unperturbed reaching movements, saccade occurrence at any time
t consistently provided an unbiased estimate of hand
position at t + 196 msec. To investigate the behavior of
this predictor during feedback error control, we applied 50 msec
random-force perturbations to the moving hand. Saccades showed a sharp
inhibition at 100 msec after perturbation. At ~170 msec, there was a
sharp increase in saccade probabilities. These postperturbation
saccades were an unbiased estimator of hand position at saccade time
t + 150 msec. The ability of the brain to guide saccades
to the future position of the hand failed when a force field
unexpectedly changed the dynamics of the hand immediately after
perturbation. The behavior of the eyes suggested that during reaching
movements, the brain computes an estimate of future hand position based
on an internal model that relies on real-time proprioceptive feedback.
When an error occurs in reaching movements, the estimate of future hand
position is recomputed. The saccade inhibition period that follows the
hand perturbation may indicate the length of time it takes for this
computation to take place.
Key words:
forward models; reaching movements; eye movements; saccades; oculomanual control; feedback delay; sensory delay
 |
INTRODUCTION |
When a reaching movement starts, the
neural commands to the arm are "feedforward" in the sense that they
rely on an internal model that predicts forces necessary to initiate
the movement (Shadmehr and Mussa-Ivaldi, 1994
; Thoroughman and
Shadmehr, 1999
). However, as the movement proceeds, these signals are
augmented by "feedback" components that take into account sensory
information from the moving limb. If a perturbation displaces the hand,
compensatory commands are produced via a "short-loop" feedback
mechanism with delays of 30-50 msec (Ghez and Shinoda, 1978
) and a
"long-loop" mechanism with delays of ~150 msec (Gielen et al.,
1988
; Petersen et al., 1998
). In theory, these delays can destabilize
the limb. The importance of proper function of the long-loop error
feedback control system in humans is illustrated in Huntington's
disease, in which damage to the basal ganglia accompanies abnormal
long-latency motor responses to somatosensory stimuli (Noth et al.,
1985
; Thompson, 1988
; Thilmann et al., 1991
). In these patients,
movements often begin normally, but slight errors result in motor
responses that produce jerky movements (Smith et al., 2000
). How does
the normal brain perform error feedback control despite long delays in
sensory feedback?
Theory suggests that the brain may cope with sensory feedback delays by
relying on an internal, neural model that predicts the effects of motor
commands on the arm (Miall et al., 1993
; Wolpert and Ghahramani, 2000
).
This is a "forward model" because it translates motor commands into
predictions of future system state, modeling the forward dynamics of
the system (Jordan and Rumelhart, 1992
). To make this prediction, the
forward model requires sensory feedback from the limb and a copy of
currently planned motor commands (efferent copy). The state predictor
might generate its output via an integration of the inputs through a
model of the physical dynamics of the limb (Bhushan and Shadmehr,
1999
). If such a system is used, motor commands that respond to a
sensed error can be based on an estimate of the future state of the
limb, rather than the potentially destabilizing alternative of where the limb was when error was sensed.
A crucial component of this theory is the idea that the brain can use
feedback and efferent copy to compute the future state of the limb. To
test this idea, experiments have been designed that ask subjects to
program motor commands to one arm as a function of their estimate of
the state of the other arm (Blakemore et al., 1998
; Witney et al.,
1999
). Here, we approached the problem differently by asking whether
subjects can use their eyes to pinpoint in real time the position of
their unseen, moving hand. We chose eye movements as a proxy for the
hypothesized state estimator because it has been suggested that the
oculomotor system has access to an efferent copy of arm motor commands
during visual tracking (Vercher et al., 1997
). By perturbing the unseen
hand and recording the saccadic response, we hoped to quantify the
estimate of hand positions by the brain during error feedback control.
 |
MATERIALS AND METHODS |
Subjects (n = 6) held the handle of a robotic
arm (Shadmehr and Brashers-Krug, 1997
) and made reaching movements in
the horizontal plane to targets that appeared at a distance of 10 cm in
random directions (Fig. 1). The handle of
the robot housed a force transducer at its base and a high-intensity
light-emitting diode (LED) at its tip. An opaque screen (12×10 inches)
was suspended 0.5 cm above the plane of the tip of the handle. A
liquid-crystal display (LCD) projector suspended from the ceiling
painted this screen. A dark, heavy cloth was draped around the screen
and prevented the subjects from viewing their arms and hands.

View larger version (33K):
[in this window]
[in a new window]
|
Figure 1.
Experimental setup. A, Subjects
used a bite bar and were instructed to try to track their unseen hand
during reaching movement. They were prevented from viewing their hands
and arms, because both arms were covered under a heavy cloth (data not
shown). The right hand held the handle of a robotic arm and moved it in
the horizontal plane. The handle housed a high-intensity LED at its
center and a force transducer at its base. The hand was below an opaque
screen. An LCD projector, held from the ceiling, painted this screen.
B, The task began with the robot bringing the hand to a
random starting position, where a start target (green
square) was displayed. The subject fixated the handle LED. The
LED was turned off, and a movement target was displayed
(yellow square). The subject saccaded to the
target. The movement target was turned off, the handle LED was turned
on, and the movement target was redisplayed. The subject fixated the
LED. A stationary random-dot pattern was displayed and the start target
was removed, signaling the subject to start the reaching movement. As
soon as movement was detected, the handle LED was turned off. At the
completion of the reach, the handle LED was turned on, the random-dot
pattern was removed, and the target square was repainted, providing
feedback to the subject. C, Two example trials. The
component of eye and hand position parallel to the direction of target
is plotted with green and black lines,
respectively. Saccade origin is marked with a red dot,
and saccade end point is marked with a blue dot.
disp., Displacement.
|
|
A trial began with the handle LED on. A start target was displayed in a
randomly selected start position, and the robot moved the handle (and
the subject's hand) to it. After the subject fixated the start
location, the handle LED and start target were extinguished, and a
final target appeared at a randomly selected direction. The subject was
required to fixate the final target for 0.5 sec (while maintaining the
handle in the start position), after which the target was extinguished
and the start target and the handle LED reappeared. The subject
returned fixation to the start target, after which a stationary
random-dot pattern filled the entire screen (~35
dots/in2). Each dot was 0.25 mm2 in size, and its position was randomly
selected for each trial. After subjects fixated the start target for
0.5 sec, the target was extinguished so that only the random-dot
pattern and the LED were visible. This signaled to the subject to begin
movement to the remembered location of the target. As soon as movement
initiation was detected (encoder resolution of handle position was
better than 0.05 mm, and movement initiation was detected using a fixed 0.02 m/sec threshold), the handle LED was extinguished. Therefore, during the movement, the handle LED was off and the target was not
displayed. The only visible information was the pattern of random dots.
After completion of movement, the random dots disappeared, the handle
LED was turned on, and the target reappeared. The target color provided
feedback regarding the timing of the movement, and the position of the
LED with respect to the target provided feedback regarding the accuracy
of the movement. If the hand reached the target in 0.9 ± 0.05 sec, the target exploded and made a pleasing sound. It turned red if
the hand arrived at the target too soon and blue if it arrived too late.
After ~150 unperturbed trials, on pseudorandomly selected trials
(probability of 58%), a 50 msec, 25 N force pulse pushed the hand in
one of two directions perpendicular to the target direction at either
200, 250, 300, or 350 msec after movement initiation. Force-pulse
latency was determined pseudorandomly. As before, no auditory or visual
cues were available in these trials.
We recorded eye position at 100 Hz using an infrared camera and light
source (iView system; SMI Corp., Berlin, Germany) mounted on a
helmet that was itself tracked using a Polhemus tracker (Polhemus Corp., Colchester, VT). To stabilize the recordings, subjects used a
bite bar anchored to the floor. We performed a two-part calibration
procedure. Each block of ~40 trials began with saccades to 24 points
painted on the plane immediately above the hand. For each trial within
the block, we checked on the calibration of the eye system with the
hand position (as reported by the robot) by using the fixation period
when the LED at the handle was on and the fixation period when the
target was painted. Trials were aborted if eyes were not within 7.5 mm
of the presented target. False starts also resulted in the rejection of
that trial.
Subjects received training before experiments to familiarize them with
the setup. In the pre-experiment session, robot motors were always off,
and subjects were asked to try to look at the perceived position of
their hand during unseen reaching movements. A typical subject's
performance began with a single saccade to the remembered location of
the target as the reach took place. With additional practice, they
produced multiple saccades during the reach. Subjects never received
feedback regarding the accuracy of their saccades during the reach. We
began the experiments once the subjects were able to consistently make
at least two saccades per reaching movement. Once the experimental
session began, however, we did not exclude any trials other than those
with false starts or poor calibration.
 |
RESULTS |
We instructed the participants to look at their hand to the best
of their ability without vision of their hand or the target as they
reached to the remembered location of a target (Fig. 1).
We begin by presenting the results of the unperturbed trials. The reach
took place in 0.78 ± 0.18 sec and was accompanied by a sequence
of 3.50 ± 1.1 saccades (mean ± SD). The trajectory of the
hand was usually straight, but it varied in speed from movement to
movement. An example of two trials that differed in speed is shown in
Figure 1C. The timing and placement of saccades appear to
correspond to the trajectory of the hand in each trial. However, the
timing and placement properties of saccades varied greatly across the
movements, resulting in a broad distribution in the probability of
saccades during the reach (Fig.
2A) and a wide
scattering of saccade origins and end points (Fig.
2B).

View larger version (23K):
[in this window]
[in a new window]
|
Figure 2.
Saccades during reaching movements in unperturbed
trials. A, Probability (prob.) of
saccade occurrence calculated in 10 msec time bins as a function of
hand movement time. Each bin indicates the probability that in a single
trial, a saccade would occur at that time bin. Reaches were to various
target directions. B, Mean trajectory of the reach (± 1 SD; yellow line) is shown by the black
line and is represented along directions parallel or
perpendicular to the target direction. Saccades are represented by
vectors of eye position change: this vector has an origin (red
dots) and an end point (blue dots).
C, Average timing, origins, and end points of the first
three saccades (red and blue lines; ± 1 SD). The SD on timing of saccade end points is identical to the SD of
saccade origin and is not shown for clarity. The green
line is the average eye position across all trials and
corresponds to the continuous representation of the discrete saccade
data. Black lines show average hand position. A saccade
end point is occasionally not equal to the origin of the subsequent
saccade because of small amounts of smooth pursuit.
Dir., Direction.
|
|
Because there were, on average, approximately three saccades during the
reach, we began our analysis by considering the position of these
saccades with respect to the hand. On average, saccades were initiated
shortly after the hand passed the point of eye fixation and were made
to a point that led the hand position (Fig. 2C). The end
point of the first saccade was, on average, ahead of the hand toward
the target (Fig. 2C). This eye position was maintained
(although there were sometimes very small amounts of smooth pursuit,
usually <4 mm) until the hand passed the point of fixation by ~7 mm,
at which point a second saccade was generated. Note that the hand is
not visible during the movement. After hand position progressed beyond
the current eye position, a third saccade was generated. The data in
Figure 2C describe the temporal and spatial distribution of
all first, second, and third saccades, regardless of the number of
saccades that there might have been in a given reaching movement.
However, there were many trials with only two saccades and other trials
with four saccades (the probability that a trial would have a given
number of saccades is shown in Fig.
3A). It is not clear whether
the second saccade in a two-saccade trial should be combined with the
second saccade in a four-saccade trial. This variability in the number
of saccades per trial results in the counterintuitive observation that
in Figure 2C, the average origin of the third saccade is
actually slightly less than the average end point of the second
saccade. Indeed, the broad distribution of saccade timings (170-220
msec in Fig. 2C) suggests that they were not generated in a
rigid temporal pattern. To investigate this further, we examined
saccade generation probability as a function of time for trials with
two, three, and four saccades separately. In each type of trial, we
found a broadly distributed probability function peaking at ~280
msec, with little evidence of periodicity (Fig. 3B),
reinforcing the idea that saccade timing is highly variable and
saccades are not generated with rigid temporal structure.

View larger version (19K):
[in this window]
[in a new window]
|
Figure 3.
Saccades during reaching movements in unperturbed
trials. A, Percentage of reaching movements that
accompanied a given number of saccades. B, Probability
(prob.) of saccade occurrence calculated in 10 msec time bins for reaching movements that had two, three, or four
saccades.
|
|
Because our objective here is to compare the behavior of the eyes with
that of the hand, we thought that rather than averaging all first,
second, etc., saccades, a better approach might be to divide reaching
movements into small temporal bins and analyze the behavior of saccades
that began in each bin. For instance, we analyzed the end point, with
respect to the hand, of saccades generated at different times into the movement.
We considered all saccades that took place in a given 10 msec time bin.
For each of these saccades, we compared the eye position e(t) with the hand position
h(t) in the corresponding reaching movement. We
represented these positions as displacements toward the target (i.e.,
scalar quantities). The result is an error measure e(t)
h(t) for each
saccade. In principle, the measure could be positive or negative
depending on values of e(t) and
h(t) in a given trial. We averaged this error
across all saccades in each 10 msec time bin. When all time bins were
considered, the result was a measure of error between saccade end
points and hand positions as a function of time into the movement. We
extended this analysis by considering the possibility that the saccade
was related not to the current position of the hand (current meaning
the time bin in which the saccade occurred) but perhaps to where the
hand was in the past or would be in the future. This measure is
described by the formula e(t)
h(t +
). We considered
ranging from
300 to 500 msec. For example, when
= 0, the saccade end point was compared with hand position at the time of that saccade in the corresponding movement. When
= 200, the saccade end point was compared with hand position 200 msec after saccade time.
For a given 10 msec time bin, we found the value of
for which the
average of the quantity e(t)
h(t +
) crossed zero (the average was over all
the saccades that took place in that time bin). The result is shown in
Figure 4A. On the
y-axis of this figure, we have the time bins in which
saccades took place. On the x-axis, we have the time shift,
, that was imposed on the hand trajectory. We found that up to 600 msec into the reaching movement, the average error between eye and hand
was consistently zero if the hand trajectory was shifted back by ~200
msec (196 ± 31 msec). This means that on average, although
saccade end point was a poor estimator of hand position at saccade
time, it estimated hand position at a fairly consistent time in the
future. This method of analysis suggested that saccades at any time (up
to 600 msec into the movement) on average predicted hand position at
time t + 196 msec.

View larger version (18K):
[in this window]
[in a new window]
|
Figure 4.
Saccades during reaching movements in unperturbed
trials. A, Eye position at saccade end point and hand
position are represented as scalar quantities
e(t) and
h(t), indicating position along
the direction of the target. The figure illustrates the distribution of
the time delays, , for which the error measure
e(t) h(t + ) had an average value of zero
across all saccades that took place in a 10 msec time bin. Each 10 msec
time bin is considered independently, and the corresponding is
found. The clusters at 196 ± 31 msec, indicating that
e(t) was an unbiased estimator of
h(t + ) at 196 msec.
B, The SD of the error measure
e(t) h(t + ) at the for which the
measure had zero mean. Time of saccade refers to the time at which the
saccade ended.
|
|
Although there were saccades that took place after 600 msec, there were
no values of
for which the quantity e(t)
h(t +
) crossed zero; therefore, this
resulted in no data points for these saccades in Figure
4A. A closer look at these data showed that saccades
that took place after 600 msec overestimated final hand position by an
average of 11 mm, resulting in e(t)
h(t +
) to be positive for all
.
Figure 4A displays the time
for which saccades
that took place at a given time t had an average error
e(t)
h(t +
)
that was zero. It is informative to ask for the distribution about this
zero mean. To illustrate this, in Figure 4B we
plotted the SD of e(t)
h(t +
) at the time
at which the average
of this quantity was zero. Results showed that
e(t)
h(t +
)
had, on average, an SD of 11 mm about its mean of zero at optimal time
. Using the cumulative distribution function of the error quantity, this SD implies that on any given trial, a saccade at time t
predicted hand position at t + 196 msec (the average of the
optimum
s) to within 5 mm with a probability of 35.1%.
Another way to estimate
is to quantify how
R2 (coefficient of
determination, equivalent to percentage variance explained by the
regression) between e(t) and
h(t +
) varies with change in
. This is
illustrated in Figure 5. We considered a
range of
between
200 and +500 msec and found that
R2 reached a peak value of 0.88 at +150 msec.

View larger version (10K):
[in this window]
[in a new window]
|
Figure 5.
Saccades during reaching movements in unperturbed
trials. Eye position at saccade end point and hand position are
represented as scalar quantities
e(t) and
h(t), indicating position along
the direction of the target. The figure illustrates
R2 between eye position at saccade
end point e(t) and hand position
at time of saccade t + [(i.e.,
h(t + )] for all saccades. The
maximum R2 is at = 150 msec.
Time of saccade refers to the time at which the saccade ended.
|
|
The method of comparison of eye and hand data used to generate Figure 4
weighs equally the
s that were estimated across different times in
finding an average
of 196 msec. This method is blind to the fact
that the likelihood of saccade generation was not uniform across
different saccade times (Fig. 2A). The method used to
generate Figure 5, however, is more influenced by the relatively large
number of saccades that occurred at ~200 msec. Although the results
of the two methods are similar, the former method of analysis is more
consistent with the hypothesis that regardless of when a saccade is
generated, it should predict hand position at a specific time in the
future. Therefore, we used the average
of 196 msec to ask how much
of the variability in hand trajectory from trial to trial was reflected
in the variability in the saccades.
For each saccade that took place during the interval
200 to 600 msec,
we compared eye position with hand position at saccade time + 196 msec
(Fig. 6). We fitted a linear function to
the hand and eye data and found a slope of 1 with bias of nearly zero
for both x- and y-components of the data.
R2 in both cases was >0.85.
The cluster of points around zero in the x- and
y-components of this figure occurs because some of the
movements were either horizontal or vertical, producing no change along
the x- and y-directions, respectively.

View larger version (20K):
[in this window]
[in a new window]
|
Figure 6.
Saccades during reaching movements in unperturbed
trials. Eye position at saccade end point and hand position were
represented as two-dimensional quantities indicating position in a
Cartesian coordinate system with respect to origin of the reaching
movement. The figure illustrates the position of the end point of a
saccade and hand position in the corresponding reaching movement at
saccade time + 196 msec for all saccades in all trials. Time of saccade
refers to the time at which the saccade ended. There is a cluster of
points at zero because a number of reaching movements were vertical or
horizontal and had no x- or y-components.
h, Hand position; e, eye position; m,
slope of the line; b, crossing point.
|
|
The above data were all collected in conditions in which the hand
freely performed a reaching movement. We subsequently quantified the
behavior of the eye when the hand was suddenly perturbed. As before,
subjects made reaching movements to targets, but now on randomly
selected trials (probability of 58%), a 50 msec 25 N force pulse
pushed the hand in one of two directions perpendicular to the intended
movement direction at one of four possible times into the movement. Two
perturbed trials are shown in Figure 7. In both cases, the first saccade leads the hand and has an end point
that is along the direction of the target. However, the perturbation
significantly alters the expected trajectory of the hand. Remarkably,
the second saccade is not to a point where the hand was when it was
perturbed, but rather to a point that the hand will visit in the
future. The eyes fixate that location until the hand passes it; then a
third saccade is generated to a location that again leads the hand.

View larger version (20K):
[in this window]
[in a new window]
|
Figure 7.
Two representative trials during which the hand
was perturbed with a force pulse. The gray and
black lines represent the trajectories of eye and hand,
respectively. Gray dots locate the end point of each
saccade. Black dots indicate hand position at the time
of origination of that saccade. For example, e1 is the end point of the
first saccade and h1 is the position of the hand at the start time of
that saccade. pos., Position; sac,
saccade.
|
|
The combination of movement directions and force pulses produced a wide
range of hand displacements (the mean of the maximum hand displacement
across movements was 40.2 mm). The probabilities of saccades during
perturbed and unperturbed trials are shown in Figure
8A. We found that a
force pulse at 200 msec was followed by a sharp dip in saccade
probability, reaching a minimum at 320 msec. When the hand was not
perturbed, there was a peak in probabilities at ~230 msec (Fig.
2A). The time between the pulse onset and the dip was
similar regardless of pulse timing. Pulses at 200, 250, 300, and 350 msec produced a probability minimum at latencies of 120, 90, 90, and
120 msec. A peak in saccade probability immediately followed the
minimum. The pulses produced a peak in saccade probabilities at
latencies of 180, 170, 160, and 180 msec after pulse onset. Therefore,
it appeared that the perturbation to the hand caused a significant
inhibition of saccades at 100 msec, followed by an increased
probability of saccades at ~170 msec after perturbation onset.

View larger version (28K):
[in this window]
[in a new window]
|
Figure 8.
Trials in which movements were perturbed with a
force pulse. A, Saccade probability as a function of
movement time. The first gray bar indicates the period
(50 msec) during which the reach was perturbed with a force pulse. In
all subplots, the second bar is positioned at
perturbation (pert.) onset + 150 msec and is 50 msec wide. This period consistently coincides with a peak in
postperturbation saccade probabilities. B, Eye positions
at the saccade end point for saccades that occurred 150-200 msec after
onset of the pulse were represented as two-dimensional variables and
were multiplied by a matrix [a b; b a] to best estimate hand position
at t + with respect to the time, t,
of the saccade in the same trial. Parameters a and
b are shown (±95% confidence interval). When
a = 1 and b = 0, the saccade
vector remains unscaled and unrotated. A rotation,
arctan(b/a), and scaling,
,
of the saccade vector was needed to estimate hand position at all
delays except at = 150 msec. For this time lead, the actual
saccades were an unbiased estimator of hand position. C,
Force pulses were perpendicular to the direction of the target and
displaced the hand by various amounts in each movement. The end points
of saccades that occurred 150-200 msec after the pulse are plotted
versus hand position in the same movement at saccade time + 150 msec. perp. dir, Perpendicular direction;
m, slope of the line; b, crossing point.
|
|
We compared these postperturbation saccades (saccades that occurred at
150-200 msec after the pulse onset) with hand positions in the same
movement. Unlike our analysis in Figures 4 and 5, in which eye and hand
positions were viewed as one-dimensional quantities (displacement
toward the target), because of the perturbation, here the eye and hand
positions were represented as two-dimensional quantities (position
parallel and perpendicular to the direction of target). For all
saccades during the interval 150-200 msec after perturbation, we
compared e(t) with the corresponding
h(t +
). At a given
, we multiplied eye
position by matrix [a
b; b
a] to best estimate h(t +
) in the same
trial. For all
s other than 150 msec, we found that the
relationship between eye and hand required a scaling
] and rotation [arctan(b/a)] of the eye position
vector (Fig. 8B). However, at
= 150 msec, the rotation angle was zero and the scale was 1.06. Therefore,
saccades that occurred "in response" to the perturbation appeared
to be an unbiased estimator of where the hand would be 150 msec in the future.
However, considerable movement-to-movement variations existed for error
that was caused by the pulse in the trajectory of the hand. Depending
on the direction of motion of the hand and its speed, force pulses,
which were perpendicular to the direction of motion, produced a large
range of perpendicular displacements (
70 to 70 mm). Did the saccades
predict this variation from trial to trial? In Figure 8C, we
compare the saccades that took place 150-200 msec after pulse onset
with the error (perpendicular displacement) that was recorded in the
trajectory of the hand in the same movement 150 msec after the saccade.
The slope of the fit is 1.08, bias is
1.6 mm, and
R2 is 0.80. This suggests that
when there were unpredictable pulses that displaced the hand from the
nominal trajectory, a saccade that occurred in response to the pulse
was, on average, an unbiased estimator of this displacement at saccade
time + 150 msec and accounted for 80% of the trial-to-trial variance.
It seems reasonable that the brain programmed these postpulse saccades
on the basis of feedback that it had received from the perturbed arm
and an internal model that predicted where the hand would be in the
future. If, indeed, the prediction was based on an internal model, then
changing the dynamics of the arm unexpectedly after the perturbation
should make the saccades inaccurate. For example, a resistive (or
assistive) field introduced after the 50 msec pulse should cause a
saccade that takes place 150-200 msec after pulse onset to
overestimate (or underestimate) future hand position.
We performed two separate experiments in which we introduced a viscous
force field that either assisted or resisted the subject's hand. As
before, the pulse was present in only some of the trials (50%), and
the field was engaged after the pulse in only a fraction of these
pulsed trials (25%). We observed a postpulse minimum in saccade
probability at 120 msec, followed by a peak at 150-200 msec (Fig.
9A). For each postpulse
saccade, we compared its end point with hand position at 150 msec after
the saccade. In trials in which a field was present after the pulse,
these saccades were inaccurate. A resistive field caused saccade
overestimation of the effect of the pulse (Fig. 9B). An
assistive field caused underestimation of the effect of the pulse.

View larger version (17K):
[in this window]
[in a new window]
|
Figure 9.
In these trials, a viscous force field that either
assisted or resisted the movement of the hand was introduced
immediately after the offset of the force pulse to the hand.
A, Saccade probabilities. The first gray
bar indicates pulse period (50 msec). The second gray
bar indicates a perturbation (pert.)
period 150-200 msec after pulse onset. B, The end
points of saccades that occurred 150-200 msec after the pulse are
plotted versus hand position in the same movement at saccade time + 150 msec. Saccades overestimated hand position in resistive trials and
underestimated hand position in assistive trials. perp.
dir, Perpendicular direction.
|
|
 |
DISCUSSION |
Cortical motor commands that are generated in response to a
perturbation reach the arm muscles 100-150 msec after perturbation onset (Petersen et al., 1998
). For a moderate-speed movement that is
completed in 500 msec, these delays are dangerously long, because they
can destabilize the limb. How is stability maintained? One suggestion
is that the brain may cope with sensory feedback delays by relying on
an internal model that predicts the future state of the arm (Miall et
al., 1993
; Wolpert and Ghahramani, 2000
). According to this theory, the
brain programs motor commands on the basis of the prediction, rather
than basing commands on where the arm was when it was perturbed
(Bhushan and Shadmehr, 1999
). We thought that if the brain can predict
the future state of the arm, then perhaps eye movements can serve as a
proxy for this prediction.
Our rationale for this conjecture was the wealth of data suggesting
that smooth-pursuit eye movements are influenced by commands to the
arm. For one-dimensional arm movements, it has been demonstrated that
when the eyes track a visual target attached to the hand, the
smooth-pursuit eye movements that result are more closely linked to the
motion of the target (Steinbach, 1969
; Gauthier and Hofferer, 1976
;
Koken and Erkelens, 1992
) and can track a faster target (Gauthier et
al., 1988
; Vercher et al., 1993
) than if the target were driven by an
external source. This has suggested that the smooth-pursuit control
system of the eyes uses an efferent copy of arm motor control signals
to anticipate the direction and timing of arm movements (Vercher et
al., 1996
; Scarchilli and Vercher, 1999
).
Given these data, we reasoned that subjects might be able to express an
estimate of the state of their arm through programming of their eye
movements. However, in smooth pursuit, visual information regarding the
state of the limb is always present. We eliminated this information so
that the behavior of the state estimator could be observed when the
only feedback was through proprioception. Preventing viewing of the
moving hand prevents generation of smooth pursuit and results in
saccadic eye movements. If the target of the movement is visible, the
eyes saccade to the target and maintain fixation until the end of the
reaching movement (Neggers and Bekkering, 2001
). Therefore, we
eliminated target information during the reach and asked subjects to
try to look at their unseen hand.
Some characteristics of reaching movements were stereotypical: the
trajectory of the hand was straight, with a bell-shaped speed profile
(Morasso, 1981
). Subjects instructed to look at their unseen hand
during reaching might have been expected to generate saccades
stereotypically. We did not find this to be the case. The probability
of generating a saccade was widely distributed during the movement.
Although on average a 10 cm, 0.8 sec movement had approximately three
saccades, the timing and position of these saccades varied greatly. On
average, the end point of a saccade would lead the hand, and the eyes
would remain there until the hand passed the fixation point, at
which time another saccade would be generated (Fig. 2C).
This finding is similar to the observation made by Johansson et al.
(2001)
in that the eye movement was apparently dependent on the
occurrence of a kinematic event for the hand. However, whereas some
reaching movements had two saccades, similar reaching movements in
other cases had three or four. We therefore began by assuming that the
timing of the saccade, rather than its serial order, was the relevant
variable to analyze and tested the hypothesis that the end point of the
saccade predicted hand location with a specific latency. For each
saccade, we computed the distance between saccade end point and hand
position (delayed by some latency,
). To allow for the possibility
that the eye position related to hand position at some time in the
future (or past), we computed the distance
e(t)
h(t +
) for
all
. For each t, we found the
that produced an
average of zero distance for saccades that took place at that time.
When all times were considered, the
clustered along a straight line
centered on 196 msec. It appeared that saccades were an unbiased
estimator of hand position at t + 196 msec.
Although the estimator was unbiased for this particular
, it was
quite noisy. The saccades estimated hand position with an error of <5
mm with a probability of only ~35%. The distribution of the error
about its zero mean was broad throughout the movement period (Fig.
4B). In preliminary experiments, we found that the projection of a stationary random-dot pattern with a modest density (35 dots/in2) on a plane immediately above the
hand during the reaching movement was useful because it facilitated
generation of saccades during the reach. It is possible that the noise
in estimation of hand position was because the saccades moved to a dot
that was closest to the estimate of hand position. However, the SD of
the error was more than twice as high as would be expected if errors
were solely a result of dot density. When we compared
e(t) with h(t + 196 msec)
for all saccades, we found a linear relationship with a slope of 1. Saccades predicted >85% of the variance in the hand data.
For the eyes to be able to estimate future hand position in real time
as a reaching movement takes place, one theory suggests that the brain
may use an internal model that relies on sensory feedback and efferent
copy (Scarchilli and Vercher, 1999
). This is a particular example of
the forward-model theory (Jordan and Rumelhart, 1992
; Miall et al.,
1993
), in which an internal model of the dynamics of the arm
(mathematically represented by a differential equation that relates
force input to acceleration output) is integrated from an initial
condition specified by the delayed proprioceptive feedback (or visual
feedback, if available) with a forcing function specified by the
efferent copy (Bhushan and Shadmehr, 1999
). The importance of the
theory is that it provides the means by which a controller can
effectively respond to perturbations despite the long delays in
sensorimotor pathways. A strong prediction of the theory is that if a
brief perturbation displaces the arm at unpredictable instances, the
state estimator should still be able to predict the future state of the
limb by combining the delayed sensory feedback with efferent copy.
In randomly selected trials and at random times during the selected
trials, we applied a 50 msec force pulse to the hand. The direction of
the pulse was always perpendicular to the direction of the target, but
it could be either clockwise or counterclockwise (randomly selected).
Because of the anisotropy of the inertia and stiffness of the arm, the
pulses produced a very large range of hand displacements. The range of
perpendicular displacements was ~140% of the distance traveled in
the unperturbed movement, and the distribution within this range was
fairly uniform (Fig. 8C). Therefore, the perturbations
produced a large variance in the trajectory of the hand.
In response to the perturbation, we observed saccadic inhibition at a
latency of ~100 msec. Choi and Guitton (2002)
recently observed that
when the head was perturbed during free eye-head movements toward
remembered targets, ongoing saccades were interrupted and, after a
pause period, a new saccade was generated. Fixation neurons in the
rostral pole of the superior colliculus were activated in response to
the head perturbation, terminating the saccade. In the current task, it
appears that during the postperturbation period, a saccade gating
mechanism sensitive to proprioceptive error from the arm caused
previously planned saccades to be aborted.
At ~150-200 msec after perturbation onset, the probability of
saccades increased from its low by approximately eightfold (Fig. 8A). The error caused the ongoing or planned saccades
to be inhibited and a new saccade to be computed. The computation did
not result in a saccade to the position where the hand was when it was
perturbed. Rather, the resulting saccade was to where the hand would go
in the near future. The analysis of the relationship between the postperturbation saccades and hand positions in the same movement found
a remarkable correlation between saccade end point and hand position at
saccade time + 150 msec. Thus, the postperturbation depression is
interpreted as the point at which the proprioceptive information
regarding the perturbation is integrated into the state estimator. The
time between the perturbation onset and saccade generation ~170 msec
later is a period during which we speculate that the behavior of the
eyes is most influenced by a forward model of the arm. Because the
saccades that responded to the perturbation predicted hand state 150 msec after the saccade, the brain appears to compute the future state
of the hand for a time interval that is nearly equal to the delays in
sensorimotor pathways.
In summary, we found that saccades were unbiased, real-time estimators
of the future position of the hand. Immediately after a brief
perturbation, the brain appears to be able to predict where the limb
will be in the near future. It seems plausible that the long-latency
motor commands that are computed in response to an error take into
account such an estimator. As subjects learn to control their arms in
novel dynamics, these estimators are thought to adapt (Shadmehr and
Brashers-Krug, 1997
). It remains to be seen whether adaptation of the
internal model for the arm results in changes in eye movements.
 |
FOOTNOTES |
Received March 28, 2002; revised June 10, 2002; accepted June 12, 2002.
G.A. was supported by a Bozelle Fellowship. This work was also
supported by a postdoctoral fellowship from the National Institutes of
Health (NIH) to O.D., a Distinguished Postdoctoral Fellowship from the
Johns Hopkins University Department of Biomedical Engineering to O.D.,
and grants from the NIH (NS37422) and the Office of Naval Research
(N000140110534) to R.S. This work is part of a Masters Thesis submitted
by G.A. to the Johns Hopkins University Department of Biomedical Engineering.
Correspondence should be addressed to Reza Shadmehr, Johns Hopkins
School of Medicine, 419 Traylor Building, 720 Rutland Avenue, Baltimore, MD 21205. E-mail: reza{at}bme.jhu.edu.
 |
REFERENCES |
-
Bhushan N,
Shadmehr R
(1999)
Computational architecture of human adaptive control during learning of reaching movements in force fields.
Biol Cybern
81:39-60[ISI][Medline].
-
Blakemore SJ,
Goodbody SJ,
Wolpert DM
(1998)
Predicting the consequences of our own actions: the role of sensorimotor context estimation.
J Neurosci
18:7511-7518[Abstract/Free Full Text].
-
Choi WY, Guitton D (2002) Activity on the motor map of the
primate superior colliculus during head-perturbed large gaze shifts.
Neural Control of Movement Society, Vol 12.
-
Gauthier GM,
Hofferer JM
(1976)
Eye tracking of self-moved targets in the absence of vision.
Exp Brain Res
26:121-139[ISI][Medline].
-
Gauthier GM,
Vercher JL,
Mussa-Ivaldi FA,
Marchetti E
(1988)
Oculo-manual tracking of visual targets: control learning, coordination control, and coordination model.
Exp Brain Res
73:127-137[ISI][Medline].
-
Ghez C,
Shinoda Y
(1978)
Spinal mechanisms of the functional stretch reflex.
Exp Brain Res
32:55-68[ISI][Medline].
-
Gielen CC,
Ramaekers L,
van Zuylen EJ
(1988)
Long-latency stretch reflexes as co-ordinated functional responses in man.
J Physiol (Lond)
407:275-292[Abstract/Free Full Text].
-
Johansson RS,
Westling G,
Backstrom A,
Flanagan JR
(2001)
Eye-hand coordination in object manipulation.
J Neurosci
21:6917-6932[Abstract/Free Full Text].
-
Jordan MI,
Rumelhart DE
(1992)
Forward models: supervised learning with a distal teacher.
Cogn Sci
16:307-354.
-
Koken PW,
Erkelens CJ
(1992)
Influences of hand movements on eye movements in tracking tasks in man.
Exp Brain Res
88:657-664[ISI][Medline].
-
Miall RC,
Weir DJ,
Wolpert DM,
Stein JF
(1993)
Is the cerebellum a Smith predictor?
J Mot Behav
25:203-216[ISI][Medline].
-
Morasso P
(1981)
Spatial control of arm movements.
Exp Brain Res
42:223-227[ISI][Medline].
-
Neggers SFW,
Bekkering H
(2001)
Gaze anchoring to a pointing target is present during the entire pointing movement and is driven by a non-visual signal.
J Neurophysiol
86:961-970[Abstract/Free Full Text].
-
Noth J,
Podoll K,
Friedemann HH
(1985)
Long-loop reflexes in small hand muscles studied in normal subjects and in patients with Huntington's disease.
Brain
108:65-80[Abstract/Free Full Text].
-
Petersen N,
Christensen LOD,
Morita H,
Sinkaer T,
Nielsen J
(1998)
Evidence that a transcortical pathway contributes to stretch reflexes in the tibialis anterior muscle in man.
J Physiol (Lond)
512:267-276[Abstract/Free Full Text].
-
Scarchilli K,
Vercher JL
(1999)
The oculomanual coordination control center takes into account the mechanical properties of the arm.
Exp Brain Res
124:42-52[ISI][Medline].
-
Shadmehr R,
Brashers-Krug T
(1997)
Functional stages in the formation of human long-term motor memory.
J Neurosci
17:409-419[Abstract/Free Full Text].
-
Shadmehr R,
Mussa-Ivaldi FA
(1994)
Adaptive representation of dynamics during learning of a motor task.
J Neurosci
14:3208-3224[Abstract].
-
Smith MA,
Brandt J,
Shadmehr R
(2000)
The motor dysfunction in Huntington's disease begins as a disorder in error feedback control.
Nature
403:544-549[Medline].
-
Steinbach MJ
(1969)
Eye tracking of self-moved targets: the role of efference.
J Exp Psychol
82:366-376[ISI][Medline].
-
Thilmann AF,
Schwarz M,
Topper R,
Fellows SJ,
Noth J
(1991)
Different mechanisms underlie the long-latency stretch reflex response of active human muscle at different joints.
J Physiol (Lond)
444:631-643[Abstract/Free Full Text].
-
Thompson PDB
(1988)
The coexistence of bradykinesia and chorea in Huntington's disease and its implications for theories of basal ganglia control of movement.
Brain
111:223-244[Abstract/Free Full Text].
-
Thoroughman KA,
Shadmehr R
(1999)
Electromyographic correlates of learning internal models of reaching movements.
J Neurosci
19:8573-8588[Abstract/Free Full Text].
-
Vercher JL,
Volle M,
Gauthier GM
(1993)
Dynamic analysis of human visuo-oculo-manual coordination control in target tracking tasks.
Aviat Space Environ Med
64:500-506[Medline].
-
Vercher JL,
Gauthier GM,
Guedon O,
Blouin J,
Cole J,
Lamarre Y
(1996)
Self-moved target eye tracking in control and deafferented subjects: roles of arm motor command and proprioception in arm-eye coordination.
J Neurophysiol
76:1133-1144[Abstract/Free Full Text].
-
Vercher JL,
Lazzari S,
Gauthier G
(1997)
Manuo-ocular coordination in target tracking. II. Comparing the model with human behavior.
Biol Cybern
77:267-275[ISI][Medline].
-
Witney AG,
Goodbody SJ,
Wolpert DM
(1999)
Predictive motor learning of temporal delays.
J Neurophysiol
82:2039-2048[Abstract/Free Full Text].
-
Wolpert DM,
Ghahramani Z
(2000)
Computational principles of movement neuroscience.
Nat Neurosci
3:1212-1217.
Copyright © 2002 Society for Neuroscience 0270-6474/02/22177721-09$05.00/0
This article has been cited by other articles:

|
 |

|
 |
 
D. Thura, D. Boussaoud, and M. Meunier
Hand Position Affects Saccadic Reaction Times in Monkeys and Humans
J Neurophysiol,
May 1, 2008;
99(5):
2194 - 2202.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
F. Danion and F. R. Sarlegna
Can the Human Brain Predict the Consequences of Arm Movement Corrections When Transporting an Object? Hints from Grip Force Adjustments
J. Neurosci.,
November 21, 2007;
27(47):
12839 - 12843.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. C. Simani, L. M. M. McGuire, and P. N. Sabes
Visual-Shift Adaptation Is Composed of Separable Sensory and Task-Dependent Effects
J Neurophysiol,
November 1, 2007;
98(5):
2827 - 2841.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
V. Gritsenko, N. I. Krouchev, and J. F. Kalaska
Afferent Input, Efference Copy, Signal Noise, and Biases in Perception of Joint Angle During Active Versus Passive Elbow Movements
J Neurophysiol,
September 1, 2007;
98(3):
1140 - 1154.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
L. A. Mrotek and J. F. Soechting
Target Interception: Hand-Eye Coordination and Strategies
J. Neurosci.,
July 4, 2007;
27(27):
7297 - 7309.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
A. Battaglia-Mayer, M. Mascaro, and R. Caminiti
Temporal Evolution and Strength of Neural Activity in Parietal Cortex during Eye and Hand Movements
Cereb Cortex,
June 1, 2007;
17(6):
1350 - 1363.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
U. J. Ilg and S. Schumann
Primate Area MST-l Is Involved in the Generation of Goal-Directed Eye and Hand Movements
J Neurophysiol,
January 1, 2007;
97(1):
761 - 771.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
L. Ren, A. Z. Khan, G. Blohm, D.Y.P. Henriques, L. E. Sergio, and J. D. Crawford
Proprioceptive Guidance of Saccades in Eye-Hand Coordination
J Neurophysiol,
September 1, 2006;
96(3):
1464 - 1477.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
M. Synofzik, P. Thier, and A. Lindner
Internalizing Agency of Self-Action: Perception of One's Own Hand Movements Depends on an Adaptable Prediction About the Sensory Action Outcome
J Neurophysiol,
September 1, 2006;
96(3):
1592 - 1601.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
H. Gomi, N. Abekawa, and S. Nishida
Spatiotemporal tuning of rapid interactions between visual-motion analysis and reaching movement.
J. Neurosci.,
May 17, 2006;
26(20):
5301 - 5308.
[Abstract]
[Full Text]
[PDF]
|
 |
|

|
 |

|
 |
 
S. Vaziri, J. Diedrichsen, and R. Shadmehr
Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback.
J. Neurosci.,
April 19, 2006;
26(16):
4188 - 4197.
[Abstract]
[Full Text]
[PDF]
|
 |
|