 |
 Previous Article | Next Article 
The Journal of Neuroscience, 2001, 21:RC196:1-4
RAPID COMMUNICATION
Rotational Remapping in Human Spatial Memory during Eye and
Head Motion
W. Pieter
Medendorp1, 2, 3,
Michael A.
Smith1, 2, 3,
Douglas B.
Tweed1, 2, 4, and
J. Douglas
Crawford1, 2, 3
1 Canadian Institutes of Health Research Group for
Action and Perception, 2 Centre for Vision Research, and
3 Department of Psychology, York University,
Toronto, Ontario, Canada M3J 1P3, and 4 Departments
of Physiology and Medicine, University of Toronto, Toronto, Ontario,
Canada M5S 1A8
 |
ABSTRACT |
The brain uses vision and other senses to compute the locations of
objects relative to the body, and then must update these locations when
the body moves. How geometrically sophisticated is this internal
updating? It has been suggested that updating simply shifts the stored
locations of all objects uniformly, by a common vector, when the eye or
head turns. For horizontal and vertical turns, a uniform shift would
often approximate the real changes in location of objects in front of
the subject. But for torsional rotations, a shift would be inadequate:
accurate updating would call for a more geometrically exact remapping,
not shifting but rotating the stored locations through the inverse of
the rotation of the eye in space. Here we asked human subjects to make
eye saccades to remembered targets after torsional head rotations. Their accuracy showed that spatial updating works in the torsional dimension and operates by rotation rather than shifting.
Key words:
remapping; coordinates; spatial perception; eye-head
movements; three-dimensional; human; spatial memory; rotation; rotational geometry
| |
INTRODUCTION |
From
its sensory inputs, the brain computes the locations of surrounding
objects. How is this geometric information handled? A central problem
is that vision reports the locations of objects relative to the retina
(10° right of the fovea, for instance, or 5° below), but these
coordinates become obsolete as soon as the eyes move. Nevertheless the
brain manages to keep track of objects in the dark, using remembered
visual information, despite motion of the eyes or head. Humans and
monkeys can look accurately to the remembered location of a flashed
light, even when another eye or head movement intervenes between the
target flash and the look (Hallet and Lightstone, 1976 ; Sparks and
Mays, 1983; McKenzie and Lisberger, 1986; Israel and Berthoz, 1989;
Pelisson et al., 1989; Schlag et al., 1990; Ohtsuka, 1994; Zivotofsky
et al., 1996; Blouin et al., 1998; Herter and Guitton, 1998).
How could this space constancy be realized? Two mechanisms have
been proposed. The brain may convert visual information from its
original, retinal frame to a more stable frame, computing and storing
the locations of objects relative to the Earth (or torso or inertial
frame), so that the stored coordinates remain correct when the eyes and
head move (Mays and Sparks, 1980; Andersen et al., 1985; Soechting et
al., 1991). Alternatively, the brain may store locations of objects in
a retinal frame (Goldberg and Bruce, 1990; Duhamel et al., 1992a;
Walker et al., 1995). In that case, it would have to recompute all
those locations every time the eyes moved, but there would be
compensating advantages (Henriques et al., 1998). For either mechanism,
visual spatial information must be remapped. Eye-centered locations
must be continuously updated or they must be converted to a more stable
frame and later converted again, to limb-, eye-, or head-related
coordinates for motor planning.
How geometrically sophisticated is this remapping? It has been
suggested that updating might work with simple, geometrically crude
operations (e.g., shifting the stored locations of all objects uniformly, by a common vector, when the eye turns) (Goldberg and Bruce,
1990; Duhamel et al., 1992b; Quaia et al., 1998). But a uniform shift
merely approximates the geometrically exact remapping, which would
involve rotating the stored locations through the inverse of the
rotation of the eye in space, turning them all 30° left when the eye
turns 30° right (Henriques et al., 1998; Smith and Crawford, 2001).
Can spatial memory perform operations of this kinematic complexity?
Here we test whether visual memory can be rotationally remapped. The
clearest test involves torsional eye and head motion (clockwise or
counterclockwise, or in other words right or left ear down), because in
this case the exact remapping differs markedly from a uniform shift.
When the eye turns clockwise, then the world turns counterclockwise on
the retina. Objects above the fovea move leftward in the retina-fixed
frame, and objects below move rightward; objects right and left of the
fovea move up and down; so no uniform shift can match even
approximately the true, rotary transformation. Surprisingly, however,
spatial updating has never been studied in the torsional dimension.
Here we asked human subjects to make eye saccades to remembered targets
after torsional eye and head rotations.
| |
MATERIALS AND METHODS |
All experiments were approved by the Human Participants Review
Subcommittee of York University. Our subjects were six healthy volunteers aged 21-31. They were seated and restrained to minimize trunk motion. The left eye was patched. We measured the
three-dimensional (3-D) orientations of the right eye and the head
using search coils (Skalar, Delft, The Netherlands) in three
mutually orthogonal magnetic fields generated by field coils 2 m
across. The three voltages from each coil were sampled at 100 Hz and
converted into 3-D eye and head positions, expressed as quaternions,
and two-dimensional pointing directions (Tweed et al., 1990).
Calibration and accuracy were as described previously (Henriques et
al., 1998; Klier and Crawford, 1998).
Subjects faced a black tangent screen 2 m away in complete
darkness. Green light-emitting diodes (2 cd/m2) mounted on this screen served as
targets. The target array consisted of 29 diodes: a central fixation
light directly in front of the subject, 4 cardinal targets (20°
right, left, above, and below center), and 24 other targets distributed
evenly among the four quadrants (six in each) at random locations
between 15° and 25° eccentricity.
Experimental paradigm. In each trial, the subject began by
fixating the illuminated central target (F) while
looking straight ahead (Fig. 1). Subjects
were then instructed to rotate the head to one of five torsional
orientations. These orientations were indicated by verbal commands,
using clock-face numbers 10, 11, 12, 1, and 2; 10 o'clock meant
maximum counterclockwise head torsion (~45°), 12 o'clock meant
upright (i.e., no head rotation was required), and 2 o'clock meant
maximum clockwise torsion. While the subject maintained this head
position, ~3 sec after the start of the trial, one of the peripheral
targets (T) flashed for 100 msec. Then the subject
straightened his or her head to the upright 12 o'clock position
while still fixating the central light. Approximately 2 sec later, the
central light switched off and an audio tone cued the subject to look
to the remembered location of the flashed target, keeping the head
still. [When the head was tilted, the eyes counter-rolled slightly,
~4.7° on average for 45° of head torsion, as in other studies
(Haslwanter et al., 1992). By having subjects hold the head upright
during their gaze shifts, we eliminated any effect of counter-roll on
their saccades (Klier et al., 1998).] The entire trial lasted ~7
sec. Targets and head orientations were selected randomly. For each of
the five head orientations, the four cardinal targets were tested three
times each and the other 24 targets were tested once each, for a total
of 180 trials. Note that the 12 o'clock target enabled us to obtain
the subject's memory-guided saccades in the absence of updating.
Typically eight trials per subject were excluded from additional
analysis because the subject failed to follow the above
instructions.
View larger version (33K):
[in this window]
[in a new window]
|
Figure 1.
A typical trial illustrating the experimental
paradigm. Top, The torsional orientation (in degrees) of
the eye in space (black) and the head
(gray). Bottom, The pointing
direction of the eye (Gv, Gh);
thin gray lines (Tv, Th)
indicate the direction of the target. Thick black boxes
mark the durations of the fixation target (F) and
the flashed target (T). See Materials and Methods
for explanation. The vertical dashed lines indicate the time
interval during which the target was presented.
|
|
In a control test, the subject was seated with his or her head
upright and saccaded from the central target to each of the peripheral
targets, which were illuminated for 1.5 sec. In this way, we
obtained the ideal gaze directions for each of the targets.
| |
RESULTS |
Figure 2 shows the gaze trajectories
of subject S.M. toward the four remembered cardinal targets (Fig. 2,
 ) for the control condition (i.e., without head rotation; Fig. 2,
left) and after a head tilt of ~45° (Fig. 2,
right). In the control condition, this subject often
undershot targets, but his saccade directions were accurate. In the
test condition (Fig. 2, right), with an intervening head
rotation, the subject viewed the flashed targets from a 45° clockwise
head tilt before righting the head and finally performing the saccades.
As the figure shows, saccades were accurate for all four targets,
indicating almost perfect compensation for the head and eye
rotation.
View larger version (20K):
[in this window]
[in a new window]
|
Figure 2.
One subject's performance in the control
condition (left) and in the "after head-tilt"
condition (right). Saccades to the four cardinal
targets () are shown. In the control condition the subject made no
head movements before saccading to the remembered target. In the after
head-tilt condition, the subject perceived the target at a 45°
rightward head tilt, rolled his head back to upright, and subsequently
made the saccade. In both conditions, saccadic directions were
accurate.
|
|
What would have happened in the test condition if the subject's
compensation had been imperfect? When the head is tilted 45° clockwise, the eye is tilted ~40° clockwise relative to space, so
if the subject had not corrected at all for the intervening eye
rotation in space (i.e., if there had been no remapping) his saccades
would have been misdirected by ~40° counterclockwise. (For example,
if the head is tilted 45° clockwise, and the eye is tilted 40°
clockwise in space, and a target appears in the 12 o'clock position on
a space-fixed clock face, then the retina will see the target on a
meridian tilted 40° counterclockwise. If the subject rights his head
and then makes a saccade along that retinal meridian, his directional
error will be 40°.) If the subject had corrected for the rotation of
the eye in the head, rather than its rotation in space, he would have
been off by 45° counterclockwise. If he had corrected for the
rotation of the head in space rather than the rotation of the eye, he
would have misaimed by ~5° clockwise.
Figure 3 shows the data of all our
subjects, plotting directional error versus the intervening head
rotation. Because eye responses often consisted of several saccades, we
measured the directional error of the first saccade toward the
remembered target. Subjects differed in their average error, varying
between 2.2° and 7.5°. But across all subjects, the data scattered
around zero, showing no systematic directional errors.
View larger version (33K):
[in this window]
[in a new window]
|
Figure 3.
Directional errors of all subjects plotted against
the size of the intervening head rotation (black dots).
Subjects varied in their accuracy, but none showed a systematic
relationship between direction error and head rotation: regression
lines (thick lines) had slopes near zero. The slopes
would have equaled 0.9 (thin solid line) if subjects had not
corrected for the torsion of the eye in space (because of a
small ocular counter-roll, eye torsion in space is 90% of head torsion
in space). The dashed lines indicate updating for eye in
head torsion but not for head torsion in space.
|
|
Using linear regression, we quantified the relationship between
directional error and head rotation. If there had been no updating for
the torsion of the eye in space, the line would have had a slope of
~0.9. If the updating had corrected for the torsion of the eye in the
head but not the torsion of the head in space, the slope would have
been 1.0; if it had corrected for the torsion of the head but not the
torsion of the eye, the slope would have been  0.1. Perfect updating
would yield a slope of 0. The results favor the last hypothesis: across
all subjects the slopes ranged between 0.02 and 0.02, indicating
almost ideal performance. On average, the slope was not significantly
different from 0 (t test; p = 0.9) but was
significantly different from 0.1 (t test;
p < 0.001) and from 0.9 and 1 (t test;
p < 0.001).
Could our subjects have achieved this accuracy by shifting, rather than
rotating, the stored locations of the targets? Geometrically, the shift
that best approximates a torsional rotation is the null shift (no shift
at all), because any other uniform motion would improve the spatial
information on one side of the fovea but make it less accurate on the
other side. So at best, updating by shifting would lead to errors like
those shown in Figure 3 for the no-remapping strategy, with its slope
of 0.9. Figure 3 shows, then, that our subjects were far more accurate
than is consistent with updating by a uniform shift. Their updating
closely approximated the optimal, rotational remapping.
| |
DISCUSSION |
Our subjects made almost no systematic directional errors when
looking to remembered targets after intervening torsional eye and head
movements. It has already been shown that spatial memory copes well
with horizontal and vertical eye and head motion. Our results show that
it also corrects for torsional motion. Because these experiments were
performed in total darkness, the system must have relied solely on
extra-retinal signals. Together with previous results, we can now say
that these signals account for all three dimensions of eye and head
rotation: horizontal, vertical, and torsional. Here we consider the
physiological and computational implications of this finding.
Some authors have suggested that remapping might operate by very simple
geometry, merely shifting all stored locations by the same, common
vector whenever the eyes turn (Goldberg and Bruce, 1990; Moschovakis
and Highstein, 1994). This uniform shift would approximate fairly well
the real changes of location during most horizontal and vertical eye
movements, but it would cause a pattern of cumulative inaccuracy for
eye movements between eccentric targets. When tested on this task,
subjects do not show the predicted errors (Smith and Crawford, 2001).
Their accuracy suggests that updating is more sophisticated than a
simple shift, that it more closely approximates the geometrically exact
remapping, rotating stored locations through the inverse of the
rotation of the eye. Smith and Crawford (2001) studied horizontal and
vertical saccades with the head fixed. In the present study, we have
tested their scheme in a far more dramatic and previously untested
situation. We show accurate, rotational remapping in a more
computationally demanding task, during eye and head motions in the
torsional dimension. Together with the results obtained by Smith and
Crawford (2001), our findings suggest that spatial updating operates by
rotational remapping in all three dimensions (horizontal, vertical, and
torsional) and accurately combines information about the movements of
the eyes and head. Remapping may work with efference copy or vestibular inputs (Blouin et al., 1998). It may operate entirely in retinal coordinates, it may use other head- or Earth-fixed coordinates, or it
may involve mixed or distributed representations (Pouget and Sejnowski,
1997). But our results show that the net result is an accurate,
rotational remapping of object locations.
Our data suggest new experimental predictions for the neural structures
that participate in remapping, such as the posterior parietal cortex,
the frontal cortex, and the superior colliculus (SC) (Goldberg and
Bruce, 1990; Duhamel et al., 1992a; Walker et al., 1995). Figure
4 shows the predicted neurophysiological consequences of torsional remapping in the SC (Munoz et al., 1991; Klier et al., 2001). A space-fixed target a, flashed when
the eye is turned 45° counterclockwise, stimulates the left side of the retina. Therefore, it is represented on the left SC. But after the
eye rotates upright, the remembered target is now on the right relative
to the retina. In the collicular map, then, its representation must
cross the midline, from the left to the right SC (Munoz et al., 1991;
Walker et al., 1995; Pouget and Snyder, 2000). At the same time, target
b should cross in the opposite direction, from the right
side of the retina to the left, and therefore from the right SC to the
left. That is, a torsional motion of the eye or head should rotate the
activity pattern on the collicular map, moving different active sites
in opposite directions across the midline of the brain.
View larger version (22K):
[in this window]
[in a new window]
|
Figure 4.
Rotational remapping in the collicular map. A
space-fixed target a (10° leftward and 30° upward)
that is flashed when the eye is turned 45° counterclockwise
stimulates the left side of the retina. Therefore it is
represented on the left SC. But after the eye rotates upright, the
remembered target is now to the right relative to the retina. In the
collicular map, then, its representation must cross the midline, from
the left to the right SC (see a'). At the same time, target
b (10° rightward and 30° downward in space) should
cross in the opposite direction, from the right SC to the left (see
b').
|
|
| |
FOOTNOTES |
Received Sept. 17, 2001; revised Oct. 17, 2001; accepted Oct. 18, 2001.
This work was supported by grants from the Canadian Natural Sciences
and Engineering Research Council. W.P.M. is supported by the Human
Frontier Science Program. M.A.S. holds an Ontario Graduate Scholarship.
D.B.T. is a Canadian Institutes of Health Research Scientist.
J.D.C. is supported by the Canadian Research Chair Program. We thank J. Martinez-Trujillo for comments on this manuscript.
Correspondence should be addressed to Dr. W. P. Medendorp,
Department of Psychology, York University, 4700 Keele Street, Toronto, Ontario, Canada M3J 1P3. E-mail: pieter{at}yorku.ca.
This article is published in
The Journal of Neuroscience, Rapid Communications Section,
which publishes brief, peer-reviewed papers online, not in print. Rapid
Communications are posted online approximately one month earlier than
they would appear if printed. They are listed in the Table of Contents
of the next open issue of JNeurosci. Cite this article as:
JNeurosci, 2001, 21:RC196 (1-4). The
publication date is the date of posting online at
www.jneurosci.org.
| |
REFERENCES |
-
Andersen RA,
Essick GK,
Siegel RM
(1985)
Encoding of spatial location by posterior parietal neurons.
Science
230:456-458.
-
Blouin J,
Labrousse L,
Simoneau M,
Vercher JL,
Gauthier GM
(1998)
Updating visual space during passive and voluntary head-in-space movements.
Exp Brain Res
122:93-100.
-
Duhamel JR,
Colby CL,
Goldberg ME
(1992a)
The updating of the representation of visual space in parietal cortex by intended eye movements.
Science
255:90-92.
-
Duhamel JR,
Goldberg ME,
Fitzgibbon EJ,
Sirigu A,
Grafman J
(1992b)
Saccadic dysmetria in a patient with a right frontoparietal lesion. The importance of corollary discharge for accurate spatial behaviour.
Brain
115:1387-1402.
-
Goldberg ME,
Bruce CJ
(1990)
Primate frontal eye fields. III. Maintenance of a spatially accurate saccade signal.
J Neurophysiol
64:489-508.
-
Hallet PE,
Lightstone AD
(1976)
Saccadic eye movements to flashed targets.
Vision Res
16:107-114.
-
Haslwanter T,
Straumann D,
Hess BJM,
Henn V
(1992)
Static roll and pitch in the monkey: shift and rotation of Listing's plane.
Vision Res
32:1341-1348.
-
Henriques DYP,
Klier EM,
Smith MA,
Lowy D,
Crawford JD
(1998)
Gaze-centered remapping of remembered visual space in an open-loop pointing task.
J Neurosci
18:1583-1594.
-
Herter TM,
Guitton D
(1998)
Human head-free gaze saccades to targets flashed before gaze-pursuit are spatially accurate.
J Neurophysiol
80:2785-2789.
-
Israel I,
Berthoz A
(1989)
Contribution of the otoliths to the calculation of linear displacement.
J Neurophysiol
62:247-263.
-
Klier EM,
Crawford JD
(1998)
The human oculomotor system accounts for 3-D eye orientation in the visual-motor transformation for saccades.
J Neurophysiol
80:2274-2294.
-
Klier EM,
Wang H,
Crawford JD
(2001)
The superior colliculus encodes gaze commands in retinal coordinates.
Nat Neurosci
4:627-632.
-
Mays LE,
Sparks DL
(1980)
Saccades are spatially, not retinotopically coded.
Science
208:1163-1164.
-
McKenzie A,
Lisberger SG
(1986)
Properties of signals that determine the amplitude and direction of saccadic eye movements in monkeys.
J Neurophysiol
56:196-207.
-
Moschovakis AK,
Highstein SM
(1994)
The anatomy and physiology of primate neurons that control rapid eye movements.
Annu Rev Neurosci
17:465-488.
-
Munoz DP,
Pelisson D,
Guitton D
(1991)
Movement of neural activity on the superior colliculus motor map during gaze shifts.
Science
251:1358-1360.
-
Ohtsuka K
(1994)
Properties of memory-guided saccades toward targets flashed during smooth pursuit in human subjects.
Invest Ophthalmol Vis Sci
35:509-514.
-
Pelisson D,
Guitton D,
Munoz DP
(1989)
Compensatory eye and head movements generated by the cat following stimulation-induced perturbations in gaze position.
Exp Brain Res
78:654-658.
-
Pouget A,
Sejnowski TJ
(1997)
Spatial transformations in the parietal cortex using basis functions.
J Cogn Neurosci
9:222-237.
-
Pouget A,
Snyder LH
(2000)
Computational approaches to sensorimotor transformations.
Nat Neurosci [Suppl]
3:1192-1198.
-
Quaia C,
Optican LM,
Goldberg ME
(1998)
The maintenance of spatial accuracy by perisaccadic remapping of visual receptive fields.
Neural Netw
11:1229-1240.
-
Schlag J,
Schlag-Rey M,
Dassonville P
(1990)
Saccades can be aimed at the spatial location of targets flashed during pursuit.
J Neurophysiol
64:575-581.
-
Smith MA,
Crawford JD
(2001)
Implications of ocular kinematics for the internal updating of visual space.
J Neurophysiol
86:2112-2117.
-
Soechting JF,
Flanders M,
Helms Tillery SI
(1991)
Transformation from head- to shoulder-centered representation of target direction in arm movements.
J Cogn Neurosci
2:32-43.
-
Sparks DL,
Mays LE
(1983)
Spatial localization of saccade targets. I. Compensation for stimulation-induced perturbations in eye position.
J Neurophysiol
49:45-63.
-
Tweed D,
Cadera W,
Vilis T
(1990)
Computing three dimensional eye position quaternions and eye velocity from search coil signals.
Vision Res
30:97-110.
-
Walker MF,
Fitzgibbon J,
Goldberg ME
(1995)
Neurons of the monkey superior colliculus predict the visual result of impeding saccadic eye movements.
J Neurophysiol
73:1988-2003.
-
Zivotofsky AZ,
Rottach KG,
Averbuch-Heller L,
Kori AA,
Thomas CW,
Dell'Osso LF,
Leigh RJ
(1996)
Saccades to remembered targets: the effects of smooth pursuit and illusory stimulus motion.
J Neurophysiol
76:3617-3632.
Copyright © Society for Neuroscience 0270-6474//$05.00/0
This article has been cited by other articles:
|
|
|
|
 
M. Ruiz-Ruiz and J. C. Martinez-Trujillo
Human Updating of Visual Motion Direction During Head Rotations
J Neurophysiol,
May 1, 2008;
99(5):
2558 - 2576.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Van Pelt and W. P. Medendorp
Updating Target Distance Across Eye Movements in Depth
J Neurophysiol,
May 1, 2008;
99(5):
2281 - 2290.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
E. M. Klier, B. J. M. Hess, and D. E. Angelaki
Human Visuospatial Updating After Passive Translations in Three-Dimensional Space
J Neurophysiol,
April 1, 2008;
99(4):
1799 - 1809.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Glasauer and T. Brandt
Noncommutative Updating of Perceived Self-Orientation in Three Dimensions
J Neurophysiol,
April 1, 2007;
97(4):
2958 - 2964.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Van Pelt and W. P. Medendorp
Gaze-Centered Updating of Remembered Visual Space During Active Whole-Body Translations
J Neurophysiol,
February 1, 2007;
97(2):
1209 - 1220.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
E. M. Klier, B. J. M. Hess, and D. E. Angelaki
Differences in the Accuracy of Human Visuospatial Memory After Yaw and Roll Rotations
J Neurophysiol,
April 1, 2006;
95(4):
2692 - 2697.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. D. Crawford, W. P. Medendorp, and J. J. Marotta
Spatial Transformations for Eye-Hand Coordination
J Neurophysiol,
July 1, 2004;
92(1):
10 - 19.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
W. P. Medendorp, D. B. Tweed, and J. D. Crawford
Motion Parallax Is Computed in the Updating of Human Spatial Memory
J. Neurosci.,
September 3, 2003;
23(22):
8135 - 8142.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. T. Baker, T. M. Harper, and L. H. Snyder
Spatial Memory Following Shifts of Gaze. I. Saccades to Memorized World-Fixed and Gaze-Fixed Targets
J Neurophysiol,
May 1, 2003;
89(5):
2564 - 2576.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
Memory
TOP
ABSTRACT
INTRODUCTION
