Abstract
The concurrent planning of sequential saccades offers a simple model to study the nature of visuomotor transformations since the second saccade vector needs to be remapped to foveate the second target following the first saccade. Remapping is thought to occur through egocentric mechanisms involving an efference copy of the first saccade that is available around the time of its onset. In contrast, an exocentric representation of the second target relative to the first target, if available, can be used to directly code the second saccade vector. While human volunteers performed a modified double-step task, we examined the role of exocentric encoding in concurrent saccade planning by shifting the first target location well before the efference copy could be used by the oculomotor system. The impact of the first target shift on concurrent processing was tested by examining the end-points of second saccades following a shift of the second target during the first saccade. The frequency of second saccades to the old versus new location of the second target, as well as the propagation of first saccade localization errors, both indices of concurrent processing, were found to be significantly reduced in trials with the first target shift compared to those without it. A similar decrease in concurrent processing was obtained when we shifted the first target but kept constant the second saccade vector. Overall, these results suggest that the brain can use relatively stable visual landmarks, independent of efference copy-based egocentric mechanisms, for concurrent planning of sequential saccades.
- chunking
- delayed saccade double-step task
- error propagation
- localization errors
- parallel programming
- reference frame
Introduction
Sequential movements, often performed in the absence of sensory feedback (Keele, 1968), are thought to be planned even before the start of the first movement (Lashley, 1951). Behavioral evidence of such parallel programming in the context of saccadic eye movements are the very short intersaccadic intervals (shorter than typical saccade latencies) that are sometimes observed between sequential saccades elicited in the double-step task (Becker and Jürgens, 1979; McPeek et al., 2000; Ray et al., 2004; Sharika et al., 2008). Second saccades in these sequences have been proposed to be based on visual information available before the end of the first saccade, thus requiring a visuomotor transformation of either the target/goal or the motor vector corresponding to the second saccade plan (Hallett and Lightstone, 1976).
One mechanism that might enable such a visuomotor transformation for concurrent processing of sequential saccades is predictive remapping (Murthy et al., 2007; Phillips and Segraves, 2010), which computes the anticipated gaze displacement/position following an impending saccade using the latter's motor efference around the time of its onset (Duhamel et al., 1992; Li and Andersen, 2001; Nakamura and Colby, 2002; Sommer and Wurtz, 2004a,b). Alternatively, it has been suggested that in a relatively stable environment, representations of target location in either allocentric (relative to a fixed location) or exocentric (relative to another object) frames of reference may also operate to generate accurate movements despite intervening saccades or reach movements (Hayhoe et al., 1992, 2003; Dassonville et al., 1995; Karn et al., 1997; Obhi and Goodale, 2005; Krigolson et al., 2007; McGuire and Sabes, 2009; Byrne and Crawford, 2010). In the context of sequential saccades, exocentric representations of the second target can, in principle, allow the updating of second saccade vector independent of the efference copy, and therefore, be used for concurrent planning of sequential saccades. Indeed, it has been argued that immediate and delayed movements are processed differentially in an egocentric and exocentric frames of reference, respectively (Hu and Goodale, 2000; Westwood et al., 2000; Zimmermann et al., 2013), although both are likely to participate in generating stable and accurate target representations for movements (Byrne et al., 2010; Thompson et al., 2012; Schütz et al., 2013).
The aim of this study was to investigate the relative contributions of egocentric and exocentric representations in the concurrent processing of sequential saccades. We used a modified double-step task in which the second target occasionally changed location during the first saccade. Previous work (McPeek et al., 2000; Sharika et al., 2008) has shown that the frequency of second saccades to the original location of the second target in these trials is a measure of parallel programming. By changing the first target location in a small fraction of trials well before the efference copy of the first saccade can influence predictive planning, we tested the relative roles of exocentric and egocentric coding on the degree of concurrent planning as well as the degree to which the two concurrently planned saccades were linked together.
Materials and Methods
Subjects and recording setup.
Eye movements of 11 healthy human volunteers (5 males and 6 females), with normal or corrected to normal vision, were recorded with their heads stabilized by means of chin, temple and forehead rests. All participants gave their informed consent in accordance with the institutional human ethics committee of National Brain Research Centre and the Declaration of Helsinki. Subjects were monetarily rewarded for all sessions.
Experiments were computer-controlled using TEMPO/VIDEOSYNC software (Reflective Computing) that displayed visual stimuli, sampled and stored eye position with other behavioral parameters. Eye position was recorded with an infrared pupil tracker running at 240 Hz (ISCAN) that interfaced with TEMPO software (Reflective Computing) in real-time. The spatial resolution of our system was ∼0.01° and the median saccadic accuracy, as estimated by the SD of saccadic end-points across three successive trials to single targets presented in the task, was ∼0.7° (Sharika et al., 2013). Slower drifts in the eye-tracker system were inspected throughout the course of the experiment and were corrected by calibrating the tracker at the start of each half of the session and also resetting the gaze locations during the intertrial period when found to be consistently (>3–5 trials) off the calibrated center of the electronic windows at the start of the trial or at the end of saccades. All stimuli were presented on a computer screen (15 inch, 60 Hz refresh rate) placed 90 cm in front of the subject. Stimuli were calibrated with a Minolta CA-96 colorimeter.
Task and stimuli.
The delayed saccade double-step task used in this study was designed to test the role of the first saccade target as an exocentric reference for concurrent processing of the second saccade. For this purpose, although the majority of trials consisted of two targets being presented one after another, in small fractions of test trials, one or both these targets were shifted to new locations much before the efference copy of the first saccade was known to affect predictive planning of sequential saccades (∼300 ms before first saccade onset). The impact of shifting a stable reference on concurrent planning, while leaving the use of efference copy intact, was studied by examining the second saccade end-points following a shift in the second target location during the first saccade. Because second saccades to the original location of the second target were indicative of their preparation having begun before the end of the first saccade, the proportion of second saccades to the original versus new second target location was used as a metric of concurrent preparation (Sharika et al., 2008, 2013). We describe below the different types of trials that were interspersed randomly throughout the course of a session.
Approximately 79% of the trials in each session were “no-shift trials” (Fig. 1A) in which following fixation on a white square (0.15° × 0.15°) at the center of the screen, the first, green target (0.3° × 0.3°; 10.6 cd/m2) was presented at one of the four diagonal positions specified by a radial distance of 10.8° and polar angles of 45°, 135°, 225°, or 315° from the fixation point (Fig. 1B, top row). Thereafter, following a random delay (known as target-step delay) ranging from 10 to 210 ms, a second, red target (0.3° × 0.3°; 10.4 cd/m2) was presented randomly at any one of the two adjacent positions of the first target which would be at a radial distance of 7.6° and polar angles of either 0°, 90°, 180°, or 270° from the fixation point (Fig. 1B, middle and bottom rows show respectively the two possible second target locations for each first target position shown in the top row). While both targets, once presented, remained on the screen till the end of the trial, subjects were instructed to maintain fixation at the center till the disappearance of the fixation spot at the end of hold-time (∼1000 ms from the first target presentation) and then make a sequence of saccades to the green and red targets in the order of their presentation, as soon as possible (Fig. 1A, right). The long hold-time was used to encourage exocentric encoding of the second target with respect to the first target because such a representation has been previously shown to develop slowly over time (Hu and Goodale, 2000; Westwood et al., 2000; Zimmermann et al., 2013).
Schematic of a no-shift trial. A, Temporal sequence of events: fixation at a white square at the center of the screen is followed by the presentation of the first, green target. After a random target step delay (TSD; 10–210 ms), a second, red target (magenta square here) is presented at an angular distance of 45° from the first target. Subjects are instructed to maintain fixation until the white square at the center disappears after a hold time of 1000 ms from the presentation of the first target. This serves as a signal to make the first saccade to the green target followed by another one to the red target as soon as possible. Right, Dotted lines with arrowhead show the sequence of correct saccadic responses in such a trial. B, Top row, Probable locations of the first target at polar angles 45°, 135°, 225°, or 315° with respect to fixation. Middle and bottom rows, respectively, show the two probable locations of the second target for each first target location shown in the top row. The numbers in each case represent the polar angles of the second target with respect to fixation.
In addition to no-shift trials, seven types of test trials (identical to a no-shift trial except in ways specified below) were randomly interleaved with a probability of ∼3% each (amounting to 21% of total trials) in any given session (Table 1). They were as follows:
Concurrent planning probe trial (or simply, probe trial). To assess the natural tendency for concurrent planning of sequential saccades across subjects, the second, red target in these trials was stepped to a new position during the execution of the first saccade, where it remained until the end of the trial (Fig. 2). If second saccades were planned only after the end of the first saccade, i.e., when the second target has shifted, they are expected to always end at the new location of the second target. In contrast, in trials where the second saccade is planned in parallel, before the end of the first saccade, they are expected to at least sometimes end at the old location of the second target.
Typically, the second target shift occurred approximately half-way into the first saccade (grand average across subjects = 35 ± 0.8 ms from saccade onset; Table 2). Only those trials, in which the target shifted strictly during the execution of the first saccade, i.e., after it began but before it ended, were used in the analysis. The ‘shifted’ position of the second target (referred, hereafter, as the new position of the second target) was always the position adjacent to the first target in the same quadrant, and hence, at a polar angle of 90° from the original position of the second target (referred, hereafter, as the old position of the second target). Note that the old and the new positions of the second target were always equidistant from the first target so that the frequency of saccades to either of them was not biased by the distance to be covered. In addition, the luminosity of the targets was kept low to minimize any after-shift flash effects.
First target shift probe trial. The role of the first target as an exocentric reference for the concurrent planning of second saccades was tested by shifting the first target in these trials to a new position during the hold-time (grand average across subjects ∼82 ms before the disappearance of fixation box; Table 3) and on average >300 ms before the first saccade (see Table 9) and then shifting the second, red target to a new position during the execution of the first saccade (Table 2). The shifted position of the first target was always at a quadrant adjacent to that of the original first target location which also included the second target in it (Fig. 3).
First target shift control trial. To examine the effect of only shifting the first target on saccadic behavior, and thus serving as a control for ‘first target shift probe’ trials described above, the first target in these trials was shifted to a new position during the hold-time (grand average across subjects ∼82 ms before the disappearance of the fixation box; Table 4) and approximately >300 ms before the first saccade (see Table 9) where it stayed on till the end of the trial.
Both targets shift (same vector) probe trial. To test whether a second saccade plan, processed concurrently via exocentric encoding, could be executed independently of the first saccade plan or whether it was linked to the first saccade plan during preparation, the first and second targets in these trials were shifted to the adjacent quadrant during the hold-time (grand average across subjects ∼82 ms before the disappearance of fixation box (Table 3) and approximately >300 ms before the first saccade (see Table 9) such that the second saccade vector was maintained. The second, red target was subsequently stepped to a new position during the execution of the first saccade (Fig. 4).
Both targets shift (reverse vector) probe trial. As a control for ‘both targets shift (same vector) probe’ trials described above, the first and second targets in these trials were shifted to the diametrically opposite quadrant during the hold-time (grand average across subjects ∼83 ms before the disappearance of fixation box (Table 3) and approximately >300 ms before the first saccade (see Table 9), such that the original second saccade vector was reversed. Similar to (4), the second, red target was subsequently shifted to a new position during the execution of the first saccade. It may be noted that these trials are, in principle, equivalent to the first target shift probe trials, in that both the first and second saccade vectors were modified after the shift during the hold-time, though the second target location remained the same in the first target shift probe trials.
To prevent the anticipation of a second target shift during the first saccade in (4) and (5) above, the following two trials were introduced in a session.
Both targets shift (same vector) control trial. As a control for ‘both targets shift (same vector) probe’ trials described in (4) above, the first and the second targets in these trials were shifted to an adjacent quadrant during the hold-time (grand average across subjects ∼ 82 ms before the disappearance of fixation box; Table 4) and approximately >300 ms before the first saccade (see Table 9), such that the second saccade vector was maintained following the shift. The critical difference between (4) and these trials is the absence of the subsequent second target shift during the execution of the first saccade.
Both targets shift (reverse vector) control trial. As a control for ‘both targets shift (reverse vector) probe’ trials described in (5) above, the first and second targets in these trials were shifted to the diametrically opposite quadrant during the hold-time (grand average across subjects ∼ 81 ms before the disappearance of fixation box; Table 4) and approximately >300 ms before the first saccade (see Table 9), such that the original second saccade vector was reversed. It may be noted that these trials are, in principle, equivalent to the first target shift control trials in that both the first and second saccade vectors were modified after the shift, though the second target location remained the same in the first target shift control trials. The critical difference between (5) and these trials is the absence of the subsequent second target shift during the execution of the first saccade.
List of questions that were examined using the different types of probe and control trials in the delayed saccade double-step task (see text for details)
Sequence of events in a probe trial aligned to their representative time of occurrence. A, Top, Following fixation, the first green target and the second, red target (magenta square here) appear just as in a no-shift trial (Fig. 1) and subjects are instructed to maintain fixation till the white square at the center disappears. During the execution of the first saccade, the second target shifts to a new location in the same quadrant, 45° away from the first target. An unfilled, magenta square shows the old position of the second target for the convenience of the reader. Dotted arrow denotes mid-execution of the first saccade toward the first target. Bottom, Solid vertical line denotes the beginning of the trial. Horizontal lines trace the time of presentation of the fixation spot (F), first target (FT), second target (ST) and the shift of the second target during the first saccade (TS). B, Bottom, Eye positions corresponding to the horizontal (red) and vertical (blue) components of the first saccade are plotted for a sample of probe trials. Whereas the broken black vertical line represents the average end of the first saccade, the broken magenta vertical line denotes the mean time of the second target shift during the execution of the first saccade. Inset, Distribution of the times at which the second target shifts (blue) and the first saccade ends (orange), aligned to the onset of the first saccade (zero on the x-axis).
Mean and SEM for the time of the second target shift during the first saccade (with respect to its onset) for all valid and correct probe trials, first target shift probe trials, both targets shift (same vector) probe trials, and both targets shift (reverse vector) probe trials
Mean and SEM for the time of first and second targets' shift during the hold-time (with respect to the end of hold-time/disappearance of fixation spot) for all valid and correct first target shift probe trials, both targets shift (same vector) probe trials, and both targets shift (reverse vector) probe trials
Sequence of events in a first target shift probe trial aligned to their representative times of occurrence. A, All notations are the same as that in Figure 2 except that the first target (green square) shifts to a new position ∼82 ms before the end of hold-time. Bottom left, FTS is the time of first target shift in the trial. The shift is always to the adjacent quadrant in which the second target is already present. Top right, Dotted arrow represents the mid-execution of the first saccade to the shifted position of the first target. B, Distribution of the times at which the first target shifts (blue) and the first saccade begins onsets (magenta), aligned to the end of hold-time (black broken vertical line), in first target shift probe trials for a representative subject.
Mean and SEM for the time of first and second target shifts during the hold-time (with respect to the end of hold-time/disappearance of fixation spot) for all valid and correct first target shift control trials, both targets shift (same vector) control trials, and both targets shift (reverse vector) control trials
Sequence of events in a both targets shift (same vector) probe trial aligned to their representative times of occurrence. A, All notations are the same as that in Figure 2 except that the first and second targets shift to new locations in the adjacent quadrant, such that the second saccade vector is maintained. Bottom left, FTS and STS trace the time of first and second target shifts, respectively, ∼ 82 ms before the end of hold-time. Top right, Dotted arrow represents the mid-execution of the first saccade to the shifted location of the first target. Distribution of the times at which the first (red) and the second (blue) targets shift and the first saccade begins (magenta), aligned to the end of hold-time (black broken vertical line at zero), in (B) both targets shift (same vector) probe trials and (C) both targets shift (reverse vector) probe trials for a representative subject.
Trials were scored as successful and conveyed to subjects by auditory feedback, only if following fixation (within an electronic window of 3.7° × 3.7° centered on the fixation box) throughout the hold time, subjects made the first and second saccades, respectively, to the on-screen positions of green and red targets at the end of hold-time (fixating the respective targets within an electronic window of 5.6° × 5.6° centered on the targets). Trials in which the second target was shifted during the first saccade, only saccades to the new, and not old, second target position were conveyed as correct via online auditory feedback to dissociate the effects of positive feedback on the concurrent planning of saccades. However, for offline evaluation of performance in these trials, all second saccades that were directed to the old second target position were also considered correct.
All eight types of trials were randomized such that subjects could not predict or anticipate the appearance of target shifts. The overall percentage of test trials was kept low so that the performance strategy of subjects could be dictated by how they plan the sequence of saccades on the majority of trials, i.e., the no-shift trials. On average, each session lasted for ∼45 min to 1 h in which subjects performed ∼550 trials with a 5–10 min break in between two halves of the session. Subjects were given some practice trials (∼50) before data were collected in the first session. The total number of sessions ranged from three to six for each subject to obtain sufficient number of different trials to perform the analyses. Off-line analysis was done using MATLAB (MathWorks). The analog eye position data were smoothened and blinks removed. A velocity threshold of 40°/s was used to demarcate the beginning and end of saccades. All blink-perturbed saccades were eliminated from analysis. All statistical tests were done using MATLAB. Normality of data was checked using the Lilliefors test and nonparametric tests used wherever normality was violated.
Results
The delayed-saccade double-step task was used to assess the extent to which exocentric and efference copy-based representations contribute toward concurrent planning of sequential saccades. In the majority of the trials, two targets were presented one after another and subjects were asked to make a sequence of saccades to them, following a substantial fixation hold-time. In a small random fraction of trials, one or both targets were shifted to a new location much before (typically ∼300 ms before the first saccade) the efference copy of the first saccade has been reported to be available to the oculomotor system for predictive planning. The impact of upsetting exocentric encoding of second target on concurrent planning was then tested using trials in which the second target was further shifted during the first saccade. The proportion of second saccades to the old versus new location of the second target was used to measure the extent of concurrent planning following the loss of a stable reference, while the use of efference copy-based information was left intact.
Task performance
Table 5 shows the total number of trials collected for each subject. Of these trials, only those in which the subject maintained fixation throughout the hold-time, produced blink-free saccades, and the presentation of stimuli conformed to the design of the task (described in Materials and Methods) were considered “valid” for further analysis. Tables 5, 6, and 7 indicate the number of valid and correct trials for different trial types. A trial was defined as correct if the first saccade was made to the first target (green) present on the screen at the end of hold-time, whereas the second saccade was made to the red target present on the screen either at the end of the hold-time or at the end of the first saccade (i.e., to the location of the second target before or after the shift during the first saccade) in a probe trial. Also, data from only trial types wherein the subject performed above chance (Tables 6 and 7, No. of Correct Trials columns, not bold) were included in the analysis. The minimum number of trials analyzed per condition for any subject was 14 (subject UR, no. of correct both targets shift (same vector) probe trials).
The total number of collected trials, valid trials, correct trials and the percentage of valid trials that were correct as per the criteria described in Results
The number of valid and correct no-shift trials, first target shift control trials, both targets shift (same vector) control trials and both targets shift (reverse vector) control trials
The number of valid and correct probe trials, first target shift probe trials, both targets shift (same vector) probe trials and both targets shift (reverse vector) probe trials
Incorrect responses were also analyzed separately to examine any patterns in inaccuracy. Table 8 describes the data for all types of probe trials (i.e., trials in which the second target was shifted during the first saccade), because subjects, in general, made a greater number of errors in these trials as compared to all types of control trials. The table shows the number of trials in which subjects made an incorrect first saccade, as well as those in which a correct first saccade was followed by an incorrect second saccade. Remarkably, 98.8% of all these “incorrect” second saccades across subjects were small amplitude saccades that ended within the same target window as the first saccade, followed by a third saccade to the new location of the second target 75% of the times. These figures suggest that of the trials in which the subjects made a correct first response (i.e., 81% of the total valid trials presented in Tables 7 and 8), a majority of the incorrect second saccades were likely to be a result of insufficient concurrent preparation rather than noncomprehension of task instructions.
For all subjects, the number of valid but incorrect probe trials, first target shift probe trials, both targets shift (same vector) probe trials and both targets shift (reverse vector) probe trials
Concurrent preparation of the second saccade
We used probe trials (Fig. 2) to examine whether second saccades in the delayed saccade double-step task can be prepared in parallel with the first saccade. The hypothesis tested was that if the second saccades can be planned only after the end of the first saccade (by which time the second target has already shifted to a new location), they should always be directed to the new location of the second target. On the other hand, if the second saccade planning can begin before the end of the first saccade, instances of second saccades to the old position of the second target should be observed as well (McPeek et al., 2000; Sharika et al., 2008, 2013). Consistent with the former hypothesis, we found many trials across subjects in which the second saccades were directed at the new position of the second target. However, for all subjects we also observed saccades directed at the old position of the second target where a target no longer existed (Fig. 5). Because saccades to a location midway between the old and new second target positions were not observed in this task, we used the correct probe trials to plot the percentage of second saccades to the old versus new location of the second target for each subject (Fig. 6A). Consistent with the notion of concurrent planning and despite the fact that the second saccades to the old second target location were not reinforced during the task, they occurred on average 17.9 ± 4.4% of the times across subjects (n = 11; minimum = 2.5%, maximum = 45.5%), whereas the rest of the second saccades were directed at the new location of the second target location consistent with a serial processing strategy.
Behavior in probe trials. Eye position traces for a sample of probe trials in different subjects show first saccades (black) to the first target (green) and second saccades (blue) directed at the old (broken magenta) and new (solid magenta) locations of the second target. Squares represent the electronic windows used for classifying saccades based on their end-point.
Concurrent processing in the delayed-saccade double step task. A, Bar graphs showing the percentage of second saccades to the old versus new position of the second target in all valid and correct probe trials (see Results for selection criteria) for all 11 participants. B, Estimating Bcosϴ or the propagation of error in sequential saccades. Left and Middle, The green and magenta squares represent the first target and old location of the second target, respectively, in a probe trial. Whereas solid green and magenta arrows denote the first and second saccades, respectively, the broken black arrows, vector A (left) and vector B (middle), represent the localization errors in the first and second saccades, respectively. Right, Vectors A and B are brought together to meet at a point O with angle ϴ between them. The projection of B on A (Bcosϴ) is a measure of the magnitude of localization error in second saccade, which is in the direction of the localization error in the first saccade. C, Plot showing a significant positive correlation between mean Bcosϴ for second saccades directed at the old second target location in probe trials and the frequency of these saccades for nine subjects (see Results for selection criterion).
Concurrent planning of the second saccade has been previously associated with short intersaccadic intervals (ISIs) defined as the duration between the end of the first saccade and the onset of the second (Becker and Jürgens, 1979; Ray et al., 2004; McPeek et al., 2000; Sharika et al., 2008, 2013). Hence, if second saccades to the old location of the second target were processed in parallel, their ISIs are expected to be shorter than the ISIs associated with second saccades to the new second target location. We found this to be true [mean ± SEM: ISI (old) = 202.6 ± 12.4 ms; ISI (new) = 341.2 ± 14.5 ms, two-tailed Wilcoxon signed rank test; ISI (old, new): p = 0.001, n = 11]. In fact, the ISIs of second saccades to the old location of the second target were also significantly shorter than those of no-shift trials [mean ± SEM: ISI (no-shift) = 229.2 ± 9.7 ms, two-tailed Wilcoxon signed rank test; ISI (old, no-shift): p = 0.04, n = 11]. This is particularly consistent with the idea of second saccades to the old location of the second target being concurrently planned because the first saccade latencies in these trials (RT1, defined as the time between the disappearance of the fixation spot and the onset of the first saccade) were not statistically different from the RT1s of either no-shift trials [Tables 6⇑–Tables 8; mean ± SEM: RT1 (old) = 198.9 ± 16.8 ms; RT1 (no-shift) = 197 ± 9.8 ms, two-tailed paired t test; RT1 (old, no-shift): p = 0.85, n = 11], or trials in which the second saccades were directed at the new second target location [mean ± SEM: RT1 (new) = 196.5 ± 9.9 ms, two-tailed paired t test; RT1 (old, new): p = 0.83, n = 11). Also, the distributions of first saccade latencies in no-shift and probe trials were not significantly different for any subject (p > 0.05, two-sample Kolmogorov–Smirnov test on data from each subject).
As a further test of concurrent processing across subjects, we examined the localization errors of saccades that were planned in parallel with the first saccade (i.e., second saccades to the old second target location). Because an efference copy of the first saccade is likely to generate a relatively accurate estimate of how far the eyes will be from the second target following the first saccade, second saccades planned using the efference copy have been shown to compensate for localization errors in the first saccades (Bock et al., 1995; Ditterich et al., 1998; van Donkelaar and Müri, 2002; Sommer and Wurtz, 2004b; Collins, 2010; Joiner et al., 2010). On the other hand, in the absence of such information about the first saccade, second saccades planned using an exocentric representation of the second target would not account for the localization errors made in the first saccade. Therefore, errors in the first saccade are expected to propagate.
We computed an index for the magnitude of first saccade errors that was propagated, or in other words, not compensated by the second saccade. If the localization error in the first saccade with respect to the center of the first target is denoted by a vector A and that in the second saccade with respect to the center of the second target is represented by vector B, the magnitude of first saccade error that was propagated in the second saccade was estimated by computing the second saccade error in the direction of the first saccade's error or Bcosϴ (ϴ being the angle between vectors A and B; Fig. 6B; Collins, 2010). We calculated the average propagation of localization error in second saccades to the old location of the second target for all subjects with more than one such probe trial and examined its relationship with their respective tendency for concurrent processing. Although concurrent planning based on the efference copy of the first saccade predicts a decrease in error propagation with an increase in the frequency of second saccades to the old location of the second target across subjects, exocentric encoding of the second target predicts the propagation of first saccade errors to increase with an increase in the tendency for concurrent planning. We found the correlation between the average propagation of localization error and the frequency of second saccades to the old location of the second target (Fig. 6C) to be significantly positive across subjects (n = 11; slope = 0.04; Pearson's correlation coefficient, r = 0.62; p = 0.04). This is contrary to what is expected if concurrent planning of sequential saccades is completely based on the efference copy of the first saccade alone. In fact, these results are consistent with the possibility of an exocentric representation of the second target being used for concurrent planning of the second task in the delayed saccade double-step task.
Exocentric encoding in the concurrent planning of saccades
We used first target shift probe trials to examine the role of the first target as an exocentric reference for computing the second saccade vector in parallel. Using data from only those subjects who performed these trials accurately above chance (Table 7), we found that on an average the first target was shifted ∼338 ± 18 ms (grand mean across subjects ± SEM, n = 8; minimum = 263 ms, maximum = 420 ms) before the onset of the first saccade. This figure was obtained by adding the average duration between the first target shift and disappearance of the fixation spot (82 ms) to the mean first saccade latency of these trials across subjects (see Table 9). Shifting the first target toward the end of hold-time increased the latency of first saccades in these trials compared with those of probe trials [Tables 9; mean ± SEM: RT1 (probe trials) = 198.2 ± 10.6 ms; RT1 (first target shift probe trials) = 256.2 ± 18.1 ms, two-tailed paired t test; RT1 (probe, first target shift probe trials): p = 0.003, n = 8], which is consistent with the idea of preparing a new first saccade following the cancellation of the original plan which was no longer task relevant. Also, although the distribution of first saccade latencies was significantly different between probe and first target shift probe trials for six of eight subjects (with p = 0.02 or less when significant, two-sample Kolmogorov–Smirnov test), the same was true between no-shift and first saccade shift control trials for 9 of 11 subjects (with p = 0.03 or less when significant, two-sample Kolmogorov–Smirnov test).
If concurrent planning of saccades was solely based on the efference copy/corollary discharge of the first saccade and the latter is known to effect a change in cortical receptive fields not >100 ms before the onset of the first saccade (mean = 24 ms postsaccade onset; Sommer and Wurtz, 2004a), we hypothesized that shifting the location of the first target >300 ms before the first saccade should not upset the extent of concurrent planning in first target shift probe trials and should, thus, be equivalent to that observed in probe trials. In contrast, shifting the first target toward the end of hold-time is likely to make it an unstable reference for concurrent planning via exocentric encoding of the second target (Byrne and Crawford, 2010; Zimmermann et al., 2013), reducing the frequency of second saccades to the old second target location in first target shift probe trials compared with probe trials. Consistent with the latter hypothesis, we found that all eight participants showed a decrease in the percentage of second saccades to old second target position in the first target shift probe trials versus probe trials (Fig. 7B) and this decrease was statistically significant [mean ± SEM: Old (first target shift probe trials) = 6.6 ± 2.9%; Old (probe trials) = 20.6 ± 5.8%; two tailed Wilcoxon signed rank test: p = 0.008, n = 8].
Behavior in first target shift probe trials. A, Eye position traces for a sample of first target shift probe trials in different subjects. Notations are same as in Figure 6. Green broken square represents the electronic window of the first target before it was shifted during the hold time. B, Bar graphs showing the frequency of second saccades to the old versus new second target position in first target shift probe trials (yellow) versus probe trials (brown) for eight participants (see Results for selection criteria).
We also used the first target shift control trials and no-shift trials to compare the effect of shifting the first target on the propagation of localization errors in the second saccades. Because of the shift in the first target location, we expected to see lesser exocentric encoding-dependent propagation of error in the second saccades of first target shift control trials versus no-shift trials. We found this to be true [mean ± SEM: Bcosϴ (no-shift trials) = 0.56 ± 0.08, Bcosϴ (first target shift control trials) = 0.47 ± 0.09, two tailed, Wilcoxon signed rank test: p = 0.04, n = 11]. Together, these results represent, to our knowledge, the first clear evidence for the use of exocentric encoding of the second target, as distinct from the efference copy of the first saccade, in the concurrent planning of sequential saccades.
Relationship between concurrently prepared saccade plans
We used data from those subjects who performed both targets shift (same vector) probe trials accurately above chance to examine if the second saccades planned in parallel via exocentric encoding are linked to the first saccade plan during preparation or can be executed independently of the first saccade. The two targets were shifted on an average ∼309.4 ± 26.1 ms (grand mean across subjects ± SEM, n = 6; minimum = 229 ms, maximum = 406 ms) before the onset of the first saccade. This estimate was obtained by adding the average duration between the shifting of the two targets and disappearance of the fixation spot (83 ms) to the mean first saccade latency of these trials across subjects (Table 9). We tested whether the concurrent planning affected by shifting both the first and second targets but maintaining the second saccade vector is comparable to that affected by shifting only the first target, such that the second saccade vector is no longer the same (i.e., first target shift probe trials analyzed in the above section). If concurrent planning based on exocentric representations necessitated concatenation of individual motor plans, the change in the frequency of second saccades to the old second target position by shifting both targets, but maintaining the second saccade vector should be the same or greater than the decrease observed by shifting only the first target during hold-time. In contrast, if concurrent planning based on exocentric representations allowed the execution of the second saccade vector independent of the first saccade plan, the decrease in concurrent planning expected as a result of shifting both targets, but maintaining the second saccade vector [i.e., the frequency of second saccades to the old second target position in both targets shift (same vector) probe trials relative to probe trials] would be smaller than the decrease observed when only the first target is shifted (i.e., the frequency of second saccades to the old second target position in first target shift probe trials relative to probe trials).
The mean and SEM for first saccade latencies or reaction times (RT1) of all eight types of trials in the delayed saccade double-step task where subjects performed these trials correctly above chance
As expected, there was a significant reduction in the percentage of second saccades to the old location of the second target in both targets shift (same vector) probe trials compared with probe trials [Fig. 8A; n = 6, mean ± SEM: Old (probe trials) = 17 ± 6.8%; Old (both targets shift (same vector) probe trials) = 7.9 ± 3.7%, one tailed, pairwise t test; Old % (probe trials > both targets shift (same vector) probe trials), p = 0.046]. On comparing this reduction to that obtained for first target shift probe trials across all six subjects (Fig. 8B), we found that for half of these subjects (UR, HS, and SS), the reduction in the frequency of second saccades to the old location of the second target was similar in the two trial types suggesting that maintaining the second saccade vector did not allow the independent execution of the concurrently planned second saccade. For the other half of the subjects (RS, IP, and MK), the decrease in the frequency of second saccades to the old location of the second target was lesser in the both targets shift (same vector) probe trials when compared with that in the first target shift probe trials, suggesting that the concurrently planned second saccade could be executed independently at least to some extent on account of the second saccade vector remaining the same in these subjects.
Behavior in both targets shift (same vector) probe trials. A, Frequency of second saccades to the old versus new second target position in both targets shift (same vector) probe trials (blue) versus probe trials (purple) for six subjects (see Results for selection criteria). B, Effect of disrupting the use of exocentric reference but maintaining the second saccade vector on concurrent planning of second saccades is examined by plotting the decrease in the frequency of second saccades to the old second target location in first target shift probe trials relative to probe trials (on the x-axis) against the similar decrease in both targets shift (same vector) probe trials relative to probe trials (on the y-axis) for the same six subjects as above. Red broken line denotes the line of unity.
Discussion
We show for the first time the role of exocentric representations, as distinct from efference copy-based egocentric mechanisms, in the concurrent planning of sequential saccades and how this may allow programming of two saccades as a single action before the first movement starts.
Concurrent processing of sequential saccades
A measure of parallel programming of sequential saccades is the inverse relation between ISI and the reprocessing time (RPT; i.e., the time between the presentation of the second target and the onset of the first saccade), which is the time available for concurrent planning (Becker and Jürgens, 1979; McPeek et al., 2000; Ray et al., 2004). However, this relation is an average measure across trials. In previous work (Sharika et al., 2008), we have shown the frequency of second saccades to the old target location in probe trials (known as target-shift trials in the study) to increase monotonically with RPT. In addition, because it is only saccades that are sufficiently prepared in parallel that cannot be aborted despite a shift in the target location, second saccades to the old target location are a reliable measure of concurrent processing. This is in contrast to the classic double-step task where the timing of remapping and planning of the second saccade is ambiguous and could have occurred even after the onset of the first saccade. Together, using a relatively stable and predictable task, we could isolate a subset of trials (second saccades to the old target location) in which remapping is predictive and, whose frequency, across subjects, correlated well with the average propagation of localization errors that was used as an index of the extent of exocentric encoding (Hayhoe et al., 1992; Dassonville et al., 1995; van Donkelaar and Müri, 2002).
Exocentric and egocentric representations enable concurrent planning
Although the role of exocentric cues has been previously examined in relation to the localization of postsaccadic targets and in visual perception (Hallett and Lightstone, 1976; Matin et al., 1982; Dassonville et al., 1992, 1993, 1995; Hayhoe et al., 1992, 2003; Honda, 1993; Karn et al., 1997; Obhi and Goodale, 2005; Krigolson et al., 2007; McGuire and Sabes, 2009; Byrne and Crawford, 2010; Tatler and Land, 2011), their contribution to concurrent preparation of sequential saccades has not been assessed before. We examined this by studying the frequency of second saccades to the old target location after presumably upsetting its exocentric encoding but leaving the use of an efference copy-based egocentric representation (in oculocentric or craniocentric coordinates) for concurrent planning intact. A critical assumption underlying this conclusion is that the efference copy for predictive remapping is available only around the onset of the first saccade. If, however, subjects were to automatically generate a saccade plan to the first target during the hold-time, the reduction in parallel programming may be attributable to a very early egocentric-based remapping signal instead of exocentric encoding. We believe this is unlikely because remapping, as assessed from either perceptual/saccadic mislocalization or neural activity, has been found to be specifically linked to saccade onset (maximum ∼100 ms before it) and not to target presentation (Matin et al., 1982; Honda, 1989, 1993; Duhamel et al., 1992; van Donkelaar et al., 1992; Dassonville et al., 1995; Umeno and Goldberg, 1997; Ross et al., 2001; Nakamura and Colby, 2002; Medendorp et al., 2003; Sommer and Wurtz, 2004a,b; Melcher, 2007; Parks and Corballis, 2010; Burr and Morrone, 2011; Hall and Colby, 2011; Rolfs et al., 2011; Wurtz et al., 2011). In addition, given that the first saccade latencies in the task (Table 9) fall within the normal range, there is sufficient scope for the natural expression of efference-based planning of second saccade to occur. Instead, we suggest that the significant reduction in parallel programming by manipulation of the first target is a consequence of the disrupted exocentric encoding of the second target. Indeed, the presence of stimuli for substantial duration is a common aspect of tasks that have demonstrated spatiotopic coding of stimuli (Hu and Goodale, 2000; Melcher and Morrone, 2003; Dean and Platt, 2006; Zimmermann et al., 2011, 2013).
Nevertheless, other reasons could also explain our results. For example, the decrease in the frequency of second saccades to the old second target location in first target shift probe trials may be a consequence of additional attention or perceptual requirements due to the change in the first target location. We think it is unlikely because the first saccade latencies of first target shift control trials were long enough to overcome these extra requirements (Table 9) and efference copy-based concurrent planning being linked to the onset of the saccade, and not to the target or target shift, should not have been compromised. It is also possible that shifting the first target toward the end of hold-time triggered a change of cognitive set or strategy which discouraged concurrent planning of the second saccade (Sharika et al., 2013). This is unlikely considering that 79% of trials in a session were no-shift trials and an additional 9% were first target shift control or both targets shift control trials where it would still be behaviorally relevant to plan the second saccade in parallel. In contrast, it is more likely that the task used in this study (characterized by long target display durations and high-frequency of no-shift trials) prompted the use of the first target as a reliable reference for representing the second target location and subsequent preparation of the second saccade in parallel. However, we did observe evidence for predictive planning based on egocentric representations in trials where second saccades were directed to the old target location, despite changes in first target position (particularly clear in subjects, MK, GA, and HS). Overall, our results are consistent with both egocentric and exocentric representations guiding concurrent planning of sequential saccades.
Exocentric encoding of targets and chaining of responses
One disadvantage of planning movements based on egocentric representations is that they need to be constantly updated or transformed after any preceding movement in the sequence. Exocentric representations, on the other hand, being coded in external space and relative to other target locations, do not require such constant updating, and can thus chain multiple stimuli or responses into a coherent action. The behavioral correlates of such concatenation of sequential plans into a single package, also known as chunking (Lashley, 1951; Sternberg, 1969), are longer first movement latencies that increase in proportion to the number of movements in the chunk as well as the smaller intermovement durations within a chunk of movements (Rosenbaum et al., 2007; Verwey et al., 2010).
A sequence of saccades to visible, as well as remembered, stimuli locations has been previously proposed to be organized as chunks with their first saccade latencies being longer than the interval between the saccades (Becker and Fuchs, 1969; Inhoff, 1986; Zingale and Kowler, 1987; Ditterich et al., 1998; Hayhoe et al., 2003). Fixed amplitude saccades have also been shown to be preplanned as part of a sequence and executed without compensating for the end-point errors of the preceding saccades (Doré-Mazars et al., 2006). In the current study, although some signatures attributed to chunking were seen, such as the smaller intersaccadic intervals in probe trials with second saccades to the old target location and a correlation between the propagation of errors and the degree of concurrent preparation (Fig. 6C), we did not observe the first saccade latencies to be any longer than their intersaccadic intervals, even for subjects who were unable to execute the second saccade independently of the first one in both targets shift (same vector) probe trials (Fig. 8B). This may be explained by the presence of the long hold-time following the presentation of targets in the delayed-saccade double-step task, which could potentially allow considerable preprogramming of saccades to occur, even before the signal to execute the sequence was made explicit.
The use of exocentric encoding for chunking was tested by maintaining the relative displacement between the two targets but changing their absolute locations. A significant decrease in concurrent processing observed in this case suggests that maintaining the second saccade vector did not help preserve the extent of concurrent planning (Fig. 8A). Also, a striking relation between the loss of parallel programming when the second saccade vector was maintained versus when it was not suggested that a disruption of exocentric coding tended to disrupt chunking (Fig. 8B). Thus, exocentric representations were likely to allow during the course of the hold-time an association between the first and second saccades such that if the first saccade is cancelled due to changes in the first target location, the second saccade is cancelled too. However, the heterogeneity in performance observed across individual subjects suggests that the oculomotor system may be equipped with the flexibility of using exocentric representations in multiple ways during concurrent planning of sequential saccades.
Footnotes
This work was supported by the Department of Biotechnology, Government of India. We thank both reviewers for their comments and suggestions.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr Aditya Murthy, Centre for Neuroscience, Indian Institute of Science, Bengaluru 560012, Karnataka, India. aditya{at}cns.iisc.ernet.in