Abstract
As we shift our gaze to explore the visual world, information enters cortex in a sequence of successive snapshots, interrupted by phases of blur. Our experience, in contrast, appears like a movie of a continuous stream of objects embedded in a stable world. This perception of stability across eye movements has been linked to changes in spatial sensitivity of visual neurons anticipating the upcoming saccade, often referred to as shifting receptive fields (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997; Nakamura and Colby, 2002). How exactly these receptive field dynamics contribute to perceptual stability is currently not clear. Anticipatory receptive field shifts toward the future, postsaccadic position may bridge the transient perisaccadic epoch (Sommer and Wurtz, 2006; Wurtz, 2008; Melcher and Colby, 2008). Alternatively, a presaccadic shift of receptive fields toward the saccade target area (Tolias et al., 2001) may serve to focus visual resources onto the most relevant objects in the postsaccadic scene (Hamker et al., 2008). In this view, shifts of feature detectors serve to facilitate the processing of the peripheral visual content before it is foveated. While this conception is consistent with previous observations on receptive field dynamics and on perisaccadic compression (Ross et al., 1997; Morrone et al., 1997; Kaiser and Lappe, 2004), it predicts that receptive fields beyond the saccade target shift toward the saccade target rather than in the direction of the saccade. We have tested this prediction in human observers via the presaccadic transfer of the tilt-aftereffect (Melcher, 2007).
Introduction
The tilt-aftereffect occurs when an oriented grating (adaptor) is viewed for a prolonged duration. A subsequently presented test grating (probe) is then perceived as tilted away from the orientation of the adaptor. This repellent effect is explained by an unbalanced population response to the probe due to neural adaptation toward the orientation of the adaptor. The tilt-aftereffect is strongest when probe and adaptor are presented at the same retinotopic location but, immediately before a saccade, the tilt-aftereffect for an adaptor presented close to the locus of fixation can be transferred to the saccade target (Melcher, 2007). This transfer has been interpreted as evidence of a presaccadic shift of receptive fields or feature detectors in the ventral pathway of the human visual system. An important consequence of this interpretation is the question about the nature of the presumed receptive field shifts. Two cases deserve particular consideration. Feature detectors might shift in the direction of a saccade, consistent with the idea that the receptive field of neurons is updated in anticipation of the eye movement to its postsaccadic, i.e., future receptive field location (Duhamel et al., 1992; Sommer and Wurtz, 2006; Wurtz, 2008), generalizing the concept of predictive remapping from the pure spatial domain to the domain of feature selectivity. Alternatively, feature detectors might shift toward the saccade target, as suggested by cell recordings in monkey V4 (Tolias et al., 2001) and the phenomenon of perisaccadic compression (Morrone et al., 1997; Ross et al., 1997; Lappe et al., 2000; Kaiser and Lappe, 2004; Hamker et al., 2011). These different predictions allow us to dissociate between receptive field shifts toward the future, postsaccadic position and toward the saccade target if we present the adaptor in the periphery (Fig. 1A).
The subject's task was to judge the orientation of a briefly flashed probe stimulus which was shown after the presentation of an adaptor stimulus. The spatial layout of the stimulus arrangement is shown in Figure 1B (see Materials and Methods for details). To distinguish between an update of the tilt-aftereffect and a transfer toward the saccade target, the adaptor position was chosen above and slightly to the right of the saccade target. A probe could be presented either at the adaptor position, at the future position, or at the saccade target position. Each trial of the saccade condition consisted of an adaptation period followed by a rapid eye movement triggered by a displacement of the fixation point. In the fixation condition subjects were also required to judge the orientation of the probe, but to keep fixation throughout a trial.
Materials and Methods
Subjects and apparatus.
Three subjects, two of them authors, with normal or corrected to normal vision participated in this study. The experiment was conducted in an illuminated room (140 lx). Stimuli were viewed on a 22 inch CRT (iiyama Vision Master Pro 514) with a display size of 40 × 30 cm from a distance of 51 cm. The monitor was run with a temporal resolution of 80 Hz and a spatial resolution of 2046 × 1530 pixels driven by a Power Mac 7.4. Stimuli were generated in MATLAB (MathWorks) and eye position was monitored using Eyelink II (SR Research).
Visual stimuli and procedure.
Both the adaptor and probe consisted of oriented Gabor-gratings with a spatial frequency of 0.8°/cyc, a Gaussian envelope with a σ of 1°, and Michelson contrast of 0.94. They were presented on a gray background with a luminance of 40 cd/m2 (Minolta LS-110). The adaptor was always presented at the adaptor position with an eccentricity of 18.03° (x = 15°, y = 10°). The probe could be presented at the adaptor position, or at the future position with an eccentricity of 26.93° (x = 25°, y = 10°), or at the saccade target position with an eccentricity of 11.75° (x = 11.4°, y = 2.86°). The adaptor was either tilted −20° to the left or 20° to the right from vertical in blocked conditions. The probe orientation was randomly chosen out of 9 different possibilities. When the probe was presented at the future position or the saccade target position, its orientation ranged from −4° to 4° in steps of 1°. When it was presented at the adaptor position, its orientation ranged from −8° to 8° in steps of 2°. The larger orientation range of the Gabor-gratings presented at the adaptor position was necessary because of the larger tilt-aftereffects found at that position. In the saccade condition subjects initially fixated a small white dot (0.5° in diameter) with a luminance of 136 cd/m2. After 500 ms the adaptor appeared and remained on the display for 30 s if it was the first trial of a block and thereafter remained on for 3 s. After a random delay between 100 and 200 ms the fixation point was displaced 10° to the right serving as the signal for the subjects to initiate a saccade. After a further k ms the probe was flashed for 50 ms and subjects indicated whether they perceived the probe as tilted to the left or right from vertical. Note that k was adjusted individually for each subject during the experiment to present the probe shortly before saccade onset. The fixation condition was identical to the saccade condition with the exception that the initial fixation point remained stationary and no saccade had to be executed. In both conditions subjects have been instructed to attend the fixation spot and to ignore the adaptor. One block lasted for ∼15 min. After an initial training phase subjects typically conducted 2–4 blocks a day in no particular order over several months until the required amount of measurements as described below was achieved.
Data analysis.
For data analysis trials were included that matched the following criteria. In the fixation condition subjects had to keep fixation in a circular region of 2° in radius centered at the fixation point throughout the whole trial. In the saccade condition the saccade had to start within a region of 2° in radius around the fixation point and to land in 2° radius around the saccade target. Furthermore, saccadic latencies had to be larger than 100 ms and shorter than 400 ms. Probe offset had to occur before saccade onset but not before 100 ms before the eye movement, that is, for all valid trials probes were presented presaccadically. Valid trials were then used to estimate psychometric functions for each subject in each condition for each position and adaptor orientation yielding a total of 12 psychometric functions per subject. To allow for a robust estimate of the respective psychometric functions 22 measurements were required for each of the 9 probe orientations (Wichmann and Hill, 2001). To calculate the tilt-aftereffect in the different conditions the following logistic regression was used:
where P(r) is the probability of a given response, in our case a rightward response, c is the set of experimental conditions—unique combinations of saccade instruction (saccade or fixation) and probe position (adaptor position, future position, saccade target position)—that was used in the regression. θ is the probe orientation, Ic is an indicator variable that is 1 for trials in c and 0 for other trials, and Lc is an indicator variable that is 1 for adaptor orientation of 20° in c and 0 for the rest of trials. βc0 are regression coefficients that define the center of the psychometric function (point of subjective equality) for −20° adaptor orientation in condition c. βc1 are coefficients that define the shift of the psychometric function (tilt-aftereffect) for 20° adaptor orientation in c. The S coefficient is the common slope of the psychometric functions. Similar results were obtained by assuming variable slopes. Because the tested range of probe orientations differed between the adaptor position and the future and saccade target position, we performed two separate fits: one that included trials where the probe was presented at the adaptor position, and another that included trials where the probe was presented either at the future or the saccade target position. A maximum likelihood fitting procedure was used to estimate the coefficients and their SEs. The quality of the fits was measured with R2, one minus the fraction of unexplained variance for the data points and the respective psychometric functions. All fits were reasonably ranging from 0.95 to 0.99. To estimate the change of the tilt-aftereffect between different conditions, we redefined I and L, and rearranged the β coefficients to create new coefficients that corresponded to the estimated quantities.
Results
Figure 1C qualitatively shows the predictions of receptive field shifts toward the future position and receptive field shifts toward the saccade target. If receptive fields shifted toward the future location the tilt-aftereffect of probes presented immediately before saccade onset should be larger at the future position than at the saccade target position. If instead receptive fields shifted toward the saccade target, the tilt-aftereffect should be larger at the saccade target position than at the future position. Figures 2 and 3 show the results of the experiment. In the saccade condition (Fig. 2A) the tilt-aftereffect of each subject (p = 3.89 × 10−7, p = 4.77 × 10−15, p = 1.83 × 10−7) and also the combined data (p < 1 × 10−16) are larger at the saccade target position than at the future position (see Data analysis for statistical details) as it can also been seen in Figure 3 depicting the psychometric functions of the combined data. As for the tilt-aftereffect in the fixation condition the difference between the saccade target position and the future position is only significant for one subject (S1) with p = 1.44 × 10−5. For the other two subjects and for the combined data the difference is not significant (p = 0.45, p = 0.07, p = 0.06). Compared with the tilt-aftereffect in the fixation condition (Fig. 2B) the tilt-aftereffect in the saccade condition shows the same qualitative trend, an increase at the saccade target position and a decrease at the future position (Fig. 2C), resulting in a significant change of the difference of the tilt-aftereffect between the positions at the single subject level (p = 1.34 × 10−13, p = 1.26 × 10−8, p = 0.02) and in the combined data (p = 1.11 × 10−16). The tilt-aftereffect at the adaptor position was strong in all subjects in the fixation condition (10.58°, 8.78°, 8.87°), and similar in strength in the saccade conditions (10.21°, 10.76°, 9.03°) with the following p-values of the presaccadic change for the single subjects p = 0.3, p = 0.01, p = 0.42 and the combined data p = 0.14.
Hypothetical relationship between receptive field shifts and the tilt-aftereffect. A, Schematic illustration of two possible types of presaccadic receptive field dynamics. Two cells with their current receptive fields (CRF) are shown. For a current receptive field close to fixation, both types shift toward the future receptive field (black dashed circles), and shifts toward the saccade target (gray solid circles) are qualitatively similar and can hardly be distinguished since both change in the direction of the saccade. For a receptive field above the saccade target, both types can be dissociated since a presaccadic shift toward the future receptive field consists of a change in the direction of the saccade, whereas a shift toward the saccade target consists of a change orthogonal and against the saccade direction. Note the depicted receptive fields are intended to qualitatively illustrate the direction of the change in receptive field location and do not provide information about the absolute size. B, To the left, the spatial layout of stimuli used to measure the presaccadic tilt-aftereffect is shown. The position of the adaptor is abbreviated as AP, while FP and SP denote the future position and saccade target position, respectively. C, The two predictions for the tilt-aftereffect (TAE) made by shifts toward the future receptive field and shifts toward the saccade target. If a shift toward the future receptive field was true, the presaccadic tilt-aftereffect measured at the future position should be larger than the tilt-aftereffect at the saccade target position. In contrast, if a shift toward the saccade target was true, the tilt-aftereffect measured at the future position should be smaller than the tilt-aftereffect at the saccade target position.
Results for the future position (FP) and the saccade target position (SP) for all single subjects (S1, S2, S3) and the combined data. A, Tilt-aftereffect (TAE) as measured immediately before a saccade. Error bars indicate SEs. The tilt-aftereffect is significantly stronger for all subjects and the combined data at the saccade target position. B, Tilt-aftereffect measured during continuous fixation serving as baseline. C, Difference of the tilt-aftereffect in the saccade and fixation condition. The tilt-aftereffect tends to increase at saccade target position and to decrease at the future position. This differential effect is significant for all subjects and the combined data.
Psychometric functions of the combined data for the future position (FP) and the saccade target position (SP). A, Saccade condition. B, Fixation condition. Solid curves correspond to the leftward oriented adaptor (−20°) and dashed curves to the rightward oriented adaptor (20°) relative to vertical (0°). Compared with the fixation condition, the tilt-aftereffect, i.e., the separation of the psychometric functions, increases at the saccade target position and decreases at the future position.
A subsequent control experiment revealed no significant tilt-aftereffect for probes at the fixation point (x = 1.4°, y = 2.86°) in the fixation condition when the adaptor was presented at the original position of the main experiment (x = 15°, y = 10°). It is thus unlikely that our observed effects at the saccade target are caused by a parallel remapping of receptive fields from the fovea to the saccade target. The observed tilt-aftereffect for the three subjects and the combined data in this control is 0.09° with p-values of p = 0.28, p = 0.33, p = 0.34, and p = 0.23.
Discussion
The observed pattern of the presaccadic changes of the tilt-aftereffect qualitatively resembles the one of perisaccadic compression, revealed by localizations of briefly flashed stimuli around the time of saccades (Morrone et al., 1997; Ross et al., 1997; Lappe et al., 2000; Kaiser and Lappe, 2004). Both are directed toward the saccade target and start before the eye movement.
This pattern of the tilt-aftereffect supports an alteration of receptive fields closer toward the saccade target rather than to their respective future positions. From an electrophysiological point of view the receptive field of a neuron refers simply to the region in visual space that causes a neural response if stimulated appropriately. A shift of the receptive field thus occurs when the spatial sensitivity of the neuron is modulated. As this study does of course not allow to directly infer a change in receptive fields, a possible interpretation of our observations can be made by a recent computational model of perisaccadic perception (Hamker et al., 2008). In this model a mandatory presaccadic attentional focus on the saccade target region (Hoffman and Subramaniam, 1995; Deubel and Schneider, 1996), implemented as a corollary discharge from oculomotor control areas, causes a focal, nonuniform, neural gain alteration which in turn leads to anticipatory receptive field shifts, similar as observed by cell recordings in monkey V4 (Tolias et al., 2001), and to perisaccadic compression. If this were true, one would expect that more adapted cells participate in the orientation judgment of the probe—leading to an increased tilt-aftereffect at the saccade target position, since their receptive field is drawn from the adapted area in direction to the saccade target.
The attention-induced receptive field changes can also account for the observation made by Melcher (2007). When probe and adaptor are both presented at the saccade target the tilt-aftereffect is reduced before an eye movement compared with a fixation condition using the same stimulus configuration. This observation is hard to explain by a concept of spatial attention that considers only a change in response sensitivity but can easily be explained by the additional notion of a shift of receptive fields that is induced by the gain change when one takes the entire population response into account: more peripheral receptive fields of rather unadapted cells shift closer toward the saccade target and thus reduce the tilt-aftereffect on the population level.
As far as the direction and magnitude of a presaccadic receptive field shift is concerned, the model predicts a dependency on the current receptive field location, which varies for different regions of visual space (Hamker et al., 2008; Zirnsak et al., 2010). For example, receptive fields located close to the fovea show presaccadic changes along the saccade direction, but shorter than it would be expected by a complete transfer to the future, postsaccadic receptive field (Zirnsak et al., 2010). This is consistent with the observation that the tilt-aftereffect of a probe presented at the intermediate position between the initial fixation and the saccadic target is even stronger than the tilt-aftereffect of a probe at the saccade target (Melcher, 2007). Receptive fields located above and beyond the saccade target, on the other hand, show a presaccadic change toward the saccade target (Hamker et al., 2008; Zirnsak et al., 2010) rather than parallel to the saccade vector toward the future receptive field. This is consistent with the results of the present study.
The design of our study allowed us to compare two locations, the saccade target and the future stimulus position. The pattern of presaccadic changes of the tilt-aftereffect supports a shift of receptive fields closer toward the saccade target rather than to their respective future positions. The exact pattern of receptive field modulations might be even richer. If receptive field shifts result from dynamic gain alterations of neurons, various types of receptive field changes are possible (Hamker et al., 2008; Zirnsak et al., 2010). Depending on the exact properties of the corollary discharge signal and possible lateral inhibitory connectivity receptive field shifts might be complete, i.e., involve a simultaneous increase of sensitivity at a previously nonresponsive location and decrease at previously responsive location, or incomplete. In the latter case, receptive fields may stretch, become transiently bimodal, or shift with different latencies or in different proportions. The remaining sensitivity to stimuli presented in the current receptive field (Sommer and Wurtz, 2006), as indicated by the strong tilt-aftereffect at the adaptor position in the saccade condition, may be explained by any such incomplete shift.
To summarize, the results of our study add to the idea that trans-saccadic perception involves dynamic changes in the receptive field structure (Melcher and Colby, 2008; Wurtz, 2008; Burr et al., 2010). Particularly, it appears that brain areas involved in feature analysis are actively drawn in to process the future fixation in greater detail. While this seems particularly relevant before eye movements (overt shifts of attention), similar receptive field dynamics have been observed during covert shifts of attention (Connor et al., 1996, 1997; Womelsdorf et al., 2006, 2008; Anton-Erxleben et al., 2009), suggesting a general mechanism of dynamic allocations of processing resources to attended locations. Our previous studies suggest that a minimum receptive field size beyond the one of V2 is required to observe sufficiently large receptive field shifts (Hamker et al., 2008). Since the paradigm used in this study requires the identification of form, a likely candidate would be the ventral pathway from the level of V4 onwards. Indeed it has been shown that oculomotor areas such as the frontal eye field can affect the gain of cells in area V4 of the macaque (Moore and Armstrong, 2003) and induce changes in receptive fields (Armstrong et al., 2006). Compared with other recent studies which explicitly investigated the updating of sustained spatiotopic (Golomb et al., 2008; Pertzov et al., 2010; Rolfs et al., 2011) or transient (Mathôt and Theeuwes, 2010) visual attention at locations different from the saccade target, our effects might be interpreted as a result of a mandatory attentional focus toward the saccade target. For example, Rolfs et al. (2011) observed a presaccadic sensitivity increase at the saccade target, consistent with the assumption of a mandatory attentional focus toward the saccade target, but in addition a presaccadic updating of the sustained attentional focus toward its new retinotopic location. Our study did not attempt to test this location which is in the opposite direction to the eye movement, since our subjects have been instructed to ignore the adaptor, particularly because this is likely to further complicate the final pattern of receptive field shifts.
To conclude, our results thus suggest the ventral pathway does not participate in the subjective experience of spatial stability by a feature-selective updating of receptive fields toward their future location. Instead, it may play a role in maintaining object continuity across saccades by focusing the processing resources on the object of interest already before the eyes start to move.
Footnotes
We gratefully acknowledge support by the German Science Foundation (DFG HA 2630/4-1, DFG LA 952/3-2), the European Commission (FP7-ICT: Eyeshots), and the Federal Ministry of Education and Research (BMBF 01GW0653). We thank Rufin Van Rullen and Heiner Deubel for valuable comments on a previous version of this manuscript.
- Correspondence should be addressed to Fred H. Hamker, Department of Computer Science, Artificial Intelligence, Chemnitz University of Technology, Straβe der Nationen 62, 09107 Chemnitz, Germany. fred.hamker{at}informatik.tu-chemnitz.de
References
In this issue
