Abstract
The ability to perceive the visual world around us as spatially stable despite frequent eye movements is one of the long-standing mysteries of neuroscience. The existence of neural mechanisms processing spatiotopic information is indispensable for a successful interaction with the external world. However, how the brain handles spatiotopic information remains a matter of debate. We here combined behavioral and fMRI adaptation to investigate the coding of spatiotopic information in the human brain. Subjects were adapted by a prolonged presentation of a tilted grating. Thereafter, they performed a saccade followed by the brief presentation of a probe. This procedure allowed dissociating adaptation aftereffects at retinal and spatiotopic positions. We found significant behavioral and functional adaptation in both retinal and spatiotopic positions, indicating information transfer into a spatiotopic coordinate system. The brain regions involved were located in ventral visual areas V3, V4, and VO. Our findings suggest that spatiotopic representations involved in maintaining visual stability are constructed by dynamically remapping visual feature information between retinotopic regions within early visual areas.
SIGNIFICANCE STATEMENT Why do we perceive the visual world as stable, although we constantly perform saccadic eye movements? We investigated how the visual system codes object locations in spatiotopic (i.e., external world) coordinates. We combined visual adaptation, in which the prolonged exposure to a specific visual feature alters perception, with fMRI adaptation, where the repeated presentation of a stimulus leads to a reduction in the BOLD amplitude. Functionally, adaptation was found in visual areas representing the retinal location of an adaptor but also at representations corresponding to its spatiotopic position. The results suggest that an active dynamic shift transports information in visual cortex to counteract the retinal displacement associated with saccade eye movements.
Introduction
How we maintain our sense of visual space in the face of permanent eye movements is a central question in visual neuroscience (Burr and Morrone, 2011; Crespi et al., 2011). Saccadic eye movements, which are continuously performed in everyday life, constantly alter the relationship between retinal and external space. Integrating the steadily changing retinal images into a spatially coherent representation of the outside world requires information about how retinal coordinates relate to the external world. Several hypotheses have been advanced to explain how visual space is stabilized across eye movements: Objects may generally be coded in real-world coordinates, thereby rendering visual space invariant to displacements of the eye. Electrophysiological studies, however, failed to identify explicit spatiotopic maps of external space (Wurtz et al., 2008). Functional imaging studies in humans reported mixed evidence for spatiotopic processing. Although area MT (d'Avossa et al., 2006; Crespi et al., 2011) and lateral occipital cortex (McKyton and Zohary, 2007) have been reported to contain object representations in external world coordinates, other studies seriously questioned the existence of spatiotopic maps in the human brain (Gardner et al., 2008; Golomb and Kanwisher, 2012; Merriam et al., 2013).
It is also known that retinotopic space is dynamically updated after every saccade. The underlying mechanism is commonly referred to as remapping and suggests that visual areas are predictively informed about an upcoming saccade allowing for compensation of retinal displacement (Duhamel et al., 1992; Nakamura and Colby, 2002).
We combined functional and behavioral adaptation to investigate the neural mechanisms underlying the formation of gaze-invariant information in the human brain. In behavioral adaptation, prolonged exposure to an adapter stimulus induces a measurable aftereffect that alters perception of a subsequently shown probe stimulus (Clifford et al., 2000). A saccade between the presentation of the adapter and a probe stimulus allows dissociating retinal and external space coordinates: Following the saccade, the probe stimulus can either be shown at the spatiotopic position, which matches the adapter location on the screen; or at the retinotopic position, which matches the adapter location on the retina. Measurable adaptation effects are expected at both the retinotopic and the spatiotopic positions (Melcher, 2005; Zimmermann et al., 2013).
The mechanisms generating behavioral adaptation are used in the fMRI-adaption paradigm. In fMRI adaptation, the repeated presentation of similar stimuli (adapter and probe) leads to a reduction of the BOLD amplitude (Kourtzi and Huberle, 2005; Krekelberg et al., 2006). We used adaptation to a tilted grating, which has been shown to produce reliably fMRI adaptation during a fixation task (Boynton and Finney, 2003; Murray et al., 2006).
Spatiotopic representations may result from dynamically remapping retinotopic information in conjunction with eye movements. In that case, saccades and the retinal displacements caused are expected to induce measureable shifts of activation within retinotopic visual regions. In particular, a saccade moving an adapted location from the right to the left visual field is expected to shift neural adaptation to the opposite hemisphere. As a consequence, activation of a probe patch shown at the same spatiotopic location as the adapter should be reduced, even though this particular retinotopic representation was never exposed to the adapter stimulus.
Materials and Methods
The study consisted of three different experiments, which compared spatiotopic adaptation with adaptation at a retinotopic (Experiment 1) or alternatively at a control position (Experiment 2). In addition, Experiment 3 tested the feature specificity of spatiotopic adaptation.
Participants.
Eighteen participants (mean age: 28 years, 11 male, 7 female) took part in Experiment 1. Twelve participants (mean age = 29 years, 5 male and 7 female) took part in Experiment 2. Seventeen participants (mean age = 26 years, 1 male and 4 female) took part in Experiment 3 (mean age = 27 years, 6 male and 11 female). All subjects had normal or corrected-to-normal vision and no dyschromatopsia. Written informed consent was obtained before the experiment in accordance with the Declaration of Helsinki. Participants were remunerated for their time. The study was approved by the ethics committee of the German Society of Psychology.
Apparatus.
In the fMRI experiments, stimuli were presented on a 30 inch LCD screen mounted behind the scanner and viewed via a mirror system installed on top of the head coil. Viewing distance was 245 cm, resulting in a visual field of 13.8° × 8.2°. Stimuli were presented on the monitor with a vertical frequency of 60 Hz at a resolution of 1024 × 768 pixels on a homogeneously gray background. Functional imaging data were acquired by means of a 3-T TRIO MRI system (Siemens).
Eye movement measurements.
Eye movements were monitored in all experiments by the MRI-compatible Eyelink 1000 system (SR Research), which samples gaze positions with a frequency of 2000 Hz. Viewing was binocular, but only the dominant eye was recorded. The system detected a start and an end of a saccade when eye velocity exceeded or fell below 22°/s and acceleration was above or below 4000°/s2. At the beginning of each experimental session, a 9 point calibration and validation procedure was conducted. If the calibration did not meet the specified criteria, calibration was repeated until it was successful.
Procedure.
Each trial started with the presentation of the fixation point, located 6° left of the screen center and at the horizontal meridian (see Fig. 1A). Subjects were required to keep their gaze directed to the fixation point until it disappeared. Simultaneously with the appearance of the fixation point, a tilt adapter was shown 3° to the left and 2° above screen center. Tilt-adapter stimuli were grating patches oriented at 15° of 100% contrast, vignetted within a circular Gaussian envelope. Their spatial frequency was 0.8 cycles/° and the visible stimulus ∼2.7° in diameter. Orientation and position of the adapters remained constant across all trials. In the no-adapter condition, only the fixation point without an adapter was presented. Participants were instructed to keep their gaze at the fixation point. After 3000 ms, the saccade target (black rectangle, 1° × 1°) appeared. For 1000 ms, the fixation point and saccade target were simultaneously visible; thereafter the fixation point disappeared. This was the go-signal for participants to initiate a saccade to the saccade target. Average eye movement traces are shown in Figure 1B–E. Trials were excluded from analysis in which saccade landing positions deviated >3° from the saccade target location. After the saccade, a probe patch was presented for 50 ms to measure the adaptation aftereffect. Probe patches were identical to the adapter patches, except that their orientation changed across trials. The onset of the probe patch relative to the disappearance of the tilt-adapter patch was jittered across trials in 8 steps (normally distributed) between 2.5 s and 8.5 s. This was done to separate functional activation induced by the adapter from that generated by the test patch and to improve the sampled temporal resolution. Subjects had to indicate tilt orientation by reporting whether (via a two finger button press) the patch appeared tilted clockwise or counterclockwise relative to the horizontal. In each trial, a single probe patch was presented. It was either shown at the spatiotopic position (i.e., the same location as the adapter stimulus) or at the corresponding retinotopic position (i.e., the location which matches the adapter position in retinotopic coordinates), which was 3° to the right of the screen center. The probe patch orientation varied according to the adaptive PEST algorithm (Taylor and Creelman, 1967), which homed in on the point where the grating orientation appeared to be horizontal. Subjects had to give their response within 1000 ms after probe presentation otherwise the trial was excluded from analysis. Adaptation and no-adaptation trials were presented in separate blocks, and each experimental session consisted of two blocks. In all experiments, a 2 × 2 design was implemented with the factors adaptation (adapter condition/no-adapter condition) and test position, which differed in the three experiments. In each condition, 40 trials were measured. Intertrial intervals were jittered in 8 steps between 2.5 s and 8.5 s, normally distributed. The presentation order of the blocks was randomized across subjects. Trials in which saccades were either <2° or started after probe onset were excluded from analysis.
Experiment 1: adaptation in spatiotopic and retinotopic location.
In Experiment 1, the probe patch was either shown at the spatiotopic or the retinotopic position. The spatiotopic position matched the location of the adapter, 3° to the left and 2° above screen center. The retinotopic position matched the adapter position in retinotopic coordinates, which was 3° to the right and 2° above screen center.
Experiment 2: adaptation in spatiotopic and in control location.
In Experiment 2, we also measured adaptation at a spatiotopic and at a neutral control location. This experiment was identical to the spatiotopic adaptation experiment, except that the test Gabor patch was shown either at the spatiotopic position or at a control position, which was located under the position of the adapter patch (3° to the left and 2° below screen center).
Experiment 3: test of feature selectivity.
In Experiment 3, we investigated first the feature specificity of spatiotopic adaptation, and second controlled for the spatial selectivity of the tilt adaptation. In the previous experiments, adaptation effects were compared with a baseline condition where no adapter was shown. In principle, adaptation effects could be driven by all visual features constituting the adapter (e.g., contrast, phase, or size). Accordingly, Experiment 3 tested whether adaptation was specific for the feature tilt. To test for feature specificity, we compared Gabor patches, which were identical in all aspects, except for their orientation. In particular, we compared adaptation as induced by Gabor patches, which had either an oblique or an orthogonal orientation. The latter replaced our no adaptor control condition from the previous experiments and was chosen because orthogonal adapter orientations led neither to perceptual aftereffects (Blake et al., 1985) nor to neural changes in the BOLD signal (Krekelberg et al., 2006). In other words, the two adapter conditions were designed to generate identical adaptation, except for the visual feature orientation.
We contrasted two adapter conditions to test the spatial selectivity of adaptation. It might be argued that tilt adaptation takes place in purely retinotopic coordinates and that the spatiotopic effects measured by us are the result of a spreading of adapter activation. In this view, the saccade could have smeared adapter activation across large parts of visual cortex. In the feature selectivity experiment, we used Gabor patches with a neutral orientation, flanking the patch in the spatiotopic position. These flanking patches would interrupt any global spreading of adapter activation. Under this condition, aftereffects in the spatiotopic position can only arise if adaptation is spatially selective.
The experimental setup was identical to the first two Experiments. Subjects were required to perform a saccade and judge the orientation of a subsequently shown probe patch. The only difference was that, in Experiment 3, two adapter conditions were contrasted. A 2 × 2 design was implemented with the factors adapter orientation (orthogonal/oblique) and test position (spatiotopic/control). In the orthogonal-adapter condition, three adapters with a vertical orientation were presented simultaneously (see Fig. 4A, top). In the oblique-adapter condition, three adapters were shown: two of them were oriented vertically, and the spatiotopic adapter in the middle was oriented 105°. We used two flanking adapters to restrict adaptation of the middle patch to its particular position and to cancel out a possible spread of adaptation to other locations. Contrasting activation between the orthogonal and the oblique condition should lead to behavioral and neural adaptation only for the middle adapter. This difference should reveal the neural activation, which is specific only to the visual feature tilt because all other features are identical between conditions. We tested the spatiotopic position (Fig. 5A, red “S”) and a control position (Fig. 5A, red “C”), which are crucial to test spatial selectivity. If adaptation can be found at the spatiotopic location but not at the control location, then adaptation cannot result from a global adaptation spread.
fMRI data acquisition.
A 3 tesla TRIO MRI system (Siemens) was used to obtain functional imaging data. Functional images were acquired by means of a T2*-weighted EPI sequence with a TR of 2.2 s and a TE of 30 ms. The field of view was 200 mm, using a 64 × 64 matrix with an in-plane. The voxel size of each volume was 3 × 3 × 3 mm3 with each consisting of 36 axial slices; 923 volumes were measured in the spatiotopic adaptation experiment, 711 volumes in the control adaptation experiment, and 1020 volumes in the feature selectivity experiment.
fMRI data preprocessing.
Imaging data were preprocessed and analyzed using the Statistical Parametric Mapping software SPM8 (Wellcome Department of Imaging Neuroscience, London; www.fil.ion.ucl.ac.uk/spm/software/spm8). The first six images that were acquired before a steady BOLD signal was reached were omitted from further analysis. Images were spatially realigned to the seventh volume to correct for interscan movement and subsequently rerealigned to the mean of all images. Data were normalized to standard MNI space as extracted with the segmentation function. The normalized images were then spatially smoothed with an 8 mm FWHM Gaussian kernel. Separately measured anatomical images (voxel size 1 × 1 × 1 mm3) were obtained with a standard T1-weighted 3D MPRAGE sequence.
fMRI data analysis.
In the fMRI experiments, onset regressors were defined at the single-subject level, representing separately the time points when probe stimuli were presented (separately for the four different conditions resulting from our 2 × 2 design). Two additional regressors were included in the model representing the presence of adapters in the spatiotopic and the retinotopic condition. Also, a regressor was defined representing trials that involved missing responses or incorrect saccade behavior. The BOLD response related to each regressor was modeled using a canonical hemodynamic response function and its time derivative (using SPM8). Finally, the six movement parameters obtained from the realignment procedure were included into the model. Initially, first level contrasts for the four different conditions were specified by setting the regressor of interest to 1 and all others to 0. The resulting images were then taken to the second level and were entered into an ANOVA using a flexible factorial design as implemented in SPM8. Next, differential contrasts were calculated at the second level and were reported with a statistical threshold of p < 0.05 at the cluster level (p < 0.001 cutoff at the voxel level). Results are reported using a corrected threshold (family wise error correction) of p < 0.05 at the cluster level (p < 0.001 cutoff at the voxel level). BOLD signal time courses were extracted at the maximum activated voxel.
In our analysis, we first calculated the main effects for the different locations using the following contrasts: for Experiment 1: (1) retinotopic > spatiotopic, and (2) spatiotopic > retinotopic; and for Experiment 2: (1) control > spatiotopic, and (2) spatiotopic > control. To identify regions from which to extract signal time course data in an additional analysis, we restricted our analysis to those regions in visual cortex, which specifically code the spatiotopic, the retinotopic, or the control adapter position. We therefore inclusively masked the main effect adapter > no adapter by the corresponding main effect, which was cluster level corrected, p < 0.05 (cutoff at the voxel level, p < 0.001). Thus, the functional contrast defining the mask and the contrast that was actually tested represent orthogonal factors within a full-factorial design. For the analysis of the signal time course data, we used the differential group activations to select the relevant voxels. For each subject, we extracted the first eigenvariate time course of all voxels within the predefined region. Event-related averaging was performed by averaging the signals evoked by single events within a time window of 20 s after the event. Time course data were then averaged, first within each subject across trials, and thereafter across all subjects. This was performed for every experimental condition and for each subject. Subsequently, these averages were integrated across subjects generating a single grand average for each condition. The trial-wise jittering between adapter offset and probe onset increased the temporal resolution of the time course data and justified a temporal interpolation resulting in time bins of 0.5 s.
Retinotopic mapping.
Phase-encoded retinotopic mapping was conducted on all subjects in Experiment 2 using standard checkerboard rotating wedges to map polar angles of early visual regions (Engel et al., 1994; Sereno et al., 1995; DeYoe et al., 1996). High-contrast radial checkerboard patterns were presented as 60° wedges. Each run rotated clockwise through 8 cycles with a period of 60 s/cycle. Subjects fixated at the center of the display and pressed a button every time the checkerboard briefly turned red (which occurred at a randomly chosen time within a cycle). fMRI data were preprocessed in SPM8, including spatial realignment as well as spatial smoothing using a Gaussian kernel of 4 mm FWHM. Data were then coregistered to the structural scan for each participant. A fast Fourier transform was applied to each time series, and the phase and power at the stimulation frequency (8 cycles/scan for polar scan) were extracted. An F-statistic indicating the significance of the visual response was calculated by dividing the power at the fundamental frequency of the stimulus by the average power across all frequencies.
We used freesurfer (Fischl, 2012) for the segmentation to generate the gray and white matter surfaces for each individual. Then, we projected the Fourrier transformation volumes (real and imaginary volumes) of each subject to the gray matter surface with a projection fraction of 0.52 (using command mr_vol2surf) to create a paint map of the polar stimulus. Based on the paint map, we could distinguish the visual field boundaries, which were delineated manually following standard phase-reversal criteria (Sereno et al., 1995; DeYoe et al., 1996, Engel et al., 1997; Larsson and Heeger, 2006; Swisher et al., 2007). Bilateral ROIs were created for areas V1, ventral and dorsal V2, and ventral and dorsal V3. To create a probabilistic map, all these ROIs were transformed to the fsaverage template. Calculating the probability value of each surface point for each area resulted in a probabilistic map. We thresholded the probability map to 0.3, which means that at least three participants have the same area for the given surface point, which leads to a maximum probability map (MPM, to prevent any confusion we refer to this as “SpatioMPM”).
The group result of the spatiotopic condition (the t-map volume in the MNI space) was projected to the fsaverage brain resulting in a patch on the surface. This patch was used for further analysis by calculating the overlap with SpatioMPM and MPM (Abdollahi et al., 2014).
Results
We investigated transsaccadic adaptation both behaviorally and functionally. The respective correlates of adaptation were determined by means of two different conditions: In the adapter condition, an adapter with an orientation of 105° was shown for 3000 ms. While the adapter was presented, participants were asked to keep their gaze at the fixation point and to perform the saccade after the adaptation period. Similarly, in the no-adapter condition, participants had to fixate the fixation point for 3000 ms, but the rest of the screen remained blank. Behaviorally, a tilt adapter, as used in the present experiment, is expected to alter tilt perception; for instance, an objectively horizontally oriented probe subjectively appears to be tilted clockwise or counterclockwise. At the neural level, such adapted probe activation should be reduced compared with an equivalent condition without adaptation. Mean tilt thresholds were calculated by averaging probe patch orientations across trials and subjects.
Eye movement data analysis
Subjects were well able to keep their gaze directed at the fixation point and to perform the required eye movements (Fig. 1B–E). Eye positions in the period from trial start until execution of the required saccade did not deviate >0.94° (SD) from the center of the fixation point. There were no significant differences between conditions (two-way ANOVA). Saccades, which had to be performed 1000 ms after presentation of the saccade target, were on average very accurate and overshot the target by only 0.04° (SD 0.87°). A two-way ANOVA indicated no significant differences between conditions. Average saccade latencies were 332 ms (SD 134.6 ms), which is a typical latency duration in delayed saccade paradigms.
A, Schematic description of the trial sequence. A trial started with the presentation of a fixation point (FP). After 1 s, an oriented adapter was shown 3° to the left of and 2° above screen center for 3 s. Then a saccade target (ST) was shown for 1 s, simultaneously with the fixation point. When the fixation point disappeared, the subject was required to perform a saccade. After on average 3.5 s, an oriented probe was presented in one of three possible locations: the spatiotopic (S) position, the retinotopic (R) position, or the control (C) position for 100 ms. B, C–E, Average eye position over the course of the trials in the spatiotopic (green) and the retinotopic (red) adaptation and baseline conditions. Gray bars represent the positions of the fixation point (at −6°) and the saccade target (at 0°).
Experiments 1 and 2
fMRI, whole brain data
To identify the neural mechanism underlying the behavioral aftereffects, we performed an fMRI whole-brain analysis. In a first step, we identified the brain regions involved in processing the probe stimuli at the retinotopic or the spatiotopic location. For this purpose, we calculated the main effects for the different locations using the following contrasts: (1) spatiotopic > retinotopic, (2) retinotopic > spatiotopic, and (3) control > spatiotopic. As illustrated in Figure 2, in the spatiotopic condition, probes were presented in the left visual field and hence induced significantly stronger activation in the contralateral (right) visual cortex. Likewise, probes in the retinotopic condition were shown in the right visual field and hence significantly activated left visual cortex. Probes shown in the control position activated the upper part of the right visual cortex (cluster level corrected, p < 0.05; cluster-forming threshold: p < 0.001, at the voxel level). Visual activation occurred in ventral areas, which is to be expected because we had presented the adapter and the probe stimuli in the spatiotopic and the retinotopic condition in the upper visual field.
Bold activation in response to probe presentations resulting from contrasting the conditions spatiotopic > retinotopic (green) and retinotopic > spatiotopic (red).
fMRI adaptation
We compared activation induced in the different no-adapter conditions with its respective adaptation counterparts to substantiate that the tilt adapter induced a measurable reduction of the BOLD amplitude. This was done separately for the spatiotopic, the retinotopic, and the control condition (i.e., spatiotopic: no adapter > adapter; retinotopic: no adapter > adapter; control: no adapter > adapter). Results of the whole-brain analysis are shown in Figure 3D for the spatiotopic and Figure 3G for the retinotopic location. To select regions of interest for the extraction of β values at those locations, which responded to the presentation of the spatiotopic, the retinotopic, and the control probe, we generated a binary mask. The binary mask was generated from the corresponding orthogonal main effect, p < 0.05 (cluster level corrected, cluster-forming threshold: p < 0.001, at the voxel level). We inclusively masked activation in each condition and extracted β values at the position of maximal activation.
A, Schematic illustration of the probe positions in the spatiotopic (S), the retinotopic (R), and the control (C) positions. B, E, H, Average time courses of the BOLD amplitude extracted at the spatiotopic, the retinotopic, and the control locations. Gray represents time courses from no-adapter sessions. Other colors represent adapter sessions. Error bars indicate SEM. C, F, I, Results from the behavioral tilt adaptation. Subjects had to estimate the orientation of the probe patch relative to the horizontal midline. Tilt adaptation magnitude was calculated by subtracting perceived tilt in the adapter sessions from perceived tilt in the no-adapter sessions for each subject. Same color conventions as in A. Error bars indicate SEM. D, G, fMRI adaptation in the visual cortex for probes shown in the spatiotopic location (green) and in the retinotopic location (red).
In the control condition (i.e., control: no-adapter > adapter), no significant fMRI adaptation was observed at the control probe location (cluster level corrected, p < 0.05; cutoff at the voxel level, p < 0.001). The time courses of the BOLD amplitudes extracted from all voxels within the cluster in the control no-adapter (shown in gray) and in the control adapter conditions (shown in blue) are shown in Figure 3B. In the spatiotopic condition (i.e., spatiotopic: no-adapter > adapter), significant fMRI adaptation was observed at the spatiotopic probe location (p < 0.05 (cluster level corrected, cluster-forming threshold: p < 0.001, at the voxel level). Fig. 3E shows the time course of the BOLD adaptation in the spatiotopic condition for the no-adapter condition (gray color) and the adapter condition (green color). BOLD signal time courses presented were binned with a bin width of 500 ms before being averaged across trials and subjects. Time courses were locked to probe onset, and the interval between adapter offset and probe onset was jittered across trials. Because the period between adapter offset and probe onset was jittered across trials, it is unlikely that these time courses reflect activation by the adapter stimulus. Significant adaptation was also found at the retinotopic probe location. The results show that the adapter produced significant BOLD reduction in the retinotopic condition after execution of a saccade (cluster level corrected, p < 0.05; cutoff at the voxel level, p < 0.001). This finding is illustrated in Figure 3H, which shows the average time courses of the BOLD amplitudes extracted from all voxels within the cluster in the retinotopic no-adapter (shown in gray) and in the retinotopic adapter conditions (shown in red).
Retinotopic mapping
Retinotopic mapping was conducted to functionally localize the spatiotopic remapping processes. Accordingly, we determined individual visual area borders for three areas: V1, V2, and V3. Then we produced a maximum probability map from these data, to which we from here on refer to as “SpatioMPM.” We used a threshold of 30% for our SpatioMPM, meaning that at least three individuals for any given point of surface of the template brain have the same retinotopic area. To determine the precise localization of our spatiotopic fMRI adaptation (Fig. 3D), we projected the average neural adaptation data into the SpatioMPM (Fig. 4A, red). There was a 25% overlap of activation in the spatiotopic condition with ventral area V3. To confirm this, we used another freely available MPM (Abdollahi et al., 2014), which delivered similar results: 28% of the activation in the spatiotopic condition overlapped with ventral V3. This confirms that there is a strong agreement between SpatioMPM and MPM (Fig. 4A,B). The MPM also shows that the rest of the activation in the spatiotopic condition falls within areas V4 (29% overlap) and VO (26% overlap).
A, Results from the retinotopic mapping analysis. Outlines of the spatiotopic adaptation results (red) are superimposed. Individual borders of areas V1, V2d, V2v, V3d, and V3v from the right hemisphere were determined according to the results of a phase-encoded mapping. The number of activated voxels for the spatiotopic effect in each area was counted for each subject. The highest number of activated voxels was found in dorsal area V3. B, Visual areas as identified by a cytoarchitectonic-based atlas (Eickhoff et al., 2005) and outlines of the spatiotopic adaptation results (red) superimposed.
Behavioral
In Experiment 1, we measured adaptation at the spatiotopic and the retinotopic location. In Experiment 2, we measured adaptation at the spatiotopic and a control location. To compare adaptation at these locations statistically, we pooled the results obtained at the spatiotopic location from the two experiments. First, we report behavioral and neural adaptation in a control position, which should not be affected by adaptation (Fig. 3A). No significant behavioral adaptation was observed in this position (Fig. 3C).
As expected, the tilt adapter effectively altered tilt perception. Importantly, adaptation was found at both the spatiotopic and the retinotopic adapter location (Fig. 3F,I). Tilt adaptation magnitude was calculated by subtracting perceived tilt in the adapter trials from perceived tilt in the no-adapter trials for each subject. On average, the adapter patch significantly altered tilt perception of the probe patch at the respective spatiotopic location by 2.79° (SEM 1.79°). Similarly, at the retinotopic location, tilt adaptation of 2.68° (SEM 1.59°) was observed. A two-way ANOVA revealed a significant main effect of adaptation (adapter present vs absent) but no effect of space (spatiotopic vs retinotopic) (adaptation: F(1,17) = 4.35, p = 0.04; space: F(1,17) = 0.92, p = 0.34). Thus, the behavioral data demonstrate that tilt adaption was effective at both the retinotopic and the spatiotopic position.
Experiment 3
In Experiments 1 and 2, we investigated spatiotopic adaptation with the standard procedure used in most behavioral setups (Melcher, 2005; Turi and Burr, 2012; Zimmermann et al., 2013). We found a broad cluster of activation in occipital cortex. In Experiment 3, we aimed to test spatial and feature specificity of this adaptation. Evidence for spatial selectivity of our tilt adaptation was already provided by the results of the control condition in Experiment 2. However, neural adaptation may spread along the horizontal direction. The findings from the spatiotopic adaptation could theoretically be explained by a mechanism that is purely retinotopic in nature but involves a relatively wide spatial spread of adaptation across the visual field, including the spatiotopic location. If global spreading of adaptation occurred, this should have involved not only both the spatiotopic and the retinotopic locations, but also the space between those positions. We therefore tested fMRI adaptation for a probe in the spatiotopic location (Fig. 5A, red “S”) and a control position (Fig. 5A, blue “C”) located in between the spatiotopic and the retinotopic position. To prevent any putative spreading, the adapter was flanked by two additional adapters, which in all conditions were shown with an orientation orthogonal to the probe (Fig. 5A). Thus, adaptation induced by the oblique adapter could not spread across the visual field.
A, Schematic illustration of the adapter patches in Experiment 3. Two conditions were compared. In the first condition (top left panel), all adapters were oriented vertically. In the second condition (top right panel), the spatiotopic adapter in the middle was oriented obliquely and the other vertically. The probe patch was presented in one of two possible locations: the spatiotopic location (marked by the red S) or the control location (marked by the blue C). B, Results from the behavioral tilt adaptation in the spatiotopic condition for the no-adapter (gray) and the adapter trials (red), and in the control condition for the no-adapter (gray) and the adapter trials (blue). C, fMRI adaptation in the visual cortex for probes shown in the spatiotopic location (red). No significant fMRI adaptation was found in the control location.
In Experiment 3, we also wanted to test the feature specificity of spatiotopic adaptation. In the previous experiments, we compared neural activation from sessions with adapter and without adapter. This design was chosen to increase adaptation-related neural activation. For instance, the adapter presented had a particular spatial frequency, a specific contrast, and a particular size. Accordingly, in addition to the tilt adaptation effect tested behaviorally, these additional features may also have induced neural adaption that may have contributed to spatiotopic neural adaptation observed in the fMRI data. We therefore contrasted two conditions with adapters that differed only in their orientation. In the first contrast, an adapter was presented with an orientation orthogonal to the probe patch (Fig. 5A). It has previously been established that with orthogonal orientation between probe and adapter no-adaptation occurs, neither behaviorally (Blake et al., 1985) nor neurally (Boynton and Finney, 2003). In the second contrast, the adapter was shown with an oblique orientation (105°) relative to the probe, similar to the previous experiments and was thus expected to induce adaptation. Any differential activation between both contrasts would specifically reveal spatiotopic adaptation of the particular visual feature orientation because all remaining stimulus features were kept constant between conditions.
Behavioral adaptation (Fig. 5B) was significantly stronger at the spatiotopic than the control location, as revealed by a significant interaction effect (two-way repeated-measures ANOVA, F(1,16) = 5.356, p = 0.035). To investigate neural adaptation, we calculated a whole-brain analysis using the maximum probability map (see Retinotopic mapping) as an inclusive mask. Neural activation resulting from this analysis is shown in Figure 5C. A two-way ANOVA showed a significant interaction effect (F(1,16) = 8.248, p = 0.005), thus revealing stronger neural adaptation in the spatiotopic than in the control condition. This excluded the possibility that the spatiotopic effect was only the signature of a global adaptation spread. Accordingly, the spatiotopic adaptation is indeed specific for visual feature information and carries activation related to visual orientation across hemispheres.
Discussion
We used behavioral and neural adaptation to investigate how visual space is stabilized across saccades. We found evidence for a transfer of spatiotopic information between visual areas as indicated by both our behavioral and functional data. First, at the behavioral level, and in agreement with our previous report (Zimmermann et al., 2013), adaptation aftereffects were observed after a saccade was executed. Second, fMRI adaptation was shifted from one to the opposite hemisphere consistent with a transsaccadic information transfer of adapter activity (Duhamel et al., 1992). This effect was seen in visual areas V3, V4, and VO. Furthermore, a control experiment confirmed that the effect was spatially selective and specific to the feature tilt. The spatial selectivity rules out that a global spread of adaptation may account for the occurrence of aftereffects at the spatiotopic location.
Spatiotopic adaptation
In our experiment, the presaccadic adapter stimulus was located in the right visual field and was thus processed and coded within the left hemisphere. The saccade after adapter offset shifted the relation between retina and external space, such that the location that previously contained the adapter (in terms of spatiotopic coding) fell into the left visual field. Because this shift occurred after adapter offset, the adapter stimulus itself was never actually processed in the right hemisphere. Nevertheless, strong fMRI adaptation effects were observed at that particular retinotopic location in the right hemisphere. This finding implies that neural adaption generated in the left hemisphere was interhemispherically transferred and became effective in the right hemisphere. This shift of neural adaption activity might result from an active process, which accounts for the behavioral spatiotopic aftereffects observed in this and previous studies (Melcher, 2005; Turi and Burr, 2012; Zimmermann et al., 2013). Although our adapter and our probe stimuli activated expectedly ventral areas, our retinotopic mapping analysis revealed that the shift of adapter activity occurred in dorsal visual area V3 and in higher areas visual, including V4 and VO. Electrophysiological results have demonstrated the existence of receptive field shifts in area V3 (Nakamura and Colby, 2002), thus providing a neural basis for the transfer of visual feature information that we observed. The involvement of early visual areas is consistent with the feature specificity of the spatiotopic tilt aftereffect. The feature orientation is well suited for a probe stimulus, since due to the radial bias, fMRI activity is 20% stronger for stimuli with radial orientation (Sasaki et al., 2006). However, it is unlikely that the radial bias can account for our spatiotopic adaptation since trials with and without adaptation were matched with regard to eye movements. Contrasting no adapter and adapter trials should therefore cancel out any putative and selective radial bias effects.
Retinotopic adaptation
In addition to spatiotopic behavioral and fMRI adaptation, we found adaptation for probe stimuli presented at a position retinotopically matched to the adapter location. In the retinotopic condition, both the presaccadic adapter stimulus and the postsaccadic test stimulus were processed by the same hemisphere (here: the left hemisphere). fMRI adaptation in early visual areas for probes in the retinotopic position are in good agreement with earlier reports of orientation-specific adaptation under ocular fixation (Boynton and Finney, 2003) where adaptation was observed along the hierarchy of visual areas (V2-V4). The finding that adaptation remains in retinotopic coordinates seems, at least at first sight, to be inconsistent with the notion of receptive field shifts. Buildup of activity at the new position of the cell should, in principle, correlate with a decline at its old location. In the current experiment, we did not simply measure neural activation correlated with the presentation of a stimulus but neural activation following visual adaptation. Triggered by the incidence of the saccade, a remapping process actively transported adapter activity to the opposite hemisphere, to compensate for the spatial displacement of the retina. This is consistent with our findings of two adapted locations: the retinotopic location, which matches the position of the adapter before saccade execution; and the spatiotopic location, which matches the position of the adapter after saccade execution. Earlier studies showed that spatial attention operates on the basis of retinotopically organized maps and hence needs dynamic updating to compensate for eye movements (Golomb et al., 2008). It has been demonstrated that updating of a spatiotopic position takes some time, which is consistent with a slow buildup of transsaccadic orientation adaptation (Zimmermann et al., 2013).
Alternative explanations
Several results from our experiments confirm that tilt adaptation was spatially selective and argue against an account of a global spread of adaptation. In this view, behavioral and neural adaptation at the spatiotopic position would be seen merely because neural adapter activity spreads over large parts of visual cortex. Our data provide strong evidence against such an account. First, we observed that the presentation of a probe in a neutral but distance matched position below the adapter location neither produced an aftereffect nor any significant neural adaptation. Second, we used adapters, which were flanked by patches with a neutral adaptation. These flankers blocked any spreading from the retinotopic to the spatiotopic location. Nevertheless, behavioral and neural adaptation at the spatiotopic position was observed.
General implications
How does the brain transform visual information from an initial retinal representation into a spatiotopic format? A simple solution would be that visual space as such is coded in real-world coordinates. Such a gaze-invariant map could be generated by directly linking retinotopic coordinates and current eye positions. Building up a spatiotopic map from retinotopic input would require knowledge about the position of the eyes. Visual neurons, whose responses are modulated by changes in eye position, have been reported by several studies (Trotter et al., 1992; Guo and Li 1997; Dobbins et al., 1998; Trotter and Celebrini, 1999; Rosenbluth and Allman, 2002; Durand et al., 2010). Response modulation by eye position signals in visual cortex has been found in studies using electrophysiological recordings (Galletti and Battaglini, 1989) and brain imaging (Merriam et al., 2013). The mapping of these neurons is retinotopic; however, the amplitude of their response is modulated by eye position. A population of neurons with different receptive fields and different gain fields represents retinal stimulus location and eye position simultaneously. Spatiotopic location information could then be read from this representation, as modeling studies have suggested (Zipser and Andersen, 1988; Galletti et al., 1993). Imaging studies have reported evidence for external space maps in some visual areas, such as MT (d'Avossa et al., 2006; Crespi et al., 2011) or the lateral occipital cortex (McKyton and Zohary, 2007). Importantly, the buildup of an external space map requires attention (Crespi et al., 2011). Crespi et al. (2011) asked subjects to perform an attention demanding task on stimuli displayed at the fovea. BOLD responses evoked by moving stimuli irrelevant to the task were mostly tuned in retinotopic coordinates. However, where subjects could attend easily to the motion stimuli, BOLD responses were tuned not in retinal but in external coordinates.
If the effects observed in our study resulted from eye position, activation would have varied with changes in gaze direction. However, in our differential contrasts, changes in gaze direction were identical and their effects therefore should have cancelled out each other. A spatiotopic map also predicts that fMRI adaptation would have to be observed in the same hemisphere that had been adapted by the adapter stimulus. The same neurons that had been adapted would respond to the probe stimulus despite the intervening saccade. However, in our Experiment 3, which was particularly designed to test the spatial selectivity, we found fMRI adaptation in the hemisphere that was not adapted by the adapter stimulus. This finding indicates that spatiotopic adaptation is not governed by eye position. It rather suggests that adapter activation must have been actively transported to the opposite hemisphere. Indeed, receptive field shifts, indicative of remapping, have been observed in area V3 (Nakamura and Colby, 2002). This is consistent with Merriam et al. (2003) who showed that, triggered by saccade execution, a memory trace of a visual stimulus is updated in parietal cortex across hemispheres and that the visual cortex has access to this information (Merriam et al., 2007). Our findings extend these results, by demonstrating that transsaccadic remapping carries visual feature information.
In conclusion, our study provides clear evidence for spatiotopic adaptation both behaviorally and at the neural level in ventral visual areas V3, V4, and VO. Transsaccadic remapping carries visual feature information to compensate for the retinal displacement.
Footnotes
This study has been supported by the Deutsche Forschungsgemeinschaft, DFG (ZI 1456/3-1).
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Eckart Zimmermann, Cognitive Neuroscience, Institute of Neuroscience and Medicine (INM-3), Research Centre Jülich, Germany. ec.zimmermann{at}fz-juelich.de
References
In this issue
