Abstract
Attending to a specific spatial location modulates responsivity of neurons with receptive fields processing that part of the environment. A major outstanding question is whether attentional modulation operates differently for the foveal (central) representation of the visual field than it does for the periphery. Indeed, recent animal electrophysiological recordings suggest that attention differentially affects spatial integration for central and peripheral receptive fields in primary visual cortex. In human electroencephalographic recordings, spatial attention to peripheral locations robustly modulates activity in early visual regions, but it has been claimed that this mechanism does not operate in foveal vision. Here, however, we show clear early attentional modulation of foveal stimulation with the same timing and cortical sources as seen for peripheral stimuli, demonstrating that attentional gain control operates similarly across the entire field of view. These results imply that covertly attending away from the center of gaze, which is a common paradigm in behavioral and electrophysiological studies of attention, results in a precisely timed push–pull mechanism. While the amplitude of the initial response to stimulation at attended peripheral locations is significantly increased beginning at 80 ms, the amplitude of the response to foveal stimulation begins to be attenuated.
Introduction
The foveal representation of visual space in early visual cortex differs from the representation of the periphery in terms of contrast sensitivity, receptive field sizes, and response dynamics. There is evidence for feedback from higher cortical areas into foveal visual cortex for objects presented in the periphery (Williams et al., 2008). A recent animal electrophysiology study showed that attention reduces spatial integration for cells with centrally located receptive fields, while increasing spatial integration for cells with peripheral fields (Roberts et al., 2007). However, it is unclear how attention modulates processing of central visual input in humans. A number of functional magnetic resonance imaging (fMRI) studies (Paus et al., 1995; Tootell et al., 1998; Brefczynski and DeYoe, 1999; Somers et al., 1999) showed attentional modulation of the blood oxygenation level-dependent (BOLD) response for centrally presented inputs within early visual areas, but from these studies it is not clear when this attentional modulation occurs.
In contrast to fMRI, electroencephalography (EEG) has a very good time resolution and therefore lends itself to answer this question. In EEG recordings, spatial attention to peripheral locations modulates sensory processing in hierarchically early visual cortex (e.g., Hillyard and Anllo-Vento, 1998; Kelly et al., 2008). This effect can be described in terms of sensory gain control. According to this view, spatial attention enhances or reduces neural responses in sensory areas without changing their timing. The generators of gain-modulated responses to peripheral stimuli are well documented (Heinze et al., 1994; Woldorff et al., 1997; Martínez et al., 1999). Two recent event-related potential (ERP) studies (Eimer, 2000; Handy and Khoe, 2005) indicated that, unlike for peripheral stimuli, there is no attentional gain control for visual stimuli presented in the fovea. Paradoxically, in one of these studies (Handy and Khoe, 2005), consistent cueing effects were found on reaction time to both locations, foveal and parafoveal, providing behavioral evidence that attentional selection must have occurred at some level. The ERP technique, however, is only one way of obtaining the brain's impulse response function to a given (attended) stimulus. Another is the VESPA technique (Lalor et al., 2006), in which the intensity (or potentially any property) of a stimulus is modulated on each monitor refresh. With this technique, visual evoked responses can be estimated from continuously present stimulation streams, thereby removing the need for flashing stimuli. Therefore we are able to assess the effects of allocating attention among simultaneously presented stimuli in the absence of bottom-up capture effects of sudden stimulus onsets necessitated by the ERP technique (Lalor et al., 2007). In addition, the VESPA stimuli better approximate real environmental circumstances where objects seldom (if ever) flash on and off repeatedly. Using these stimuli, we expected to find early attentional modulation of foveal inputs.
Materials and Methods
Subjects.
Eighteen healthy volunteers (11 female) participated in this study, which was approved by the Institutional Review Board of the City College of New York. All subjects gave written informed consent to participate in this study and received a modest fee. All had normal or corrected to normal vision.
Stimuli and procedure.
A stimulus consisted of four rectangles arranged in the quadrants of a 1.8 × 1.8° fixation cross (Fig. 1). During every 5 s trial, the luminance of these four rectangles was modulated on each monitor refresh (60 Hz) between 0.1 and 118 cd/m2 using a Gaussian random process with ±3 SDs spanning the luminance range (Lalor et al., 2006). One of these stimuli was located at the center of the monitor and the other 2.2° above. Since the stochastic waveforms modulating the two stimuli were completely uncorrelated, we could estimate the brain's impulse response function to each individual stimulus concurrently. For a block of 20 trials subjects continuously fixated on one of these stimuli. Between 800 and 1200 ms before each trial, an arrow cue (height 0.5°, presented 0.75° from center of fixation) indicated to which stimulus subjects had to attend. Therefore subjects either had to attend to the fixated or covertly attend to the nonfixated stimulus. The task consisted of detecting up to three targets in the luminance modulation of the precued stimulus and to ignore the uncued stimulus. We continuously adjusted the detectability of targets by changing their duration to keep subjects at performance levels of ∼60% correct responses using an up-down transformed rule.
Recording.
We recorded 168 EEG channels using a ActiveTwo (BioSemi) system sampled at 512 Hz, while at the same time tracking the subjects' eyes with an EyeLink 2K system (resolution 0.01°; SR Research) running at 500 Hz.
Eye-tracking analysis.
We filtered the raw eye-tracking data using a low pass of 15 Hz to eliminate high-frequency errors. On this filtered data, we removed all trials in which the subjects' eyes moved >1° from center of fixation. This procedure also detected all trials with eyeblinks. Such a strict criterion is necessary, since the VESPA technique has allowed us here to present two continuous stimuli concurrently. Thus, if there were eye movements during a trial, the calculation of the response to the fixated location would include periods where participants were fixating the surround or even the other stimulus. We aborted the experiment of one participant who had very large eye movements in every trial, detectable by visual inspection during recording. For analysis of eye-tracking and EEG data, we removed the first 500 ms of each trial. Doing so removed regions with small eye movements at the onset of the stimuli. Only trials in which subjects' eyes were stable were used for further EEG analysis. Subjects who had less than an average of 100 trials remaining for each condition after this eye-movement rejection step were excluded from further analysis. This left us with 12 subjects.
EEG analysis.
The recorded EEG data were referenced offline to the nasion and filtered between 1 and 40 Hz using Butterworth filters. We then calculated the two VESPAs, i.e., the impulse response functions using the two known monitor luminance signals and the measured EEG signal for each channel using linear least-squares estimation. As in previous studies, this was done using a 500 ms sliding window (Lalor et al., 2006). It is important to note that the meaning of this time interval is slightly different from the time intervals over which VEPs are typically plotted. Unlike the VEP, the VESPA time interval is not determined with relation to a specific discrete event occurring at time 0. Instead, each time point on the time axis can be interpreted as being the relative time between the continuous EEG and the continuous input intensity signal. Therefore, the VESPA at −100 ms, for example, indexes the relationship between the input intensity signal at any time point and the EEG 100 ms earlier; obviously this should be zero. As another example, the VESPA at +100 ms indexes how the input intensity signal at any time point affects the EEG 100 ms later.
Dipolar sources of brain activity were estimated using a four-shell ellipsoidal head model as implemented in BESA (Megis Software; version 5.1.8). The software fits ellipsoidal compartments for brain, CSF, bone, and skin to the electrode locations. This method assumes that there are a limited and distinct number of active brain regions over the evoked potential epoch, each of which can be approximated by an equivalent dipole. Dipole generators are placed within the ellipsoidal volume conductor model and overlaid on and adjusted to a segmented structural MRI (in this case, an averaged brain). The forward solution to this dipole configuration is tested against the observed experimental data. When not fixed, the positions and orientations of the dipoles are iteratively adjusted to minimize the residual variance between the forward solution and the observed data (Scherg and Picton, 1991; Scherg and Berg, 1996). We used the difference wave between the two attention conditions to localize the P1 attention effect. For the lower fixation spot the time range for estimation was 70–90 ms, while for the upper fixation spot it was 66–86 ms. A pair of dipoles constrained to be mirror-symmetrical in location but allowed to vary in orientation was fitted. The initial starting position of dipoles was randomly chosen and the solution was restarted form multiple starting positions to ensure stability.
Statistical analysis.
The VESPA method produces one impulse response function for each experimental subject. To assess the significance of differences in VESPA amplitude around the P1 component, we therefore applied a statistical test across subjects. For each subject, we normalized the VESPA for the selected electrodes and determined the mean amplitude within the P1 timeframe. The mean amplitudes for attended and unattended conditions were then compared using the nonparametric Kolmogorov–Smirnov test. Please note that comparisons were always done for the fixated stimulus and that the only difference on any comparison was whether or not the stimulus was attended or unattended.
Results
Behavioral results
Participants maintained fixation on one of two identical but independently modulated stimuli arranged one above the other (Fig. 1) and were required to detect momentary luminance divergences in either the fixated or peripheral stimulus, according to a precue (for an example video, see supplemental material, available at www.jneurosci.org). We continuously adjusted target duration to keep subjects' performance stable at ∼60%. Subjects initially improved in detecting targets and then maintained stable levels for several blocks (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). In the later blocks performance tended to decline, and thus, target duration usually increased back toward the starting level. This pattern of target durations indicates that participants were able to perform the task and remained engaged in it throughout.
Stimuli and task. a, Throughout an experimental block, participants fixated on the center of one of two stimuli. Before every trial, an arrow indicated which stimulus the participants had to attend to, either the fixated or the parafoveal stimulus. b, During a 5 s trial, the luminance of each stimulus was changed by its own stochastic waveform on each monitor refresh. The participants performed a difficult task on the attended stimulus and ignored the unattended one. Normally the luminance of all four quadrants in each stimulus modulated with an identical time course. The task consisted of counting the number of times one of the four quadrants of the attended stimulus momentarily diverged in luminance from the other three. We present six example frames with a target in the upper right quadrant of the upper stimulus of frame n.
Temporal properties of attentional modulation
The VESPA technique derives the brain's impulse response function from the recorded EEG response and the known stimulus brightness waveform. Analyzing the influence of attending away from the fixation point, we do not find an attentional modulation in the time frame of the C1 component of the classical VEP (all p > 0.18). However there is a significant attention-dependent modulation of the VESPA for foveal stimuli between 80 and 100 ms after stimulus at the average of four central occipital electrodes (p < 0.019, n = 12) (Fig. 2a,b). This modulation of the VESPA is present for both possible fixation locations and occurs in the time range of the P1 component of the classical ERP. Our results clearly demonstrate that spatial attention modulates visual processing of foveal stimuli in the same time range found for peripheral stimuli.
Temporal properties of attentional modulation. a, Modulation of grand average VESPA at central occipital electrodes with attention when subjects fixated on the upper visual stimulus. b, Map of voltage differences between attended and unattended foveal stimuli at 86 ms after stimulus onset. The black ellipse encircles the central occipital electrodes used in a and c, as well as for the statistical comparison. c, Modulation of grand average VESPA at central occipital electrodes with attention when subjects fixated on the lower visual stimulus.
Cortical sources of attentional modulation
To further investigate the nature of this attentional modulation, we estimated the location of its cortical generators using equivalent dipole models. If the attentional modulation for foveal stimuli is governed by the same mechanisms as for peripheral stimuli, then the sources should be in comparable regions of extrastriate cortex. Indeed, localizing the peak of the difference between attended and unattended foveal stimuli revealed sources in extrastriate cortex, spatially close to sources previously described for attentional modulation for peripheral stimuli (Heinze et al., 1994; Woldorff et al., 1997; Martínez et al., 1999). For the lower fixation location, we localized the peak of the difference wave between 70 and 90 ms to the middle occipital gyrus (Fig. 3) (Talairach coordinates: x: ±35, y: −80, z: 4; explained variance: 77%). We find sources more ventral and medial (Talairach coordinates: x: ±19, y: −78, z: −5; explained variance 92%) for the difference wave of the upper stimulus between 66 and 86 ms (supplemental Fig. 2, available at www.jneurosci.org as supplemental material). These differences in location of cortical sources between upper and lower visual field are in agreement with earlier results (Heinze et al., 1994; Woldorff et al., 1997). As for the temporal profile of activation, the cortical sources of the attentional modulation for foveal visual stimuli are comparable to the sources for peripheral stimuli.
Dipole representations of sources of attentional modulation for fixation on lower stimulus. a, c, Transversal sections at z = 2. b, Coronal section at y = −80. d, Sagittal section at x = −35.
Discussion
Implications for covert attention
Covertly attending away from the center of gaze is one of the most commonly used techniques to study attentional modulation in behavioral and electrophysiological experiments. In behavioral attentional studies, participants are cued to covertly attend one of several possible locations at which a target might show up. There are numerous reports of reduced reaction times (e.g., Posner, 1980) and lower perceptual thresholds (e.g., Bashinski and Bacharach, 1980) to covertly attended stimuli, when the cue is valid compared to invalid cues. Electroencephalographic studies consistently show an enhancement of the visual evoked potential to covertly attended peripheral stimulation beginning at ∼80 ms (e.g., Heinze et al., 1994; Woldorff et al., 1997; Hillyard and Anllo-Vento, 1998). Our finding that processing resources begin to be withdrawn from foveal inputs at the same time as the enhancement of peripheral inputs starts, suggests that there is a coupling between these two processes. That is, our data point to a precisely timed push–pull mechanism of spatial visual attention.
Methodological innovation
The discrepant results between our study and earlier ERP studies (Eimer, 2000; Handy and Khoe, 2005) is very likely attributable to the nature of the stimuli. In those studies, isolated suddenly onsetting stimuli were used. It is well known, however, that such sharply onsetting, flashed stimuli automatically attract attention and thereby may override or weaken the effect of endogenous attention. The VESPA technique, in contrast, minimizes the involvement of exogenous attention for two reasons. First, the stimuli are continuously present and subjects consistently attend to the cued stimulus to carry out the experimental task. Second, the onset and duration (in this case 5 s) of luminance modulation is the same for both cued and uncued stimuli, which are displayed simultaneously. As such, these stimuli are considerably less likely to interfere with the subjects' endogenous deployment of attention [see Lalor et al. (2007) for a discussion of the use of the VESPA for assessing purely endogenous attention]. In addition, the VESPA better approximates real environmental circumstances where objects seldom (if ever) flash on and off repeatedly.
Attentional modulation in primary visual cortex
No significant attentional modulation of the C1 component was found, which peaked at ∼65 ms in the current data. This component is thought to mainly reflect activity in primary visual cortex (V1) (Clark et al., 1995; Foxe and Simpson, 2002; Kelly et al., 2008). Therefore one could assume that attentional modulation of centrally presented visual input does not occur during initial afference of V1. However, it is important to point out that this study was not explicitly designed to test this issue and there are several plausible alternate explanations for the lack of attentional modulation in this time range. There is a substantial intersubject variability in cortical geometry in and around the calcarine fissure, with primary visual cortex divided into subsulci and seldom conforming to the canonical cruciform representations common in textbooks (see e.g., Stensaas et al., 1974). As such, foveal stimulation can lead to considerable variation in C1 topographies and amplitudes across subjects, and this high variance needs to be explicitly accounted for when assessing modulations of the C1 (see e.g., Kelly et al., 2008). In addition, it has also been shown that modulation of the C1 component depends on high perceptual load of the visual attention task (Fu et al., 2009; Rauss et al., 2009). Since there was only one distracter stimulus in the current design, perceptual load may not have been sufficiently taxed.
Spatial and temporal attentional modulation
Attention cannot only be cued to spatial locations, objects, or stimulus features, but also to temporal aspects of a task (i.e., when it is that something is likely to occur). Several studies using temporal cueing paradigms have examined temporal attentional modulation in the fovea. While one study obtained no temporal attentional modulation of the P1 component (Miniussi et al., 1999), more recent studies have reported a larger P1 for foveated stimuli appearing at attended moments (e.g., Correa et al., 2006). Thus, as with spatial attention, temporal orienting can also modulate early processing at the center of gaze and an interesting future question will be whether these two types of modulation are governed by the same neural processes.
Attentional modulation in the fovea
Our study provides the first empirical evidence of attentional gain modulation at the center of gaze. However it is not the first study to show an influence of spatial attention on foveal processing. Functional neuroimaging studies have shown attentional modulation of BOLD response in brain areas representing the foveal visual field. Due to the restrictions of fMRI, however, these studies could not show the temporal dynamics of the attentional modulation and thus could not ascertain whether it is due to bottom-up or top-down processing (Logothetis, 2008). In addition, since some of these studies used periodic presentation of stimuli or periodicities in the task, the hemodynamic signals could have been influenced by an entrainment to experimental structure (Sirotin and Das, 2009). Here we show that the attentional modulation at the center of gaze occurs early (starting at ∼80 ms) in extrastriate visual cortical areas, with the same latency and cortical generators as previously described for peripheral stimuli.
Footnotes
-
This study was supported by a grant from the United States National Science Foundation to J.J.F. (BCS0642584). E.C.L. was supported in part by an Irish Research Council for Science, Engineering & Technology Government of Ireland Postdoctoral Research Fellowship.
- Correspondence should be addressed to Prof. John J. Foxe: Children's Evaluation and Rehabilitation Center, Departments of Pediatrics and Neuroscience, Albert Einstein College of Medicine, Van Etten Building–1C, 1300 Morris Park Avenue, Bronx, NY 10461. john.foxe{at}einstein.yu.edu
