Abstract
Previous studies raise the hypothesis that attentional bias in the phase of neocortical excitability fluctuations (oscillations) represents a fundamental mechanism for tuning the brain to the temporal dynamics of task-relevant event patterns. To evaluate this hypothesis, we recorded intracranial electrocortical activity in human epilepsy patients while they performed an audiovisual stream selection task. Consistent with our hypothesis, (1) attentional modulation of oscillatory entrainment operates in a distinct network of areas including auditory, visual, posterior parietal, inferior motor, inferior frontal and superior midline frontal cortex, (2) the degree of oscillatory entrainment depends on the predictability of the stimulus stream, and (3) the attentional phase shift of entrained oscillation cooccurs with classical attentional effects observed on phase-locked evoked activity in sensory-specific areas but seems to operate on entrained low-frequency oscillations that cannot be explained by sensory activity evoked at the rate of stimulation. Thus, attentional entrainment appears to tune a network of brain areas to the temporal dynamics of behaviorally relevant event streams, contributing to its perceptual and behavioral selection.
Introduction
Attentional modulation of brain activity often manifests as an amplification of trial-averaged stimulus-evoked responses in sensory areas, including primary auditory and visual cortices (Motter, 1993; Luck et al., 1997; Mehta et al., 2000; Bidet-Caulet et al., 2007). The “oscillatory selection” hypothesis (Schroeder and Lakatos, 2009) proposes that the brain can accomplish these gains by a mechanism that exploits the large-scale neuronal excitability shifts that occur during low-frequency neuroelectric oscillations (Shu et al., 2003; Lakatos et al., 2005). When task-relevant stimuli occur in a rhythmic stream, attention can selectively amplify their neuronal representation by appropriately biasing the high excitability phases of ongoing oscillations so that they align with the events in the stream. Results to date strongly support the hypothesis of oscillatory selection in primary sensory areas (Lakatos et al., 2008, 2009).
Classical attention effects on evoked activity, however, are not limited to primary sensory cortices but have been shown to occur all along the sensory pathways and beyond (Luck et al., 1997; Martínez et al., 1999; Kastner and Ungerleider, 2000; Mehta et al., 2000). We hypothesized that, if attentional entrainment of low-frequency oscillations is indeed a fundamental strategy for controlled processing, it should generalize to a subset (network) of higher order sensory and association cortices. Another key prediction of the oscillatory selection hypothesis is that the strength of the attentional bias should increase with stimulus predictability.
To test these hypotheses, we recorded the electrocorticogram (ECoG) of epileptic patients chronically implanted with subdural grids and depth electrodes for surgical treatment of their medically refractory seizures. The electrode coverage included much of the lateral, superior, and frontal surfaces of the neocortex (see Fig. 1). We adapted the paradigm previously used in monkeys (Lakatos et al., 2008): patients were presented with interleaved auditory and visual stimulus streams in phase opposition, each having a mean rhythm of 1.5 Hz, and were required to pay attention to one or the other in different blocks. In two patients, we also varied the predictability of the stimulation. We presented stimuli near threshold to reduce the contribution of evoked responses relative to ongoing oscillations. This was necessary because rhythmic sensory stimulation evokes large rhythmic sensory responses (steady-state evoked potentials) in auditory and visual cortices (Picton et al., 2003; Vialatte et al., 2010) that will tend to align the phase at the stimulation frequency, counteracting any effect of attention on entrained oscillations.
Our findings are consistent with the idea that rhythmic low-frequency activity plays a mechanistic role in neuronal representation and perceptual/behavioral selection of an attended sensory stream. We show that attention shifted the phase of the 1.5 Hz oscillation entrained to the stimulation over a large neocortical network, including areas implicated in numerous other aspects of controlled processing and that the size of the effect increases with stimulus predictability. Moreover, the pattern of findings indicates that this oscillatory entrainment reflects a substantial reorganization of endogenous oscillatory activity these areas, rather than mere feedforward evoked activity at 1.5 Hz.
Materials and Methods
Patients.
Recordings were obtained in seven female patients with medically intractable focal epilepsy undergoing intracranial EEG recording [five patients at the Columbia University Medical Center/New York-Presbyterian Hospital (CUMC) and two patients at the North Shore–Long Island Jewish Hospital (LIJ)] to help identify the epileptogenic zone (i.e., the tissue that must be removed to obtain seizure control). The study was approved by the Institutional Review Boards of Columbia University Medical Center and North Shore–LIJ Health Systems. Informed consent was obtained from each patient before the experiment. More comprehensive information about the patients is available in supplemental Tables 2 and 4 (available at www.jneurosci.org as supplemental material).
Stimuli.
Auditory standard stimuli were 33 ms, 440 Hz pure tones and visual standard stimuli were gray-filled circles spanning 10° of visual angle on a computer screen. Auditory target stimuli differed from standards in frequency (targets were higher). Visual target stimuli differed from standards in luminance (targets were brighter). Standard stimuli were delivered with an intensity/contrast just above detection threshold and target stimuli at discrimination threshold. For both auditory and visual stimulation, the detection threshold was established before the main experiment using the method of limits. In some cases, the intensity or the contrast was slightly increased during the main experiment if the patient could not complete the task properly.
The deviations in frequency/luminance of the auditory/visual targets were established using a two forced-choice procedure (separately for auditory and visual targets): the patients were presented with a standard (at detection threshold) followed by either another standard or a target from five predefined frequency levels and had to indicate whether the stimuli were identical or different. There were 10 trials for each of the six levels. The level of the target for the main experiment was the one that gave ∼75% correct responses. The level of the target was adjusted during the experiment, as the performance of the patients tended to get better at the task.
Procedure.
Patients sat comfortably in their hospital bed. The stimulation screen (17 inches, 60 fps LCD monitor) stood on an overbed table at ∼80 cm from the patient. Sounds were delivered via two open-field speakers positioned on either sides of the screen.
In the main experiment, patients were presented with alternating auditory and visual stimuli that were separated by a random interstimulus interval (ISI). The intermodal ISI had a normal distribution with a mean of 333 ms and SD of 75 ms, which resulted in a mean unisensory ISI of 667 ± 150 ms for both the auditory and visual streams. In the “attend visual” condition, the patient was instructed to attend the visual stream (i.e., detect visual targets by clicking a mouse while ignoring auditory targets). In the “attend auditory” condition, the patient had to attend the auditory stream (i.e., respond to auditory targets while ignoring visual targets). In the main (jittered ISI) experiment, stimuli were presented in blocks of 200 stimuli (100 auditory and 100 visual), including eight visual and eight auditory targets. Attend visual and attend auditory blocks were alternated and the total number of blocks depended on the state and motivation of the patient (between 8 and 12 blocks). Behavioral results are given in supplemental Table 3 (available at www.jneurosci.org as supplemental material).
Two patients (S5 and S6) also performed additional blocks of stimuli in which stimuli were presented at a regular intermodal ISI of 333 ms, without the random jitter. The experiment was otherwise identical. In patient S5, the jittered and regular ISI experiments were performed on different days, whereas in patient S6 both experimental conditions were recorded within the same session, with experimental blocks of each condition alternated.
Acquisition.
Intracranial EEG was acquired from up to 127 electrodes per patient (supplemental Table 1, available at www.jneurosci.org as supplemental material), at 500, 1000, or 2000 Hz/channel with 24-bit precision (0.5–125 Hz bandpass filtering) using a clinical video-EEG system (XLTek). Subdural macroelectrodes were platinum disks 4–5 mm in diameter and arranged in linear (strips) or matrix arrays (grids) with 1.0 cm center-to-center spacing. Depth electrodes were tubular in shape, 2.3 × 0.5 mm with 5 mm center-to-center spacing. For CUMC patients, the reference was an inverted electrode strip positioned over the electrode grid, with electrical contacts facing the dura. For LIJ patients, both the reference and ground electrodes were attached to the skull, on the frontal bone approximately at midline. When showing “unfiltered” signal (see Figs. 2, 3; supplemental Figs. 1, 2, available at www.jneurosci.org as supplemental material), signals from patient L4 and L5 were actually low-pass filtered under 30 and 15 Hz, respectively, because of high levels of 60 Hz noise.
Analysis.
All computations were implemented using Matlab (The MathWorks). Statistics on circular variables were performed using Daniel Rizutto's PhasePACK package for Matlab.
Exclusion of epileptic activity.
Single trials were reviewed visually by a clinical neurophysiologist (C. A. Schevon) and trials including nonphysiological artifacts, persistent abnormal activity (e.g., excess delta slowing), or frequent interictal epileptiform on any of the channels were excluded. Channels that were not excluded by this procedure but were identified as belonging to the epileptogenic zone according to the clinical evaluation or were found to display interictal abnormalities outside of the experimental EEG samples are identified in supplemental Figure 6 (available at www.jneurosci.org as supplemental material). Among the 76 electrodes showing frequent interictal activity and 83 showing occasional interictal activity, respectively, 49 and 70 were included in our analysis and thus represented 20% (119 of 595) of the analyzed electrodes. However, these electrodes did not significantly bias our subsequent analysis of phase. Supplemental Figure 6 (available at www.jneurosci.org as supplemental material) compares the spatial distribution of interictal electrodes (red and orange) with that of electrodes significantly influenced by attention (green). Only four electrodes presenting frequent interictal activity (purple) and seven electrodes presenting occasional interictal activity (pink) showed a significant effect in our main analysis, that is ∼15% (11 of 72) of electrodes presenting a significant phase shift.
Entrainment and effect of attention on slow oscillations.
To isolate slow oscillations at the sensory stimulation frequency (1.5 Hz), each trial was bandpass filtered between 1.2 and 2 Hz using a second-order Butterworth filter in the forward and reverse directions and the instantaneous phase at 0 ms was extracted for each trial using the Hilbert transform. We used a 3000 ms window centered on each stimulus onset to avoid edge effects. All subsequent analyses were separately conducted on auditory standard trials (“auditory-locked analysis”) and visual standard trials (“visual-locked analysis”).
To identify electrodes where 1.5 Hz oscillations were entrained to one of the unisensory stimulation streams, the distribution of instantaneous phases across trials was tested for uniformity using Rayleigh's test. We also computed the mean resultant length of phase vector across trials [also called intertrial coherence or phase-locking value (PLV)] in both the attend auditory and attend visual conditions.
To assess the effect of selective attention on the phase of entrained 1.5 Hz oscillations, we compared the instantaneous phase between the attend visual and attend auditory using Fisher's test for equality of the circular mean of two independent samples (Fisher, 1993; Rizzuto et al., 2006). In this test, a statistic Yr that reflects the absolute phase difference is compared with the χ2 distribution with 1 df. However, it turned out that, for most electrodes, the χ2 distribution underestimated the distribution of differences under the null hypothesis. Therefore, we devised a randomization version of the test (Watson, 1983) that estimated the actual distribution under the null hypothesis that the two samples are drawn from the same distribution using permutations: we pooled the NA auditory attention trials and the NV visual attention trials, drew at random NA and NV trials, and computed the statistic Yr. This procedure was repeated 10,000 times, which yielded a distribution of Yr under the null hypothesis. We then computed the proportion of values that are greater than the Yr value computed on the actual data, which is an exact estimate of the probability that the two samples have been drawn from the same distribution. The randomization test was consistently more conservative than the parametric test.
Multiple testing procedure.
Since we did not have strong hypotheses on the anatomical location of effects, it was necessary to correct for multiple testing over electrodes. This was done for each patient independently using a two-step procedure.
First, we used the Rayleigh test of nonuniformity of phase across trials to reduce the number of channels on which the subsequent test of equality of mean phase would be performed: only channels at which the distribution of phase at 0 ms was significantly nonuniform at p < 0.05 (i.e., showed entrainment) in both attend auditory and attend visual conditions were retained for subsequent analysis. We kept electrodes for which this was true either in the visual-locked or the auditory-locked analysis.
In the subsequence test for equality, we used a spatial cluster randomization methods (Maris and Oostenveld, 2007): for each randomization, we selected clusters of contiguous significant (p < 0.05) electrodes. Electrodes were considered contiguous if they were <13 mm apart, which keeps together vertically and horizontally but not diagonally contiguous grid electrodes. We computed the distances between electrodes by digitizing the location of the electrodes from the postimplantation computed tomography (CT) scan (see below). For each cluster (that sometimes included only one electrode), we computed the sum of Yr values and took the maximum value over all clusters for this randomization. The 10,000 randomizations yielded a distribution of the maximum value one can observe under the null hypothesis when computing the sum of Yr on a cluster of contiguous significant electrodes. We considered as significant in the actual data all the clusters whose sum of Yr had <5% probability to happen under the null hypothesis. We only report results for electrodes in which the phase difference was significant after correction for multiple testing, either for the auditory-locked or the visual-locked analysis.
High-frequency phase-locked activity.
Because the ISI was very short in our experimental paradigm, there was no flat baseline and it was therefore difficult to identify event-related activity using trial-averaged wavelet power or even evoked potentials at most recording sites. However, because the ISI was slightly jittered from trial to trial, phase-locked evoked responses appeared as poststimulus event-related increases in PLVs, at frequencies ranging from the stimulation rate to at least 30 Hz.
To compute PLVs, we first computed the wavelet decomposition of all trials between 0.48 to 213 Hz from 1500 ms before stimulus to 1500 ms after stimulus. PLV at each time/frequency point were computed across all trials for a given condition as the complex average of the wavelet phase. Only values significant at the Rayleigh test of nonuniformity were considered (without correction for multiple testing) and displayed.
The obvious limitation of PLVs is that they are not sensitive to event-related increases in the power of non-phase-locked oscillations (induced oscillations). Therefore, our measure of high-frequency activity was limited to phase-locked components (phase-locked increases in power or phase resets) that would appear as increases in trial-averaged evoked potentials and can therefore be considered equivalent to classical evoked potentials.
PLVs at frequencies higher than the stimulation rate (>1.5 Hz) could also be used to distinguish between genuine entrained oscillations at 1.5 Hz and steady-state evoked potentials at the same rate. Indeed, entrained oscillation PLVs should be limited to the 1.5 Hz band, whereas steady-state PLVs should also spread to higher frequencies because evoked potential are nonsinusoidal and therefore have a greater frequency content.
Correlation between attentional phase shift and high-frequency phase-locked activity.
We computed the Spearman rank correlation across electrodes between the size of the attentional phase shift at 1.5 Hz and an index of phase-locked activity at higher frequency. The size of the attentional phase shift was computed as the absolute phase difference between the attend auditory and the attend visual conditions. A unique high-frequency PLV index was computed as the proportion of significant PLVs in a spatiotemporal window encompassing 41 frequency bands from 3.15 to 28.9 Hz, and time samples from 50 to 350 ms poststimulus in the attended condition. For both values, we chose the maximum value between the auditory-locked and the visual-locked analysis. For the main results (see Fig. 4), we computed the correlation coefficient across electrodes showing a noncorrected significant attentional phase shift. In supplemental Figure 4 (available at www.jneurosci.org as supplemental material), we also show this correlation analysis for corrected significant phase shift electrodes and all electrodes showing an entrainment.
Effect of attention on high-frequency (3–30 Hz) phase-locked activity.
To study the effect of attention on the PLVs at higher frequencies, we selected electrodes that showed significant phase difference at 1.5 Hz and that recorded signal from either auditory or visual cortex, using the following criteria: (1) they were spatially close to the known anatomical location of auditory or visual cortices and (2) they showed increases in PLVs in the 3–30 Hz/50–350 ms time–frequency window only after stimulation in the corresponding modality. Indeed, since we used a jittered ISI, phase-locked activity should increase only after auditory stimulation in the auditory cortex and after a visual stimulation in the visual cortex. As shown in the examples of Figure 3 (and supplemental Fig. 2, available at www.jneurosci.org as supplemental material) and on the group maps of supplemental Figure 3 (available at www.jneurosci.org as supplemental material), increases in PLV between 3 and 30 Hz readily distinguish between regions activated by auditory and visual stimuli. The selected electrodes for each modality-specific region are indicated in Figure 4B. We then ran paired one-sided permutation tests across electrodes using the PLV index described earlier after stimulation in the relevant modality as a measure. For electrodes over the auditory cortex, we compared this proportion after auditory stimulation in the attend auditory and the attend visual conditions, with the hypothesis that it would be larger in the attend auditory condition. For electrodes over the visual cortex, we hypothesized that PLVs after visual stimulation would be larger in the attend visual than in the attend auditory condition.
Effect of stimulation predictability.
For the regular ISI condition in patients S5 and S6, we computed the phase of the 1.5 Hz the same way as in the jittered ISI conditions. This yielded the number of electrodes showing a significant entrainment (as defined above). For the effect of attention on the phase of the 1.5 Hz rhythm at 0 ms, we performed the statistical cluster analysis only for those electrodes that showed a significant entrainment in the jittered ISI conditions, so that the outcome of the statistical test is comparable between the two conditions. To compare the value of the phase shift, we took all the electrodes that showed a significant entrainment in both the regular and the jittered ISI conditions and compared the absolute phase difference between attentional conditions using a paired permutation test across electrodes, both for the auditory-locked and the visual-locked analyses. Both tests yielded similar results and only the visual-locked analysis is reported.
Localization of the ECoG electrodes.
The location of the electrodes relative to the cortical surface was determined using Bioimagesuite software (http://www.bioimagesuite.org) and custom-made Matlab software. First, postimplantation CT scans were thresholded to identify the electrode locations in the coordinate system of the CT scan. As a first approach to localization relative to the cortex, we coregistered the CT scan volume to the preoperative magnetic resonance (MR) volume by linear transformation using an automated procedure and transformed the coordinates of the electrode into the coordinate system of the preoperative MR. Because of the tissue compression and the small degree of midline shift (usually <2 cm) that typically occurs during subdural array implantation, the result of this coregistration is approximate (Dalal et al., 2008).
Since we focus on group analysis in this paper, we normalized the coordinates of each individual patient's implantation using the Talairach normalization method (Talairach and Tournoux, 1988). We then transformed this normalized coordinate system to the coordinate system of the MR images of the Montreal Neurological Institute (MNI) single subject, using the inverse of the Talairach normalization procedure. Electrodes were displayed on a three-dimensional reconstruction of the cortical surface of the MNI brain [segmented using FreeSurfer software (http://surfer.nmr.mgh.harvard.edu)]. Talairach normalization introduces additional error in the relative localization of electrodes on the cortical surface.
To reduce these errors, each patient's electrode implantation was then adjusted (rotated and translated) to match constraints provided by cortical neurostimulation mapping (localization of the tongue and the hand motor representation, Broca's area when available), responses to auditory and visual stimuli (localization of the strongest auditory responses and/or visual sensory response when available), and direct anatomical information provided by intraoperative photographs. For display purposes, X-coordinates of the left electrodes were multiplied by −1 so as to appear on the right hemisphere. All electrodes were finally projected to the nearest and most outward cortical surface vertex using point projection centered on the closest point to the electrode location on a two-dimensional frame located inside the right hemisphere and following its broad shape.
We assessed the quality of coregistrations using two sets of functional data independent from the main effect of interest. First, we displayed the group maps of high frequencies phase-locked to auditory or visual stimulation and ensured they were found in anatomically defined auditory and visual cortex, respectively (supplemental Fig. 3, available at www.jneurosci.org as supplemental material). Second, we displayed the group maps of the electrical neurostimulation mapping data acquired as part of the clinical assessment in six patients and used to identify cortical regions critical for motor control and language and exclude them from resection (Hamberger, 2007) (supplemental Fig. 5, available at www.jneurosci.org as supplemental material). These clinical data were also used to refine the localization of individual electrodes described in Figures 2 and 3 and supplemental Figures 1 and 2 (available at www.jneurosci.org as supplemental material).
For density maps (see Fig. 5), the density of electrode on the cortical surface was computed using a linear combination of three-dimensional Gaussian density functions centered at each electrode location. We then took the value of this linear combination at each vertex of the cortical surface. The basis Gaussian function had a width (SD) of 50 mm. The resulting density values were then normalized by the maximum density value across vertices coded to a hue, saturation, value (HSV) color map. In Figure 5, the normalized density of significant electrodes was translated into a composite hue/brightness scale going from black (0) to red (1). The normalized density of analyzed electrodes was directly translated into a saturation scale.
Results
Effect of attention on the phase of rhythms entrained to the stimulation
A total of 595 channels devoid of epileptic activity was analyzed across patients. Among those, 197 channels (33%) showed a nonuniform phase in both the attend auditory and attend visual conditions (Fig. 1, dark gray electrodes; supplemental Table 1, available at www.jneurosci.org as supplemental material). Figure 2 shows examples of electrodes with significantly nonuniform 1.5 Hz phase at 0 ms (Rayleigh's test, p < 0.05, uncorrected) (for electrode locations, see Fig. 1). In most of these examples, the 1.5 Hz rhythm can be seen even in the unfiltered data and the scale is not very different between unfiltered and 1.5-Hz-filtered versions, indicating that it was a dominant rhythm in the EEG at these electrodes.
Location of significant 1.5 Hz entrainment and attentional phase shifts. Group maps of all the analyzed electrodes in all seven patients. The light gray electrodes did not show any effect, the dark gray electrodes showed an entrainment to the stimulation stream without an effect of attention, and the colored electrodes showed an effect of attention on the 1.5 Hz phase, either at the presentation the auditory stimulus (yellow), visual stimulus (blue), or both (green). Significant entrainments and effect of attention on the phase of the entrained rhythm were found in numerous brain areas. Data recorded at circled electrodes will be described in Figures 2 (red) and 3 (purple). Electrodes are displayed on a three-dimensional reconstruction of the cortical surface of the MNI single-subject structural MRI. Coordinates were normalized using the Talairach method and manually fitted based on stimulation mapping data and intraoperation photographs. X-coordinates of left hemisphere electrodes have been transformed to appear on the right hemisphere. Electrodes (except depth electrodes) are projected from a frame inside the hemisphere to the nearest surface vertex. The right bottom panel features a transparent cortical surface that allows one to see depth electrodes in the temporal/occipital cortex.
Examples of 1.5 Hz entrainment and attentional 1.5 Hz phase shift. This figure displays the largest attentional phase opposition of the entrained 1.5 Hz rhythm for each patient (except for patient S5, who did not show attentional shift larger than π/2). The location of each electrode is identified on Figure 1. Phase opposition of the 1.5 Hz component between the attend auditory and attend visual conditions is sometimes readily visible in the unfiltered ECoG (A) but is evident in the 1.5 Hz-filtered signal (B). The histograms of 1.5 Hz phase values for both attentional conditions (C) show that this phase difference was present at the trial level. This effect is magnified by looking at the orientation of the mean phase vector across trials (D). All depicted phase-locking values were significantly different from zero and were significantly different between the attend auditory and attend visual conditions. Distribution of 1.5 Hz phase value across trials in different attentional conditions. The location of these six electrodes is indicated in Figure 1. Mean curves and phase distributions were locked either on auditory (blue) or visual (red) events. For more examples, see supplemental Figure 1 (available at www.jneurosci.org as supplemental material).
At the group level, 72 of the 197 electrodes that exhibited entrainment (37%) (Fig. 1, colored electrodes; supplemental Table 1, available at www.jneurosci.org as supplemental material) demonstrated a significant phase shift in the visual-locked or the auditory-locked analysis (7 only in visual, 24 only in auditory, and 41 in both analyses). Figure 2 depicts clear examples, in six of seven patients, of recordings that demonstrated a phase shift close to the hypothesized 180° phase difference, as depicted by the difference in the mode of the phase distributions (Fig. 2C) and the resultant phase vectors pointing in opposite directions (Fig. 2D) (for more examples, see supplemental Figs. 1, 2, available at www.jneurosci.org as supplemental material).
In contrast with our initial expectation, 180° phase opposition was by no means the most frequent result and magnitudes of significant shifts ranged from 0.23 to 3.12 rad across patients and electrodes (see Fig. 4 below). Figure 3A–C shows examples of both near-180° and near-0° (but still significant) phase shifts.
Examples of different values of attentional phase shifts and phase-locking activity. This figure illustrates different values of attentional phase shifts for electrodes entrained to the stimulation stream. The first three columns illustrate electrodes showing a 1.5 Hz phase opposition between attend auditory and attend visual conditions (A–C). These electrodes showed no increase in PLV in frequencies >1.5 Hz after auditory (D, E) or visual (F, G) stimulation. The last two columns illustrate electrodes recording from the auditory and visual cortex, respectively. They both show a smaller (but significant) phase shift and important phase-locked activity after either auditory (D, E) or visual (F, G) stimulation, corresponding to large auditory and visual phase-locked activity (A). The locations of these five electrodes are shown in Figure 1. Note that all five electrodes show a significant PLV at 1.5 Hz, reflecting the significant 1.5 Hz entrainment that was the criterion for their inclusion in the phase difference analysis. The legends for rows A–C are the same as in Figure 2. PLVs are displayed between 0.7 and 60 Hz, between −1000 and 1000 ms. Only significant PLVs (not corrected) are displayed. The purple dashed frames indicate the time–frequency window in which the proportion of significant PLVs was measured for the correlation analysis (Fig. 4; supplemental Figs. 3, 4, available at www.jneurosci.org as supplemental material). For more examples, see supplemental Figure 2 (available at www.jneurosci.org as supplemental material).
Attentional phase shifts decrease with phase-locked sensory activity
The nonuniform distribution of phase at the frequency of stimulation (1.5 Hz) that we observe at one-third of the recording sites is attributable to two different phenomena: the regular occurrence of phase-locked evoked potentials after each sensory stimulation (in a given modality), and proper entrainment whereby a spontaneous 1.5 Hz oscillatory rhythm reorganizes its phase to follow the stimulation. These two possibilities can be distinguished because in the case of phase-locked evoked potentials, phase locking should also occur at frequencies higher than the stimulation frequency. Despite our effort to reduce evoked activity by using near-threshold stimuli, we saw clear instances of high-frequency phase-locking at some recording sites, in general over sensory cortices. The two rightmost examples of Figure 3 show examples of such electrodes, one over the auditory cortex, the other over the visual cortex, where significant phase-locking in the 1–30 Hz range can be seen after auditory and visual stimulation respectively. But in a large number of cases, high-frequency phase-locking was virtually absent despite the presence of a strong phase-locking around 1.5 Hz (Fig. 3, three leftmost columns).
Even more interestingly, Figure 3 shows that attentional phase shifts close to 180° were associated with an absence of significant phase-locking >2 Hz, whereas smaller phase differences were associated with significant increases in PLV between 1.5 and 30 Hz (for more examples, see supplemental Figs. 1, 2, available at www.jneurosci.org as supplemental material).
There was a significant negative correlation (r = −0.50, p < 10−6, Spearman rank correlation test) between the PLV index computed in a time–frequency window between 50 and 350 ms and 3 and 30 Hz (depicted as a purple rectangle in Fig. 3C–F) and the size of the attentional phase shift (Fig. 4C). The size of the attentional phase shift in the 1.5 Hz band was a monotonically decreasing function of the PLV index. Using only the 72 electrodes surviving the correction for multiple testing did not change this correlation (supplemental Figs. 3, 4, available at www.jneurosci.org as supplemental material). As can be seen on Figure 4, A and B, small phase differences associated with large PLV indices were mostly concentrated around the Sylvian fissure and over the occipital cortex, that is, over the sensory (auditory and visual) cortices.
Correlation between the size of the attentional 1.5 Hz phase shift and the phase-locked 0.7–30 Hz activity. The size of the 1.5 Hz phase difference between attend auditory and attend visual conditions (A) was inversely proportional to the phase-locked activity in a 50–350 ms/0.7–30 Hz time–frequency window after stimulation (B). Note how the attentional phase shift is minimal and high-frequency phase-locked activity is maximal at electrodes recording from auditory and visual cortex. This inverse relationship is illustrated for each patient in C in a scatter plot form. Only electrodes with a significant (not corrected) phase shift are displayed. In B, we took the maximum values between auditory and visual trials. In C, each colored scatter plot and regression line corresponds to a different patient. The black line is the regression line for all points pooled together (correlation coefficient r = −0.50; p < 10−6).
Attentional phase shift and attentional increase in high-frequency phase-locked activity
The fact that full 180° attentional phase shifts could not be observed in sensory cortices does not exclude the existence of more classical attentional effects on sensory phase-locked responses. To determine whether our paradigm also resulted in attentional modulation of phase-locked sensory response, we compared the 3–30 Hz PLV indices between the attend auditory and the attend visual, at electrodes over the auditory cortex on the one hand and over the visual cortex on the other hand. Among the 104 electrodes showing a significant (uncorrected) effect of attention on phase, 16 were located over the auditory cortex (Fig. 4B). For those 16 electrodes, the PLV index after auditory stimulation was larger for the attend auditory than the attend visual condition (40 vs 34%; p < 0.02, permutation test). Thirteen electrodes were over visual cortices (Fig. 4B). At these electrodes, the PLV index after visual stimulation was larger for the attend visual than the attend auditory condition (49 vs 42%; p < 0.05, permutation test). The same analysis, run only on the set of electrodes with corrected significant attentional phase shift, yielded significant differences, even though the number of electrodes was smaller (auditory cortex: N = 14 electrodes, 41 vs 35%, p < 0.02; visual cortex: N = 6 electrodes, 58 vs 46%, p < 0.02).
Spatial distribution of entrainment and attentional phase shift
Next, we describe the spatial distribution of significant entrainment and attention effects across patients. In human intracranial recordings, spatial sampling is sparse and guided by clinical and surgical considerations, which vary substantially between patients. Although pooling results across patients brings valuable information, the resulting group spatial sampling is inhomogeneous, and it can be misleading to simply report the spatial location of significant effect, because this procedure does not take negative results into account. To correct for this problem, we coded the density of significant electrode and the density of analyzed electrodes with independent dimensions of the color scale (Fig. 5). Colored regions indicate a higher proportion of significant effects and black regions indicate where no effects was found despite good spatial sampling, whereas whiter regions indicate a sparse coverage and colored/black regions a higher electrode density that allows a better estimation (for more details, see Materials and Methods). As illustrated in Figure 5a, the anterior frontal lobe and the anterior temporal lobe, despite extensive coverage, did not show a significant entrainment to the stimulus stream. In contrast, the precentral gyrus, the superior temporal gyrus (extending to the superior temporal sulcus), the angular gyrus, and the fusiform gyrus showed a higher proportion of significant entrainment. We will return to this contrast in Discussion.
Density of significant effects across patients. a, The proportion of analyzed electrodes showing entrainment to the 1.5 Hz stimulus stream was higher in the precentral gyrus, the superior temporal gyrus (extending to the superior temporal sulcus), the angular gyrus, and the fusiform gyrus (colored areas). In contrast, the anterior frontal lobe and the anterior temporal lobe did not show a significant entrainment to the stimulus stream, despite extensive coverage (black areas). b, Among electrodes showing entrainment, the ventral motor cortex, the top of the superior temporal cortex, the angular gyrus, and the posterior fusiform gyrus showed a higher proportion of attentional phase shifts, whereas the inferior somatosensory cortex and inferior and posterior parts of the auditory cortex did not show any. In both panels, the hue/brightness scale (from black to red) codes for the density of electrodes with significant effect (either entrainment or attentional phase shift) and the saturation (from white to black/color) codes for the density of analyzed electrodes (in b, only electrodes showing entrainment in a were included). The maximum hue and saturation values were normalized to 1 separately for each map.
Figure 5b shows that, among electrodes showing entrainment, three clearly delineated regions showed significant phase shifts with attention: the ventral motor cortex just above the Sylvian fissure, possibly extending to the facing superior temporal cortex, the angular gyrus, and the posterior fusiform gyrus. Note that some regions (like the posterior occipital cortex or the orbitofrontal cortex) that showed a large entrainment and/or attentional phase shift on individual electrodes (Figs. 2⇑–4) do not appear in these density plots because of their sparse electrode coverage. We refer the reader to the supplemental figures (available at www.jneurosci.org as supplemental material) for additional description.
Predictability increases the size of attentional phase shifts
Finally, we tested a corollary of our main hypothesis, namely that the effect of attention on the phase of the entrained rhythm should increase with the predictability of the stimuli. To get at this issue, we compared the size of the attentional phase shift when the ISI was jittered around 666 ms (main experiment described so far) and an additional condition, run in two patients, in which the ISI was always exactly 666 ms. As expected, the number of electrodes showing a 1.5 Hz entrainment to either stimulus stream increased in the regular ISI compared with the jittered ISI condition (patient S5: 40 vs 13 electrodes; patient S6: 84 vs 56 electrodes). More importantly, at the electrodes where the 1.5 Hz rhythm was significantly entrained in both regular and jittered conditions, the size of the attentional shift was significantly larger in the regular compared with the jittered ISI conditions (0.80 vs 0.57 rad; p < 10−4, paired permutation test on 59 channels) (Fig. 6).
Effect of regularity of the interstimulus interval on the size of the attentional phase shift. The size of the attentional phase shift was significantly larger when the ISI between sensory stimulation was regular and therefore the stimulation sequence predictable than when the ISI was randomly jittered. This histogram illustrates the increase in absolute phase shifts from the random to the regular ISI protocols, in two patients, at all the electrodes where the attentional phase shift was significant in both protocols. It is cumulated across the two patients. This increase was significant both at the patient level (patient S5: N = 7 electrodes; phase shift, 1.20 vs 0.66 rad; p < 0.05; patient S6: N = 52 electrodes; phase shift, 0.75 vs 0.56 rad; p < 10−3) and when data from both patients were pooled together (N = 59 electrodes; phase shift, 0.80 vs 0.57 rad; p < 10−4).
Discussion
Previous findings (Lakatos et al., 2008) raised the hypothesis that attentional modulation of low-frequency oscillatory phase may operate as fundamental mechanism of sensory/perceptual selection. The present findings provide clear confirmation of this hypothesis, identify a network of cortical areas critical to the mechanism underlying these effects, and outline several new conclusions, as well as questions and predictions for additional research.
First, we demonstrate that attentional modulation of oscillatory phase extends well beyond the primary cortical level, both recruiting a distributed network of higher order cortical areas, as well as excluding a large number of other brain areas. Specifically, we found significant attentional phase shifts over the posterior and ventral visual systems, the auditory system, the motor system, and various parts of the parietal and frontal lobes, including on their medial surfaces. Some areas were poorly sampled yet showed compelling examples of attentional phase shifts; these included the posterior occipital cortex, the parietal cortex posterior to somatosensory areas, and the inferior frontal /orbitofrontal cortex. Well covered areas with a large proportion of electrode sites demonstrating entrainment, also showing an attentional phase shift, included the inferior motor cortex (16 of 27 sites), the auditory cortex on the superior temporal gyrus (5 of 12), the posterior temporal cortex/angular gyrus (5 of 8), and the fusiform gyrus (5 of 15). It thus appears that a circumscribed network of distributed cortical areas is used to form the representation by the brain of the task-relevant event stream. We note a considerable overlap between this network and those proposed to be generally engaged during active, controlled processing tasks (Dosenbach et al., 2007; Corbetta et al., 2008; Capotosto et al., 2009), in particular, that related to “attention in time” (Coull and Nobre, 1998; Nobre et al., 2007). By analogy with other attentional functions, it is reasonable to suspect that prefrontal and parietal elements may serve executive functions. Interestingly, motor cortex showed some of the most robust and reliable entrainment effects. This fits with recent findings from intracortical recordings in humans (Saleh et al., 2010) suggesting that attentional entrainment of motor cortex to sensory stimulation rhythms is critical to effective motor responding under complex natural conditions.
Second, our findings support the idea that, at least at higher levels of processing, entrainment represents a reorganization of ongoing activity (Schroeder and Lakatos, 2009), rather than a superimposition of sensory phase-locked evoked responses. Phase-locked evoked responses can be distinguished from spontaneous oscillations because they entail stimulus-related increases in power and phase concentration at numerous frequencies in addition to any occurring at the driving frequency (Makeig et al., 2004; Shah et al., 2004). The negative correlation between the size of the attentional phase shift at 1.5 Hz and the amount of higher-frequency phase-locking suggests that attention operated on entrained rhythms that were not attributable to sensory phase-locked responses. Areas demonstrating a significant increase in phase-locking at 1.5 Hz and a large attentional phase shift, without the corresponding increase in phase-locking at higher frequencies included the motor cortex, the orbitofrontal cortex, the angular gyrus, and the parietal cortex. The spatial coverage for the angular gyrus and the inferior part of the motor cortex is dense enough to reasonably estimate the proportion of electrodes showing such a pattern to ∼70% of the analyzed electrodes (8 of 12 and 30 of 42, respectively) (Fig. 1).
An alternative explanation for this pattern of results would be the recurrence of low-frequency nonrhythmic attentional responses at the rate of stimulation. For example, apparent attention-biased entrainment in the motor cortex could reflect the recurrent generation of a contingent negative variation (CNV) (Brunia and van Boxtel, 2001) in anticipation of the next attended event. Additional experiments are required to find out whether the observed low-frequency oscillations are only apparent and reflect a “steady-state” CNV, or whether CNVs classically observed in nonrhythmic warning paradigms are in fact caused by a phase reset of frontal low-frequency oscillations by a warning cue, as we have proposed previously (Schroeder and Lakatos, 2009).
Attentional phase shifts departed the most from a true phase opposition in sensory areas, where phase-locked activity at higher frequency was also observed. In our view, this should not be interpreted as a smaller effect of attention on phase in these areas, because the mere presence of phase-locked evoked potentials superimposed to entrained oscillations would be expected to decrease the phase difference. In line with this hypothesis, focal intracortical recordings in the monkey visual cortex (Lakatos et al., 2008) have suggested that purely evoked sensory responses coexist with spontaneous slow oscillations reflecting variations in excitability on which attention can operate. It is actually remarkable that we observed any attentional phase biasing at all considering the amplitude of the phase-locked evoked responses at electrodes over relatively low-level visual regions in our data [e.g., electrode RTP2 in patient L5 (Fig. 3) and PSO6 in patient S6 (supplemental Fig. 2, available at www.jneurosci.org as supplemental material)].
It is noteworthy that numerous areas did not show any entrainment and phase opposition effects despite extensive electrode coverage. Frontal/prefrontal regions anterior to motor cortex, the anterior temporal lobe, and the hippocampal gyrus showed almost no entrainment, and, whereas the inferior somatosensory cortex and inferior-posterior parts of the auditory cortex did show phase-locked evoked responses, these areas did not show any effect of attention on the phase of the 1.5 Hz rhythm. A possible explanation for this finding is lack of sensitivity. Indeed, this is a likely explanation for the apparent lack of attention effects on hippocampal and anterior temporal regions since they were often the focus of ictal and interictal activity (supplemental Fig. 6, available at www.jneurosci.org as supplemental material). However, it hardly accounts for the absence of attentional effect in the inferior somatosensory cortex compared with both angular gyrus and inferior motor cortex, as these areas were among the least affected areas. It is possible that some of these areas are part of a network for monitoring and potentially responding to events outside of the attended stream. Non-entraining areas could support a “vigilance” (arrhythmic) mode of brain operation, which complements the “rhythmic” mode of operation addressed by the paradigm used in the present study (Schroeder and Lakatos, 2009). Additional experiments will be necessary to address that question.
If as we propose, oscillatory entrainment represents a means for the brain to exploit predictability in important event patterns, its strength should covary with event predictability. In a condition in which the stimuli were presented at exactly 1.5 Hz, without jitter, we found a significantly larger attentional phase shift. This result again favors the interpretation that the phase shift operates on ongoing oscillations entrained at 1.5 Hz, because increasing the regularity of the stimulus increases the phase-locked sensory component and should align the phases between the two attentional conditions, rather than separating them further, as we observe.
One important question is whether the attentional phase shift is a mechanism leading to attentional increases in trial-averaged evoked activity at higher frequencies. We found that, at electrodes over auditory and visual areas where there was a significant phase shift in the 1.5 Hz oscillation, transient increases in the phase concentration at frequencies between 3 and 30 Hz in response to auditory or visual stimulation were enhanced when the patients paid attention to the relevant sensory modality. Increases in phase-locking across trials would lead to increases in trial-averaged amplitude and probably correspond to classical attentional effects on sensory trial-averaged evoked responses (Woods et al., 1992; Martínez et al., 1999; Mehta et al., 2000; Bidet-Caulet et al., 2007). Although this observation does not prove a causal link between the phase effect at low frequency and increase in phase-locked activity at higher frequencies, as in the primary sensory cortices in the monkey (Lakatos et al., 2008, 2009), the pattern of findings certainly points in that direction.
We consider it unlikely that the effects we describe are specific to delta band oscillations. In our paradigm, the delta frequency was imposed by the rate of the rhythmic stimulation. The choice of this particular frequency was based on our previous study that demonstrated an attentional phase shift when using this frequency in the monkey primary visual cortex (Lakatos et al., 2008) and not on any assumption of a specific role of delta oscillation in attention. It is likely that a similar effect would be observed with different frequencies and the range of frequencies over which these effects operate will require additional experimentation.
We have suggested (Schroeder and Lakatos, 2009) that entrainment of slow oscillations could be a mechanism underlying the effects of attention to a specific point in time (Large and Jones, 1999; Lange et al., 2003; Nobre et al., 2007). It is possible that the attentional mechanism we describe in this study is specific to that circumstance. The extent to which the same mechanism can operate in other contexts would depend critically on the extent to which they contain prominent low-frequency rhythms generated by either inherent rhythm of the stimulus (e.g., biological motion or animal vocalization) or the inherent rhythmicity in the subject's own sensory sampling behavior (Rajkai et al., 2008; Bosman et al., 2009) (for review, see Schroeder et al., 2010).
Footnotes
-
This work was supported by National Institutes of Health Grants K08NS048871 and MH060358. We thank Dr. Sabine Kastner for helpful comments on a previous version of this manuscript.
- Correspondence should be addressed to either of the following: Julien Besle, Visual Neuroscience Group, University of Nottingham, Nottingham NG72RD, UK, julien.besle{at}nottingham.ac.uk; or Charles E. Schroeder, Cognitive Neuroscience and Schizophrenia Program, Nathan S. Kline Institute for Psychiatric Research, Department of Psychiatry, Columbia University College of Physicians and Surgeons, New York, NY 10032, schrod{at}nki.rfmh.org