Abstract
Recent psychophysical studies have demonstrated that periodic attention in the 4–8 Hz range facilitates performance on visual detection. The present study examined the periodicity of feature binding, another major function of attention, in human observers (3 females and 5 males for behavior, with 7 males added for the EEG experiment). In a psychophysical task, observers reported a synchronous pair of brightness (light/dark) and orientation (clockwise/counterclockwise) patterns from two combined brightness–orientation pairs presented in rapid succession. We found that temporal binding performance exhibits periodic oscillations at ∼8 Hz as a function of stimulus onset delay from a self-initiated button press in conditions where brightness–orientation pairs were spatially separated. However, as one would expect from previous studies on pre-attentive binding, significant oscillations were not apparent in conditions where brightness–orientation pairs were spatially superimposed. EEG results, while fully compatible with behavioral oscillations, also revealed a significant dependence of binding performance across trials on prestimulus neural oscillatory phases within the corresponding band. The peak frequency of this dependence was found to be correlated with intertrial phase coherence (ITPC) around the timing of button press in parietal sensors. Moreover, the peak frequency of the ITPC was found to predict behavioral frequency in individual observers. Together, these results suggest that attention operates periodically (at ∼8 Hz) on the perceptual binding of multimodal visual information and is mediated by neural oscillations phase-locked to voluntary action.
SIGNIFICANCE STATEMENT Recent studies in neuroscience suggest that the brain's attention network operates rhythmically at 4–8 Hz. The present behavioral task revealed that attentional binding of visual features is performed periodically at ∼8 Hz, and EEG analysis showed a dependence of binding performance on prestimulus neural oscillatory phase. Furthermore, this association between perceptual and neural oscillations is triggered by voluntary action. Periodic processes driven by attention appear to contribute not only to sensory processing but also to the temporal binding of diverse information into a conscious event synchronized with action.
Introduction
Psychophysical and physiological evidence increasingly suggests that the visual system processes sensory information rhythmically, as exemplified by neural oscillatory activity (VanRullen and Koch, 2003; Buzsáki, 2006; Fries, 2015; VanRullen, 2016). Recent studies show that detection performance for visual stimuli fluctuates ∼4–8 Hz relative to the timing of stimulus onset or observer action (Tomassini et al., 2015, 2017; Benedetto et al., 2016). To date, these findings have been interpreted as reflecting the periodicity of an attentional network that facilitates and speeds up stimulus processing because visual cues facilitate target detection in a periodic manner and in correlation with neural oscillatory phases (VanRullen et al., 2007; Busch and VanRullen, 2010; Landau and Fries, 2012; Fiebelkorn et al., 2013; Dugué et al., 2015, 2016). Performing a behavioral task that requires attention will allow the periodic function of attention to be more directly characterized than in a detection task.
Attention plays an essential role not only in facilitating stimulus processing but also in binding sensory information across different dimensions (Treisman and Gelade, 1980). To achieve a unified conscious perception, the brain needs to spatiotemporally integrate cross-modular information processed in different cortical streams with various timings (Zeki, 1978; Mishkin et al., 1983; Livingstone and Hubel, 1988; DeYoe et al., 1994; Van Essen and Gallant, 1994). Multiple features are bound into a conscious object or event based on the position and timing of stimulus presentation (Treisman and Gelade, 1980; Wolfe, 1994; Moutoussis and Zeki, 1997; Holcombe and Cavanagh, 2001; Bartels and Zeki, 2006; Fujisaki and Nishida, 2010). Given that reduced attention can result in feature misbinding, or illusory conjunctions (Treisman and Schmidt, 1982), attention has been considered to play a crucial role in feature binding. Holcombe and Cavanagh (2001) reported a psychophysical experiment with a dynamic conjunctive visual display to measure the temporal limits of feature binding by requiring the combination of temporally synchronous attributes. This study is in contrast to the conventional visual search paradigm that assesses feature binding less directly. Holcombe and Cavanagh's (2001) results indicated that temporal binding resolution is much lower between spatially separated features than between spatially superimposed features, thereby illustrating that slow binding for separated features is driven by attention.
By using a similar dynamic conjunctive visual display (Fig. 1), we show here that behavioral performance of attentional feature binding fluctuates periodically at ∼8 Hz with respect to a button press, as congruent with a significant dependence on neural oscillatory phases within the corresponding band. The behavioral and neural periodicities are both correlated with the peak frequency of intertrial phase coherence around the button press in individual observers. The psychophysical and physiological results suggest that an attentional network serves to bind temporally diverse visual information periodically at ∼8 Hz, subserved by neural oscillatory activity in synchrony with action.
Materials and Methods
Observers.
Seven naive participants and one of the authors (R.N.), including three females, participated in the psychophysical experiments (average age = 25.0, SD = 4.3) of the spatially separated and superimposed conditions. The same eight participants and another seven males took part in the EEG experiment (average age = 23.1, SD = 3.7) conducted only for the separated condition. All participants had normal/corrected-to-normal vision and all of them were right-handed. All experiments were conducted in accordance with the Declaration of Helsinki (2003) and were approved by the ethics committee of the University of Tokyo. All participants provided written informed consent.
Apparatus.
Images were displayed on a gamma-corrected 24 inch LCD (BENQ XL2430T; 640 × 480 pixel) with a frame rate of 100 Hz. This monitor is the successor to the BENQ XL2410T that has been proven to have sufficient temporal precision to display fast-changing stimuli common in visual psychophysics (Lagroix et al., 2012). The LCD pixel resolution was 3.0 min/pixel at a viewing distance of 70 cm and mean luminance was 71.7 cd/m2. In the EEG experiment, images were displayed on a gamma-corrected 24 inch LCD (BENQ XL2420T; 1024 × 768 pixel) with a frame rate of 100 Hz. The pixel resolution of the LCD was 1.9 min/pixel at a viewing distance of 70 cm and mean luminance was 54.0 cd/m2.
Stimuli.
In the spatially separated condition, as sketched in Figure 1, visual stimuli were two brightness–orientation pairs consisting of combinations of uniform luminance (black/white) and 0.8 cpd square-wave grating (45/135°) patterns. These patterns were enveloped by horizontal semicircles (3.2° diameter) with 1.2° vertical offsets from the central fixation point (0.4° diameter; 143.5 cd/m2), respectively. At stimulus presentation half time, the first brightness–orientation pair was replaced by the second pair, e.g., simultaneous changes from white to black along with from 135 to 45° patches. The presentation positions (upper and lower visual fields) and orders (first and second pairs) were randomly combined between brightness and orientation. The luminance contrast of the square-wave grating was full and the spatial phase was randomized across the trials.
In the spatially superimposed condition, visual stimuli were two brightness–orientation pairs (same as above) but superimposed on a single semicircular pattern (see Fig. 3A). The pattern was offset by 3.7° above the fixation point to prevent performance from reaching ceiling during the course of the staircase method. At the half time of the stimulus presentation, the first brightness–orientation pair was replaced by the second pair in the same manner as the separated condition.
Procedure.
All experiments were conducted in a dark room. Observers maintained fixation on a central point that appeared at the beginning and lasted until the end of each experimental block. Observers pressed a button with their right middle finger to start the trial. After an equally likely delay of between 50 and 810 ms in 40 ms steps, two brightness–orientation pairs were presented in rapid succession and then followed by dynamic white-noise patterns (H3.2 × V1.6°) to mask the afterimage. Observers reported the brightness–orientation pair that was presented at the same time by pressing one of two buttons on either side of the initiating button with the corresponding index or third finger in a two-alternative forced choice method. To avoid confounding responses and self-initiated actions, observers were asked to pause for a few seconds before pressing the button to start the next trial.
In the psychophysical experiments, stimulus duration was manipulated in steps of 0.05 log frame according to a classical “1 up 2 down” staircase method; duration was shortened if the response was correct in two consecutive trials but lengthened if the response was incorrect in a previous trial regardless of the delay condition. The initial value was determined for each observer by pilot experiments. Average stimulus duration was 121 ms (SD = 41 ms) in the spatially separated condition and 68 ms (SD = 53 ms) in the spatially superimposed condition. The number of trials for each delay was between 55 and 88 and was either the same or different by a few trials between delays for each observer.
In the EEG experiment, the spatially separated condition was tested with a constant stimulus duration that gave a 70% correct response rate for each observer in pilot experiments. Average stimulus duration was 123 ms (SD = 30 ms) and the average correct rate was 0.71 (SD = 0.14) across all observers. The action-to-stimulus delay was variable between 170 and 450 ms. Because we mainly aimed to examine the relationship between binding performance and neural oscillatory phases, the delay was not long enough to confirm behavioral oscillation as a function of delay. The number of trials for each delay was between 60 and 80 and was the same between delays for each observer.
Experimental design and statistical analysis: psychophysics.
To analyze behavioral oscillations, we fitted a sinusoid to individual correct response rates as a function of action-to-stimulus delay by the least-squares method with no restrictions except for initial values of free parameters: temporal frequency, phase, amplitude, and mean. Goodness-of-fit to sinusoids was tested by a nonparametric bootstrap analysis. The coefficient of determination (r2) of fit was iteratively computed based on delay values shuffled across trials with replacements for each observer. The initial values of free parameters were identical as for the observed data. Ten thousand bootstrap computations were performed to obtain a p-value as the probability of the real r2 in the reconstructed bootstrap distribution.
To ensure that the fitting was not determined by initial parameter values, we fixed the temporal frequency parameter at a variable frequency within the 1–12.5 Hz range and repeated fitting a sinusoid to the data. The initial values of free parameters were identical to those above. One-way ANOVA was used for comparisons of root mean square error (RMSE) of fits relative to the best fit between temporal frequency parameters.
To further analyze behavioral oscillations, we calculated the amplitude spectrum of correct response rates as a time-series of action-to-stimulus delay (analysis window = 50–810 ms) using the MATLAB fft function with zero padding. The significance level of spectrum amplitude was estimated by a nonparametric bootstrap analysis. For each observer, spectrum amplitude was iteratively computed with delay values shuffled across trials with replacements. Ten thousand bootstrap computations were performed to obtain a significance level at the 95th percentile of the reconstructed bootstrap distribution of maximum average amplitude across observers.
EEG recording and preprocessing.
The EEG was sampled with Ag-AgCl electrodes using appropriate sized electrode cap at Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1, O2, F7, F8, T7, T8, P7, P8, Fz, Cz, and Pz according to the international 10-20 system at a sampling rate of 1000 Hz (BrainVision Recorder, BrainAmp amplifier, EasyCap; BrainProducts). An additional electrode served as a common ground electrode placed between Fz and Fpz on the midline. All electrodes were referenced to another electrode placed between Fz and Cz, and were then re-referenced off-line to obtain the average between them. The EEG was resampled at 125 Hz, bandpass filtered between 1 and 60 Hz, and converted into epochs of between −1.5 and +2.0 s relative to the button press at the start of each trial and between −2.0 and +1.5 s relative to stimulus onset. Artifact components (i.e., eye movements) were removed by heuristic examination of independent components using the EEGALB runica function.
Experimental design and statistical analysis: EEG.
A time-frequency transform was applied to each epoch to obtain the phase angle of EEG signals for each of the 19 electrodes at various frequencies and time points using the EEGLAB newtimef function. Frequencies ranged from 1.5 to 50 Hz in 98 logarithmically spaced steps, whereas the length of the filter window changed linearly from 3 to 5 cycles. Circular-to-linear correlation was computed on the phase angle and the corresponding response accuracy (binary correct/incorrect data) across trials for each frequency, time point, electrode, and observer (Berens, 2009). Correlation values were then averaged across observers. The significance of the correlation, or the null hypothesis, was tested by a nonparametric bootstrap analysis. Circular-to-linear correlation was iteratively computed based on phase values shuffled across trials with replacements for each frequency, time point, electrode, and observer. Ten thousand bootstrap computations were performed to obtain a p value as the probability of the real average correlation in the reconstructed bootstrap distribution.
To further analyze phase-response correlation, we fitted a one-cycle sinusoid to the correct response rates in aggregate data (pooling individual data together; Benedetto et al., 2016) as a function of the corresponding neural phase (linearly binned from −π to +π) by the least-squares method with no restrictions except for initial values of free parameters: phase, amplitude, and mean. Frequency, time point, and electrode subject to this analysis were determined based on the circular-to-linear correlation (maximum average correlation). Goodness-of-fit to a sinusoid was tested by a nonparametric bootstrap analysis. R2 of fit was iteratively computed with phase values shuffled across trials with replacements. Ten thousand bootstrap computations were performed to obtain a p value as the probability of the real r2 in the reconstructed bootstrap distribution.
Intertrial phase coherence (ITPC) was also computed as a measure of the synchronization of neuronal activity locked by a button press for each frequency, time point, electrode, and observer (Tallon-Baudry et al., 1996; Delorme and Makeig, 2004) according to the following:
where n is the number of trials, Fk (f, t) is the spectral estimate of trial k at frequency f and time t, and || represents the complex norm. ITPC would be 0 for randomly distributed phases and 1 for perfect phase locking. The ITPC values were then averaged across observers. For analysis of inter-observer correlation, the null hypothesis was tested based on Pearson's correlation coefficient.
Results
Periodicity of binding spatially separate features
We examined periodicity in the temporal binding of brightness and orientation. After a variable delay from a button press (50–810 ms), two brightness–orientation pairs were presented in rapid succession (Fig. 1). Participants judged the brightness–orientation pair that was presented at the same time. If separate features are bound into a synchronous event in a periodic manner with respect to voluntary action, binding performance will fluctuate in a time-series of action-to-stimulus delay with a certain periodicity.
Stimulus presentation schematic. Participants initiate trials by pressing a button with fixation at a central point. After a variable delay from the button press (50–810 ms), two brightness–orientation pairs are presented in rapid succession (e.g., pairs of Stimulus 1 and Stimulus 2 in either the top or bottom row), and are then followed by dynamic noise mask patterns. Participants judge whether the white patch is presented at the same time as the 45° or 135° patch; if the white patch is presented with the 45° patch, then the black patch is necessarily presented with the 135° patch. The duration of each stimulus is adjusted for each participant throughout the trials by the staircase 1 up 2 down rule, which targets a success rate of 70%.
Figure 2A shows the proportions of correct responses for individual observers in the spatially separated condition. The orange curves are sinusoids fitted to data with respect to the action-to-stimulus delay. The inset circular plots indicate the sinusoids' phases (seemingly different between observers). Goodness-of-fit was confirmed by comparing the r2 with a distribution of r2 computed iteratively with bootstrapped data. In the reconstructed distribution of r2, the vertical dashed and orange lines indicate the 95th percentile and measured r2, respectively. An orange line to the right of the dashed line implies behavioral oscillation with a certain periodicity rather than biological noise. According to data, individual observers exhibit significant periodicities in the correct rates at temporal frequencies within 6.6–8.5 Hz except for S8 (S1: p = 0.048; S2: p = 0.012; S3: p = 0.001; S4: p = 0.022; S5: p = 0.006; S6: p = 0.036; S7: p = 0.040; S8: p = 0.119).
Periodic fluctuation of binding performance of spatially separate features. A, Left, Correct response rates (filled circles) of individual data fitted with a sinusoid (orange curve) as a function of action-to-stimulus delay. The best-fitting temporal frequency and phase are displayed in each panel. Right, Bootstrap distributions of r2 computed iteratively 10,000 times; dashed lines indicate the 95th percentile and orange lines indicate the real r2. B, Relative RMSE of fit estimated for different temporal frequency parameters fixed at each frequency divided by the RMSE of the best fit. Error bars represent ±1 SE across observers. C, Spectrum amplitude of correct response rates as a time-series of action-to-stimulus delays. Error bars represent ± 1SE across observers. The horizontal dashed line marks the 95th percentile of a bootstrap distribution of the maximum average amplitude computed iteratively 10,000 times.
Similarly, we found that the best fit, or minimum relative RMSE of fit, is achieved on average within 7.0–8.5 Hz when a sinusoid was fitted to the data by various temporal frequency parameters with 0.5 Hz steps, as presented in Figure 2B (ANOVA: F(23,161) = 3.788, p < 0.05). An additional spectrum analysis revealed average amplitude to be significant at ∼7.5 Hz (Fig. 2C). These results suggest that temporal binding performance of spatially separated features fluctuates with a periodicity of ∼8 Hz.
Periodicity of binding spatially superimposed features
We investigated the possibility of periodicity in binding spatially superimposed features which has previously been considered to be independent of the slow binding mechanism driven by attention (Holcombe and Cavanagh, 2001). Each pair of brightness (black/white) and orientation (45/135°) was displayed superimposed onto a single stimulus (Fig. 3A) and such two pairs were presented in rapid succession. The psychophysical results of the temporal binding task same as the separate condition are presented in Figure 3.
Binding stimuli and performance of spatially superimposed features. A, Each brightness–orientation pair was superimposed on a single stimulus and two such pairs were presented in rapid succession. B–D, Results of analyses same as Figure 2. Performance does not show significant periodic fluctuation for any individual data. Neither relative RMSE of fit (C) nor spectrum amplitude (D) is significant across the temporal frequencies analyzed.
As a result, no individual observers return significant periodicities in the correct rates (Fig. 3B; S1: p = 0.337; S2: p = 0.645; S3: p = 0.486; S4: p = 0.669; S5: p = 0.253; S6: p = 0.143; S7: p = 0.739; S8: p = 0.381). Figure 3C presents the same analyses of relative RMSE of fit (ANOVA: F(23,161) = 1.197, p = 0.25) and Figure 3D presents the spectrum, neither of which on average shows any significant periodicity. Considering the consistency of all analyses, these results should rule out the existence of periodicity in binding performance of spatially superimposed features at <12.5 Hz; the Nyquist limit of the present sampling frequency.
Correlation of EEG phase with binding performance
We subsequently examined the effect of neural oscillatory phases, specifically before stimulus onset (Busch et al., 2009; Mathewson et al., 2009; Dugué et al., 2011; Chakravarthi and VanRullen, 2012), on slow attentional binding performance. EEG was measured using 19 electrodes for 15 observers performing the same temporal binding task as with the spatially separated features. Circular-to-linear correlation was then computed between the EEG phase and the response accuracy across trials for each frequency, time point, electrode and observer. In this experiment, stimulus duration was constant at a threshold estimated in advance for each observer and the action-to-stimulus delay was variable in a 280 ms period from 170 to 450 ms after the button press.
Preliminary analysis of behavioral data exhibited periodic fluctuations of correct responses as a function of action-to-stimulus delay for 2 observers but not significant for the other 13 observers. This could be explained by the difficulty to fit a sinusoid and to estimate behavioral periodicity reliably with a small number of data points (delays) at the individual level. As noted in the Materials and Methods, behavioral analysis is not the main aim of the EEG experiment, but at least the average frequency of the best fit is 7.27 Hz (SD = 3.04 Hz) across all observers, in line with the psychophysical results.
Figure 4A shows the average correlation values from the left parietal (P3) electrode, which we selected based on the most pronounced effect. The bright color plot indicates significant values (p < 0.01). By visual examination only, a significant ∼8 Hz component appears to last from −400 to 0 ms before stimulus onset, overlapping with a period from −450 to −170 ms when observers were supposed to press a button. Poststimulus signals could not explain all aspects of the significant component starting >100 ms before the time points that contain poststimulus signals due to the analysis window. This component is localized to the P3 electrode with a peak at 8.5 Hz and −120 ms before stimulus onset (Fig. 4B). We also found that the correct response rate fluctuates cyclically depending on the EEG phase at this peak time-frequency point marginally significantly (Fig. 4C; p = 0.051) by fitting a one-cycle sinusoid to the data. From these results, binding performance seems at least partly to depend on the prestimulus neural oscillatory phase.
Binding dependence on prestimulus neural oscillatory phase and ITPC around action. A, Time-frequency representation of the average circular-to-linear correlation between neural oscillatory phase at P3 and response accuracy measured by binary correct/incorrect data. The bright color plot marks significant correlation (p < 0.01) identified by bootstrap analysis. The dashed vertical line corresponds to stimulus onset time (0 ms). The dashed curve indicates, for each frequency, the first time point whose analysis window includes the stimulus onset. B, Scalp map of circular-to-linear correlation at 8.5 Hz and −120 ms. The filled electrode point marks significant correlation (p < 0.01) identified by bootstrap analysis. C, Left, Correct response rates (filled circles) of aggregate data (n = 15) fitted with a one-cycle sinusoid (orange curve) as a function of EEG phase at P3, 8.5 Hz, and −120 ms binned linearly from −π to +π. Right, Bootstrap distribution of r2 computed iteratively 10,000 times; the dashed line indicates the 95th percentile and the orange line indicates the real r2. D, Time-frequency representation of average ITPC at P3 aligned with button press. The dashed vertical line corresponds to button press time (0 ms). E, Scalp map of ITPC at 5.1 Hz and 50 ms. F, Left, Inter-observer correlation between peak ITPC frequency within 3–15 Hz and −200 to 400 ms with respect to button press and peak circular-to-linear correlation frequency within 3–15 Hz and −600 to 0 ms relative to stimulus onset, both at electrode P3. The diagonal line shows a fitted power function: log(y) = 1.43log(x) + log(0.76). Right, Inter-observer correlation between peak ITPC frequency at P3 and the behavioral periodicity frequency of binding performance measured in the psychophysical experiment. The diagonal line shows a fitted power function: log(y) = 0.56log(x) + log(3.21).
EEG phase coherence locked with action
We further analyzed how voluntary action in the form of a button press could affect the neural processing that yields the behavioral oscillations. ITPC, or phase-locking value (Lachaux et al., 1999), relative to the observer's button press was computed for each frequency, time point, electrode, and observer.
Figure 4D shows the ITPC values at the P3 electrode averaged across all observers. A distinct ITPC component appears centered at ∼4–6 Hz from −50 to 200 ms partly before and just after the button press. This component is localized to the left hemispheric electrodes, consistent with observers using a right finger to press a button, with a peak at P3, 5.1 Hz, and 50 ms after the button press (Fig. 4E). On grounds that corresponding power components are absent in similar time and frequency bands (time-frequency representation is not shown here), the ITPC component should be attributed to phase alignment of pre-existing neural oscillations but not to distinct oscillations evoked by action (as described by Popovych et al., 2016). Importantly, we found a significant inter-observer correlation between peak ITPC frequency in the ranges of 3–15 Hz and of −200 to 400 ms with respect to button press and peak correlation frequency in the ranges of 3–15 Hz and −600 to 0 ms relative to stimulus onset at the P3 electrode (r = 0.56, t(1,13) = 2.46, p = 0.029 in log scale; Fig. 4F, left). There was no significant inter-observer correlation between peak ITPC and peak correlation values themselves (r = 0.16, t(1,13) = 0.58, p = 0.57 in log scale). Moreover, peak ITPC frequency can predict behavioral periodicity frequency between eight observers who participated in both the EEG and the psychophysical experiments (r = 0.76, t(1,6) = 2.90, p = 0.027 in log scale; Fig. 4F, right), whereas peak ITPC value does not predict behavioral periodicity frequency (r = −0.52, t(1,6) = 1.48, p = 0.19 in log scale) or amplitude (r = 0.13, t(1,6) = 0.33, p = 0.75 in log scale) in the same eight observers.
Discussion
The present study demonstrated that temporal binding performance fluctuates at ∼8 Hz with respect to a self-initiated button press for spatially separated, but not for superimposed, conjunctive features. EEG analysis also revealed a significant dependence across trials on prestimulus neural oscillatory phases. Furthermore, the peak ITPC frequency locked with the button press was highly correlated with the peak frequency of this dependence and with the behavioral periodicity frequency between observers. All the data are consistent with the notion that the temporal binding of separate features is performed in a periodic manner at ∼8 Hz based on the periodic function of an attention network and that neural oscillatory activity phase-locked to voluntary action may underlie the periodic binding performance.
Here we consider the possibility that the periodic fluctuation of correct response rate reflects a few factors other than the periodicity of feature binding. First, periodic fluctuation might be a product of periodic elements in sensory processing (e.g., mechanisms for detecting luminance) rather than of high-level attentional processes. It is however not quite probable since periodic fluctuation was not evident for binding superimposed features (a process which has been considered as involved in low-level but not attentional processes). One might consider that we cannot compare the separate and superimposed conditions because stimulus duration was different, but then we should take into account that in the separate condition stimulus duration was not correlated with behavioral frequency between observers; if anything, higher behavioral frequency was paradoxically associated with slower stimulus change (r = −0.21, t(1,6) = 0.53, p = 0.62 in log scale). Therefore, the periodic fluctuation would reflect attentional processes that possess distinct periodicities from low-level processes. Second, one might argue that observers solved the temporal binding task by comparing attribute presentation order with features in their working memories. This cannot be the case however given that stimulus duration, approximated to the discrimination threshold using a staircase method (121 ms per stimulus on average), was several times longer than the generally reported resolution of temporal order judgment of ∼30 ms (Hirsh and Sherrick, 1961; Pöppel, 1997). If one rules out the possibilities above, one may well interpret the present results as reflecting periodic processes associated with attentional feature binding.
The psychophysical results suggest the periodic function of an attention network on the present binding task. Recent psychophysical findings imply that attention samples perceptual information at a rate of ∼4–8 Hz in high-level processing stages (Landau and Fries, 2012; Fiebelkorn et al., 2013; Dugué et al., 2015). This frequency range also corresponds to the temporal resolution of tracking a single object (Verstraten et al., 2000; Holcombe and Chen, 2013). Biological visual systems input sensory information by moving gaze in a discrete manner, guided by attentional processes that can otherwise operate with covert independence from gaze (Rizzolatti et al., 1987; Moore et al., 2003; Hogendoorn, 2016). Attention is therefore expected to shift with such periodicity unless other salient events happen to capture attention. On this basis, the temporal binding of separate features may be performed through periodic attentional sampling as well. In other words, attention may explore locations of separate features alternately at ∼8 Hz, and then bind them successfully if the stimulus presentation coincides with the timing of periodic exploration. Accordingly, the stimulus duration approximated to the discrimination threshold (121 ms per stimulus on average) roughly corresponds to the behavioral periodicity, or perhaps attentional sampling, frequency of ∼8 Hz.
Periodic processes may be a consequence of neural oscillations as the most fundamental principle underlying neural information processing in the brain. Neural oscillations show various frequencies in cortical and subcortical systems, appearing to affect perceptual tasks with different periodicities (VanRullen, 2016). Computationally speaking, periodic activities can play a role in integrating temporally diverse neural representations from different cortical streams into a “present” window and thus produce a temporally unified conscious perception. Behavioral performance of integration and/or segregation tasks across visible frames depends on neural oscillatory phases within widely distributed bands (Varela et al., 1981; Cravo et al., 2015; Samaha and Postle, 2015; Milton and Pleydell-Pearce, 2016), and neuronal rhythmicity is coupled across cortical areas between different frequencies (Lakatos et al., 2005; Canolty et al., 2006; Landau et al., 2015). In this sense, stimulus duration in the superimposed condition (68 ms per stimulus on average) might be critical to the absence of behavioral oscillations because it would be difficult to discriminate two successive pairs presented within one cycle of ∼8 Hz rhythms (Wutz et al., 2016, 2018). While this speculation presupposes the existence of behavioral rhythms related to the present binding task, such behavioral rhythms might reflect the periodic processes of integration and discrimination for visual stimuli.
The EEG results suggest that neural phase-locking with action may underlie the periodic binding performance. To date, behavioral oscillations have been found relative to the timing of stimulus onset (VanRullen et al., 2007; Busch and VanRullen, 2010; Landau and Fries, 2012; Fiebelkorn et al., 2013; Dugué et al., 2015, 2016) and observer action (Tomassini et al., 2015, 2017; Benedetto et al., 2016; Bellet et al., 2017). Neurologically, it could be explained that neural information processing is modulated according to the neural oscillatory phase being reset by an external stimulus (Fiebelkorn et al., 2011; Romei et al., 2012; Mercier et al., 2013, 2015). Tomassini et al. (2017) discussed the relevance of motor command to synchronization between perception and action on the grounds that orientation detection performance is correlated with prestimulus 4 Hz neural oscillatory phases around the observer's hand movement. In common with the present study, it is still possible that spontaneous motor actions are locked to an ongoing attentional rhythm; there is no way to know whether the button press is truly randomly timed, and resets the behavioral oscillation, or whether its timing always tends to be around the same phase of a preexisting attention oscillation, which then continues after the button press with or without a phase reset. Assuming that feature binding and action are both involved in attention and that attentional selection induces oscillatory phase reset of neuronal firing patterns (Lakatos et al., 2008; Schroeder and Lakatos, 2009), we can argue for an important role of attention in mediating, or perhaps phase-resetting, neural and perceptual oscillations that are well synchronized with voluntary action.
Here we attempt to account for the frequency difference between the ITPC component aligned with the timing of action (4–6 Hz) and the periodic binding performance (∼8 Hz), although these frequencies both fall within the range of 4–8 Hz that has been presumed an attentional rhythm. The frequency difference can be explained by assuming that the periodicity of feature binding is not determined but rather affected by the action-locked component. In addition to the numerous neurophysiological phenomena of frequency entrainment (Regan, 1966; Morgan et al., 1996; von Stein and Sarnthein, 2000; Fries et al., 2002), Benedetto et al. (2016) demonstrated that the behavioral frequency of contrast detection increases from 5 to 7 Hz by reducing display luminance, and the authors interpreted the results in terms of entrainment as amplified alpha activity of ∼10 Hz. Considering the essential role of parietal cortex in feature binding (Friedman-Hill et al., 1995; Shafritz et al., 2002), periodic processes associated with feature binding might slightly shift in frequency depending on the ITPC component centered at the parietal area.
Increasing studies have reported periodic effects of attention (for review, see VanRullen, 2016) and rhythmic percepts (Chakravarthi and VanRullen, 2012). Although the periodicity varies by the stimulus and the task, feature binding would be involved in achieving a unified conscious perception. The binding problem is known to essentially assume the existence of consciousness, which is often considered a goal of cortical information processing (Tononi, 2004). Combined with perceptual experiences of discrete sampling with visual stimuli where distinct features anticipate different positions (Arnold and Johnston, 2003; Nakayama et al., 2018), the present study implies that periodic integration processes might be key in understanding consciousness.
Footnotes
This work was supported by JSPS KAKENHI Grants JP15H03461, JP16H01499, JP16H03755, and JP18H04935 to I.M. A part of this study was performed when R.N. was a JSPS research fellow, a visiting researcher at CiNet, NICT, and a visiting researcher at the University of Sydney. We thank Professor Alex O. Holcombe of the University of Sydney for helpful comments on the paper.
The authors declare no competing financial interests.
- Correspondence should be addressed to Ryohei Nakayama at ryouhei.nakayama{at}gmail.com
