Abstract
Despite their small size, microsaccades can impede stimulus detections if executed at inopportune times. Although it has been shown that microsaccades evoke both inhibitory and excitatory responses across different visual regions, their impact on the higher-level neural decision processes that bridge sensory responses to action selection has yet to be examined. Here, we show that when human observers monitor stimuli for subtle feature changes, the occurrence of microsaccades long after (up to 800 ms) change onset predicts slower reaction times and this is accounted for by momentary suppression of neural signals at each key stage of decision formation: visual evidence encoding, evidence accumulation, and motor preparation. Our data further reveal that, independent of the timing of the change events, the onset of neural decision formation coincides with a systematic inhibition of microsaccade production, persisting until the perceptual report is executed. Our combined behavioral and neural measures highlight antagonistic interactions between microsaccade occurrence and evidence accumulation during visual decision-making tasks.
SIGNIFICANCE STATEMENT When fixating on a location in space, we frequently make tiny eye movements called microsaccades. In the present study, we show that these microsaccades impede our ability to make perceptual decisions about visual stimuli and this impediment specifically occurs via the disruption of several processing levels of the sensorimotor network: the encoding of visual evidence itself, the accumulation of visual evidence toward a response, and effector-selective motor preparation. Furthermore, we show that the production of microsaccades is inhibited during the perceptual decision, possibly as a counteractive measure to mitigate their negative effect on behavior in this context. The combined behavioral and neural measures used in this study provide strong and novel evidence for the interaction of fixational eye movements and the perceptual decision-making process.
Introduction
It is increasingly apparent that microsaccades, tiny fixational eye movements, play a vital role in supporting perception (Hsieh and Tse, 2009; Martinez-Conde et al., 2013; Chen et al., 2015). However, these beneficial effects also come at a cost: to ensure perceptual stability during movement of the retina, visual activity is disrupted in the interval surrounding a microsaccade, leading to a transient increase in visual thresholds and reaction times (RTs) for stimuli appearing in that interval (Ditchburn, 1955; Zuber and Stark, 1966; Beeler, 1967; Herrington et al., 2009; Hafed and Krauzlis, 2010; Tian and Chen, 2015). Therefore, microsaccades are a potentially significant contributor to behavioral variability on a wide range of perceptual tasks, a commonly overlooked fact that has broad implications for basic and clinical research on vision (Hafed et al., 2015). However, the precise neural mechanisms that mediate microsaccadic perceptual suppression are not yet fully understood. Neurophysiological investigations of this phenomenon have been recorded from numerous visual areas under a variety of experimental contexts and, although certain areas exhibit excitatory modulations, others exhibit inhibitory modulations or a combination of the two (Gur and Snodderly, 1987, 1997; Leopold and Logothetis, 1998; Martinez-Conde et al., 2000, 2002; Dimigen et al., 2009; Herrington et al., 2009; Chen et al., 2015; Tian and Chen, 2015). Further, whereas these studies have focused primarily on the effects on sensory representations, the impact of microsaccades on the higher-level processes that translate sensory information into appropriate action has yet to be considered.
Convergent data from psychophysics, computational modeling, and neurophysiology highlight the central role of “decision variable” signals in determining the timing and accuracy of perceptual reports by accumulating sensory evidence over time up to an action-triggering threshold (Shadlen and Kiani, 2013). Such accumulation-to-bound dynamics have been demonstrated both in signals recorded noninvasively from the human brain (Kelly and O'Connell, 2015) and in the spiking activity of certain neuronal subpopulations within several sensorimotor regions of the monkey brain (Roitman and Shadlen, 2002; Huk and Shadlen, 2005; Ratcliff et al., 2007). In two of these regions, the lateral intraparietal area (LIP) and the superior colliculus, it has been reported that microsaccades occurring around the time of stimulus onset result in a brief suppression of activity (Herrington et al., 2009; Hafed and Krauzlis, 2010; Chen et al., 2015, Chen and Hafed, 2017). However, these effects were not observed specifically on neural signals reflecting evidence accumulation and, given the heterogenous firing characteristics and breadth of functional roles associated with these regions (Bisley and Goldberg, 2010; Gandhi and Katnani, 2011; Krauzlis et al., 2013; Meister et al., 2013), it cannot be inferred whether and how microsaccades may affect decision variable dynamics in particular.
Here, we sought to investigate the impact of microsaccades on perceptual decision formation in the human brain. Although microsaccades have been studied in relation to the visual responses that they evoke over occipital areas (Dimigen et al., 2009; Meyberg et al., 2015), no human study has yet established even basic aspects of their impact on neural stimulus processing, such as the suppressive effects reported in animal work. This is largely due to the technical challenges associated with parsing the complex, multicomponent responses evoked by discrete stimulus events in noninvasive recordings. We used continuous stimulation paradigms that overcome such limitations and enable the isolation of independent neural signals reflecting the key processing levels necessary for simple perceptual decisions (O'Connell et al., 2012; Kelly and O'Connell, 2013). This approach allowed us to gain three key, novel insights. First, microsaccadic neural suppression was observed in early visual cortex with a highly similar time course to that observed in single-unit recordings (Leopold and Logothetis, 1998). Second, microsaccades executed at any point during the period of decision formation cause a brief delay in decision signal buildup that accounts for their detrimental impact on perceptual performance. Finally, our data reveal that microsaccade production is systematically inhibited throughout the period of neural evidence accumulation that facilitates the sampling of visual information.
Materials and Methods
Participants
All participants gave written informed consent and all procedures were approved by the ethical review boards at the site of data collection for each task: either the ethical review board in City College of New York (Experiment 1) or the School of Psychological Sciences in Monash University Melbourne (Experiment 2). Ethical guidelines were in accordance with the Declaration of Helsinki. All participants were right handed, over the age of 18, had normal or corrected-to-normal vision, and had no history of psychiatric diagnosis, sensitivity to flickering light, or head injury resulting in loss of consciousness.
After participant rejection (see “Data exclusion” section), there was a sample size of 17 (12 males, aged 21–32 years) in Experiment 1 and 43 (26 males, aged 20–28 years) in Experiment 2.
Materials and task procedures
Both tasks were performed in a darkened room with participants seated 56 cm from a 21 inch CRT (85 Hz, 1024 × 768 resolution). Visual stimuli were presented on a dark gray background. Participants were instructed to fixate on a central 5 × 5 pixel white square at all times. They were also asked to restrict any eye movements throughout the task such as large saccades or blinks. If any participant performed such eye movements in any block, they were reminded not to do so for the subsequent blocks. An Eyelink eye tracker (EyeLink version 2.04; SR Research/SMI) recorded eye movements that were later analyzed to ensure that participants maintained fixation and to detect the presence of microsaccades.
Experiment 1: Gradual contrast change detection task.
Participants were instructed to monitor a continuously flickering (25 Hz), annular checkerboard pattern to detect intermittent targets defined by gradual contrast increases or decreases (Fig. 1A). The checkerboard stimulus extended from 3 to 8° eccentricity, with 6 tiles (alternating light and dark) spanning the radial extent and 64 tiles extending over the full 360° of polar angle. To maximize the steady-state visual evoked potential (SSVEP) signal-to-noise ratio, the annulus was divided into eight segments that flickered on-and-off with phase differences imposed to induce constructive oscillatory signal summation on the scalp (Vanegas et al., 2013). Contrast was held at a constant baseline level of 70% during interchange intervals. On target onset, contrast linearly increased (“up” blocks) or decreased (“down” blocks) from the 70% baseline by 30% over 1.2 s, reaching 100% or 40% before immediately returning to the baseline level at twice the rate, resulting in an overall target duration of 1.8 s. The interchange interval was randomly 4, 7, or 10 s. Participants performed 6 “up” blocks in which the target contrast change was upward only and 6 “down” blocks in which the target contrast change was downward only, each containing 24 targets and lasting just over 4 min. It was emphasized that participants should avoid guessing and make a right index finger mouse button press as soon as they were certain that the checkerboard had increased or decreased in contrast.
A, Contrast change detection paradigm. Participants monitored a continuously presented flickering checkerboard annulus for a gradual change in overall contrast. In each task block, the stimulus remained onscreen for ∼4 min and underwent contrast changes at pseudorandom intervals of 4, 7, and 10 s. After reaching the peak contrast change at 1.8 s, the stimulus returned to its baseline contrast level. On alternating blocks, the checkerboard stimulus underwent contrast increases or decreases and participants were cued at the start of each block regarding which change to expect. B, Experimental for the bilateral dots experiment. Participants performed a version of the task in which they monitored two peripheral patches of randomly moving dots for intermittent targets defined by a seamless step change from random to coherent motion. Each trial consisted of a pretarget period of incoherent motion lasting 1.82, 2.22, or 2.62 s, followed by a downward coherent motion target that could come up in either the left or right hemifield, upon which participants were instructed to simultaneously press both the left and right mouse buttons as quickly as possible. C, Microsaccade magnitude plotted against microsaccade velocity. This demonstrates the main sequence whereby microsaccade magnitude and velocity are highly correlated, a characteristic of saccadic behavior (Zuber et al., 1965). D, Spatial distribution of microsaccades. The center represents the start point of the microsaccade and each dot the end point of a microsaccade. The empty space in the center of the plot represents the minimum deviation from fixation required for an eye movement to be classified as a microsaccade.
Experiment 2: Bilateral dot motion task.
We then investigated whether the effects we found in the contrast-based task in Experiment 1 extended to the random dot motion task (Newsome et al., 1989; Kelly and O'Connell, 2013; Loughnane et al., 2016). Here, participants performed a version of the task in which they monitored two peripheral patches of randomly moving dots for intermittent targets defined by a seamless step change from random to coherent motion (Fig. 1B). In these data collected as part of a previously reported experiment (Newman et al., 2014; Loughnane et al., 2016), coherence levels were titrated to below ceiling performance and involved discrete trial presentations. In both the titration and experimental blocks, each trial consisted of a pretarget period of incoherent motion lasting 1.82, 2.22, or 2.62 s, followed by a downward coherent motion target that could come up in either the left or right hemifield, upon which participants were instructed to simultaneously press both the left and right mouse buttons as quickly as possible. Coherence levels were determined for each participant in the titration block using a four-down, one-up staircase designed to converge on the individualized coherence level at which the participant could achieve approximately an 85% hit rate (García-Pérez, 1998). In the titration block, 25% of trials were catch trials that contained no target and participants received visual feedback after each trial as to whether they were correct, incorrect, or pressed the button before the target had onset. The resulting individualized coherence level then remained constant throughout the experimental block. The average of the individualized coherence levels was 19.5 ± 6.7%. The experimental block included 10% catch trials of which the participant was informed before the task began; they also received negative feedback if they responded before target onset. Participants completed one block of 330 trials in total. After trial and participant rejection (see below), the average hit rate of analyzable trials during the experimental block was 81 ± 8.8%. For catch trials, the false alarm rate was 12 ± 10.5% and correct rejection rate 88 ± 10.5%.
The center of each dot motion patch was at a visual angle 10° either side and 4° below the fixation square; each patch covered 8° visual angle and consisted of 150 6 × 6 pixel white dots. The dots stimuli were flickered on and off screen every 23.5 ms, resulting in a flicker rate of 21.25 frames/s (each “frame” representing two screen refreshes). During incoherent motion, these dots were placed randomly throughout the patch on each frame. During coherent motion, a proportion of the dots were randomly selected on each frame to be displaced in either the upward or downward direction on the following frame, with a motion speed of 6°/s.
Experimental design and statistical analysis
Both Experiments 1 and 2 incorporated a within-subjects design specifically using repeated-measures ANOVA and within-subjects t tests. Any further analyses are described in the “Data analysis” section below and in the Results section. Replication of analyses from Experiment 1 on the data from Experiment 2 was used to both extend the findings across experimental paradigms and as a control against type I error.
Microsaccade detection.
Using the eye position data collected by the eye tracker, microsaccades were detected using the EEGLAB developed by Ralf Engbert and colleagues (Engbert and Mergenthaler, 2006; http://www2.hu-berlin.de/eyetracking-eeg/index.php). In this algorithm, microsaccades are defined as outliers in 2D velocity space and saccades are identified relative to the general noise level of the eye position data. Parameters for saccade detection were based on recommendations from Engbert and Mergenthaler (2006). Microsaccades were defined as eye movements in which the velocity exceeded threshold for longer than 8 ms. This velocity threshold was set at 6 SDs of the median velocity. Microsaccades were also required to be separated in time by at least 50 ms. To validate the microsaccade detection algorithm, we plotted microsaccade velocity against magnitude, known as the main sequence (Fig. 1B; Zuber et al., 1965). The majority of saccades were <1° of visual angle and microsaccades were defined as those that fell below this threshold (Martinez-Conde et al., 2013).
EEG acquisition and preprocessing.
In Experiment 1, continuous EEG was acquired from 97 scalp electrodes using a Brain Products system digitized at 500 Hz. In Experiment 2, EEG was acquired from 64 scalp electrodes using a Brain Products system digitized at 500 Hz. All data were analyzed using a combination of custom scripts and EEGLAB routines (Delorme and Makeig, 2004) in MATLAB (The MathWorks). A 0.1 Hz high-pass filter and a 35 Hz low-pass filter were applied offline. Noisy channels were interpolated (spherical spline) and the data were rereferenced to the average reference. Target epochs were extracted from the continuous data using a window of −300 ms to 1700 ms around evidence onset (contrast change in Experiment 1, coherent motion in Experiment 2) and baseline corrected with respect to −100 to 0 ms before target onset. The extra 200 ms before the baseline period allowed analysis of pretarget microsaccade rate in sliding windows. For one analysis, epochs were also created in the interchange interval, from −1500 ms to stimulus onset and baselined from −1500 ms to 1400 ms. All epochs were subjected to a current source density transformation using the CSD Toolbox for MATLAB (Kayser and Tenke, 2006).
Data exclusion.
For all experiments, trials were excluded from all analyses if they fulfilled either of the following criteria: RTs slower than 1700 ms or EEG from any channel exceeding ± 100 μV during the interval between 100 ms before evidence onset and 100 ms after response. Trials were also excluded if central fixation was broken at any stage during the epoch by blinking or eye movement >3° left or right of center throughout the whole epoch. Any participant with <30% of trials remaining were entirely excluded from all analyses. This led to the exclusion of 12 participants in Experiment 2. In addition, any participant with <15 trials in any data cell for any analysis was excluded from that analysis. This led to the exclusion of two participants in analyses comparing 0 and 1 microsaccade trials in Experiment 1 and 23 participants in the same analyses in Experiment 2.
Data analysis.
In both experiments, the central parietal positivity (CPP) was measured from three parietal midline peak electrodes, clustered around CPz using the 10–20 coordinate system. To eliminate the influence of the SSVEP and alpha power on the slow-potential CPP analyses, the waveforms were further low-pass filtered to 8 Hz using a fourth-order Butterworth filter. In Experiment 1, the SSVEP (25 Hz) was measured using the standard short-time Fourier transform (STFT) with a boxcar window size fitting exactly six cycles of the SSVEP frequency (320 ms) and 20 ms step size. The SSVEP was averaged over five electrodes centered on standard site Oz using the 10–20 coordinate system. Lateralized beta power (left hemisphere beta, LHB) was measured as the oscillatory power in the 18–30 Hz range over motor electrodes in the left hemisphere in Experiment 1. Power was measured in a sliding boxcar window of 240 ms with 20 ms step size. As an extra control analysis, all lateralized beta analyses were also performed having excluded the SSVEP frequency (25 Hz) bin in the STFT calculation. For both SSVEP and beta analyses, time–frequency calculations were also performed, this time using a wavelet method. To determine the onset of microsaccadic inhibition, CPP and lateralized beta power, we performed running sample point by sample point t tests against zero across all the participants' average waveforms. General onset was determined as defined as the first point at which the amplitude reached significance at the 0.05 level for 10 or more consecutive points, beginning at that point (Foxe and Simpson, 2002; Kelly et al., 2008).
In keeping with O'Connell et al. (2012), the SSVEP linearly tracked the physical contrast changes, but the buildup rates of the CPP and beta increased over time, indicating that they were not simply tracking the momentary sensory evidence. Following the same methods used in O'Connell et al. (2012), we formally tested the evidence accumulation hypothesis by performing separate regressions for each participant to fit the response-locked LHB and CPP signals with the SSVEP divergence from baseline and its cumulative sum beginning 0.6 s before response time. We statistically confirmed differences of accuracy of the fits via the R2 statistic of the regressions. We also performed a similar analysis whereby we regressed the actual evidence (contrast level) on each signal; the CPP, LHB, and SSVEP, to demonstrate formally the nonlinear nature of the CPP and LHB compared with the linear SSVEP.
The first step in analysis was to probe the relationship among microsaccade rate, CPP, lateralized beta (in Experiment 1), and the speed of manual response. All measurements were taken from the decision buildup period, 400–800 ms in Experiment 1 and 300–600 ms in Experiment 2 (tailored to each experiment's decision period). We sorted trials according to RT and divided them into two equal-sized bins. Importantly, RT binning was done within each trial condition (i.e., intertrial interval and target contrast direction), thus eliminating confounding factors known to have an influence on RT. Analysis of the effect of RT binning on all signals was conducted using a repeated-measures ANOVA, with factors of target contrast direction, interchange interval, and RT bin. Microsaccade rate (microsaccades per second) was calculated inside each RT bin in 100 ms sliding windows in steps of 2 ms by obtaining the number of microsaccades in a given set of trials in that 100 ms window, multiplying by 10, and dividing by the number of trials. In a further analysis to examine the duration of the effect of RT binning on microsaccade rate, running t tests were performed on each time window comparing microsaccade rates in each bin. To account for multiple comparisons across the time series, a nonparametric permutation method (Cheadle et al., 2014; Loughnane et al., 2016) was used to calculate a conservative significance threshold. The data were shuffled randomly before performing t tests across every time point up to the time of grand average RT (∼1200 ms) and then taking the most extreme t value from this series. This procedure was performed 10,000 times to create a distribution of t values with which to compare the original t value time series. The 2.5 and 97.5 percentiles of the resultant distribution provided the final two-tailed significance threshold.
In Experiment 1, SSVEP was also analyzed by RT bin, whereas in Experiment 2, the target selection signal N2c (Loughnane et al., 2016) was analyzed. With the exception of the evidence-locked N2c, all signals were analyzed by RT bin relative to both evidence onset and also relative to manual response. Finally, RT images of all electrophysiological signals were calculated by concatenating all evidence-locked single trials and sorting them by RT z-scored inside participant and experimental condition. The image was then smoothed along the y-axis in a 100-trial boxcar averaging window. The same analysis was performed on microsaccade rate, with a slight change because the data consist of a series of punctuate eye movement events. That is, microsaccade rate was first calculated across 100-trial bins and then concatenated and sorted by the average z-scored RT in those bins. Finally, we formally tested the relative alignment of saccadic suppression to contrast change or response by calculating the time at which the lowest microsaccade rate occurred in a given 100-trial bin relative to both evidence onset and response. We then compared the SDs of these two measures and performed a Brown–Forsythe test to test for significant difference in variance. The effect of hits versus misses was also examined in Experiment 2, which had enough misses in enough subjects.
The influence of microsaccades on electrophysiological signals in the decision buildup period was probed by creating epochs from −100 to 300 ms around microsaccade onset and baselining from −100 ms to 0 ms. In Experiment 1, microsaccades were chosen specifically from a window of 400–800 ms and, in Experiment 2, 300–600 ms. Only trials with RTs after these time windows were then included in analyses. These time windows were chosen as the best method by which to measure microsaccades in the early buildup of the sensorimotor transformation and not eliminate too many trials due to RT distribution. For statistical comparison, “sham” microsaccade-locked epochs were created by taking the real microsaccade timings and applying them to trials without microsaccades in the aforementioned time windows. −100 to 300 ms epochs were then taken around those sham microsaccade timings. For analysis of the CPP in this manner, it was necessary to perform a further step to exclude the potential influence of what is known as the microsaccadic lambda response (MLR) (Dimigen et al., 2009). It has been shown that microsaccades result in a positive visual-evoked potential (VEP), peaking at posterior occipital sites with a negative dipole at central parietal sites. To account for this possibility, we calculated the same microsaccade-locked epochs in the interchange interval and subtracted them from the microsaccade-locked epochs in the trial period during the evidence accumulation process, thus eliminating the MLR. This process was performed for both real and sham microsaccade-locked epochs, with the resulting subtraction waveforms analyzed in the same manner as the other signals. To fully exclude the possibility that any differences arose from diminished attention levels on trials with microsaccades, the same analysis was repeated except that the “sham” waveforms were generated by randomly shuffling the microsaccade onset times and applying these timings to trials on which a microsaccade had occurred.
Finally, we investigated whether the microsaccade-evoked neural signal modulations were consequential for behavior. To do this, we examined the plausibility of a model whereby the effect of microsaccades on RT is statistically mediated by the SSVEP, CPP, and lateralized beta (Baron and Kenny, 1986; Loughnane et al., 2016). A significant mediation in this context would support the idea that microsaccades influenced RT via the processes represented by our neural signals. The predictor variable in the model was microsaccade presence (0 or 1 in the model). The dependent variable was single-trial RT z-scored inside participant and experimental condition. The mediating variable was single-trial SSVEP, CPP, or beta amplitude (depending on the analysis) taken from the latter half of the microsaccade search window and z-scored inside participant and experimental condition. A final step to improve signal-to-noise ratio of the EEG signals was to sort single trials by z-scored RT and then average across 15-trial bins. Importantly, all analyses were also statistically significant on a single-trial basis. Bootstrapped mediation was then performed on the averaged trial bins using the Mediation Toolbox for MATLAB (M3 MATLAB toolbox; http://wagerlab.colorado.edu/tools). For all these analyses, mediation was furthermore performed across all electrodes on the scalp and the mediation effect, ab, plotted topographically to show that the effects were localized to the relevant electrodes for that signal. For the SSVEP mediation analysis, SSVEP power for contrast decrease trials was first multiplied by −1 to combine it with data for contrast increase trials.
Results
In a first experiment, participants monitored a continuously presented checkerboard stimulus for intermittent, gradual changes in its contrast (Fig. 1A). The direction of the contrast change (increase or decrease) was alternated across blocks and detections were reported via right hand button click. This allowed us to capture several distinct stages of the perceptual decision-making process. The encoding of decision-relevant sensory information in early visual regions was captured in stimulus-driven, contrast-dependent, steady-state responses (SSVEP) (Di Russo et al., 2007; O'Connell et al., 2012). In parallel, we traced decision formation in two distinct neural signals, namely effector-selective motor preparation indexed by spectral changes in the beta band (18–30 Hz) and the CPP in the event-related potential. Both of these signals have been shown to build up as a function of cumulative sensory evidence, reach a threshold before response execution, and exhibit dynamics that predict the timing and accuracy of perceptual reports (O'Connell et al., 2012), albeit with important functional differences (O'Connell et al., 2012; Kelly and O'Connell, 2013; Twomey et al., 2016). The average hit rate for these contrast changes was 89% ± 9.4%, whereas false alarms in the interchange interval were rare (median false alarm count: 8, SD = 26). The seamless nature of the contrast change/evidence onsets served to extend the duration of the decision process (mean RT, 1197 ± 104 ms), allowing the investigation of the pattern and influence of microsaccadic activity across a longer timescale than previous studies. To verify the role of evidence accumulation processes in supporting performance on this task, we fit the momentary versus cumulative contrast change values to each of the choice-relevant EEG signals and found that cumulative contrast provided a significantly better fit for the time courses of the decision signals (CPP and LHB), but not for that of the SSVEP (comparison of participant-by-participant R2 values for CPP: t(16) = −5.25, p = 7.83e-05; LHB: t(16) = 8.66, p = 1.96e-07; SSVEP: t(16) = −1.36, p = 0.19). Similarly, the cumulative sum of the SSVEP time course provided a significantly better fit for the decision signals than its absolute amplitude, consistent with evidence accumulation (comparison of R2 values for CPP: t(16) = −7.02, p = 2.98e-06; LHB: t(16) = −3.69, p = 0.002).
Although participants were instructed to maintain fixation on a central dot throughout each task block, they made saccades approximately once per second in both the interchange interval and contrast change/evidence periods. The majority of saccades fell along the horizontal meridian, had a mean duration of ∼30 ms, and fell within 1° of visual angle from fixation, much less than the distance from fixation to the inner edge of the annulus (radius of 3° of visual angle, Fig. 1C,D). Consistent with previous work, we defined microsaccades as saccades of <1° in magnitude, thus excluding 12% of all saccades that were larger. As a check on the reliability of our microsaccade detection method, we plotted microsaccade velocity against magnitude and, as expected, there was a strong correlation between these two properties, known as the main sequence (Fig. 1C; Zuber et al., 1965).
In agreement with previous studies (Herrington et al., 2009; Hafed and Krauzlis, 2010), we observed a significantly higher rate of microsaccades in a 200 ms window centered on evidence onset in trials with slower RT (fast vs slow RT bin ANOVA: F(1,16) = 17.68, p = 0.0007, Generalized Eta Squared (g.e.s.) = 0.036; Fig. 2A). However, we also found that the positive relationship between microsaccade rate and RT persisted long after evidence onset, an effect that survived correction for multiple comparisons across the time series (Cheadle et al., 2014; Loughnane et al., 2016) (600–800 ms, fast vs slow RT ANOVA: F(1,16) = 48.02, p = 3.4e-06, g.e.s. = 0.34). During this period, there was no change in saccade magnitude (baseline vs 600–800 ms time frame comparison: t(11) = 0.76, p = 0.46), whereas large saccades (>1°) were also inhibited in a similar fashion to microsaccades (baseline vs 600–800 ms timeframe comparison: t(16) = 2.18, p = 0.045). Therefore, the microsaccade rate reduction could not be attributed to participants being more likely to make larger eye movements during the decision period. Furthermore, this effect was unchanged if RTs <800 ms were excluded from the analysis. Previous studies using discrete stimulus presentations have consistently reported a steep reduction in microsaccade production at 100–200 ms after stimulus, followed by a recovery to baseline rates at ∼300–400 ms (Engbert and Kliegl, 2003; Rolfs et al., 2008; Martinez-Conde et al., 2009; Rolfs, 2009). Here, despite the elimination of sudden intensity transients, we found that the gradual contrast changes evoked a rate reduction to ∼40% of its baseline level (Fig. 2A), but one with an onset that was substantially delayed and duration prolonged compared with previous observations. Two aspects of our data indicate that the timing of this microsaccadic inhibition is coextensive with that of the decision-making process. First, a single-trial surface plot of microsaccadic rate sorted by RT (Fig. 2A) revealed that, rather than being aligned with the onset of the physical evidence, microsaccadic inhibition was time locked to the perceptual report on this task, reaching a plateau several hundred milliseconds before response. This was then confirmed statistically by testing that the cross-trial variance of the timing of the minimum microsaccadic rate was larger relative to evidence onset than relative to RT (evidence-locked SD: 324 ms vs response-locked SD: 233 ms, Brown–Forsythe value = 6.1, p = 0.015). Second, comparison of the time courses and single-trial surface plots in Figure 2 indicates that microsaccadic inhibition and the two neural decision signals, the motor-independent CPP (Fig. 2B) and effector-selective LHB activity (Fig. 2C), were closely aligned in terms of onset and duration. As discussed below, the observation that the time frame of postevidence microsaccadic inhibition is coextensive with that of the decision-making process can account for many experimental findings pertaining to this phenomenon.
A, Left, Evidence-locked microsaccade rate for fast (952 ms) and slow (1293 ms) RT bins. For all signal time series, shaded areas represent 95% confidence interval of within-subjects SE. Dashed vertical lines all mark CPP onset for each RT bin in plots A, B, and C. Middle, Response-locked microsaccade rate. Right, Surface plot of microsaccade rate sorted along the y-axis by RT z-scored inside participant, interchange interval duration, and contrast change direction to facilitate the pooling of data across participants and exclude the influence of the aforementioned experimental factors. Overlaid black line represents regular RT sorted by z-scored RT. Both signal and RT were smoothed using a Boxcar filter of 100 trials. B, Left, CPP for fast and slow RT bins. Similarly to microsaccadic inhibition, the CPP onsets at ∼370 ms, with faster buildup rate before earlier RTs (F(1,16) = 16.98, p = 0.0008, g.e.s. = 0.22). Right, Consistent with its characterization as a decision variable, the CPP reached peak amplitude at manual response. C, Left, LHB power for fast and slow RT bins. LHB power began to decrease at ∼560 ms and this decrease was stronger for faster RTs (F(1,16) = 4.5, p = 0.049, g.e.s. = 0.01). Again, to be expected for a signal related to motor preparation, the negative peak in power was aligned with manual response.
Thus far, we have shown that microsaccades affect perceptual detection times negatively and are inhibited during decision formation. We next sought to examine the impact of microsaccades on the associated neural processes. First, we compared electrophysiological data for trials containing no microsaccades in the early phase of the decision period (400–800 ms) versus trials in which one microsaccade was executed within this time window. To avoid cases in which the decision process may have been completed before microsaccade execution, we excluded trials with RT <800 ms (∼10% of trials) from these analyses. For the trials containing microsaccades, we simply extracted epochs for the CPP and LHB signals aligned to microsaccade onset. To compare these trials to trials without microsaccades, we created “sham” microsaccade onsets by randomly shuffling the real onset timings from trials with microsaccades and applying these to the neural data from trials without microsaccades. Comparing the two waveforms revealed highly similar CPP and LHB buildup rates in the immediate premicrosaccadic interval, but a significant transient interruption in their buildup during the first 100 ms after microsaccade onset. Although the signals quickly recovered to a rate-of-rise comparable to that seen on trials without microsaccades, overall, the effects of this early perturbation on signal amplitude persisted throughout the postmicrosaccadic epoch (real vs sham CPP 0–300 ms postmicrosaccade amplitude: F(1,14) = 43.72, p = 1.17e-05, g.e.s. = 0.22; real vs sham LHB 0 to 300 ms postmicrosaccade amplitude: F(1,14) = 8.95, p = 0.009, g.e.s. = 0.18; Fig. 3A). Furthermore, for the LHB analysis, we performed a time–frequency analysis to show that the difference in power between real and sham conditions was largely found in the beta range (Fig. 3B). Mediation analyses confirmed that variations in the amplitudes of the CPP and LHB fully accounted for the effect of microsaccades on RT (Fig. 3B,C).
A, Left, CPP waveform locked to real and sham microsaccade onset times. Microsaccades evoked a strong VEP component (see Fig. 4A) and therefore to rule out any contribution to CPP measurements, trials were also time locked to real and sham microsaccades occurring within the interchange interval and this activity was then subtracted from posttarget microsaccade-related activity. Green markers represent time points at which the buildup rate of the CPP differed significantly between real and sham conditions at p < 0.01. Middle, Topography of real minus sham ERP showing a relatively negative perturbation of the ERP focused on central parietal electrodes. Right, Waveforms at occipital electrodes after subtraction of activity evoked by microsaccades in the interchange interval confirm that the subtraction succeeded in eliminating the microsaccadic VEP. B, Left, Microsaccade-locked LHB for real versus sham conditions. Green lines represent the point at which LHB slope significantly differed between real and sham conditions at p < 0.01. Middle, Topography of real minus sham postmicrosaccade beta power showing a relative left hemisphere perturbation in beta power. Right, Time–frequency analysis locked to microsaccade onset showing a perturbation of power in the beta range (18–30 Hz). C, Left, Mediation parameters showing that the effect of microsaccade presence on RT was fully mediated by CPP amplitude. Left inset, Topoplot of the mediation coefficient ab showing mediation of microsaccade on RT is greatest at central parietal electrodes Right, Same plots for LHB.
We further confirmed that the CPP perturbation was not attributable to volume conduction of the microsaccadic VEP (Dimigen et al., 2009) by subtracting activity evoked by microsaccades during the interchange interval (Figs. 3A, 4A). Also, to fully exclude the possibility that these buildup rate differences arise from diminished attention levels on trials with microsaccades, the same analysis was repeated except that the “sham” waveforms were generated by randomly shuffling the microsaccade onset times and applying these timings to trials on which a microsaccade had occurred. The sham condition in this case contained randomly timed microsaccades that would cause a temporally blurred signal disruption, but this analysis nevertheless yielded consistent results (see Fig. 4B and Fig. 7C for a similar analysis in Experiment 2). Together, these results indicate that microsaccades cause a brief interruption in evidence accumulation and motor preparation and, consequently, delayed perceptual reports.
A, Left, CPP waveform locked to microsaccade onset compared with a sham condition in the CPP was aligned to sham microsaccade timings in trials where there was no microsaccade. These plots are the same as those shown in Figure 3A except that, here, the saccade-locked waveforms derived from microsaccades occurring during the interchange interval have not been subtracted out. Middle, Topographies for first 100 ms after microsaccade onset, then from 100–200 ms. Right, Microsaccadic VEP compared with sham condition. B, CPP waveform locked to microsaccade onset compared with a sham condition in which the CPP was aligned to sham microsaccade timings in the same trials, only shuffled. This is a highly conservative test of the CPP perturbation given that many of the shuffled microsaccade timings in the sham condition would be close to the real microsaccade timings. Nevertheless, the postmicrosaccade CPP was of lower amplitude compared with the sham condition (t(14) = 1.4, p = 0.17). See Figure 7C for similar analysis in Experiment 2.
We also examined whether microsaccades affected the low-level encoding of evidence in sensory cortex, which we could probe via the contrast-dependent SSVEP. Consistent with the hypothesis that the signal reflects the input to the decision process, SSVEP amplitude reliably tracked the contrast changes and predicted the timing of perceptual reports in a manner that depended on the contrast change direction (positive relationship with RT in the contrast increase condition, negative relationship for contrast decreases; Fig. 5A; RT bin × contrast change direction interaction: F(1,16) = 7.26, p = 0.016, g.e.s. = 0.29). Analysis of the SSVEP time locked to microsaccade onset revealed that it too was momentarily suppressed at ∼100 ms after onset, followed by a rebound in power above baseline at ∼230 ms (Fig. 5B). We performed a time–frequency analysis to show that the difference in power between real and sham conditions was localized to the SSVEP frequency, 25 Hz (Fig. 5B). The same effects were present in both the contrast increase and contrast decrease conditions and significantly mediated the relationship between microsaccades and RT (Fig. 5C).
A, Left, Evidence-locked SSVEP for fast and slow RT bins. Middle, SSVEP aligned with manual response. Right, Image of SSVEP sorted along the y-axis by RT separately for contrast increase and decrease trials. B, Left, Microsaccade-locked SSVEP power for real versus sham conditions. Middle, Topographies of SSVEP power at 50–120 ms and 200–300 ms postmicrosaccade. Right, Time–frequency analysis locked to microsaccade onset showing a perturbation of power at the SSVEP frequency (25 Hz). C, Left, Mediation parameters showing that the effect of microsaccade presence on RT was fully mediated by SSVEP amplitude. Right, Topoplot of the mediation coefficient ab showing mediation of microsaccade on RT is greatest at SSVEP electrodes.
To verify that the above results were not unique to our contrast change detection paradigm, we analyzed a preexisting dataset from separate experiments in which participants performed a variant of the random dot motion task (Fig. 1B). In this experiment, participants were only required to detect coherent motion that they indicated with bimanual button clicks (the experiment was originally designed to examine visuospatial asymmetries and a manuscript on this topic is in preparation). All of the key results described above were recapitulated (Figs. 6, 7): There was a positive association between microsaccadic rate and RT throughout the decision period (fast vs slow RT bin microsaccade rate, F(1,42) = 21.6, p = 3.3e-05, g.e.s. = 0.09) and the increased task difficulty also allowed us to observe a negative relationship with target detection rate, with very little microsaccadic inhibition for misses (hit vs miss microsaccade rate, F(1,36) = 30.5, p = 3e-06, g.e.s. = 0.1). No significant differences were observed when comparing RT for trials with microsaccades toward versus away from the target hemifield (t(19) = 0.73, p = 0.48; see Discussion). As shown in Figure 6, microsaccadic rate dynamics again closely mirrored those of the CPP, further highlighting how closely tied microsaccadic inhibition is to the timing and duration of the evidence accumulation process and the effect of microsaccades on CPP trajectory again accounted for the subsequent RT slowing (Fig. 7). Finally, the stepped transition from incoherent to coherent motion in this task allowed us to check the relative timing of microsaccadic inhibition and early target selection mechanisms. Loughnane et al. (2016) recently characterized a lateral øccipital target selection (N2) signal that is temporally aligned to goal-relevant stimulus feature changes and is enhanced over the hemisphere contralateral to the stimulus location (Fig. 1B). The N2 precedes the onset of neural evidence accumulation and has been shown to influence RT via modulation of the onset time and rate of evidence accumulation. The present data show that microsaccadic inhibition is aligned with the onset of this signal and persists until the completion of the decision process.
Time course of microsaccade rate reduction, low-level target detection and evidence accumulation in the random dot motion task. A, Left, Evidence-locked microsaccade rate for fast and slow RT bins. Yellow line represents microsaccade rate for misses. Middle, Response-locked microsaccade rate. Right, Image of microsaccade rate sorted along the y-axis by RT. B, N2c for fast and slow RT bins. Note the accompanying topoplot has collapsed contralateral hemisphere for both left and right hemifield targets on the right hemisphere with ipsilateral response on the left. C, CPP for fast and slow RT bins. Dashed lines represent CPP onset for each bin and are overlaid on microsaccade rate and N2 in A and B.
A, To eliminate the effect of the microsaccadic VEP, interchange interval microsaccade-locked waveforms were subtracted from both real and sham conditions. Left, Microsaccade-locked CPP compared with sham after subtracting the interchange interval waveform showing the CPP perturbation remains after subtracting away the VEP (right). Middle, Topography of real minus sham microsaccade-locked ERP showing the central parietal perturbation. B, Left, Mediation parameters showing that the effect of microsaccade presence on RT was fully mediated by CPP amplitude. Right, Topoplot of the mediation coefficient ab showing mediation of microsaccade on RT is greatest at central parietal electrodes. C, CPP waveform locked to microsaccade onset compared with a sham condition in which the CPP was aligned to sham microsaccade timings in the same trials, only shuffled (see also Fig. 4B). Again, although a highly conservative test of the CPP perturbation, the postmicrosaccade CPP was of lower amplitude compared with the sham condition (t(19) = 1.9, p = 0.07).
Discussion
Although there has been increasing recognition that microsaccades are an important source of variability in neural activity and perceptual performance in tasks that require the processing of transient visual events, the precise nature of this influence has been uncertain (Martinez-Conde et al., 2013; Hafed et al., 2015). Until now, most research on microsaccades has focused on briefly presented stimuli, with a particular emphasis on the impact of microsaccades that coincide with stimulus onset. The behavioral and neural impact of microsaccades occurring while in the process of accumulating decision evidence from a continuously presented stimulus had not been examined previously. The foregoing results reveal that, under such circumstances, microsaccades cause a brief suppression of activity at key task-relevant stages of the sensorimotor hierarchy that significantly delays perceptual reports and this detrimental influence persists throughout the period of decision formation. Therefore, our results highlight that a portion of the cross-trial variability in decision-making performance, typically captured by random variability parameters in current mathematical models, can be explained as a consequence of the effect of microsaccades on decision-relevant neural activity. This is a particularly important consideration for research that involves experimental manipulations or group comparisons that may be associated with differences in the rate of microsaccade production (e.g., in ADHD, Fried et al., 2014; and progressive supranuclear palsy, Otero-Millan et al., 2011). Our study also demonstrates that it is possible to measure the impact of microsaccades on neural activity at distinct levels of the sensorimotor hierarchy in the human brain with high temporal precision.
Using noninvasive EEG, we were able to trace the neural consequences of microsaccades at each of the key stages of decision making: contrast representation in early visual regions, evidence accumulation, and preparation of the decision-reporting action. SSVEP power was suppressed immediately after microsaccades, with a rebound above baseline at ∼150 ms; a pattern and time course that bears a striking resemblance to the microsaccadic response in V1 (Leopold and Logothetis, 1998). The negative effects of microsaccades on CPP and LHB buildup rate were also short lived, resolving within 100 ms, but this brief delay led to reduced signal amplitudes and slower perceptual reports long after the microsaccade had been completed. These observations are consistent with data from single-unit recording studies demonstrating that microsaccades that coincide with the onset of a visual stimulus elicit a transient suppression of spiking activity in areas that have been associated with decision making, including LIP and superior colliculus (Herrington et al., 2009; Hafed and Krauzlis, 2010; Chen et al., 2015). Here, however, we were able to trace directly the impact of microsaccades on the neural decision process during its evolution. Given that the SSVEP indexes the sensory input on which the contrast change detections were based, it may be that its suppression can fully account for the disruption of evidence accumulation and motor preparation signals. However, it is equally possible that these simultaneous perturbations are driven by central corollary discharge signals that are sent throughout the visual system to facilitate coordinated adjustments to retinal motion (Melloni et al., 2009). The absence of any direction-dependent effect of microsaccades would suggest the latter possibility.
At the outset, it was not certain how microsaccades might influence performance on the tasks used in these experiments given the variety of both behavioral and neural effects of microsaccades in the literature (Martinez-Conde et al., 2013). Here, however, we found that, regardless of their timing, microsaccades had a uniformly detrimental impact on perceptual performance and associated neural signals. Previous work has demonstrated that microsaccadic effects on neural signals can be both excitatory (Martinez-Conde et al., 2000; Snodderly et al., 2001; Dimigen et al., 2009; Tse et al., 2010) and inhibitory (Leopold and Logothetis, 1998; Bosman et al., 2009; Herrington et al., 2009; e.g., Hafed and Krauzlis, 2010) depending on the stimulus location with respect to microsaccade direction and depending on the neural populations and signals that are considered. Taking into account these methodological considerations is critical when comparing our findings with those of previous studies. For example, recent work has suggested a brief potentiation of visual and visuomotor responses in superior colliculus to peripheral stimuli presented immediately before a microsaccade (Chen et al., 2015), so a corresponding transient increase in decision signal buildup might have been predicted in our study. However, the premicrosaccadic enhancement reported by Chen et al. (2015) was direction dependent such that neurons with receptive fields in the direction opposite to the microsaccade exhibited either weaker enhancement (in the case of visually responsive neurons) or a suppression of activity (in the case of visuomotor neurons). Given that the stimulus used in our first experiment (a checkerboard annulus) isometrically surrounded the entire area of fixation, it is likely that these direction-dependent enhancement and suppression effects cancel out in our population-level recordings. In addition, the absence of any periodic enhancement effects in our second experiment may be explained by the fact that the bilateral dot patches were presented 10° either side of fixation and the premicrosaccadic enhancement effects of Chen et al. (2015) were much less pronounced for neurons with response fields of further than 7° eccentricity. Findings of direction-dependent effects of microsaccades in a Posner cueing task have also been reported in humans (Tian, Yoshida, and Hafed, 2016). Again, the stimuli in that study were closer to fixation, at 5° eccentricity, making it difficult to compare them with the present results.
It is clear that the antagonistic relationship between microsaccades and evidence accumulation observed in the present data cannot be assumed to generalize to all decision-making scenarios. It is well established, for example, that microsaccades serve to offset visual fading (Martinez-Conde et al., 2006; Martinez-Conde et al., 2013) and therefore it is likely possible to devise perceptual decision-making scenarios in which microsaccades could be beneficial for performance. An important challenge for future research will be to identify the parameters that determine whether microsaccades will be beneficial or detrimental to perception. For example, future work should investigate whether stimulus flicker in the contrast change task and stochastic motion in the random dot motion task serve to counteract perceptual fading effects exogenously and, consequently, negate potential longer-term benefits of microsaccades.
Our data also provide important new insights on the phenomenon of microsaccadic inhibition. It is apparent that stimulus-evoked microsaccade rate reductions are associated with several events in the brain. The most common is after the abrupt onset of a visual stimulus (Rolfs et al., 2008; Martinez-Conde et al., 2009; Rolfs, 2009). The relatively fixed timing of this stimulus-evoked inhibition (∼100–200 ms) across studies using a wide variety of stimuli has led to the view that it likely reflects a stereotyped, low-level response to a change in sensory input (Rolfs, 2009). Both experimental and modeling studies have demonstrated that microsaccadic rate inhibition can arise from endogenous mechanisms such as the anticipation of a goal-relevant stimulus (Hafed and Krauzlis, 2010; Pastukhov and Braun, 2010; Hafed et al., 2011; Engbert, 2012, Sinn and Engbert, 2016; Tian et al., 2016). The present study highlights a new source of endogenous influence on microsaccadic rate. Our tasks eliminated sudden stimulus onsets and the onset of the feature change targets was unpredictable. However, the feature changes produced a persistent rate inhibition that coincided with the period of decision formation and onset ∼600 ms before participants reported being aware of the change. The absence of any significant rate modulation in response to missed stimulus feature changes in Experiment 2 (Fig. 6A) suggests that it was not strongly dependent on bottom-up visual input, but is endogenously driven.
Using a target detection task with subtle stimulus transitions, White and Rolfs (2016) also showed that microsaccadic inhibition was absent on miss trials. The authors accounted for their results by suggesting that saccades are inhibited when a new visual stimulus reaches conscious awareness. Our study changes this interpretation slightly by showing that, in the absence of an abrupt stimulus onset, saccades are silenced during the decision-making process, up until around the decision threshold. Therefore, to the extent that the decision threshold can be taken as the moment of the “piercing of consciousness” (Kang et al., 2017), our results suggest that microsaccadic inhibition occurs before the sensory event reaches conscious awareness.
Our results are also consistent with previous empirical observations of longer-lasting microsaccadic inhibition after task-relevant and low-probability stimuli and before slow RTs (Betta and Turatto, 2006; Valsecchi et al., 2007, 2009). The authors of those studies proposed several candidate processes underlying this effect of microsaccadic inhibition, including stimulus evaluation, context-updating, and memory-related processes. Here, we found that the timing and duration of microsaccadic rate reduction closely corresponded to those of the neural decision signals in both tasks suggesting that microsaccades are inhibited actively to specifically facilitate the sampling and accumulation of sensory information, thus providing a common framework to explain these disparate phenomena.
Our results reveal close antagonistic interactions between microsaccades and decision-making processes and highlight methods for precisely measuring the effect of microsaccades on processing at the key task-relevant stages of the sensorimotor hierarchy noninvasively in humans.
Footnotes
This work was supported by the National Science Foundation (Grant BCS-1358955 to S.P.K. and R.G.O.), the European Research Council (Grant 63829 to R.G.O.), and the International Research Staff Exchange Scheme (Grant 612681 of the EU 7th Framework Programme to R.G.O.). We thank Ana Carina Pamplona, Rafael Abe, and Marco Tulio Ramalho Zoratti for help with data collection.
The authors declare no competing financial interests.
- Correspondence should be addressed to Gerard Loughnane, Trinity College Institute of Neuroscience and School of Psychology, Trinity College Dublin, Dublin 2, Ireland. loughnag{at}tcd.ie
