Abstract
It is well known that the motor and the sensory systems structure sensory data collection and cooperate to achieve an efficient integration and exchange of information. Increasing evidence suggests that both motor and sensory functions are regulated by rhythmic processes reflecting alternating states of neuronal excitability, and these may be involved in mediating sensory-motor interactions. Here we show an oscillatory fluctuation in early visual processing time locked with the execution of voluntary action, and, crucially, even for visual stimuli irrelevant to the motor task. Human participants were asked to perform a reaching movement toward a display and judge the orientation of a Gabor patch, near contrast threshold, briefly presented at random times before and during the reaching movement. When the data are temporally aligned to the onset of movement, visual contrast sensitivity oscillates with periodicity within the theta band. Importantly, the oscillations emerge during the motor planning stage, ∼500 ms before movement onset. We suggest that brain oscillatory dynamics may mediate an automatic coupling between early motor planning and early visual processing, possibly instrumental in linking and closing up the visual-motor control loop.
Introduction
The “motor” system is no longer considered a mere executor of output commands under the guide of sensory evidence, but rather an active orchestrator of the sensory analysis. The motor system (activated by both hand and eye movements) dynamically modulates the incoming sensory flow, contributing to shaping the perceptual outcome by resolving perceptual conflicts (Wohlschläger, 2000; Maruya et al., 2007), changing perceived time (Haggard et al., 2002; Morrone et al., 2005; Hagura et al., 2012; Tomassini et al., 2014), synchronizing temporal fluctuations of attention (Morillon et al., 2014), enhancing the sensitivity to object orientation changes during a grasp action (Gutteling et al., 2011), and increasing the apparent visual contrast at the movement goal location (Rolfs et al., 2013). In most of the studies just cited, the perceptual modulations occurring at the times of movement might have been mediated by attentional and decisional mechanisms since the incoming sensory information was related to the motor task (Gutteling et al., 2011; Rolfs et al., 2013; Morillon et al., 2014). At present there is no compelling evidence of a coupling between action performance and sensory sensitivity. However, to exert a precise temporal tuning of incoming sensory information, it would be beneficial if the motor and the sensory signals were coupled at very early cortical stages. To address these important questions, the present study measures the contrast sensitivity of brief visual stimuli unrelated to the instructed action, at different times during motor planning and execution. Contrast thresholds reflect the activity of the primary visual cortex, and a change of visibility would require a modulation at this cortical level (Boynton et al., 1999).
Increasing evidence shows that both sensory and motor functions are regulated by underlying rhythmic processes that reflect alternating states of neuronal excitability (Buzsáki and Draguhn, 2004; Thut et al., 2012). Correlations between trial-by-trial neural activity and behavioral outcome have shown that visual detection is either boosted or suppressed depending on the phase of the oscillatory cycle at the time of stimulus presentation (Busch et al., 2009; Dugué et al., 2011). A similar phase dependency has been demonstrated for motor responses, with faster and slower reaction times being consistently associated with specific phases of neuronal oscillations (Drewes and VanRullen, 2011). Motor evoked potentials induced by transcranial magnetic stimulation (TMS) are also modulated by the oscillatory phase and by the strength of corticomuscular coherence, suggesting that neuronal oscillations may entail a rhythmic modulation of motor excitability (van Elswijk et al., 2010; Schulz et al., 2014).
Previous studies have shown that ongoing brain dynamics can be manipulated by external stimuli inducing rhythmicity in the temporal pattern of behavioral performance. Attentional manipulations (Landau and Fries, 2012; Fiebelkorn et al., 2013) and cross-modal stimuli (Fiebelkorn et al., 2011; Romei et al., 2012) can synchronize the phase of the ongoing activity in visual areas, generating rhythmic fluctuations in visual performance.
Here we asked whether rhythmic oscillations of visual contrast sensitivity can be observed also by synchronizing the performance with the onset of a reaching and grasping action. The planning of an action develops differently over time if the action is sensory driven (reactive action) or self-paced (Jahanshahi et al., 1995; Cunnington et al., 2002; Shibasaki and Hallett, 2006). The presence of similar oscillations for the two motor tasks and before the actual onset of movement would suggest an efficient coupling between early motor planning and early visual processing.
Materials and Methods
Stimuli and procedure.
Participants sat in a dark room in front of an LCD monitor (60 Hz) at a viewing distance of 57 cm. Their motor task consisted of reaching and grasping a bar on the right side of the screen with the right hand (Fig. 1a). To understand the influence of different movement planning dynamics, we tested two different movement conditions: the action was self-initiated (Experiment 1) or triggered by an auditory tone (Experiment 2; see below). The bar and the participant's right arm were hidden from view.
The timeline of the events during each trial is illustrated in Figure 1c. The trial started with the display of dynamic visual noise and fixation point on the screen (trial onset). Participants were required to fixate a black square (0.4 × 0.4°) at the center of the screen (mean luminance, 54 cd/m2; 48 × 27° of visual field) and dynamic white visual noise (RMS contrast equal to 0.11) refreshed every second frame (16.5 ms) was displayed throughout the trial (3 s; Fig. 1b,c). At random times in the interval between ∼0.4 and ∼1.8 s from trial onset, the visual stimulus was displayed (Fig. 1c). The stimulus was a Gabor patch (spatial frequency, 1 c/deg) that was briefly presented (33 ms; two frames) randomly to the lower left or lower right of fixation with equal probability (eccentricity of 7.5° along both the horizontal and vertical orientations). Figure 1b shows a series of snapshots of the display; the third shot shows the Gabor patch. Gabor presentation times were varied randomly on a trial-by-trial basis with a resolution of ∼16 ms (one frame). Particular care was taken that for each block of trials (usually 60) a consistent number of Gabor stimuli occurred before and after movement onset, so that participants could not stereotype their movements depending on the Gabor presentation. The Gabor contrast was set around threshold; thus, participants could perceive the presence of the stimuli in ∼30% of trials on average. Given the uncertainty of the presentation time, location, and detectability of the Gabor and the fact that it (including its position) was irrelevant for the motor task, participants learned in few trials not to perform the movement in relation to the Gabor appearance.
The Gabor patch (i.e., the target stimulus) was tilted at ±45°, and participants reported its orientation with a verbal response at the end of the trial. An auditory feedback (1000 Hz; 20 ms auditory tone) was delivered in the case of an incorrect response.
A photodiode (2.3 × 2.3 cm) placed on the top left corner of the monitor was used to record the accurate timing of the visual stimulations (visual noise and Gabor). A white square (2 × 2 cm) was displayed on the screen in the position of the photodiode (hidden from view) in synchrony with the onset of the visual noise (trial onset) and again with the onset of the Gabor (Fig. 1b, top left corner of snapshots). The photodiode signals were recorded by a National Instruments data acquisition device (sampling rate, 500 Hz) providing the times of both trial onset and stimulus appearance.
Self-initiated movement: Experiment 1.
Participants performed self-initiated reaching movements with their right arm to grasp a vertical bar mounted close to the right side of the display (Fig. 1a). Subjects were free to initiate the movement at will within a 2 s interval that started ∼0.5 s and ended ∼2.5 s from trial onset, i.e., from dynamic visual noise display (Fig. 1c, bottom row). Trials were aborted (with feedback to the subjects) and successively repeated in cases of movement initiation time that were too early (less than ∼0.5 s) or too late (more than ∼2.5 s). This penalty also runs against the possible, unwanted, subject's strategy of using the Gabor stimulus presentation as a movement go signal. The movements were hidden from view by black cardboard fixated to the right side of the screen (Fig. 1a). Each trial started with the participant keeping their right hand resting on two small buttons (diameter, 0.3 cm) covered by a pasteboard representing the hand resting position. The buttons were connected to the same National Instruments data acquisition device (sampling rate, 500 Hz) used to record the photodiode signal. The time of movement onset was derived from the release of the buttons associated with the hand lifting.
A preliminary testing phase (∼30 trials) allowed participants to become familiar with the task and provided an indication of individual performance levels. Stimulus contrasts were initially set around the values yielding ∼75% of correct responses during the familiarization phase. The performance was constantly monitored by the experimenter throughout the sessions; due to the presence of learning effects, the initial stimulus contrasts were slightly changed to keep the performance level always near threshold.
Four naive participants (three females; mean age, 25 ± 1.8 years) took part in the experiment (all of them provided written informed consent).
Data were collected in separate sessions performed on different testing days. On average, 24 sessions of 60 trials each were run for each participant.
Externally triggered movement: Experiment 2.
An auditory cue instructed participants to perform reaching movements of the same type as described for the self-initiated movement condition. The sound go signal was presented at ∼560 ms (jittered by ∼160 ms) after the start of the trial. Like in the self-initiated movement condition, the Gabor was presented at random times with respect to trial onset (from ∼0.4 to ∼1.8 s). Stimulus presentation times were slightly adjusted during the experiment on the basis of the individual reaction times to optimize sampling within the desired temporal window (from ∼0.4 s before to 0.4 s after movement onset). The high predictability of movement onset time in the externally triggered condition allowed us to sample stimulus latency more effectively and collect more data by varying stimulus contrasts to calculate psychophysical contrast thresholds. Stimulus contrast varied on a trial-by-trial basis between 5 and 25%.
Two naive participants were tested in the externally triggered movement condition (one of them also took part in the self-initiated movement condition). Each participant performed 27 sessions of 60 trials each.
Data analysis.
Data were analyzed off-line. Trials were first temporally sorted on the basis of stimulus presentation time with respect to either movement onset time or trial onset time (i.e., visual noise onset). Data were then grouped in different time bins (bin size, 80 ms for individual data, 50 ms for pooled data), and for each bin the percentage of correct responses was calculated. We applied a sliding window stepped by 10 ms. Time bins including <18 and <60 trials for the individual and pooled data, respectively, were not considered in the analysis. Bin size was determined by the need to have a sufficient number of trials within each bin to provide reliable probability estimates. On average, probabilities were calculated on 32 ± 0.2 and 246 ± 29 trials per bin for the individual and pooled data, respectively. It should be noted that grouping the data within bins of 80 and 50 ms corresponds to sampling and holding the data at 12.5 and 20 Hz, which are the Nyquist frequencies of the individual and pooled analyses, respectively.
Spectral analysis of the performance was conducted using the fast Fourier transform (FFT). The behavioral time series (percentage of correct responses over time) were zero-padded (to increase frequency sampling) and then fast Fourier transformed. A nonparametric permutation test was used to determine the statistical significance of the oscillatory pattern in the visual performance. For each data set, we generated a surrogate spectral distribution by iteratively (1000 times) randomizing stimulus presentation times. Each iteration of the randomization procedure yielded a surrogate data set that was submitted to the same analysis performed on the real data set (binning, zero padding, and fast Fourier transform analysis), producing a distribution of frequency spectra (under the null hypothesis that stimulus presentation time does not have any effect on visual performance). The amplitude measures derived from the FFT output of the observed visual performance were then compared at each frequency (from 2 to 12 Hz) with the reference distribution of amplitudes. The p value for the permutation test is yielded by the proportion of values of the reference distribution exceeding the amplitude in the original data set (one-tailed thresholds of p < 0.05; Bonferroni corrected for multiple comparisons across frequencies).
For the externally triggered movement condition, data were binned and then fitted separately with cumulative Gaussian functions. Contrast thresholds were derived from the mean of the psychometric functions (75% correct responses), and standard errors of the thresholds were estimated by bootstrap simulation. Psychometric functions were fitted only if bins contained at least 30 trials.
Results
We asked participants to report the orientation of low-contrast Gabors embedded within dynamic visual noise while keeping fixation and simultaneously performing self-initiated movements with their right arm to reach and grasp an invisible bar on the right side of the display (Fig. 1a). Participants were free to initiate the movement within a 2 s interval after trial onset, i.e., the display of the dynamic visual noise and fixation point. If the movement initiated too early or too late, a feedback was given to the subject and the trial was aborted, providing the subject with the instruction to perform an additional trial. The Gabor stimuli were visible in only ∼30% of the trials and were presented at an unpredictable time and spatial position (left or right hemifield).
Experimental setup and procedure. a, Illustration of the motor and visual tasks. The motor task was reaching and grasping with the right hand a vertical bar close to the right side of the display. Both the participant's right arm and the bar to be grasped were hidden from view by black cardboard (open loop condition). The illustration of the screen shows the background visual noise, the central fixation point, and a left hemifield Gabor stimulus. The white square in the upper left corner of the screen (that was actually hidden from view by the photodiode) was displayed simultaneously to the Gabor (and also to the dynamic visual noise) and recorded by the photodiode to yield the accurate time of stimulus presentation (and of trial onset). b, An example series of snapshots of the visual display. Visual noise and the fixation point were displayed throughout the trial. At a random time from the start of the trial (i.e., from the visual noise and fixation point onset), a Gabor patch was presented for ∼33 ms (two frames) either to the lower right or to the lower left of fixation. The third snapshot shows a right hemifield Gabor as an example. c, Schematic illustration of the timeline of the events during the trial for the self-initiated condition (Experiment 1). The dynamic visual noise and fixation point were displayed for the entire duration of the trial (3 s), as shown by the upper line. The Gabor patch was presented at a random time between ∼0.4 and ∼1.8 s from the start of the trial (middle line). Participants were allowed to initiate the movement at will in a 2 s interval between 0.5 and 2.5 s from the start of the trial. Trials were aborted if movements were executed too early (before 0.5 s from the start of the trial) or too late (after 2.5 s from the start of the trial), as indicated in the third line; trial abortion and successive repetition was signaled by auditory feedback.
The individual performances in the visual orientation discrimination task (percentage correct) aligned to movement onset time (zero time) for the right (red) and left (blue) hemifield stimuli are shown in Figure 2 (left columns). For all subjects, the performance varies rhythmically by ∼30% of correct discrimination, alternating high (80–90%) and chance level (60–50%) phases in a cyclical manner. The periodic fluctuation in visual performance is present long before movement onset (approximately −500 ms) and continues during the movement over the entire tested period (up to 200 ms after movement onset). The spectral analysis of the time courses in visual performance revealed significant peaks in the theta band (3.5–8 Hz) compared with the surrogate spectral distributions derived by randomly assigning stimulus presentation times in each data set (p < 0.05, after Bonferroni correction for multiple comparisons across the 2–12 Hz frequency range; Fig. 2, right columns), with only one exception (p = 0.06, Bonferroni corrected; Fig. 2, S4 left hemifield stimuli). The oscillation frequencies for all subjects and for both visual hemifields are almost confined to the theta band; the average frequency across subjects and stimulus positions ± SD is equal to 5.8 ± 1.9 Hz. Nevertheless, the exact peak frequency changes for each participant and visual hemifield. Indeed, a pronounced difference (∼3 Hz) in the oscillation frequency between the right and left visual hemifields is observed for three of four subjects. It should be noted that for methodological reasons, we cannot exclude the presence of higher-frequency components exceeding the low alpha frequency band. The data binning procedure (sample and hold) required by the psychophysical approach causes a progressive attenuation of the frequency components up to 12.5 Hz. This may have resulted in a systematic underestimation of the alpha-band oscillatory components (see Materials and Methods).
Time courses of visual performance in the orientation discrimination task (in percentage of correct responses) temporally aligned with respect to movement onset (zero time by definition; left column graphs) and relative spectral profiles (right column graphs) are shown for the right (red) and left (blue) hemifield stimuli in the self-initiated movement condition (results for all participants). The nonparametric permutation test yields significant peaks in the theta band (3.5–8 Hz; p < 0.05, Bonferroni corrected for multiple comparisons across the 2–12 Hz frequency range), except for S4 left hemifield (p = 0.06, Bonferroni corrected), compared with the surrogate spectral distributions derived by randomly assigning stimulus presentation times in each individual data set (means and 95% confidence intervals indicated by solid and dashed gray lines, respectively). *p < 0.05, **p < 0.01, ***p < 0.001.
Each trial started by displaying dynamic visual noise. This abrupt visual stimulation at the beginning of the trial could have exerted a resetting effect on the ongoing visual activity; though this possible effect should be minimized in our paradigm since the Gabor stimuli were presented quite far away from the start of the trial (∼1000 on average, and never before 450 ms from trial onset). Additionally, subjects could have paced their movements on the basis of the visual noise appearance on the screen. To be sure that the oscillatory pattern observed represents a genuine motor synchrony, and is not a consequence of the noise visual stimulation or time-keeping processes, we also computed the visual performance by aligning the data with respect to the onset of the trial, i.e., visual noise. When the data are aligned to trial onset, visual performance shows no consistent periodicity (Fig. 3, left columns): the frequency spectra of the visual performances show no significant differences in amplitude compared with the surrogate spectral distributions generated by randomly shuffling stimulus latencies (Fig. 3, right columns).
Time courses of visual performance (in percentage of correct responses) aligned with respect to the visual noise display (i.e., trial onset) are shown for both the right (red) and left (blue) hemifield stimuli in the self-initiated movement condition (left graphs). Individual distributions of movement onset times are plotted at the bottom of each graph (bar histograms). Spectral profiles are reported for the right (red) and left (blue) hemifield stimuli, yielding in both cases no significant differences with respect to the surrogate distributions derived by randomly shuffling stimulus latencies separately for the right and left hemifield data sets (means and 95% confidence intervals indicated by solid and dashed gray lines, respectively; right graphs).
The participants could initiate the movement at their own pace (within a large time window of 2 s, from 0.5 to 2.5 s with respect to trial onset), and the movement onset times showed considerable intertrial variation, as shown by the frequency distributions reported at the bottom of the left column graphs in Figure 3 for each subject. The average movement onset times (±SD) ranged from 1.06 ± 0.19 to 1.28 ± 0.35 s across subjects. Despite the great individual variation in movement onset time, the rhythmic oscillations in performance were always present when aligning performance to movement onset (Fig. 2, time 0), and they always preceded the actual motor response by ∼500 ms.
The overall pattern of results is confirmed by analyzing the data pooled across subjects (n = 4) and stimulus positions (right and left visual hemifields). Visual performance temporally aligned with action execution oscillates in a rhythmic fashion before the onset of movement (Fig. 4, top left graph). The nonparametric permutation test yields a highly significant peak at 5.6 Hz (p < 0.0001, after Bonferroni correction for multiple comparisons across the 2–12 Hz frequency range; Fig. 4, top right graph). Remarkably, the oscillation in performance survives the great interindividual variation in movement onset times indicated by the wide frequency distribution of the bar plot of Figure 4 (movement onset time, 1.23 ± 0.3 s; mean ± SD), and, although the changes in performance are not large (∼12%), the periodic pattern is highly consistent in the pooled data (p < 0.0001, Bonferroni corrected).
Visual performance (percentage correct) calculated from the data pooled across subjects (n = 4) and stimulus positions (right and left hemifields) aligned with respect to movement onset (top left graph) and trial onset time (bottom left graph) in the self-initiated movement condition. Spectral analysis is reported in the right column graphs showing a significant peak at 5.6 Hz for the movement-locked performance (p < 0.0001, Bonferroni corrected; top right graph) and no significant oscillatory components for the performance aligned on the start of the trial (bottom right graph). The distribution of movement onset times pooled across subjects and conditions is shown by the bar plot in the bottom left graph. Movements initiated between 1500 and 2500 ms are collapsed in one single bar for illustrative reasons. ****p < 0.0001.
Contrary to the movement-locked performance, the pooled visual performance computed as a function of trial onset time does not show any significant oscillatory pattern (Fig. 4, bottom graph), reinforcing the motor-related nature of the observed perceptual modulations.
Given the great uncertainty of the Gabor stimulus due to its poor visibility, large variability in presentation times, and unpredictable location (on the left or right side), it is unlikely that subjects used it as a go signal for the reaching action. However, in principle, it is still possible. To ascertain that movements were actually self-initiated and not influenced by the Gabor presentation, we examined the individual frequency distributions of asynchrony intervals between Gabor presentation and movement onset (Fig. 5). For all participants, the distributions are rather uniform (black bars), whereas a reactive movement would have predicted Gaussian-like, not flat, distributions. Moreover, since the Gabor was often invisible, if participants systematically waited for stimulus appearance before beginning their movements, a great number of trials would have been aborted with the consequent penalty of trial repetition. Instead, the percentage of aborted trials for incorrect movement execution was virtually null, suggesting that participants did not time their motor responses based on stimulus appearance. Finally, there was a consistent number of trials in which movement started before the Gabor onset (Fig. 5, light gray bars), and the distribution of asynchrony intervals for these trials is similar to that for the intervals following the Gabor onset (black bars). Overall, we can exclude that the Gabor presentation had a relevant influence on the control of movement timing.
Individual frequency distributions of the temporal separation between movement onset and Gabor onset in the self-initiated movement condition. Negative values of Gabor-movement onset asynchrony indicate trials in which the Gabor patch was presented before movement onset (black bars), whereas positive asynchrony values indicate trials in which movement was initiated before the Gabor was presented (light gray bars).
The slow self-initiated movement onset times (on average, >1 s) and their rather large variability presumably reflect the long and variable movement preparation time. It is possible that in a reactive instead of a self-initiated movement condition, the perceptual oscillatory modulations may be different. We repeated the experiment by introducing a sound as a go signal, allowing us to strongly reduce movement onset variability. Two participants were instructed to move in response to the sound presentation, and the visual stimulus was randomly displayed within an ∼800 ms interval around the average individual reaction time (either shortly before or after sound presentation). The much higher predictability of movement onset time in the externally triggered movement condition made it possible to collect enough data by varying the contrast of the stimuli on a trial-by-trial basis and to calculate psychophysical contrast thresholds finely spaced over time.
The psychometric curves relative to three example time intervals are plotted in the bottom graphs of Figure 6 for both subjects and stimulus positions. The red and blue curves, corresponding to stimuli presented near an oscillatory peak, significantly deviate with respect to both other curves, corresponding to a preceding (black curve) and a following (gray curve) minimum of oscillatory performance, indicating lower contrast thresholds and higher visual sensitivity. Although there seems to be a trend of higher precision at the peaks of the oscillation, precision cannot be properly assessed since our paradigm did not allow us to fit psychometric functions with an equal number of data points across stimulus latencies.
Time course of visual performance (percentage of correct responses) for a subset of near-threshold contrast values aligned to movement onset time (zero time) is shown for the right and left hemifield stimuli in the externally triggered movement condition (top graphs, gray solid lines; results for the two tested subjects). The three colored stars indicate the visual contrast thresholds calculated at three abutting time intervals covering an entire oscillatory cycle. Peaks and troughs in the performance reflect relative low (contrast for the right hemifield stimuli, S4, 7.6 ± 0.35%; S5, 6.9 ± 0.5%, red stars; contrast for the left hemifield stimuli, S4, 7 ± 0.3%; S5, 7.2 ± 0.5%, blue stars; threshold ± SE) and high (contrast for the right hemifield stimuli, S4, 10 ± 0.9%; S5, 8.3 ± 0.67%, black stars; S4, 10.2 ± 0.9%; S5, 8.5 ± 0.9%, gray stars; contrast for the left hemifield stimuli, S4, 9 ± 0.7%; S5, 10.5 ± 1.3%, black stars; S4, 9.08 ± 1%; S5, 10.9 ± 1.6%, gray stars) contrast thresholds. The three corresponding psychometric functions (calculated in the same time intervals shown by the colored stars in the top graphs) indicating the proportion of trials where the Gabor orientation was judged correctly are plotted in the bottom graphs.
Figure 6 (top row) shows the time course of the orientation discrimination performance (percentage correct) calculated for a small subset (near threshold) of the overall presented stimulus contrasts as a function of movement onset time. The three colored stars superimposed to the oscillatory performance indicate the contrast thresholds derived from the psychometric functions shown in the bottom rows (Fig. 6). The relative low (contrast for the right hemifield stimuli, S4, 7.6 ± 0.35%; S5, 6.9 ± 0.5%; Fig. 6, red stars; contrast for the left hemifield stimuli, S4, 7 ± 0.3%; S5, 7.2 ± 0.5%; blue stars; threshold ± SE) and high (contrast for the right hemifield stimuli, black stars, S4, 10 ± 0.9%; S5, 8.3 ± 0.67%; gray stars, S4, 10.2 ± 0.9%; S5, 8.5 ± 0.9%; contrast for the left hemifield stimuli, black stars, S4, 9 ± 0.7%; S5, 10.5 ± 1.3%; gray stars, S4, 9.08 ± 1%; S5, 10.9 ± 1.6%) contrast thresholds reflect the corresponding peaks and troughs in the time course of performance.
Previous studies have shown that auditory stimuli can induce sound-locked oscillations in visual performance (Fiebelkorn et al., 2011). Similarly, sound has been shown to reset the phase of visual activity, producing oscillations in the rate of TMS-induced phosphenes (Romei et al., 2012). Thus, the results observed in the reactive movement condition might reflect sound locking.
Figure 7 shows the visual performance calculated on the data pooled across the two tested subjects and stimulus positions when the Gabor onset is measured with respect to movement onset (top left graph) and sound onset (bottom left graph) time. The time courses of the visual performance as well as the resulting frequency spectra are similar for the movement-locked and sound-locked data. Indeed, compared to the self-initiated movement condition, the distribution of reaction times relative to sound presentation is very narrow (Fig. 7, bar plot), indicating little temporal variability (RT, S4, 315 ± 59 ms; S5, 367 ± 88 ms; mean ± SD), as expected in a reactive task with respect to a self-paced task. Given the strong temporal correlation between sound presentation and movement onset time (r = 0.96, p < 0.0001; two-tailed Pearson correlation coefficient), which is a direct consequence of the reaction-time paradigm, a similar pattern in perceptual performance could actually survive in the sound-locked performance without being necessarily coupled to the auditory stimulation.
Visual performance (percentage correct) calculated from the data pooled across subjects (n = 2) and stimulus positions (right and left hemifields) aligned with respect to movement onset (top left graph) and sound onset time (bottom left graph) for the externally triggered movement condition. Spectral analysis is reported in the right column graphs showing a marginally significant peak at 4 Hz for the movement-locked performance (p = 0.06, Bonferroni corrected; top right graph) and no significant peaks for the performance aligned on sound presentation time (bottom right graph). The distribution of reaction times pooled across the two tested subjects is shown by the bar plot in the lower row (bottom left graph).
However, the main oscillatory peak (4 Hz) does not reach statistical significance for either the movement-locked (p = 0.06) or the sound-locked performance (p = 0.2). Since the spectral analysis and permutation test do not provide conclusive evidence, the relative contribution of sound and movement to the modulations of contrast thresholds observed in the externally triggered movement condition cannot be distinguished in our data.
Discussion
We found rhythmic oscillations in visual contrast sensitivity time locked to the execution of a voluntary reaching action. Three aspects appear particularly relevant. First, the oscillations emerge a long time before the action onset. Second, the action was not visually guided, all visual information being completely irrelevant for motor planning and execution. Third, oscillations were found for targets presented in both the left and right visual hemifields. The present results strongly suggest an automatic nonspecific coupling between early motor planning and early visual processing, possibly mediated by transient synchronization of motor and visual neuronal activities.
The observed rhythmicity in visual contrast sensitivity is the likely behavioral signature of oscillatory activity in visual areas within the theta band. Previous studies have established a firm relationship between the probability of perceiving a near-threshold visual stimulus and the prestimulus phase of ongoing cortical oscillations both in the alpha and theta frequency bands (Busch et al., 2009; Mathewson et al., 2009; Busch and VanRullen, 2010; Dugué et al., 2011). Interestingly, there is also strong evidence for early cross-modal phase modulations. Oscillatory activity in sensory cortices can be phase-reset by salient or attended stimuli in a different modality (Lakatos et al., 2009; Luo et al., 2010; Fiebelkorn et al., 2011; Romei et al., 2012), with functional consequences for perception. Phase resetting aligns brain activity across trials, leading to stimulus-locked rhythmicity in performance (Fiebelkorn et al., 2011, 2013; Landau and Fries, 2012; Romei et al., 2012). Oscillations within the theta band have been linked previously to perceptual functions (Busch and VanRullen, 2010) and also postulated to be specifically involved in sensorimotor integration functions (Bland and Oddie, 2001; Caplan et al., 2003; Rawle et al., 2012; Grent-'t-Jong et al., 2014).
Action and visual attention are tightly coupled, with visual attention selecting the feature or the portion of space relevant to the intended action (Allport, 1987; Rizzolatti et al., 1987); disentangling the two phenomena has thus proven to be difficult. Indeed, allocation of spatial and featural attention could account for the perceptual effects observed during action preparation and execution in many previous studies. For example, it may explain the shorter reaction times for congruent visuomotor information (Craighero et al., 1999), the reduced change blindness for grasp-congruent objects during movement planning (Symes et al., 2008), the perceptual enhancement of an object's visual feature relevant for the execution of a specific action (Gutteling et al., 2011), or the improved contrast sensitivity and increased apparent contrast for the reach target location relative to locations in the opposite visual hemifield (Rolfs et al., 2013). Furthermore, in the auditory domain, a rhythmic, repetitive motor act facilitates the perception of a stream of targets through attention enhancement (Morillon et al., 2014). It is possible that also in the present study it is attention allocation, rather than action preparation, that synchronizes perception. Although we cannot conclusively dissociate attention allocation from motor preparation, we think that three main reasons weaken the attentional explanation. First, attention allocation could not be finely tuned to time, as stimulus appearance was randomly varied within a large time interval (>1 s). Visual performance aligned to the start of the trial (Figs. 3, 4) did not show a consistent oscillatory pattern, indicating that temporal orienting of attention cannot have an important role in entraining perceptual oscillations (Cravo et al., 2013). Nevertheless, motor-driven attention might play a role here. Attentional orienting is known to be spatially biased according to the motor effector and movement target goal location (Baldauf et al., 2006; Eimer et al., 2006). If visual oscillations were tuned by spatially biased attentional orienting, we would have expected systematic variations between the right and left hemifield visual stimulation conditions, whereas we observed only nonsystematic differences in the oscillation frequency. Moreover, to be capable of efficiently synchronizing the perceptual rhythm, attentional allocation should be precisely locked to movement onset. This contrasts with the gradual shift of attention to the motor goal observed before saccades (Rolfs et al., 2011) and reaching actions (Jonikaitis and Deubel, 2011; but see Deubel and Schneider, 2005; Rolfs et al., 2013). It is thus more likely that the oscillations in visual performance are directly locked to action, rather than attentionally driven, suggesting a functional coupling between visual and motor rhythms. We further showed parallel modulations of visual performance and psychophysical contrast thresholds, which strongly suggests a truly perceptual—not decisional—nature of the action-locked effect, as well as its most likely early neuronal locus. In support of the latter notion, visual contrast thresholds are known to be limited by the early stages of visual analysis (Boynton et al., 1999).
The present results demonstrate that action-related processing can be intimately coupled with very early sensory functions, such as visual contrast thresholds, implicating the modulation of primary visual areas. The nonsystematic difference in the oscillation frequencies for stimuli in the two hemifields may suggest that the ongoing oscillations have a functional role, possibly of tagging the location of the incoming sensory input from different visual hemifields. However, the presence of individual differences and the emergence of a common synchronization frequency from the data pooled across subjects and hemispheres suggest that multiple frequencies within a restricted theta band are coupled with the action. Future research based on the direct measurement of electrophysiological activity, like recording electroencephalographic activity during the performance of the perceptual and motor tasks, may help to clarify this issue.
One prominent finding is that the oscillations in visual contrast sensitivity emerge already long before (∼500 ms) movement onset. The presence of oscillations prior to movement indicates that the final motor output—measured as hand displacement—cannot be itself a synchronizing event. Rather, the rhythmic visuomotor coupling must be generated earlier, probably during action preparation. The motor preparation activity, as revealed by human scalp recordings i.e., the Bereitschaftspotential, or readiness potential (Kornhuber and Deecke, 1965), starts gradually 1–2 s before the motor output. Its sources are the supplementary and cingulated motor areas (Ball et al., 1999); additional sources are premotor and motor areas generating on the scalp the negative slope component initiating about 500 ms before movement (Toma et al., 2002). This latter time window has often been associated with the consciousness of the decision to move (Libet et al., 1983) and the selection of the moving limb (Osman et al., 1992). The present data showing perceptual oscillations about 500 ms before movement are consistent with the idea that the action-locked synchronization of visual performance might be mediated by similar signals implicated in action preparation. An anticipatory corollary discharge signal has already been proposed as a mechanism to explain the complex changes in oscillatory activity and neuronal excitability observed with eye movements in electrophysiological studies (Melloni et al., 2009; Schroeder et al., 2010). In monkeys, an increase of high-frequency power and phase reset of low-frequency oscillations have been shown just after the execution of an eye movement (Rajkai et al., 2008; Bosman et al., 2009) and are suggested to be responsible for the transient perceptual enhancement measured psychophysically at the new fixation onset (Dorr and Bex, 2013). The corollary discharge signal, generated at an early stage during motor preparation, could thus keep the ongoing activity in visual areas phase locked. At present we do not know whether the corollary discharge signal is long lasting or temporally punctual, but we know that it is anticipatory. The fact that the oscillation survives the large variation in movement onset time suggests that the timing of movement must be represented with high precision in our brain, despite the gradual emerging of the readiness potential. Oscillations in the motor cortices could be a means to keep a precise representation of the timing of movement onset during the preparation phase, when the motor signals gradually build up and reach the movement threshold. Propagation of the oscillatory motor activity to visual cortices may be instrumental in linking and closing up the visuomotor loop.
Although it may be advantageous that visual activity is synchronized by the same mechanisms that subserve movement planning, there are several other mechanisms compatible with our data.
Endogenous rhythms in visual processing may be also regulating the timing of movements. Interestingly, previous evidence suggests that movement initiation is also influenced by the ongoing fluctuations in activity (Drewes and VanRullen, 2011; Schurger et al., 2012). Spontaneous visual rhythmic activity could thus drive motor behavior, cyclically modulating movement initiation time. In this respect, previous recordings of electrocortical activity in human patients have shown that the rhythmic presentation of sensory stimuli that do not require any overt motor response remarkably entrains the motor cortex (Besle et al., 2011).
Finally, the reported rhythmicity in visual perception may rather stem from an intrinsic coupling between visual and motor rhythms, not even necessarily contingent on action performance. A common neuronal rhythm dictated by some cortical or subcortical structure might jointly regulate both visual sensitivity and movement timing. Most likely, there is no unique “entrainment force,” but the oscillatory brain dynamics flexibly adapt to the current behavioral demands.
What the specific brain mechanisms are through which visual-motor oscillatory coupling is achieved remains a fascinating open question for future neurophysiological investigation. In any case, the present findings suggest that the functional interplay between motor and sensory rhythms is already present long before actual movement execution, probably closing up the motor–visuomotor loop. Oscillatory synchronization may mediate efficient information transfer between sensory and motor areas, optimizing the integration of sensory information into the unfolding motor plans. Oscillation-based mechanisms may thus provide a common frame of reference linking intention with the ensuing movement and perception.
Footnotes
This work was supported by European Research Council Advanced Grant ECSPLAIN to M.C.M.
The authors declare no competing financial interests.
- Correspondence should be addressed to Alice Tomassini, Robotics, Brain and Cognitive Sciences Department, Istituto Italiano di Tecnologia, Via Morego 30, 16163 Genova, Italy. alice.tomassini{at}iit.it
