Abstract
Beta-band (13–35 Hz) modulations following reward, task outcome feedback, and error have been described in cognitive and/or motor adaptation tasks. Observations from different studies are, however, difficult to conciliate. Among the studies that used cognitive response selection tasks, several reported an increase in beta-band activity following reward, whereas others observed increased beta power after negative feedback. Moreover, in motor adaptation tasks, an attenuation of the postmovement beta rebound follows a movement execution error induced by visual or mechanical perturbations. Given that kinematic error typically leads to negative task–outcome feedback (e.g., target missed), one may wonder how contradictory modulations, beta power decrease with movement error versus beta power increase with negative feedback, may coexist. We designed a motor adaptation task in which female and male participants experience varied feedbacks—binary success/failure feedback, kinematic error, and sensory-prediction error—and demonstrate that beta-band modulations in opposite directions coexist at different spatial locations, time windows, and frequency ranges. First, high beta power in the medial frontal cortex showed opposite modulations well separated in time when compared in success and failure trials; that is, power was higher in success trials just after the binary success feedback, whereas it was lower in the postmovement period compared with failure trials. Second, although medial frontal high-beta activity was sensitive to task outcome, low-beta power in the medial parietal cortex was strongly attenuated following movement execution error but was not affected by either the outcome of the task or sensory-prediction error. These findings suggest that medial beta activity in different spatio-temporal-spectral configurations play a multifaceted role in encoding qualitatively distinct feedback signals.
SIGNIFICANCE STATEMENT Beta-band activity reflects neural processes well beyond sensorimotor functions, including cognition and motivation. By disentangling alternative spatio-temporal-spectral patterns of possible beta-oscillatory activity, we reconcile a seemingly discrepant literature. First, high-beta power in the medial frontal cortex showed opposite modulations separated in time in success and failure trials; power was higher in success trials just after success feedback and lower in the postmovement period compared with failure trials. Second, although medial frontal high-beta activity was sensitive to task outcome, low-beta power in the medial parietal cortex was strongly attenuated following movement execution error but was not affected by the task outcome or the sensory-prediction error. We propose that medial beta activity reflects distinct feedback signals depending on its anatomic location, time window, and frequency range.
Introduction
Beta-band (13–35 Hz) oscillations have been originally associated with sensorimotor functions as they exhibit large power fluctuations with movement preparation and execution (Kilavik et al., 2013). Yet, since then, it has been shown that they are also modulated in relation to motivational and cognitive processes (Engel and Fries, 2010; Marco-Pallarés et al., 2015). Their sensitivity to reward, outcome feedback, and error had been first demonstrated in the context of response selection tasks. However, no coherent picture has been established so far. A set of MEG/EEG studies using gambling (Marco-Pallares et al., 2008a, 2009; Doñamayor et al., 2011, 2012) or reinforcement learning tasks (Cohen et al., 2007) described a high-beta power enhancement in successful rewarded trials. On the other hand, in competitive decision-making games (Cohen et al., 2009) or in tasks in which response selection errors are caused by time pressure (Koelewijn et al., 2008; Marco-Pallarés et al., 2008b; De Pascalis et al., 2012), beta power was found to be greater on erroneous trials.
Beta-band activity has also been found to be sensitive to movement execution errors. Beta-band power is attenuated following movements deviated by visual or mechanical perturbations (Tan et al., 2014; Torrecillos et al., 2015; Alayrangues et al., 2019). As kinematic errors can in turn lead to task failure (e.g., missing a target), one may wonder how, in this case, opposite error-related modulations coexist, attenuated beta power with kinematic error on the one hand and greater beta power in erroneous trials on the other hand. Moreover, the beta-power attenuation observed in motor adaptation tasks may be a response to two qualitatively distinct teaching signals, the clearly perceived kinematic error, which may activate cognitive processes (Malfait and Ostry, 2004; Mazzoni and Krakauer, 2006; Redding and Wallace, 1996; Taylor et al., 2014), and the sensory-prediction error, which drives automatic internal model updating (Jordan and Rumelhart, 1992).
So far these seemingly conflicting beta-band responses have been described in separate studies using different tasks. Here, we elicited them in the same motor adaptation task. EEG was recorded in healthy volunteers asked to shoot at a small visual target, making precise whole-arm ballistic movements. If successful, the target exploded, providing binary success/failure feedback. A visual rotation was applied in a short series of trials separated by a variable number of unperturbed trials so that participants could not predict when the perturbation would be (re)introduced. As a result, in the first trial of each series, the rotation induced a large movement deviation, and participants systematically missed the target. In the following trials, once they knew the perturbation was on, participants countered its effect using a reaiming strategy, which allowed isolating sensory-prediction error from kinematic error.
Our hypothesis was that beta responses can arise in contrasting directions (up or down modulations) during exactly the same individual trials as long as they do not overlap in their anatomic location and time window and frequency range, and differ in at least one of these three dimensions (and, thus, constitute different configurations within the spatio-temporal-spectral space of possible response patterns). First, we separated activities in medial frontal, medial parietal, and lateral central cortices using independent component analysis (ICA). Second, the use of a shooting task allowed the temporal decoupling of target explosion and movement termination. Third, we subdivided beta oscillations into low and high beta sub-bands (Nougaret et al., 2023). Without prior assumptions, we used linear mixed models to identify which spatio-temporal-spectral configurations were most sensitive to the different feedback signals. In complementary event-related desynchronization (ERD)/event-related synchronization (ERS; ERD/ERS) analyses, we focused on oscillations in specific cortical regions and frequency bands.
Contrasting successful and failed trials, we found two opposite modulations. Both were observed for high beta oscillations in the medial frontal cortex but at distinct temporal stages. Second, in the first trial of each rotation series in which large movement deviations were induced and the target missed, opposite modulations occurred within the same time window but in different medial cortical regions and beta frequency sub-bands. Therefore, we suggest that medial beta activity in different spatio-temporal-spectral configurations play a multifaceted role in encoding qualitatively distinct feedback signals.
Materials and Methods
Participants
A total of 24 healthy adults (eight females) age 26.5 years (range, 20–32) took part in the study. All participants were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971) and all had normal or corrected-to-normal vision. All participants were free of known neurologic or psychiatric disorders, gave informed consent according to a protocol approved by the Ethics Board of Aix-Marseille University, and received monetary compensation for their participation.
Experimental setup
Participants were seated with their right arm installed in a robotic exoskeleton (Kinarm, BKIN Technologies) that allows recording elbow and shoulder joint rotation in the horizontal plane. The height of the chair was adjusted so that the shoulder was abducted by ∼70°. A semisilvered mirror preventing direct vision of the arm was used to manipulate online movement visual feedback. The visual display and the cursor representing participants' index fingertip were projected onto the same plane as the (invisible) hand. Head movements were restrained by using a chin rest.
Task
Participants were asked to perform ballistic movements without online correction (Jahani et al., 2020). The movement starting position was indicated as a 0.75-cm-diameter white circle located at the center of a large concentric blue ring (10 and 14 cm radius for the inner and outer contour, respectively). Three possible targets located 5 cm away from the starting position were indicated as 0.3-cm-diameter dark gray circles at 50°, 80°, or 110° from the 0° straight-ahead direction (Fig. 1A). The size of the targets (0.3 cm diameter) was tuned during pilot experiments so that without visual perturbation participants succeeded in hitting the targets roughly half of the time.
To initiate a trial, participants had to maintain their index finger in the start circle for 2000 ms, after which they were warned to get ready (Ready signal); the start circle disappeared, and one of the three targets turned from a gray to white circle. Following a 1500 ms delay, the target was filled in white (turned on) indicating that the movement could be initiated (Go signal). Importantly, participants were informed they were not performing a reaction-time task and thus they should take all the time they needed to prepare their movement.
Participants were instructed to move through (shoot at) the target without stopping and end their movement between the inner and the outer contour of the outward ring (concentric with the start circle). They were also required to move fast enough so that the fingertip cursor reached the distance to the target (5 cm) within 250 ms. Movement onset was defined as the time when the speed of the hand exceeded 5 cm/s. The following outcome feedback was provided at the time the cursor reached the distance to the target (5 cm) hitting or missing the target: (1) The target exploded when the movement was fast and accurate enough (target hit), (2) the target turned red when the movement was fast enough but not accurate enough (target miss), and (3) the target turned green when the movement was too slow, independent of accuracy. Note that according to their verbal reports, participants viewed the explosion of the target as a rewarding experience. We augmented the experience by adding the sound of a bottle of champagne being uncorked in addition to the visual explosion.
To avoid online movement corrections, the fingertip cursor was turned off when the hand crossed the 10 cm radius inner contour of the ring. At the end of the movement, the arm was passively returned by the robot to the starting position. The fingertip cursor and the starting position circle reappeared only when the cursor was back in the starting position. Participants were asked to keep their eyes fixed on the aimed target throughout each trial. Each trial lasted ∼7 s.
We designed our task so that participants would experience a repertoire of teaching signals. The target explosion provided a binary success/failure feedback on the outcome. In the Catch trials, the unexpected introduction of the visual rotation induced both a clearly perceived kinematic error as well as a sensory-prediction error (rotated visual feedback), whereas in the rotation strategy (RS) trials participants could eliminate the uncontrolled movement deviation but still experienced the sensory-prediction error (rotated visual feedback).
Experimental protocol
The experiment was conducted in two sessions (preliminary and experimental) on two different days, during which participants performed two types of blocks, Baseline blocks of only unperturbed (baseline) trials and Mixed blocks in which a visual rotation (+30° or −30°) was applied in selected trials.
Mixed blocks structure
In Mixed blocks (Fig. 1A) unperturbed (no rotation) trials alternated with a short series of trials during which a visual rotation was applied. The visual rotation trial series always consisted of four movements to the same target, whereas the number of the no-rotation trials interleaved between two visual rotation trial series varied pseudorandomly (at least four no-rotation trials in a row). That is, participants could not predict when (on which trial) the visual rotation would be (re)introduced. As a result, in the first (Catch) trial of each rotation trial series, the hand trajectory clearly deviated, and the target was systematically missed. However, during the Preliminary session (see below), participants were informed of the properties of the rotation trial series; that is, they knew that the same visual rotation would be applied for four successive trials and then removed. Critically, they were also informed of the nature of the visual perturbation and how to counteract it.
Each Mixed block comprised 18 rotation trial series (18 Catch trials * 4 Mixed blocks = 72 Catch trials in total), and 96 no-rotation trials pseudorandomly distributed in between, for a total of 168 trials. The direction of the rotation, 30° clockwise (CW) or 30° counterclockwise (CCW), applied in the rotation trial series was kept constant throughout each Mixed block but was reversed for each new Mixed block. Half the participants started with a 30° CW Mixed block and the other half with a 30° CCW Mixed block.
Preliminary session (familiarization and learning the reaiming strategy)
During the first session participants received verbal instructions about the general task requirements. They performed at least four blocks of 20 trials with no visual rotation, followed by a block in which after four no-rotation trials, the visual rotation (clockwise or counterclockwise, counterbalanced across participants) was unexpectedly introduced for five trials. After participants had experienced the visual rotation, the experimenter explained in detail the nature of the perturbation and how they could counteract it by using a cognitive strategy, reaiming at the (clockwise or counterclockwise) neighboring target (Jahani et al., 2020). They performed two Mixed blocks, each followed by a 32-trial Baseline block (400 trials in total). EEG signals were not recorded during this session.
Experimental session
During the second session, after a 64-trial Baseline block, participants performed four Mixed blocks, each followed by a 32-trial Baseline block (864 trials in total). EEG signals were recorded throughout the session (Fig. 1B).
Between each block of trials (Mixed and Baseline) and after the 84th trial of each Mixed block, an ∼2 min break was allocated. The preliminary session lasted ∼1 h 30 min in total (including the calibration of the robot), and the experimentation session (including the placement of the EEG electrodes and the recording of their locations) lasted ∼3 h in total.
Behavioral data recording and analyses
Angular position and velocity data of the motor resolvers were collected at 1000 Hz. Signals were downsampled off-line to 100 Hz and then filtered with a second-order zero-phase shift low-pass Butterworth filter (cutoff frequency of 10 Hz). Hand position and velocity were calculated from these angular data. Kinematic data were analyzed using custom routines written in MATLAB software (MathWorks). Trials in which the hand was not maintained stably enough in the start position during the delay between the Ready and Go signals (tangential velocity > 6 cm/s) or in which the movement was initiated before the Go signal were excluded from the analyses (∼1% of trials). Movement onset was defined as the time when the tangential velocity exceeded 5 cm/s. The movement offset corresponded to the time when tangential velocity fell below 5 cm/s and remained below this value for at least 1500 ms. To quantify kinematic errors, we computed the perpendicular deviation (PD), from the straight line that connects the starting position to the target, at maximum velocity (vel; PD-vel). This measure quantifies error in the initial movement direction (feedforward component). Movement duration was also calculated. Trials that were performed too slowly (∼4%) were excluded from the analyses.
To collapse data from different Mixed blocks with opposite visual rotations (30° CW vs 30° CCW), we set the signs of the PD-vel values so that hand-path deviations in the direction of the visual rotation corresponded to positive values. Previously, we conducted preliminary analyses to test for differences between the movement errors induced by the two rotation directions.
In the statistical analyses, for the Mixed blocks we compared the trial categories corresponding to the different experimental conditions, no rotation (NR), Catch, RS, and after rotation (AR) trials (Fig. 1A; RS1, RS2, and RS3 trials were regrouped; see below). For the Baseline blocks, we distinguished between the trials in which participants successfully shot the target (B-Hit) from those in which they failed to do so (B-Miss).
Pairwise t tests or repeated-measures univariate ANOVAs followed by post hoc pairwise t tests were run on kinematic error measures (PD-vel) and movement durations. In all cases, trial category was used as a within-subject factor. Normality and sphericity assumptions were controlled and a Huynh–Feldt correction applied when appropriate. For all tests, the significance threshold was set to 0.05, and multiple-comparison corrections were performed with the Bonferroni procedure. For post hoc tests, p values are multiplied accordingly (see below, Results).
EEG data recording and analyses
EEG recording, preprocessing, and time–frequency representation
EEG activity was recorded continuously at 1024 Hz using a 64-channel Biosemi ActiveTwo system referenced to the Common Mode Sense (CMS)/Drive Right Leg contact. Electrodes were embedded into an elastic cap and distributed over the scalp according to the extended 10–20 EEG system. The electrode offsets, the voltage differences between the CMS and each active electrode, were monitored to remain within ±20 μV. For each participant, electrode locations and nasion and preauricular points were recorded by an infrared camera (Rogue Research). Electro-oculographic (EOG) activity was recorded with surface electrodes placed near both outer canthi (saccades) as well as under and above the right orbit (blinks).
EEG data were preprocessed using the free software ELAN (Aguera et al., 2011). Continuous signals were rereferenced to the average of all electrodes, filtered between 2 and 70 Hz (Butterworth order 2) and downsampled to 256 Hz. Nonstereotypical artifacts that cannot be captured by ICA (cf. Makeig et al., 1997; Delorme et al., 2007) were identified and rejected on visual data screening. Further analyzes were run using the free and open-source software FieldTrip (Oostenveld et al., 2011).
Following ICA performed on the EEG signals filtered between 2 and 70 Hz, time–frequency analyses were performed on the time courses of the obtained ICs (see below). Single-trial signals were transformed in the time–frequency domain by convolution with the complex Morlet wavelets characterized by the ratio f0/σf = 7, with f0 ranging from 2 to 50 Hz by steps of 0.5 Hz. To calculate the event-related changes in beta power, the raw power data were log transformed and then normalized relative to the average power calculated over all trials as no clear baseline period could be defined during the task (Tan et al., 2014; Torrecillos et al., 2015).
IC selection
The preprocessed EEG signals were cut into time segments extending from −4 to 3 s with respect to the outcome feedback, which covered approximately the complete trials, slightly variable in duration. The epoched EEG data were then submitted to ICA (runica algorithm).
Our goal was to identify for each participant four different ICs that would capture oscillatory activities from four cortical regions, the medial frontal cortex (IC1), the medial parietal cortex (IC2), and the left and right lateral central cortices (IC3 and IC4). For IC selection, we proceeded in two steps (Alayrangues et al., 2019; Jahani et al., 2020). First, we preselected ICs based on their topographies. For this step, we defined (a priori, before running the ICA) spatial regions of interest (ROIs); ICs that exhibited the largest weighting within one of these ROIs were preselected. To capture activity of the medial frontal cortex, we considered an ROI including electrodes F1, Fz, F2, FC1, FCz, FC2, C1, Cz. C2, for activity in the parietal medial region we used an ROI encompassing electrodes C1, Cz, C2, CP1, CPz, CP2, P1, Pz, P2, and for the left and right lateral central regions, we used two ROIs including electrodes C3, C5, CP1, CP3, CP5, P1, P3, P5 and C4, C6, CP2, CP4, CP6, P2, P4, P6, respectively.
Note that for one participant (S18; Fig. 2) we made an exception, selecting an IC with maximum weighting at electrode FC3. Then, in a second step we examined the time–frequency representation of the time courses of the preselected ICs in order to retain one IC of each type for each participant. For this step, within the trial period from −2.0 to 2.0 s relative to the outcome feedback, we examined the time–frequency representations of the time courses of the preselected ICs computed over all trials. For each individual and type of ICs, we selected the IC (most of the time only one IC per participant was preselected) exhibiting the largest variance of power for the frequency band between 17 and 40 Hz.
Dipole fitting of the ICs
We conducted dipole source analysis of the different ICs. For each participant, an equivalent current dipole model was computed for the selected ICs by using the ft_dipolefitting function in FieldTrip. Informing the forward model of the recorded locations of the EEG electrodes, dipoles were localized within a three-shell boundary element model (BEM) of the Montreal Neurologic Institute (MNI) standard human brain. For all identified individual ICs, only dipole solutions with residual variance smaller than 10% were considered.
Individual selection of the frequency bands
Theta, alpha, low-beta, and high-beta frequency bands were selected separately for each participant to take into account interindividual differences (Little et al., 2013; Torrecillos et al., 2015; Tinkhauser et al., 2017; Alayrangues et al., 2019; Meidahl et al., 2019). For each individual, the activation time courses from −3.5–2.5 s relative to the feedback of the different ICs were concatenated into a single time series, whose power spectrum was computed. In each individual power spectrum, local maxima were identified (automatically, with manual correction when needed), and defined the centers of the different frequency bands. For all participants, band widths were fixed to 3, 5, 7 and 9 Hz, respectively, for theta, alpha, low beta, and high beta. Single-trial power profiles were obtained by averaging power in each frequency band.
Trial pairwise separation using linear mixed models
The trial categories of the Mixed blocks, defined by the experimental protocol (NR, Catch, RS, and AR), were subdivided according to the performance of the participants (target hit or missed). We considered the following categories: NR-Hit, NR-Miss, Catch, RS-Hit, and RS-Miss, in addition to the B-Hit and B-Miss trials of the Baseline blocks. As participants systematically missed the target in Catch trials, these were not subdivided. In the present study, AR trials served to demonstrate the effect of the implicit sensorimotor adaptation (automatic internal model updating) driven by the sensory-prediction errors experienced in the Catch and RS trials. We do not explore any contrast with AR trials in this study.
To identify without prior assumptions which activity patterns were the most sensitive to the different types of feedback signals, we used linear mixed models to pairwise classify single trials. We considered the following pairwise classifications: (1) B-Miss versus B-Hit trials to identify oscillatory activities sensitive to the reward and/or the outcome feedback, and only trials from the Baseline blocks were considered, as in these blocks hits and misses had comparable probabilities (see below, Results); (2) Catch versus NR-miss trials to identify activities modulated by the externally induced kinematic errors and/or the sensory-prediction error, and we included only missed trials to identify these modulations independent of the task outcome; and (3) RS-hit versus NR-hit trials to identify activities sensitive to sensory-prediction error independent of kinematic error.
We assessed whether the single trials could be reliably pairwise classified based on the information contained in the activation time courses of the ICs. Pairwise classification was performed iteratively along the time axis (each 100 ms from −500 to 2500 ms relative to the feedback) to track the accuracy of the classification across trial time. For each IC and each frequency band (at each 100 ms step), single-trial activation power was averaged over 300 ms. The resulting 16 values were then used as predictor variables of a linear mixed model.
For each pairwise classification, the labels 0 and 1 were arbitrarily attached to the trial categories (e.g., in B-Miss vs B-Hit, B-Miss labeled 0 and B-hit labeled 1). The linear mixed-model equation (see Fig. 4),
For each model (computed for 100 ms steps), the data were partitioned into a training fold (80% of the dataset) and a testing fold (20% remaining data). To ensure an unbiased prediction, not influenced by the sheer incidence of each trial category, the training and testing folds were built with a balanced number of trials from both categories for each subject. Each model was trained and tested 200 times with new, randomly selected training and testing folds. The accuracy of the classification was quantified as
The same procedure was repeated but by training the models on the shuffled version of the data to calculate the chance level predictions. In the shuffled version of the training data, the category labels (yij) are shuffled so that the relationship between the predictor variables and trial category is disrupted. Each model trained on shuffled data is then tested on a testing fold of original unshuffled data. By definition, models trained on shuffled data yield chance-level prediction accuracy Ac, which the predictions achieved by the models trained on the original data can be compared with. Note that only data from participants for which all four ICs (18/23) could be identified were included in these analyses.
In general, linear mixed models assign coefficients to predictor variables based on their importance in estimating the value of the predicted variable. Here, for a given pairwise classification, different EEG predictor variables are weighted by their ability to predict the classifier decision output accurately (i.e., weaker or stronger). Moreover, some of the coefficients are positive and others negative, indicating the direction of the influence of the fluctuations of each given EEG predictor variables.
To account for variability in the estimation of individual coefficients, we not only calculated the coefficients over multiple folds of data (that ensured that the coefficients were not governed by outlier data points) but also additionally corroborated them with the t values and p values of the coefficients. The p values indicate which coefficients significantly affect the classifier decision. The EEG predictor variables with the highest t values were considered to contribute the most to the pairwise separation of interest.
The influence of the individual predictor variables can also be mechanistically quantified by systematically disrupting the correlation between the predicted and predictor variables. We applied this approach as additional tests and assessed the amount by which prediction accuracy deteriorates when the information carried by different EEG predictor variables is removed. For this purpose, we used a procedure in which the structure of the model equation remains unaltered. The information carried by different EEG predictors was disrupted by shuffling its values (to destroy its correlation with the predicted variable) rather than by removing the predictor variable itself from the model equation. That is, we shuffled, across trials and participants, the values of the terms xic,fb,ij in the model equation, where ic, fb, i, and j stand for the IC, frequency band, participant, and trial index, respectively. For instance, to test for the influence of IC1 for a given trial pairwise separation, in the training set the averaged theta power values for IC1 of all trials of all participants (xIC1,θ,ij) were shuffled and so forth for the other frequency bands α (xIC1,α,ij), low beta (xIC1,low-β,ij), and high beta (xIC1,high-β,ij). The reasoning is as follows: If prediction accuracy deteriorates more when the information carried by A is removed compared with when the information carried by B is removed, then A contributes more to the classifier prediction than B.
ERD/ERS analyses
Based on the results of the linear mixed models described above, we focused on activities in specific ICs and frequency bands. For the different activities of interest, we computed ERD/ERS profiles for different trial categories by averaging across single-trial absolute power time series. We analyzed the average power profiles within the time window going from 0 to 2.5 s relative to the feedback. Statistical analyses were computed at each sampling point (50 ms bin). Significance level was set to 0.05. The false discovery rate (FDR) method was used to correct for multiple comparisons along the time axis. For each participant, we equalized the number of trials used to calculate the power profiles of the categories to be contrasted. Data from participants with <20 trials by trial category, after rejecting artifacts, were excluded from the analyses, which we respecify below.
Results
Our goal was to test whether beta-band responses observed in different task contexts, and seemingly conflicting modulations, could in fact coexist as they correspond to different configurations within a space of possible spatio-temporal-spectral oscillatory response patterns. To this aim, we designed a motor task in which participants experienced different feedback signals, binary success/failure feedback, kinematic error, and sensory-prediction error. Participants were asked to shoot at a visual target small enough that even without any perturbation (in the Baseline blocks), they missed it half of the time. When the movement was precise enough, the explosion of the target provided a binary feedback of success. When the visual rotation was unexpectedly introduced (in the Catch trials), participants experienced both a large kinematic error and a sensory-prediction error, whereas when they successfully countered it (in the RS-hit trials), only the sensory-prediction error was elicited.
Behavioral performance
One participant was excluded from all analyses (behavioral and EEG) because of noisy EEG data; hence, data from 23 participants were analyzed. As expected, different sizes and directions of kinematic errors were observed in the different conditions. This is visible in Figure 1C, presenting the hand-path PD-vel averaged across participants. Statistical analyses were performed on the PD-vel values and collapsed across Mixed blocks (see above, Materials and Methods). Data were collapsed following preliminary analyses revealing no significant difference between the effects observed for the opposite 30° CW and 30° CCW rotations. Indeed, a two- (rotation direction, 30° CW and 30° CCW) by-five (type of trials, Catch, RS1, RS2, RS3, and AR) two-way repeated-measures ANOVA revealed no main effect of rotation direction (F(1,22) = 2.260, p = 0.134) and no significant interaction effect (F(4,88) = 0.743, p = 0.564), although a significant main effect of trial category was observed (F(4,88) = 247.1, p < 0.0001). Also, data from the different rotation strategy trials (RS1, RS2, and RS3) were regrouped into RS trials, following preplanned (uncorrected p values) pairwise comparisons revealing no significant differences (RS1 vs RS2, t(22) = −1.4627, p = 0.1577; RS1 vs RS3, t(22) = −2.0593, p = 0.0515; RS2 vs RS3, t(22) = −2.0713, p = 0.0503).
Task, experimental protocol, and behavioral data. A, Participants were instructed to shoot, without stopping, at one of three possible visual targets. In rotation-trial series, the cursor representing the index fingertip was displayed rotated by 30° CW or 30° CCW, relative to its real position. The rotation-trial series always consisted of four trials separated by a variable number of NR trials (blue-gray) in which the cursor displayed the real position of the hand. In the first rotation trials (Catch, red), the visual rotation was unexpectedly (re)introduced. In the three following trials (RS1–3, green), participants knew the rotation would be applied and used the reaiming strategy to counter it. Participants were also told that after the fourth rotation trials the visual rotation would be removed (AR, blue). Each Mixed block consisted of 18 rotation-trial series. B, In the experimental session, participants performed four Mixed blocks, in which the direction of the visual rotation was kept constant at 30° CW or 30° CCW. The rotation directions alternated over the four Mixed blocks, with the order counterbalanced across participants. The mixed blocks were preceded and separated by Baseline blocks of 64 and 32 no-rotation trials, respectively. C, Left, Initial movement-direction error data averaged across participants (n = 23), calculated as the perpendicular hand-path deviation measured at tangential velocity peak (PD-vel). Error bars indicate SE. Colors used for the individual trials represent the different trial categories in A. Positive values correspond to errors in the CW direction. Right, The values of the PD-vel observed at the group level (n = 23) for the different categories of trials of the Mixed blocks (Catch, RS, and AR) and the successful and failed trials of the Baseline blocks (B-hit and B-miss). The sign of the errors observed for the opposite visual rotation (CW and CCW) has been flipped so that data could be collapsed (see above, Materials and Methods). Box plot represents statistical results. Middle mark indicates the median. Edges of the box represent the 25th and 75th percentiles. Whiskers extend to the extreme data. Each participant's data are plotted individually. Kinematic errors observed for the Catch and AR trials significantly differed from those observed in all other trial categories (***p < 0.0001, Bonferroni corrected), with deviations in the direction of the visual rotation for Catch trials and deviations in the opposite direction for AR trials.
Group data for each trial are presented on the left in Figure 1C. Large kinematic errors in the direction of the visual rotation were observed for Catch trials, confirming that participants did not predict when the perturbation would be reintroduced. Clear initial movement-direction errors were also visible for the AR trials, with deviations in the opposite direction (aftereffects) reflecting implicit sensorimotor adaptation activated in response to the sensory-prediction errors experienced in the rotation trial series, composed of the Catch and RS trials.
A repeated-measures ANOVA on the PD-vel conducted on the Mixed blocks data confirmed a significant effect of trial categories (NR, Catch, RS, and AR) on movement accuracy (F(3,66) = 391.38, p < 0.0001). Post hoc comparisons revealed that hand-path deviations in the Catch and AR trials differed significantly from those in all other trial categories (Catch vs NR, RS, and AR, respectively, t(22) = 32.7129, p < 0.0001; t(22) = 39.4770, p < 0.0001; t(22) = 39.4770, p < 0.0001; AR vs NR, RS, respectively, t(22) = 8.1572, p < 0.0001; t(22) = 8.4535, p < 0.0001) with deviations in the direction of the visual rotation for Catch trials and deviations in the opposite direction for AR trials. Kinematic errors did not differ significantly between the NR and the RS (NR vs RS, t(22) = −2.8650, p = 0.0540). Yet, movement direction was more variable in RS than in NR (F(22,22) = 9.0060, p < 0.0001). These results are summarized on the right in Figure 1C.
In addition, for the NR and RS trials, we conducted preplanned comparisons (uncorrected p values) to test whether kinematic errors differ significantly from zero. Hand-path deviations differed significantly from zero for the NR trials (t(22) = −4.7849, p = 0.0001), with a bias in the direction opposite to the visual rotation. This was likely because of aftereffects because of the implicit sensorimotor adaptation (automatic internal model updating) that may not have completely washed out between rotation series. No significant bias was found for the RS trials (t(22) = 1.6354, p = 0.1162), confirming that in general participants properly counteracted the visual rotation by applying the reaiming strategy. Yet, as indicated, reaches in RS trials were substantially more variable than in the NR trials. In RS trials participants hit the target in only 18.25% of trials, whereas they did so in 42.63% of NR trials (in Catch trials and AR trials, 0% and 16.61% of successes were observed, respectively).
In the Baseline blocks, participants successfully hit the target in 49.96% of the trials. In these blocks, performed in the absence of perturbation, failures (target missed) not only represented general variance in the movement direction but also a systematic bias in the direction opposite to the direction of the visual rotation applied in the preceding Mixed block (Fig. 1C, left and middle). This was confirmed by preplanned comparisons with zero (B-Miss, t(22) = −5.1325, p < 0.0001). In fact, such a systematic bias was also observed for the successful trials (B-hit, t(22) = −2.8570, p = 0.0092). As for the NR trials between the rotation series in the Mixed blocks, this bias most likely reflected the persistence of implicit sensorimotor adaptation aftereffects.
We also analyzed movement duration. For the Mixed blocks (NR, Catch, RS, and AR trials), a repeated-measures ANOVA revealed a significant effect of trial category (F(3,66) = 7.3884, p = 0.0002). Post hoc pairwise comparisons indicated that Catch trials had significantly longer durations than NR trials (t(22) = −4.1774, p = 0.0023; NR, 567 ± 81 ms; Catch, 606 ± 101 ms) and RS trials (t(22) = 3.2280, p = 0.0234; RS, 580 ± 80 ms). No other pairwise contrast survived multiple comparison correction (AR, 586 ± 99 ms).
In summary, first, the behavioral data confirmed that participants could not predict when the rotation would be (re)introduced. In the Catch trials, all systematically witnessed their hand movement deviate and miss the target by a large margin. Second, when participants knew they would have to counteract the visual rotation (RS trials), the average direction error did not differ significantly from zero. However, on these trials movement direction was significantly more variable than when the rotation was not applied (NR trials). Third, substantial aftereffects were observed on the removal of the visual rotation (AR trials), confirming that in the rotation trials (i.e., Catch and RS trials) participants experienced a sensory-prediction error, driving implicit sensorimotor remapping (internal model updating). Finally, in the Baseline blocks, in which all trials were performed under unaltered conditions (no visual rotation), participants hit the target on average nearly half of the time, which is essential to control for the effects of success probability.
EEG oscillatory responses
To disentangle activities from functionally distinct cortical regions that are mixed at the sensor level, we used ICA. Our aim was to identify four ICs in each participant that would capture oscillatory activity, respectively, in the medial frontal cortex, whose activity has long been known to be sensitive to error and reward (Marco-Pallarés et al., 2015; Cavanagh and Frank, 2014); in the medial parietal cortex, whose role is more uncertain but has been shown to be involved in high-level cognitive and attentional processes (Cavanna and Trimble, 2006); and in bilateral central regions, that is, the left and right sensorimotor cortices.
Figure 2 presents the topographies of the four different ICs selected for each participant. Not all ICs could be identified in all participants. The IC1, IC2, and IC3 (left) and IC4 (right) lateral central ICs could be identified in 21/23, 21/23, 20/23, and 21/23 participants, respectively. (Empty spaces in Fig. 2 indicate the participants for whom the corresponding IC could not be identified.) The corresponding individual dipole solutions for the ICs, together with the group averaged dipoles, are presented in Figure 3. Dipole centroids were localized, for the medial frontal ICs, to the supplementary motor area (MNI coordinates, X = 4, Y = −5, Z = 55); for the medial parietal ICs, to the precuneus (MNI coordinates, X = 2, Y = −62, Z = 50); and for the lateral central ICs, to the left and right sensorimotor areas (MNI coordinates, X = −32, Y = −36, Z = 62 and X = 33, Y = −32, Z = 61, respectively).
Individual IC topographies. Topographies of the ICs identified for each participant for the IC1, IC2, and left (IC3), and right (IC4) lateral central cortex.
Dipoles fitted for individual IC. Estimated dipole locations within a three-shell BEM of the MNI standard brain for the medial frontal (green), medial parietal (purple), and left and right lateral central (blue) ICs. Small spheres represent dipoles for each participant. Large spheres represent centroids of the individual dipoles for the different types of ICs.
We distinguished oscillatory activities in the low- and high-beta frequency ranges, as these two beta frequency sub-bands could have different sensitivity patterns. The frequency bands selected for each participant are presented in Figure 4 (mean ± SE, 5.09 ± 0.13 Hz, 10.78 ± 0.19 Hz, 20.80 ± 0.34 Hz, 29.01 ± 0.59 Hz, for theta, alpha, low beta and high beta, respectively). For illustration purposes, the plots of one participant's data are superimposed. The power spectrum calculated for the time courses of the four different ICs concatenated, used to define the center of the frequency bands, is plotted (thick black line) together with those for each IC separately (thin colored lines). This illustrates that in addition to substantial interindividual variability power spectra also varied between ICs within participants.
Single-trial power profiles were obtained by averaging 3, 5, 7, and 9 Hz wide bands, for the theta, alpha, and low-beta and high-beta frequency bands, respectively, which were used as input to the linear mixed models. We performed pairwise separations of trial categories based on single-trial power values averaged over successive time windows (width, 300 ms) to identify, without prior assumptions, the oscillatory activities most sensitive to the different teaching signals (binary feedback, kinematic error, and sensory-prediction error).
Individual frequency band selection and linear mixed-model design. Top, Model equation; parameters are described in the text. Bottom left, Frequency band individually selected for the theta, alpha, low- and high-beta bands. White lines in the middle of the box plots indicate the medians. Edges of the box represent the 25th and 75th percentiles. Whiskers extend to the extreme data. Each participant's data are plotted individually. For illustration purposes, the spectrograms of the data of one participant (S02) are overlaid. The spectrogram computed for the activation time courses of the different ICs concatenated into a single time series, used to select the center of the different frequency bands, is shown (thick black line), together with the spectrogram computed for the activation time-courses of each IC separately (IC1, IC2, IC3, and IC4, green, purple, and blue thin lines, respectively). The extent of each frequency band is indicated by a shaded area around the participant's (S02) specific theta, alpha, and low- and high-beta bands. The thick-colored dashed lines indicate the center of the frequency bands. Upper and lower limits of the frequency bands are indicated by thick colored dotted lines; band widths were fixed to 3, 5, 7 and 9 Hz, respectively, for theta, alpha, low beta, and high beta. Bottom right, Individual topographies for each type of ICs.
B-Hit versus B-Miss trials
We first evaluated how well trials all performed in the same unaltered condition (without visual rotation) but resulting in different feedback; that is, B-Hit versus B-Miss trials could be discriminated on the basis of the activities in the different spatio-temporal-spectral configurations. (Single trials of the two categories by one participant (S02) are plotted for illustration purposes in Fig. 5C.) Figure 5A shows the pairwise classification accuracy obtained by the linear mixed model, as well as the p value for the model as a whole (the significance of the model compared with a null model), calculated at each 100 ms step. Over the time windows extending from 200 to 1200 ms and 1400 to 1800 ms relative to the feedback, classifications based on the actual data were significantly more accurate (p < 0.01 with Bonferroni-correction) than those based on the shuffled data. The highest prediction accuracy (56.1% accuracy) was obtained for the model fitted with data from the time windows centered at 100 ms and 200 ms relative to feedback.
Pairwise classification and power profiles for the trial categories B-Hit versus B-Miss. A, Accuracy of the prediction of the trial categories B-Hit versus B-Miss based on the actual data (dashed blue line) and based on the shuffled data (dashed green line) for each model, computed at each 100 ms step, taking as input data from a 300-ms-wide time window. Red dots indicate the models for which the accuracy was significantly higher (p < 0.01, two-tailed t test, Bonferroni corrected) when using the actual data relative to using the same but shuffled data. The p values associated with the overall models are overlaid (dashed black line). B, Regression coefficients for each of the 16 predictors of the models, 4 ICs (IC1–IC4) * 4 frequency bands (theta, alpha, low beta, high beta). Significant coefficients (p < 0.01, two-tailed t test) are highlighted in red boxes. Time = 0 corresponds to the time of the outcome feedback. C, Illustrative hand paths of single trials of the two categories by one participant (S02). D, Group averaged topographies together with power profiles for the IC1 (top) and the IC2 (bottom) aligned to the task outcome feedback. Power profiles obtained by averaging the individually selected ICs and frequency bands separately for the trial categories, B-Hit (green) and B-Miss (red). The period during which power differed significantly (p < 0.05, two-tailed t test; FDR corrected) between trial categories is indicated in gray. Small black dots indicate sampling point of uncorrected significant difference.
Figure 5B presents the t values of all predictor variables in the models. Statistically significant (p < 0.01) coefficients are highlighted in a red box. High-beta activity in IC1 from 200 to 300 ms, and then from 800 to 900 ms and 1200 to 1300 ms contributed significantly to the prediction. Interestingly, the sign of the regression coefficient flipped between the two time windows, reflecting a brief transient increase in power for B-Hit trials versus B-Miss trials at a short latency (200–400 ms), followed by an opposite modulation (decrease in power) at a longer latency and for a more prolonged period (800–1000 ms). Neither low-beta-band activity in the same region, nor low or high beta in other regions contributed significantly to the pairwise separation.
However, at a very short latency (200–500 ms), theta activity in IC1 also contributed significantly to trial category discrimination, with enhanced power for B-Miss compared with B-Hit. This was followed by significant transient contributions of theta activity in IC2 and IC4 cortices (200–600 ms), with a reversed sign, that is, this time, with greater power in B-Hit trials than in B-Miss trials.
In short, for IC1, significant t values were found for the theta and high-beta variables; whereas, for IC2 and IC4, significant t values were found for the theta component only (and for shorter time periods). For IC3, significance was reached for none of the components. This suggested that oscillatory activity in the IC1 contributed most to trial categories separation.
To test this, we assessed how shuffling the data of each IC affected prediction accuracy (see above, Materials and Methods). At each time point (from −0.5 to 2.5 s relative to feedback; 31 time points – 100 ms steps), we computed the median of the prediction accuracies computed for the 200 training folds run for each of the different models, IC1 shuffled, IC2 shuffled, IC3 shuffled, IC4 shuffled, and No shuffle. These values were then subjected to a repeated-measures ANOVA, which was found to be statistically significant (F(4,30) = 5.8852, p = 0.0002). Post hoc pairwise comparisons (with Bonferroni-corrected/multiplied p values) revealed that shuffling the IC1 data induced a significant drop in prediction accuracy (IC1 shuffled vs No shuffle, t(30) = −3.3618, p = 0.0212), whereas shuffling the components of IC2, IC3, or IC4 did not have a significant impact on prediction accuracy assessed globally over the trial time course (t(30) = −2.4602, p = 0.1987; t(30) = −2.7099, p = 0.1102; t(30) = −1.0176, p = 1, for IC2 shuffled vs No shuffle, IC3 shuffled vs No shuffle, and IC4 shuffled vs No shuffle, respectively). The drop induced by shuffling IC1 was the largest 100 ms after the feedback, with a decrease in accuracy from 0.5605 ± 0.0050 (No shuffle) to 0.5022 ± 0.0033 (IC1 shuffled).
To corroborate these results, we supplemented them with additional ERD/ERS analyses focused on the oscillatory configurations that were found by the mixed model to be most relevant. Figure 5D shows group-averaged power profiles of activities in the IC1 (top row) and IC2 (bottom row) cortices. In each case, paired t tests were performed at each sample point (50 ms bins), and the number of trials was equalized between categories (B-Hit versus B-Miss trials) in each participant (mean ± SE of the number of trials for each category, 62.76 ± 4.98 and 63.81 ± 4.60 for IC1 and IC2, respectively).
As expected, independent of movement outcome (i.e., for both trial categories), in the IC1 increased power in the theta band was observed shortly before and during movement execution, whereas power in the low-beta and high-beta bands was reduced before and during movement and showed a clear rebound after its completion. Contrasting the power profiles for the two categories of trials confirmed that theta power (Fig. 5D, left column) was significantly enhanced (with FDR correction for multiple comparisons along the time axis) in B-Miss trials compared with B-Hit trials at a very short latency (−200 to 300 ms). Furthermore, the average high-beta profiles (Fig. 5D, right column) showed that the two opposite modulations revealed by the linear model coincided with the end of the desynchronization period (200–400 ms) and the postmovement beta rebound (750–1000 ms), respectively.
Activity in IC2 was more weakly modulated by movement execution than in the IC1. Theta power profiles exhibited hardly any fluctuation related to movement execution (movement onset on average 200.2 ± 8.5 ms before the outcome feedback), and desynchronization and postmovement beta rebound in low- and high-beta ranges were substantially weaker in the parietal region than in the frontal region. A two-way repeated measures ANOVA run on the power peak-to-peak values revealed a significant effect of the cortical region (F(1,17) = 11.591, p = 0.0034), with larger modulations (peak-to-peak values) for IC1 than for IC2 (IC1, 59.127 ± 8.417; IC2, 39.468 ± 5.655% power change). No main effect of the frequency bands (F(2,34) = 1.2535, p = 0.2984) or interaction effect (F(2,34) = 2.1535, p = 0.1316) was found. Yet, although both beta frequency sub-bands showed no sensitivity to reward and/or outcome feedback, parietal theta power increased significantly during movement (150–500 ms) in B-Hit trials compared with B-Miss trials.
In summary, oscillatory activity in the IC1 contributed most to separating B-hit trials from B-miss trials. Theta power was higher for failed trials (B-Miss) than for successful trials (B-Hit). More critically, we found that in this region activity in the high-beta range was affected differently depending on the temporal stage in the trial. Shortly after the target explosion, high beta exhibited a brief but robust increase (∼200 ms) for B-Hit trials compared with B-Miss trials, whereas at a longer latency, coinciding with the postmovement rebound, high-beta power was greater for failed trials (B-Miss) than for successful (B-Hit) trials for an extended period. In contrast, activity in the IC2 region was modulated only in the theta range, with enhancement for successful trials peaking shortly after theta power in the IC1 had reached its maximum for failed trials.
Catch versus NR-miss trials
In the previous contrast, we classified trials that were all performed under the same unperturbed conditions but had different task outcome. Here, we aimed to isolate the response to the large kinematic error induced by the unexpected visual rotation from that to the task outcome (as identified in the previous section). To this end, we contrasted the Catch and NR-miss trials in which the kinematics of the movement differed considerably but resulted in the same outcome, task failure. (Single trials of the two categories by one participant (S02) are plotted for illustration purposes in Fig. 6C.) The results of the pairwise separation are shown in Figure 6A. Pairwise classification accuracy and the p value for the model as a whole are shown by the blue and black dashed lines, respectively. The same measures for models computed on a smaller dataset (see below) are plotted in light blue and gray dashed lines, respectively. The best prediction was obtained for the time window centered at ∼100 ms (60.8% accuracy) relative to the feedback.
Pairwise classification and power profiles for the trial categories Catch versus NR-miss. A, Accuracy of the prediction of the trial categories, Catch versus NR-miss, based on the actual data (dashed blue line) and the shuffled data (dashed green line) for each model, computed at each 100 ms step, with input data taken from a 300-ms-wide time-window. Red dots indicate the models for which the accuracy was significantly higher (p < 0.01, two-tailed t test, Bonferroni corrected) when using the actual data relative to using the same but shuffled data. The p values associated with the overall models are overlaid (dashed black line). Also overlaid are the prediction accuracy (light blue and green dashed lines) and the fit (gray black line) achieved by the linear models based on the reduced data (see above, Materials and Methods). B, Regression coefficients for each of the 16 predictors of the models, 4 ICs (IC1–IC4) * 4 frequency bands (theta, alpha, low beta, high beta). Significant coefficients (p < 0.01, two-tailed t test) are highlighted in red boxes. Coefficients significant according to a threshold (α = 0.06) adjusted for the size of the dataset are highlighted in yellow dotted boxes. Time = 0 corresponds to the time of the outcome feedback. C, Illustrative hand paths of single trials of the two categories by one participant (S02). D, Group averaged topographies together with power profiles for the IC1 (top row) and the IC2 (bottom row) aligned to the task outcome feedback. Power profiles obtained by averaging the individually selected ICs and frequency bands separately for the trial categories, Catch (red) and NR-miss (blue-gray). The period during which power differed significantly (p < 0.05, two-tailed t test; FDR corrected) between trial categories is indicated in gray. Small black dots indicate sampling point of uncorrected significant difference.
The weights and statistical significances of all predictors are shown in Figure 6B. Statistically significant (p < 0.01) coefficients are highlighted in red boxes. In addition, coefficients estimated using a smaller dataset (see below) with p values <0.06 are highlighted in dashed yellow boxes. This threshold (α = 0.06) was defined to take into account the difference in the size of the dataset used for the current contrast, Catch versus NR-miss, and the following one, RS-hit versus NR-hit (see below). It was determined using Cohen's d effect size coefficient (|dmin| = 0.061) yielding p < 0.01 for the classification Catch versus NR-miss trials. Low beta in IC2 contributed significantly to the separation of trial categories in time windows extending, respectively, from 400 to 500 ms, 900 to 1200 ms, and 1600 to 1800 ms relative to feedback. Alpha activity in the same region also contributed significantly for prolonged periods ranging from 500 to 700 ms and 1300 to 2200 ms. In addition, at a very long latency, high-beta activity in the IC1 also weighted significantly (500–600 ms and 1900–2200 ms). In all cases, the coefficients corresponded to lower power in Catch trials than in NR-miss trials.
Furthermore, at a short latency, low-frequency theta modulations in the two medial regions (IC1 and IC2) weighted significantly, this time with higher power in Catch-miss trials than in NR-miss trials. These observations suggested that oscillatory activities in the two medial regions (IC1 and IC2) contributed most to the separation of the Catch trials from the NR-miss trials. We tested this by assessing how shuffling the data corresponding to the different ICs affected prediction accuracy (see above, Materials and Methods). At each time point (from −0.5 to 2.5 s relative to feedback; 31 time points – 100 ms steps), we computed the median of the prediction accuracies computed for the 200 training folds run for each of the different models—IC1 shuffled, IC2 shuffled, IC3 shuffled, IC4 shuffled, and No shuffle. A repeated-measures ANOVA run on these values was found to be statistically significant (F(4,30) =17.4954, p < 0.0001). Post hoc pairwise comparisons (with Bonferroni-corrected p values) revealed that shuffling the IC1 or IC2 data induced a significant drop in prediction accuracy (IC1 shuffled vs No shuffle, t(30) = −4.5335, p = 0.0009; IC2 shuffled vs No shuffle, t(30) = −6.8677, p < 0.0001), whereas shuffling the components of IC3 or IC4 did not have a significant impact on prediction accuracy assessed globally over the trial time course (IC3 shuffled vs No shuffle, t(30) = −2.3028, p = 0.2840; IC4 shuffled vs No shuffle, t(30) = −2.0578, p = 0.4839). The drop induced by shuffling the IC1 components was the largest at 0 ms after the feedback, with a decrease in accuracy from 0.5991 ± 0.0038 (No shuffle) to 0.5438 ± 0.0047 (IC1 shuffled). The drop induced by shuffling the IC2 components was the largest at 1100 ms after the feedback, with a decrease from 0.5507 ± 0.0038 (No shuffle) to 0.5161 ± 0.0046 (IC2 shuffled).
The corresponding ERD/ERS analyses conducted on activities in the IC1 and IC2 cortices, again the most relevant, are presented in Figure 6D (mean ± SE of the number of trials for each category, 61.14 ± 2.09 and 61.52 ± 2.05 for IC1 and IC2, respectively). In both regions, the general patterns of movement-related fluctuations in the different frequency bands were the same as those described above for the unperturbed trials of the Baseline blocks. In response to the unexpected introduction of the visual rotation, activities in both medial regions were significantly affected but in a clearly different manner.
In the IC1 region, a large modulation was found in the theta range (Fig. 6D, left column), with greater theta power in Catch trials compared with NR-miss trials around the feedback time (−200 to 250 ms). In the same region, a modulation was also observed at a very long latency (1950–2100 ms) with a significant decrease in the high-beta range for the Catch trials. In the IC2, oscillations exhibited sensitivity to the sudden introduction of the visual rotation in all frequency ranges (alpha band not shown) except in the high-beta band. In the theta range, the power modulation pattern was similar to that observed for the IC1, with a significant increase around the feedback time (−150 to 350 ms) followed by a significant decrease (650–800 ms). In contrast, whereas low-beta activity in IC1 remained unaffected by the sudden perturbation, a lasting effect was seen in IC2, with a significant low-beta decrease extending from 300 to 2000 ms after feedback.
To sum up, oscillatory activities in the two medial regions (IC1 and IC2) contributed most to the separation of the Catch trials from the NR-miss trials, clearly showing sensitivity to the unexpected (re)introduction of the visual rotation. In the theta range, the visual perturbation induced similar fluctuations in both regions. However, the effects on other frequency bands clearly differed between frontal and parietal medial cortices. In the IC2, long-lasting effects were found on alpha and low-beta power, whereas high-beta oscillations remained unaffected. In contrast, in the IC1, only high-beta power showed some response at very long latency. Oscillatory activities in the lateral regions (IC3 and IC4) did not contribute significantly to the trial classification.
RS-hit versus NR-hit trials
We supplemented the previous results with a third set of analyses to resolve an important ambiguity. When a visual rotation is introduced in an unexpected manner, participants typically experience two qualitatively different error signals, a clear movement deflection and a sensory-prediction error. To determine whether the effects we found for the contrast Catch trials versus NR-miss trials were because of the sensory-prediction error driving implicit sensorimotor adaptation (updating of the forward internal model), we pairwise separated RS-hit and NR-hit trials. Indeed, in the RS-hit trials participants experienced sensory-prediction error but no kinematic error. Here, one has to keep in mind that sensory-prediction error and kinematic error are not the same. Some kinematic errors do not entail sensory-prediction error (Diedrichsen et al., 2005; Torrecillos et al., 2015), and conversely sensory-prediction error can be elicited without kinematic error, as in our manipulation and other studies (Mazzoni and Krakauer, 2006; Taylor and Ivry, 2011). (Single trials of the two categories by one participant (S02) are plotted for illustration purpose in Fig. 7C.) As can be seen in Figure 7A, prediction accuracy was poor for this classification, especially from 1000 ms onward, that is, during the postmovement beta rebound period. There was also a drop in the fit of the model (compared with a null model) during this period, making interpretations based on the estimated regression coefficients less reliable. In Figure 7B, the significant coefficients (highlighted in yellow dotted boxes) are indicated relative to an adjusted threshold (α = 0.06) to take into account the different size of the dataset (total number of trials 1190 and 2236 for the comparisons RS-hit vs NR-hit trials and Catch vs NR-miss, respectively). As indicated above, the training and testing folds were built with a balanced number of trials from both categories for each subject.). The adjusted threshold was determined using Cohen's d effect size coefficient (|dmin| = 0.061) yielding p < 0.01 for the classification Catch versus NR-miss trials. Short-lived and limited contributions by theta and low- and high-beta activities in IC2, and high-beta power in IC3, with power decreased in the NR-hit trials relative to the RS-hit trials showed up.
Pairwise classification and power profiles for the trial categories RS-hit versus NR-hit. A, Accuracy of the prediction of the trial categories RS-hit versus NR-hit based on the actual data (dashed blue line) and based on the shuffled data (dashed green line) for each model, computed at each 100 ms step, taking as input data from a 300-ms-wide time-window. Red dots indicate the models for which the accuracy was significantly higher (p < 0.01, two-tailed t test, Bonferroni corrected) when using the actual data relative to using the same but shuffled data. The p values associated with the overall models are overlaid (dashed black line). B, Regression coefficients for each of the 16 predictors of the models, 4 ICs (IC1–IC4) * 4 frequency bands (theta, alpha, low beta, high beta). Significant coefficients (p < 0.06, two-tailed t test; threshold adjusted for the size of the dataset) are highlighted in yellow dashed boxes. Time = 0 corresponds to the time of the outcome feedback. C, Illustrative hand paths of single trials of the two categories by one participant (S02). D, Group averaged topographies together with power profiles for the IC1 (top row) and IC2 (bottom row) aligned to the task outcome feedback. Power profiles obtained by averaging the individually selected ICs and frequency bands separately for the trial categories, RS-Hit (green) and NR-Hit (blue-gray). The period during which power differed significantly (p < 0.05, two-tailed t test; FDR corrected) between trial categories is indicated in gray. Small black dots indicate sampling point of uncorrected significant difference.
In other words, the differences in the outcome patterns observed for the two contrasts cannot be attributed to variations in the sizes of the datasets used (2236 and 1190 trials, respectively). Let's briefly recall how this was demonstrated. When we used a significance threshold of α = 0.01 to determine which coefficients significantly influenced the decision of the classifier, we identified several coefficients for the Catch versus NR-miss trials contrast (Fig. 7B, highlighted in red). However, for the RS-hit versus NR-hit trials contrast, no significant coefficients were found. Nonetheless, it remained possible that the differences in result patterns were linked to variations in dataset sizes (i.e., more power for the contrast Catch versus NR-miss trials than for the contrast RS-hit versus NR-hit trials).
To rule out this possibility, for the contrast Catch versus NR-miss trials, we completed our results using a dataset of the exact same size as the one available for the contrast RS-hit versus NR-hit trials with a significance threshold adjusted according to Cohen's d effect size coefficient, which was α = 0.06. For the contrast RS-hit versus NR-hit trials, in the same way, we considered the adjusted threshold α = 0.06. In the case of the RS-hit versus NR-hit trials contrast, even with the adjusted threshold (α = 0.06), the outcome pattern noticeably diverged from what was observed in the Catch versus NR-miss contrast. Specifically, during the postmovement beta rebound period, the low-beta activity in the IC2 did not significantly contribute to the differentiation between trial types. This stands in contrast to the Catch versus NR-miss contrast, where this effect remained robust even after the sample size was reduced to 1190 trials, resulting in a reduction in statistical power.
Figure 7D presents the average power profiles for the two trial categories, computed from the data of 15/21 participants who got at least 20 RS-hit trials (mean ± SE of the number of trials for each category, 44.13 ± 3.54 and 42.53 ± 3.66 for IC1 and IC2, respectively). Consistent with the results of the linear model, no significant effect of the trial category was observed during the postmovement beta rebound. Similarly, no significant theta-band short-latency modulations were found in the medial frontal cortex.
As indicated, the dataset used for the present contrast was smaller than the one available for the contrast Catch versus NR-miss, for which we found a significant decrease of the postmovement beta rebound in the Catch trials relative to the NR-miss trials. Therefore, we recomputed the linear models for this latter pairwise classification including exactly the same number of trials (i.e., 1190 trials). The prediction accuracy and the fit achieved by the linear models based on this reduced dataset are presented in Figure 6A (light blue and light green dashed lines, for the actual and shuffled data, respectively). One can see that reducing the size of the dataset had a modest effect on the prediction accuracy and that the overall fit of the model remained nearly unchanged.
We also recomputed the average power profiles for the contrast Catch versus NR-miss with a dataset identical in size with the one used for the contrast RS-hit versus NR-hit trials. The general pattern of modulation remained unchanged, with a significant short-latency theta-power increase and a significant decrease of alpha and low-beta power in the IC2 during the postmovement period (data not shown) for the Catch trials compared with the NR-miss trials.
In brief, when contrasting the RS-hit versus NR-hit trials, we found no significant decrease of the postmovement alpha or low-beta rebound for the RS-hit trials in which participants experienced sensory-prediction error but no kinematic error relative to the NR-hit trials.
Discussion
Our hypothesis was that careful distinction of the space, time, and frequency of the oscillatory activities may be the key to reconcile apparent discrepancies in the literature. We report findings that help give insight into a more unified view. First, within a single task, elicited by the same outcome feedback (hit or miss), we found two opposite high-be modulations in the medial frontal cortex, each akin to the seemingly incompatible responses reported in separate studies (Cohen et al., 2007; 2009; Koelewijn et al., 2008; Marco-Pallares et al., 2008a; Marco-Pallarés et al., 2008b, 2009; Doñamayor et al., 2011, 2012; De Pascalis et al., 2012). Second, although medial frontal high-beta activity was sensitive to the outcome feedback, low-beta power in the medial parietal cortex was strongly attenuated following salient kinematic error but was not affected by either task outcome or sensory-prediction error.
Bidirectional modulations in the medial frontal cortex elicited by task outcome feedback
In response selection tasks, opposite beta-band modulations in the medial frontal cortex have led De Pascalis et al. (2012) to conjecture that “the apparent discrepancies in the feedback valence effects among studies may be because of differences in task demands.” We found bidirectional beta-band modulations in this region when comparing successful and failed trials (B-Hit vs B-Miss).
Shortly after the target explosion, high-beta power was stronger for a brief period for successful trials, which might be attributed to the rewarding aspect of the positive feedback. Participants reported that it was pretty satisfying to watch and hear the target exploding, an aspect well identified by video game designers. High-beta-band response to reward has been extensively studied by Marco-Pallarés et al. (2015). Although mostly using gambling tasks in which no learning is involved, they propose that it reflects a mechanism for monitoring unexpected positive outcomes in learning-related contexts. Selectively sensitive to salient and novel positive events in the environment, it would mediate interactions between attentional and emotional systems and would be influenced by subcortical activity constituting a key mechanism in frontostriatal coupling (Marco-Pallarés et al., 2015). On the other hand, as expected, theta power was enhanced in the failed trials (Cavanagh and Frank, 2014). Indeed, it has been proposed that the two frequency bands have complementary roles in reinforcement learning. Although increased theta power encoding negative feedback may weaken erroneous stimulus–response associations, enhanced beta power may bolster correct/rewarding associations (Cohen et al., 2011). In contrast to this first beta response, the beta rebound was higher in failed trials than in successful trials. An analogous modulation was reported by Koelewijn et al. (2008) in a task where participants made fast-paced/quick button presses based on cues. According to them, stronger rebound reflects an active response inhibition that typically follows the detection of an erroneous action (Ridderinkhof et al., 2004). Based on similar observations, Marco-Pallarés et al. (2008b) reached similar conclusions.
Here, high-beta power in erroneous trials coincides with the postmovement beta rebound. Diesburg and Wessel (2021) adapted the two-stage model of action stopping for humans, originally developed by Schmidt and Berke (2017) for rodents. In this model, a short-latency salience-related Pause process is triggered whenever a task relevant stimulus elicits an attentional reorientation, whereby motor inhibition is exerted globally. In contrast, a long-latency stop-specific Cancel process is deployed only when outright stopping is appropriate in the task context. Although underpinned by two complementary basal ganglia pathways, beta activity may be a signature of the Pause and Cancel processes as it relates to both early, nonselective, corticospinal excitability suppression and the later deployment of selective inhibition (Lavallee et al., 2014).
One may have noticed the similarities between the events susceptible to trigger the Pause process and those expected to elicit the high-beta response thoroughly studied by Marco-Pallarés et al. (2008b). In fact, the only difference may well lie in their valence. This means that given the salience of the target explosion the short latency transient increase in beta power may be viewed as a reflection of a Pause process, which, however, in our case, would not be followed by a Cancel process. On the other hand, the data presented by Koelewijn et al. (2008) and Marco-Pallarés et al. (2008b), as well as our own, suggest that the Cancel process can, symmetrically, be decoupled from the Pause process. However, in trying to decide between these two interpretations, one might stumble on the shared neural features of novelty and reward processing (Bunzeck et al., 2010; Steiger et al., 2022).
Low-beta activity in medial parietal cortex is modulated by salient kinematic error
In motor adaptation tasks, the postmovement beta rebound is attenuated following kinematic errors induced by external perturbations (Tan et al., 2014; Torrecillos et al., 2015). Here, the unexpected (re)introduction of the visual rotation also invariably led to task failure. To identify activities sensitive to kinematic error independent of task outcome, we contrasted trials that clearly differed in kinematics but resulted in the same negative task outcome (Catch vs NR-miss trials). Postmovement rebound was significantly attenuated in alpha and low-beta bands in the medial parietal cortex, that is, in a frequency range and spatial location clearly distinct from those of the task outcome-sensitive beta activity discussed above.
Although the frequency range of this response is consistent with the report by Tan et al. (2014), who discovered this error-related response, the spatial location may seem at odds with it. These authors analyzed the reduction of the postmovement beta rebound at electrodes located over the sensorimotor cortex contralateral to the moving hand (C3, Cz), at which they recorded the largest beta rebound. However, the site of its maximum amplitude may not coincide with the location of its maximum modulation. Here, high beta in the medial frontal region exhibited the largest rebound, which was, however, not affected by kinematic error. Moreover, in sensor space, activations from different sources are mixed. In Alayrangues et al. (2019), through ICA, we found (surprisingly) that the attenuation of the beta rebound visible contralaterally in sensor space was actually because of a modulation in the medial parietal cortex.
To establish which processes are reflected by this modulation, it is necessary to determine the nature of the error signal that causes it, the clearly perceived movement deviation or the sensory-prediction error. To solve this issue, we taught the participants how to counteract the visual rotation to obtain trials in which they experienced sensory-prediction error (Mazzoni and Krakauer, 2006; Taylor et al., 2014) but no uncontrolled movement deviation. Compared with trials performed without rotation (RS-hit versus NR-hit), no decrease in beta power was visible in these trials. This complements a previous finding. In Torrecillos et al. (2015) we demonstrated that sensory-prediction error was not necessary to cause a decrease in the postmovement beta rebound. Here, we show that sensory-prediction error is not sufficient to cause it. Therefore, we attribute the decrease of the postmovement beta rebound to the clearly perceived movement deviation, triggering an orienting response and cognitive processes. Furthermore, one may note that the same reasoning applies to the ample theta-band modulations elicited in the Catch trials, which also vanished when the large kinematic errors were suppressed.
We dipole localized the medial parietal activity in the precuneus. Compared with medial frontal areas, this region, activated under a wide range of cognitive demands, has long remained notoriously enigmatic. Nevertheless, there is now consensus that it constitutes a major connectivity hub (Cavanna and Trimble, 2006; Tomasi and Volkow, 2010, 2011; van den Heuvel and Sporns, 2013; Utevsky et al., 2014; Popov et al., 2018; Lyu et al., 2021), with an exceptionally high metabolic rate (Raichle et al., 2001). Specifically, it has recently been proposed that as the posteromedial hub of the default-mode network (DMN), it plays a key role in integrating external information with internal representation (Lyu et al., 2021). The idea has emerged that although widely considered responsible for internally oriented cognition the DMN is also engaged during goal-directed tasks (Spreng et al., 2014; Elton and Gao, 2015; Vatansever et al., 2015).
The decrease in alpha and low beta that we observed is consistent with experimental and theoretical work suggesting that alpha/beta desynchronization is essential to the capacity of the neocortex to represent information (Hanslmayr et al., 2016, 2019). Specifically, that neural desynchronization in the alpha/beta band is positively related to the richness of represented information (Hanslmayr et al., 2012).
Limitations and open questions
Although we believe our results help with discerning the different beta-band responses described in the literature more coherently, there is still some way to go before we have a comprehensive view. Evidently, given the critical role of subcortical activities, namely, in reward and sensory-prediction error processing, a unified picture is difficult to obtain solely based on cortical surface EEG recording. The absence of information on subcortical activity in our linear models may explain in large part the limited level of prediction accuracy achieved.
Finally, here, we have considered oscillations occurring at different locations, times, and frequencies as different independent variables. However, they may be jointly comodulated within modes at the system level of oscillatory dynamics (Atasoy et al., 2016). In this view, individual oscillatory processes could not be considered as independently reflecting different aspects of sensorimotor behavior as they would be intrinsically coupled. On the contrary, their precise sequence of occurrence, coordinated through space, time, and frequency, would arise as a signature of the trajectory of the collective system being adjusted to the task being performed and adapted in response to external feedback.
Footnotes
This work was supported by Agence Nationale de la Recherche Grant ANR-18-CE37-0018-01. We thank Amirhossein Jahani for data collection and Julie Alayrangues and Flavie Torrecillos for participating in pilot data recording.
The authors declare no competing financial interests.
- Correspondence should be addressed to Nicole Malfait at nicole.malfait{at}univ-amu.fr or Antoine Schwey at antoine.schwey{at}univ-amu.fr
