Abstract
The ability to adapt behavior after erroneous actions is one of the key aspects of cognitive control. Error commission typically causes people to slow down their subsequent actions [post-error slowing (PES)]. Recent work has challenged the notion that PES reflects adaptive, controlled processing and instead suggests that it is a side effect of the surprising nature of errors. Indeed, human neuroimaging suggests that the brain networks involved in processing errors overlap with those processing error-unrelated surprise, calling into question whether there is a specific system for error processing in the brain at all. In the current study, we used EEG decoding and a novel behavioral paradigm to test whether there are indeed unique, error-specific processes that contribute to PES beyond domain-general surprise. Across two experiments in male and female humans (N = 76), we found that both errors and error-unrelated surprise were followed by slower responses when response–stimulus intervals were short. Furthermore, the early neural processes following error-specific and domain-general surprise showed significant cross-decoding. However, at longer intervals, which provided additional processing time, only errors were still followed by post-trial slowing. Furthermore, this error-specific PES effect was reflected in sustained neural activity that could be decoded from that associated with domain-general surprise, with the strongest contributions found at lateral frontal, occipital, and sensorimotor scalp sites. These findings suggest that errors and surprise initially share common processes, but that after additional processing time, unique, genuinely error-specific processes take over and contribute to behavioral adaptation.
SIGNIFICANCE STATEMENT Humans typically slow their actions after errors (PES). Some suggest that PES is a side effect of the unexpected, surprising nature of errors, challenging the notion of a genuine error processing system in the human brain. Here, we used multivariate EEG decoding to identify behavioral and neural processes uniquely related to error processing. Action slowing occurred following both action errors and error-unrelated surprise when time to prepare the next response was short. However, when there was more time to react, only errors were followed by slowing, further reflected in sustained neural activity. This suggests that errors and surprise initially share common processing, but that after additional time, error-specific, adaptive processes take over.
Introduction
Behavioral adjustments after errors are a hallmark of cognitive control (Mars et al., 2005; Danielmeier and Ullsperger, 2011; Ullsperger et al., 2014; Wessel, 2018). The most well established of these post-error adjustments is post-error slowing (PES) of reaction times (Laming, 1968; Rabbitt and Rodgers, 1977). However, the nature, function, and neural origin of PES remain controversial (Notebaert et al., 2009; Purcell and Kiani, 2016; Ullsperger and Danielmeier, 2016; Wessel, 2018). The traditional view of adaptive error processing theories is that PES reflects a strategic adjustment of subsequent behavior, either because of improved performance monitoring (King et al., 2010; Cavanagh et al., 2011), extended perceptual processing (Laming, 1968), motor inhibition (Guan and Wessel, 2022; Ridderinkhof et al., 2002; Ridderinkhof, 2002), or a fine-tuned speed-accuracy trade-off (Botvinick et al., 2001). Indeed, some studies report that PES is related to an increase in post-error accuracy, providing support for adaptive theories (Forster and Cho, 2014; Hajcak et al., 2005; Ullsperger and Szymanowski, 2004). In contrast, more recent maladaptive theories have suggested that PES may not be a functional behavior at all. According to these theories, PES reflects impaired behavior, for example, because of a processing bottleneck (Jentzsch and Dudschig, 2009) or because of an orienting response that diverts attention from the primary task (Houtman and Notebaert, 2013; Notebaert et al., 2009).
The behavioral and neural overlap between error and surprise processing provides supporting evidence for the maladaptive theories. For example, when errors are the expected action outcome, behavioral slowing takes place after correct responses (Notebaert et al., 2009). In line with the proposal that PES is merely a side effect of a surprise-related orienting response, trial-to-trial behavioral modeling has supported the fact that both PES and the slowing of actions after surprising stimuli share a common underlying process (Parmentier et al., 2019). On the neural level, this is supported by findings of shared neural activity between error and surprise processing, which is observable in both scalp EEG (Gentsch et al., 2009) and fMRI (Wessel et al., 2012). Together, such findings suggest that a much more domain-general process takes place after action errors—one that is, in fact, not driven by the erroneous nature of the action at all (Alexander and Brown, 2011).
As such, it is an open question whether there is a genuine error processing system in the human brain, or whether brain activity after errors is merely driven by the surprising nature of errors and thus reflects domain-general surprise processing (Alexander and Brown, 2011; Houtman and Notebaert, 2013; Notebaert et al., 2009; Wessel et al., 2012). Accordingly, PES might merely reflect post-surprise slowing, making the term a misnomer. Therefore, here we aimed to compare error and surprise processing using EEG-based neural decoding techniques in a novel paradigm that allowed us to distinguish behavioral and neural processes uniquely related to error processing from those that are also found after domain-general surprise.
Our predictions were motivated by the adaptive orienting theory of error processing (AOT; Wessel, 2018), a theoretical framework that seeks to bridge the gap between adaptive and maladaptive theories on error processing (Fischer et al., 2018; Purcell and Kiani, 2016). The AOT makes explicit predictions regarding the role of domain-general surprise processing after errors, as well as potentially error-specific processes. According to AOT, error processing occurs in two stages. The first is an automatic cascade of processes that involves inhibition and attentional orienting toward the source of the expectancy violation. That stage is purportedly shared between error processing and the processing of domain-general surprise. Second, in the case of errors, a controlled processing cascade then follows, which involves slower and more deliberate processes geared toward tuning the task set and improving subsequent behavior.
Based on these two purported stages, the AOT also makes specific predictions regarding the origin of PES; specifically, it proposes that PES can have different contributing causes. First, if a motor response is required while the initial automatic cascade of processes is active, PES will be mostly because of inhibition and distraction caused by the error (prediction taken from Notebaert et al.'s, 2009, orienting account of PES; see above). This would be the case on trials with a short response–stimulus interval (RSI) between the erroneous response and the next response-imperative stimulus, that is, when the next stimulus has to be processed during the initial inhibition-orienting phase. However, at sufficiently long RSIs, which allow the surprise-related inhibition-orienting phase to subside, the AOT holds that any remaining PES is because of error-specific strategic processes.
Here, we aimed to test whether there are unique operations related to error processing that are not shared by the processing of domain-general surprise. In addition, we aimed to test the AOT hypotheses regarding the origin of PES. We designed a new experimental paradigm in which action-related expectation violations could be because of one of two reasons, namely, (1) an error or (2) an unexpected perceptual outcome of a correct response. We then manipulated the RSI (short, medium, and long) to test whether both errors and unexpected action outcomes of correct responses would be followed by action slowing at short RSI, whereas at longer RSIs, only errors would be followed by slowing.
In a first behavioral experiment, we confirmed this prediction of the AOT, suggesting there are unique error-specific processes that contribute to PES over and above domain-general surprise. In a second experiment, we replicated this finding and identified unique neural processes that contributed to the sustained slowing observed after errors via multivariate pattern analysis (MVPA) of scalp EEG.
Materials and Methods
Participants
For experiment 1, 39 healthy young adults participated in the study (mean age, 18.39 years; SD = 0.67; 1 left-handed, 31 females). For experiment 2, 38 healthy young adults participated. All participants were recruited via the university online scheduling system for research in psychology (SONA) and via university mass e-mails. We did not have influence on the demographics of the signups, leading to an imbalance with regard to sex in our final convenience sample. One participant was excluded from the analysis because of a technical problem during the EEG recording session, leaving 37 participants (mean age, 21.11 years; SD = 3.38; three left-handed, 22 females). Participants were either paid $15 per hour or received course credit for their participation. All participants had normal or corrected-to-normal vision and did not report any psychological medications. Before the study session, they signed written informed consent forms. The study was approved by the ethics committee at the University of Iowa (#201511709) and conducted in accordance with the Declaration of Helsinki.
Stimuli and experimental paradigm in experiments 1 and 2
Each trial in the experiment was a combination of a flanker task and a novelty-oddball task, based on the paradigm from (Wessel et al., 2012). On each trial, participants were presented with an arrow version of the Eriksen flanker task (Eriksen and Eriksen, 1974). Stimuli included left and right arrows (< and >, respectively). Each trial started with a presentation of five arrow stimuli in the middle of the screen (Fig. 1). Participants were asked to respond to the direction of the arrow in the center (e.g., the target arrow) while ignoring the surrounding four arrows on the screen (e.g., the flanker arrows). Participants were instructed to press the Q key with their left index finger in response to left arrows and the P key with their right index finger in response to right arrows on a standard QWERTY keyboard. In congruent trials, the direction of the target arrow and the flanker stimuli were identical (e.g., ⋘≪ or ⋙≫), whereas in incongruent trials, the target arrow and the flanker indicated incompatible responses (e.g., ≪>≪ or ≫<≫). Congruent and incongruent stimuli were equiprobable.
Participants were asked to make a response within a response window. If participants failed to respond in time, the message FASTER! in a burgundy color was presented in the middle of the screen for 900 ms, followed by a blank screen for 1100 ms in experiment 1 and 1500 ms in experiment 2. In experiment 2, a fixation cross was presented in the middle of the screen for 800 ms before the next trial began. Otherwise, immediately after the response was made in time, one of three stimuli—standard, target, or a novel image—was presented in middle of the screen. On most trials, participants were given a standard upward-pointing triangle as a visual action outcome (i.e., standard action outcome). Participants were told this was a feedback of their button press, independent of the correctness of their response. Furthermore, they were instructed that on some occasions (specifically, on two congruent correct trials randomly selected within a block, although this was not divulged to the participants), a downward-pointing triangle (i.e., the target) would be presented in the middle of the screen. Participants were instructed to signal this occurrence by pressing the space bar. If they responded to the target, they were given the feedback “Correct!” If they missed the target, the feedback “You took too long to respond!” would be presented to them for 1200 ms followed by a blank screen for 1700 ms in experiment 1 and 1200 ms in experiment 2. In experiment 2, a fixation cross was presented in the middle of the screen for 800 ms before the next trial began. The purpose of this instruction was to ensure that participants should monitor the visual outcome of their action closely on every trial. The key factor was that on some correct trials (matched in frequency to each participant's error rate), a novel image was unexpectedly presented (i.e., surprising action outcome). These stimuli were chosen from Snodgrass and Vanderwart's (1980) picture set. Emotionally evocative stimuli (e.g., insects) were excluded from a set of images. Images were black-and-white silhouettes and were comparable in size with each other and the size of the upward/downward triangle. These surprising trials were dynamically matched to each participant's performance, in that (1) the frequency of surprising action outcome was matched to the error rate of a participant and (2) the congruency of novel and erroneous trials as well as that of the previous and the next trials were fully matched. This was done to match the potential surprise associated with the surprising action outcome with that of an error. Thus, there were four types of trials in the task—correct trials with standard action outcome (i.e., standard trials), erroneous trials with standard action outcome (i.e., action error trials), correct trials with surprising action outcome (i.e., surprise trials), and rare congruent correct trials with a target outcome (i.e., target). The standard (correct), action error, and surprise trials were our interests in the analysis. The surprising action outcome was presented for 200 ms, whereas the standard action outcome was presented for 300 ms in experiment 1. In experiment 2, both standard and surprising action outcome were presented for 300 ms. The action outcome was embedded with three RSIs, 350 ms (short), 1700 ms (medium), and 3000 ms (long), followed by a blank screen throughout the intervals.
In experiment 1, participants performed up to two blocks of 36 trials before the main task. In experiment 2, the practice was shortened to one block of 24 trials because of the elongated study session. In the main task, a block consisted of 72 trials each. Between blocks, participants were given a summary of their performance in the previous block, consisting of the mean response time (RT; ms), the error rate (percentage), miss rate (percentage), and the number of successful target detections. Each block was separated by a self-paced break. To dynamically match the number of surprise trials to the error trials, the participants had to undergo a varying number of trials; experiment 1 had 804.923 trials (SD = 60.777; range, 10–12 blocks), experiment 2 had 1103.73 trials (SD = 67.821; range, 13–16 blocks). This was because of the matching of the frequency of surprise trials with the error rate; if at the end of the experiments, surprise trials still had to be presented to match the number of errors, additional trials had to be appended (the surprise trial itself, embedded in a sequence of nonsurprise trials to match the error rate). If additional errors were made in that time period, this process was reset until the number of errors and surprise trials, as well as their respective rates, were successfully matched. This could take longer in some participants and with some discrepancy in the congruency ratio compared with others. The dynamic matching the frequency of error and surprise trials was effective in both experiments as revealed in Pearson correlations (experiment 1, r = 0.987, p < 0.001; experiment 2, r = 0.988, p < 0.001).
Throughout the task, a response window was adaptively adjusted to participant performance to induce sufficient erroneous trials. The response window for the next block decreased by 25 ms if both error rate and miss rate were under 10% in experiment and by 20 ms if the error rate was under 13% and the miss rate was under 10% in experiment 2. If the error rate exceeded 25%, or the miss rate exceeded 10%, the response window for the next block increased by 25 ms. In addition to this blockwise adjustment, the response window in experiment 2 was also adjusted within each block. (However, this adjustment did not make any difference in participants' overall error rates between the two experiments, t(74) = −0.33, p = 0.743, independent samples t test.)
All participants performed a task in a dim experimental room, and stimuli were presented on a white screen. Experiment 1 was programmed with JavaScript, CSS, jQuery, and PHP programming languages, whereas experiment 2 was programmed with MATLAB version 2017b and Psychtoolbox-3 (Brainard, 1997) to enable EEG recording.
Behavioral analyses
Anticipatory responses (<200 ms), target trials, and miss trials (RT greater than response window) were excluded from the analysis. This led to overall 10.144% and 8.776% of exclusion rate in experiment 1 and 2, respectively. Only incongruent trials were included in the analyses since congruent errors (and hence, congruent surprise trials) were very rare. We first report the incongruent error rate (percentage) as well as the mean RT differences between trial types. For the examination of post-trial action slowing, trial triplets (N − 1, N, N + 1 trials) were considered, in which the N − 1 (pre-event response) and N + 1 (postevent response) trials were always correct. The correct pre-event and postevent trials were always followed by a standard action outcome. This method avoids any confounds from trials preceding the trials of interest. The post-trial action slowing was calculated following two expectancy violations, action errors (i.e., PES) and surprising action outcome [i.e., postsurprise slowing (PSS)] at three RSIs, respectively. It was quantified in percentage change from postcorrect RT as follows: Post-trial action slowing (%) =
This enabled us to normalize each behavioral adjustment index and to facilitate the comparison between PES and PSS at each RSI; Experiment1, triplets for action errors, 47.462 (SD = 19.766); surprise, 59.154 (SD = 22.919); experiment 2, triplets for action errors, 66.622 (SD = 23.07); surprise, 90.784 (SD = 30.102). To this end, the degree of post-trial action slowing (percentage) from each participant was subjected to a three-by-two repeated-measures ANOVA with RSI (short, medium, long) and expectancy violation (action error, surprise) as within-subjects factors. This was followed by planned paired-samples t tests (two sided) to see whether PES and PSS were comparable at each RSI. We further tested the degree of PES and PSS with one-sample t tests (two sided) at each RSI to see whether post-trial action slowing occurred. We also report post-trial action slowing calculated with the traditional nontriplet method (Schroder et al., 2020; Extended Data Fig. 2-1).
EEG recording (experiment 2)
EEG data were recorded using a 64-channel actiCHamp amplifier (Brain Products). Ground and reference electrodes were placed at AFz and Pz, respectively. EEG data were collected at a sampling rate of 500 Hz. Electrode impedance was kept at <10 kΩ throughout the recording session.
EEG preprocessing
Following data acquisition, EEG data preprocessing was conducted using custom MATLAB scripts and the EEGLAB toolbox (Delorme and Makeig, 2004). Individual data were imported and filtered with high-pass 0.3 Hz and low-pass 40 Hz filters. Nonstereotypical artifacts and noisy channels were visually inspected and removed from the data. The data were subsequently rereferenced to the common average and subjected to a temporal infomax independent component analysis (ICA; Bell and Sejnowski, 1995) with extension to sub-Gaussian sources (Lee et al., 1999). Before running ICA decomposition, rejected channels were interpolated using a spherical spline interpolation (Ferree, 2006). Independent components representing stereotypic artifact activity (e.g., ocular, muscle, electrode artifacts) were visually inspected together with the automated IC classifier embedded in EEGLAB (ICLabel; Pion-Tonachini et al., 2019) and removed from the data. The remaining components were backprojected into channel space to reconstruct artifact-free channel data and were subjected to further analyses.
Multivariate pattern analysis across time
We conducted a multivariate pattern analysis of the whole-scalp EEG activity using the MVPA-Light toolbox (Treder, 2020). The preprocessed individual data were downsampled to 250 Hz and segmented with an epoch of −300 to 3200 ms with respect to response onset to fully cover the longest RSI period (i.e., 3000 ms). Each epoch was baseline corrected with a period of −300 to −100 ms relative to the response to control for differences between error and correct trials (e.g., standard correct and surprise) that emerge during stimulus processing. As with behavior, only incongruent trials were included in the analysis. We identified the three trial types of interest (e.g., standard, action error, and surprise) with subsequent correct response trials and equated the number of trials for each trial type to avoid biasing the classifier in the following comparisons.
We trained and tested three classifiers for pairwise comparisons—action error versus standard, surprise versus standard, and action error versus surprise. To this end, linear discriminant analysis (LDA) was performed on the trial-to-trial data for each participant. The decoding for each time point used a sixfold cross-validation procedure; trials from each participant were randomly separated to six sets and the one set was held out as test data (1/6), whereas their remaining trial data (5/6) were used for training. This procedure was repeated six times until each group served as a testing dataset. We iterated this entire procedure five times, and the decoding performance from each procedure was then averaged, producing a time-resolved decoding performance vector. For this analysis, we only used trials from the medium and the long RSIs. Because participants did not know the length of the RSI, early period processing is identical across all three RSIs, and including both the medium and long period was a good compromise between trial count for inclusion and post-trial time period covered. Participants were only included in these analyses if they had at least 20 trials per condition for classification (N = 34, three exclusions). Overall, 65.147 (SD = 17.557; range, 26–99) trials per condition for each participant were included in the decoding analysis.
Decoding performance was calculated using the area under the curve (AUC). Each participant's AUC per time point was then submitted to paired-samples Wilcoxon signed-rank test against chance-level performance (0.5). The resulting p value was corrected for multiple comparisons using cluster-based permutation tests (1000 iterations, p < 0.05) as implemented in MVPA-Light toolbox (Treder, 2020; Maris and Oostenveld, 2007).
Testing for overlap between error-related and surprise-related processing
In addition to testing the different trial conditions for unique neural activity using the MVPA decoding technique mentioned above, we also aimed to replicate previous reports of overlapping neural activity between errors and surprise immediately after their occurrence (Gentsch et al., 2009; Wessel et al., 2012) using a cross-decoding analysis (e.g., generalization between two classifications). To this end, we first assembled two datasets for each participant, one containing action errors and a matched number of standard trials, and one containing surprise trials and a matched number of standard trials. A different set of standard correct trials was used for each classifier to not bias the analysis toward findings overlap. As we focused on the time period immediately after the response onset, all three RSIs were used in this analysis (as the difference between the RSI conditions does not occur until later). Single-trial ERPs were epoched from −100 to 350 ms (e.g., up to the shortest RSI) relative to the response, and baseline corrected with the same baseline window of −300 to −100 ms used in the main decoding analysis.
Cross-decoding analysis enabled us to explore the time course of shared processing in two different datasets (King and Dehaene, 2014). For example, if an LDA classifier trained for action error versus standard could also classify surprise versus standard, this would indicate that shared processing occurs in both classifications. Thus, we investigated our prediction by training one classifier with one dataset (e.g., action error vs standard) and then tested the other dataset (e.g., surprise vs standard) and vice versa.
A time-by-time temporal generalization matrix was obtained for each participant. As this process is stochastic because of random assignments of trials for each dataset, we repeated this process 10 times, and the obtained decoding performance was averaged. Each participant's AUC per time point was then submitted to a paired-samples Wilcoxon signed-rank test against chance level. Statistical significance was again tested with cluster-based permutation tests just as for the MVPA analysis.
Brain–behavior relationship between classifier performance and corresponding postbehavior adjustments with searchlight analysis
We investigated the relationship between the decoding performance of a given classifier and post-trial action slowing in the time-by-channel space using a searchlight analysis embedded in the MVPA-Light toolbox (Treder, 2020). This also enabled us to investigate channel contributions underlying the overall decoding performance across time periods together. In doing so, we first built channel clusters by defining neighbors based on 2D coordinates of channel locations. Channel neighbors were defined with a cutoff value of 0.20 so that the distance between channels closer than the cutoff were considered as spatial neighbors (Oostenveld et al., 2011). The average cluster size was 8.53 channels (range, 4–11). The same LDA classifiers used in the main analyses were applied with channel clusters, resulting in decoding performance at channel × time space.
After the channel dimension was averaged, each time point in the decoding was then smoothed with a 30 time point window and correlated to post-trial slowing using Spearman's correlations—decoding performance from action error versus standard classifier and PES, action error versus surprise and PES-PSS difference, and surprise versus standard and PSS. To identify statistically significant correlations, we report time windows in which significance (p < 0.05) was found for at least 10 consecutive time points, 40 ms). This threshold was established to highlight only meaningful and temporally extended relationships between the behavioral measurement and the decoding accuracy. We then unfold brain–behavior relationships within the detected windows in channel space. Within each specified window of significant correlation, each channel was separately correlated with post-trial slowing.
Hypotheses
Behaviorally, we predicted behavioral slowing after both action errors and surprise at short RSIs. At long RSIs, we predicted only errors to be followed by slowing. On the neural level, we first predicted that action errors and surprise would share initial processing stages. Specifically, we predicted that processing in the first ∼100 ms after action errors would match processing ∼200–300 ms after surprising action outcomes (given previous reported findings of shared neural generators underlying the error-related negativity and novelty-N2 ERPs, which take place during these time periods; cf. Wessel et al., 2012). Finally, we predicted that error-specific neural processes would emerge after this initial period of overlap, in line with the AOT (Wessel, 2018). Moreover, we predicted that these processes would relate to the error-specific PES that we expected to find at longer RSIs.
Data availability
Datasets and related analysis scripts are available on Open Science Framework at https://osf.io/mxh8a/.
Results
Experiment 1: behavior
All participants successfully monitored the visual action outcomes, resulting in no misses on target trials. The mean incongruent error rate was 24.089% (SD = 8.916). There was a significant difference in mean RTs for three trial-types (F(2,114) = 37.216, p < 0.001, η2p = 0.395). The following post hoc tests indicate that RTs for surprise trials (mean = 428.511 ms, SD = 33.13) and standard trials (mean = 431.339 ms, SD = 30.138) were comparable (t(38) = −0.403, psidak = 0.97), whereas RTs for error trials (mean = 377.55 ms, SD = 29.58) were significantly faster than RTs for surprise trials (t(38) = −7.262, psidak < 0.001) and faster than standard trials (t(38) = −7.665, psidak < 0.001), indicating the typical pattern of premature erroneous responses shown in interference tasks.
Post-trial action slowing analysis
Post-trial action slowing is shown in Figure 2A. The degree of post-trial action slowing (percentage) was subjected to three-by-two repeated-measures ANOVA. This analysis revealed a main effect of RSI (F(2, 76) = 8.5, p < 0.001, η2p = 0.183). The main effect of expectancy violation was also significant (F(1,38) = 38.384, p < 0.001, η2p = 0.503), as was the interaction between RSI and expectancy violation (F(2,76) = 5.602, p = 0.005, η2p = 0.128). Planned comparisons for post-trial action slowing following action errors (PES) and surprise (PSS) showed that at the short RSI, there was no difference between PES and PSS (t(38) = 1.139, p = 0.262, Cohen's d = 0.182). Moreover, both PES (mean = 4.933%, SD = 5.701) and PSS (mean = 4.016%, SD = 3.403) were significantly increased compared with zero at that RSI (PES, t(38) = 5.403, p < 0.001, Cohen's d = 0.865; PSS, t(38) = 7.37, p < 0.001, Cohen's d = 1.18). At both longer RSIs, however, paired-samples t tests revealed that PES was significantly greater than PSS [at medium RSI, PES vs PSS, 3.09% (SD = 5.298) versus −0.821% (SD = 5.577), t(38) = 5.512, p < 0.001, Cohen's d = 0.883; at long RSI, PES vs PSS, 3.136% (SD = 5.605) vs −1.199% (SD = 4.706), t(38) = 4.796, p < 0.001, Cohen's d = 0.768]. Moreover, PES was significantly increased compared with zero at the medium RSI (t(38) = 3.642, p < 0.001, Cohen's d = 0.583) and the long RSI (t(38) = 3.494, p = 0.001, Cohen's d = 0.559). However, PSS was not significantly different from zero neither at the medium RSI (t(38) = −0.919, p = 0.364, Cohen's d = −0.147) nor at the long RSI (t(38) = −1.592, p = 0.12, Cohen's d = −0.255). Rather, PSS was numerically decreased at both longer RSIs.
Figure 2-1
Post-trial action slowing (percentage) calculated with the traditional method. The degree of post-trial action slowing following action error and surprising outcome at three RSIs, short, medium, and long. A–C, Results from (A) experiment 1 (N = 39), (B) experiment 2 (N = 37), and (C) pooled data (N = 76) for experiments 1 and 2. Download Figure 2-1, EPS file.
Together, these results confirm our hypothesis; at short RSIs, action errors and surprising outcomes of otherwise correct responses lead to a comparable slowing of reaction times. However, for errors specifically, this slowing persisted even at longer RSIs. This shows that post-error slowing appears initially to be because of the surprising nature of the error, after which error-specific processes take place, which are not explained by surprise alone. In experiment 2, we aimed to replicate these findings in an EEG experiment, allowing us to additionally investigate neural signatures of these hypothesized error-specific mechanisms.
Experiment 2: behavior
Participants performed the same task as experiment 1 while undergoing EEG recording. All participants successfully monitored visual action outcomes, showing no misses on target trials. There was a significant difference in mean RTs for for standard correct, action errors, and surprising trials (F(2,108) = 24.631, p < 0.001, η2p = 0.313). Following post hoc tests indicate that RTs for surprise trials (mean = 387.925 ms, SD = 34.167) and standard trials (mean = 393.494 ms, SD = 31.65) were comparable (t(36) = −0.771, psidak = 0.826), whereas RTs for error trials (mean = 347.086 ms, SD = 26.916) were significantly faster than RTs for surprise trials (t(36) = −5.656, psidak < 0.001) and RTs for standard correct trials (t(36) = −6.427, psidak < 0.001). In sum, these results replicate those of experiment 1.
Post-trial action slowing analysis
Post-trial action slowing is shown in Figure 2B. The degree of post-trial action slowing (percentage) was subjected to a three-by-two repeated-measures ANOVA. This analysis revealed a main effect of RSI (F(2,72) = 23.553, p < 0.001, η2p = 0.396), as well as a main effect of expectancy violation (F(1,36) = 4.552, p = 0.04, η2p = 0.112), as was the interaction between RSI and expectancy violation (F(2,72) = 23.004, p < 0.001, η2p = 0.39). Planned comparisons between post-trial slowing following action errors (PES) and surprising action outcome (PSS) were followed to test our main hypothesis at each RSI. Unlike in experiment 1, at the short RSI, there was a significant difference between PES and PSS (t(36) = −2.539, p = 0.016, Cohen's d = −0.417), indicating greater PSS compared with PES. Importantly however, both PES and PSS were significantly increased compared with zero [PES, 4.069% (SD = 4.393), t(36) = 5.635, p < 0.001, Cohen's d = 0.926; PSS, 6.071%, (SD = 5.042), t(36) = 7.323, p < 0.001, Cohen's d = 1.204]. Replicating experiment 1, at longer RSIs, paired-samples t tests revealed that PES was significantly greater than PSS at the medium and at long RSIs [at medium RSI, PES vs PSS, 1.42% (SD = 4.612) vs −1.026% (SD =3.959), t(36) = 3.855, p < 0.001, Cohen's d = 0.634; at long RSI, PES vs PSS, 1.549% (SD = 4.507) vs −1.331% (SD = 3.165), t(36) = 4.44, p < 0.001, Cohen's d = 0.73]. Moreover, PES was significantly increased compared with zero at the medium RSI (t(36) = 1.873, p = 0.034, one sided, Cohen's d = 0.308) and the long RSI (t(36) = 2.091, p = 0.044, Cohen's d = 0.344). In contrast, PSS was not significantly different from zero at the medium RSI (t(36) = −1.577, p = 0.124, Cohen's d = −0.259). At the long RSI, PSS was rather significantly decreased from zero (t(36) = −2.558, p = 0.015, Cohen's d = −0.42). In sum, overall, these results replicate the findings from experiment 1, with the exception of the short RSI showing a slight increase in PSS compared with PES.
Pooled experiments 1 and 2 data
We also pooled the data from both experiments (N = 76) to identify the most reliable effect size estimates for the behavioral effects of interest with maximal statistical power. The analyses are the same as above. We found a main effect of RSI (F(2,150) = 28.426, p < 0.001, η2p = 0.275), a main effect of expectancy violation (F(1 5) = 31.965, p < 0.001, η2p = 0.299) and a significant interaction (F(2,150) = 21.706, p < 0.001, η2p = 0.224). Planned comparisons revealed that at the short RSI, there was no significant difference between PES and PSS (t(75) = −0.862, p = 0.392, Cohen's d = −0.099). Both PES and PSS were significantly increased compared with zero at the short RSI [PES, 4.512% (SD = 5.091), t(75) = 7.727, p < 0.001, Cohen's d = 0.886; PSS, 5.016%, (SD = 4.375), t(75) = 9.996, p < 0.001, Cohen's d = 1.147]. PES was significantly greater than PSS at the medium RSI [PES vs PSS, 2.227% (SD = 5.014) vs −0.921% (SD = 4.826), t(75) = 6.637, p < 0.001, Cohen's d = 0.761] and at the long RSI [PES vs PSS, 2.363% (SD = 5.129) vs −1.263% (SD = 4.004), t(75) = 6.433, p < 0.001, Cohen's d = 0.738]. At the medium RSI, PES was significantly increased from zero (t(75) = 3.959, p < 0.001, Cohen's d = 0.454); the same was true at the long RSI (t(75) = 4.017, p < 0.001, Cohen's d = 0.461). In contrast, PSS was not significantly different from zero at the medium (t(75) = −1.663, p = 0.10, Cohen's d = −0.191). At the long RSI, PSS was significantly decreased from zero (t(75) = −2.751, p = 0.007, Cohen's d = −0.316), indicating response speeding.
To sum up, throughout experiments 1 and 2, our data consistently showed that both action errors and surprise produced significant slowing at the short RSI, whereas only errors further showed significant slowing at longer RSIs. In experiment 2, we then aimed to identify neural signatures of the error-specific processes that lead to the sustained PES at the longer RSIs.
Experiment 2: EEG multivariate pattern analysis
Sustained decoding of post-error neural activity from both standard and surprise activation
Decoding performances for the three classifiers of interest (errors vs standard, surprise vs standard, and errors vs surprise) are shown in Figure 3. Significant clusters are marked on the top of the plot.
Figure 3-1
Topographical maps for decoding performance of three classifiers. Decoding performance across time × channel with searchlight analysis are shown. For visualization, time-by-channel topographical maps were averaged within each 500 ms window. Top to bottom, The rows represent three classifiers, action error versus standard, surprise versus standard, and action error versus surprise. The columns indicate the respective time bins. Note that each column has a consistent range on the color bar, with the minimum decoding performance value set at 0.5 (chance level). The maximum value varies across the columns. Download Figure 3-1, EPS file.
Figure 3-2
Temporal generalization matrix for three classifiers. Overall, classifiers for action errors (vs standard and vs surprise) showed sustained neural activity during the postevent period, whereas surprise versus standard classifier showed time-specific activity in the early time period after response onset (time = 0 ms). Significant time periods are shown (p value corrected with cluster-based permutation tests; 1000 permutations, p < 0.05). Download Figure 3-2, EPS file.
Figure 3-3
Cross-decoding analysis results for action error-to target (left) and surprise-to-target (right). We found a significant cluster for each generalization matrix. Cross-contrast classification revealed the most prevalent overlap between the time period around response ∼300 ms after errors and the time period ∼250–350 ms following target (left). We also found that classifiers for the surprise versus standard ∼80–350 ms time period could generalize to classify the surprising target from standard ∼80–120 ms, which was relatively short. Given that both surprise and targets were infrequent, but the surprise trials needed sustained processing as they were novel in every trial, whereas the targets were not, the relatively short decodable time periods for target versus standard suggests that it predominantly captures the infrequency character of both surprising outcomes. Download Figure 3-3, EPS file.
Three major patterns can be discerned from these multivariate classification analyses. First, action errors showed unique activity that distinguished them from both surprise and standard trials even before the response. This is likely because errors are the only trial type in which the actual response does not match the intended response and is in line with prior reports of error-preceding activity (Ekman et al., 2012; Hajcak et al., 2005; Steinhauser and Steinhauser, 2019).
Second, action errors and surprise differed from standard trials, and from each other, in the early postevent time period. Although the error-standard and surprise-standard differences were expected given the known early EEG activity after errors [error-related negativity/positive error (ERN/Pe) complex; Falkenstein et al., 1991, 2000; Gehring et al., 1993] and surprise (N2/P3 complex; Courchesne et al., 1975), the difference between errors and surprise trials is unexpected at a superficial glance because previous studies have shown substantial overlap between error and surprise-related neural activity (Gentsch et al., 2009; Wessel et al., 2012). However, as mentioned above, we expected the overlapping activity played out at different times following both events, which we confirm in further analyses below.
Third, and most importantly, although surprise trials ceased to produce any discernible activity from standard trials after ∼1100 ms postevent, error-related activity remained decodable from both standard- and surprise-related activity throughout the epoch. This shows that errors are followed by unique, sustained activity that is not attributable to surprise alone, confirming our hypothesis (Extended Data Figs. 3-1, 3-2). We subsequently investigated whether this unique activity after errors was related to the sustained PES found at longer RSIs (see below).
Cross-condition decoding reveals overlapping early processing between errors and surprise
As indicated by prior work (Gentsch et al., 2009; Guan and Wessel, 2022; Wessel et al., 2012), we hypothesized that errors and surprise would share common neural processing early after each event. However, early error and surprise processing play out at slightly different time scales, indicated by the fact that known EEG differences between errors and correct trials, such as the ERN, emerge ∼50 ms following the action, whereas it takes ∼100–150 ms longer for first differences between visual surprise and standard trials, such as the N2/P3 complex, to emerge. Importantly, past work has indicated that ERN and N2 share a neural generator (Gentsch et al., 2009; Wessel et al., 2012). This would mean that the significant decoding difference between errors and surprise trials reported in the previous paragraph may be misleading, as the processing should overlap but not align in time. Therefore, we performed a cross-decoding analysis to see whether a classifier trained for one contrast (e.g., action errors vs standard) during the early postresponse time period (−100–350 ms around the response) would generalize to another contrast (e.g., surprise vs standard).
The temporal generalization matrices from these analyses are shown in Figure 3B. Both comparisons (decoding surprise vs standard contrast by using the action error vs standard classifier and vice versa) revealed complementary findings. First, as expected, cross-contrast classification revealed the most prevalent overlap between the time period of ∼40–220 ms after errors and the time period of ∼200–350 ms following surprise (Fig 3B, orange rectangles), confirming our a priori hypothesis. In response to a reviewer's suggestion, we also checked whether the infrequent, but not surprising, target could be decoded from standard by using the action error versus standard classifier and the surprise versus standard classifier. This indeed yielded comparable results (Extended Data Fig. 3-3). Interestingly, there was also a weaker but significant period of significant cross-decoding between the time period immediately following the response (0–60 ms) on surprise trials and the time period of ∼40–240 ms following errors.
Sustained error-specific neural processes explain PES at later RSIs
Decoding performance from three classifiers was correlated with the corresponding behavioral indices —PES, PSS, and the PES-PSS difference from the long RSI (Fig. 4, bottom, thicker line indicates significant time points). A significant positive relationship between PES and the decoding performance for action error versus standard started to emerge at ∼900 ms, specifically, in the following time windows: 916–1068, 1080–1140, 1488–1680, and 2360–2844 ms. The relationship between the PES-PSS difference and decoding performance for action errors versus surprise classifier revealed significant periods at 1504–1724 and 2428–2576 ms. Finally, there was a significant negative relationship between PSS and decoding performance for the surprise versus standard classifier before response onset at −152 to −104 ms and after ∼1312 ms (1312–1564, 1676–1744, and 1848–1916 ms).
Once the significant time windows were revealed, we further aimed to unfold brain–behavior relationships within the detected time windows in channel space. Within each specified window averaged across time points, each channel decoding performance was correlated to behavior. The topographical maps for correlation coefficient are shown in Figure 4B. Significant channels under (critical puncorreced < 0.01) are marked with solid black dots.
For the relationship between PES and the action errors versus standard classifier, sensorimotor, left frontolateral, parietal, and occipital channels showed significant correlations. The left frontolateral channels showed significant relationship early (e.g., the first window, 916–1068 ms). For the relationship between PES-PSS difference and its corresponding decoding performance (i.e., action errors vs surprise), again, occipital and sensorimotor channels were dominant.
For the relationship between PSS and its corresponding decoding performance (i.e., surprise vs standard), the negative relationships showed peaks in the right frontolateral and parietal channels. Note that we did not find any significant channels (under puncorreced < 0.01) in the preresponse window (and therefore, its topographical figure is not shown in Fig. 4B).
Discussion
In this study, we combined a novel experimental paradigm with time-resolved multivariate decoding of whole-scalp neural recordings to provide evidence for unique error-related processes in the human brain. Previous work questioned whether the neural dynamics and behavioral adjustments after errors are genuinely attributable to the erroneous nature of the performed action or whether more domain-general surprise processing could account for post-error changes in behavior and neural activity (Alexander and Brown, 2011; Notebaert et al., 2009; Parmentier et al., 2019; Wessel et al., 2012). The present work resolves this debate by providing both behavioral and neural evidence for genuine error-specific activity in the human brain, notably by decoding multivariate brain activity at precise time scales using scalp EEG.
First, we used our experimental paradigm to identify a unique and novel distinction in behavior after surprise and errors. Previous studies had been unable to disentangle whether post-error slowing is because of the surprising nature of the action outcome or because of the fact that an error was committed (Castellar et al., 2010; Houtman and Notebaert, 2013; Notebaert et al., 2009; Parmentier et al., 2019). Indeed, we found that both errors and surprise led to slowing of subsequent responses when the response–stimulus interval was short. At longer intervals, however, only errors showed a sustained slowing of RTs. Importantly, post-error and post-surprise slowing at the short RSI was identical in magnitude (pooled data, t(75) = 0.862, p = 0.39, Cohen's d = 0.099; Bayes factor, BF01 = 5.542, indicating moderate evidence for no difference; Van Doorn et al., 2021), ruling out the alternative explanation that surprising events in the current experimental paradigm were not as salient as errors. We also took great care to match the frequency of occurrence of these trials through our behavioral paradigm. In sum, our results support the interpretation that surprise-related slowing, in contrast to PES, is not sustained when more processing time is provided. We suggest that this is because behavioral slowing on post-error trials strategically addresses the cause of the error (overly speeded responding, evident by the typical pattern of faster error compared with correct trial RT), whereas using such a strategy would not be functional when the unexpected action outcome was because of external circumstances (i.e., on surprise trials). Hence, it is adaptive to slow responses after errors, even when sufficient processing time is available to fully process the outcome, whereas the same is not true for surprise. These findings were explicitly motivated and a priori formulated in the adaptive orienting theory of error processing (Wessel, 2018).
We then used the same experimental paradigm in combination with multivariate pattern analysis of neural responses recorded via scalp EEG. This methodology allowed us to use a whole-scalp decoding approach to the differences (and commonalities) between error and surprise processing with high temporal resolution. This is important because the behavioral results indicated that the common and unique processes associated with surprise and error processing likely play out in a close temporal sequence, suggested by the importance of RSI in whether either trial type elicited differences in overt behavior (Jentzsch and Dudschig, 2009). As such, alternative methods, notably fMRI, would have lacked the temporal resolution to identify potential disjunctions. Indeed, our EEG-based MVPA analysis showed two main findings, both of which highlight the importance of the temporal sequence of processes.
First, we demonstrated that action errors and surprise share neural processes, although these play out at different times. Specifically, classifiers trained to distinguish action errors from standard trials were able to also decode surprise from standard trials in a later time window (and vice versa). The two distinct time windows in question align with two ERP components, namely, the ERN/Pe components (Falkenstein et al., 1991, 2000; Gehring et al., 1993) for errors and the N2/P3 complex (Courchesne et al., 1975) for surprise, respectively. This is in line with previous work showing that these two ERPs are explained by the same independent component (Wessel et al., 2012). Our current findings thus confirm the involvement of a shared neural system in processing action errors and domain general surprise, presumably signaling the need for cognitive control detected by the action monitoring system (Alexander and Brown, 2011; Cavanagh and Frank, 2014). Interestingly, there was some weaker overlap between error and surprise decoding even sooner after the response (beginning right after the response on surprise trials and ∼50 ms after errors; Fig. 3B, pink box) We can only speculate on the nature of this overlap because unlike for the ERN/Pe–N2/P3 overlap period, we did not have an a priori hypothesis regarding this time range (and given its much lower cross-decoding strength compared with the ERN/Pe–N2/P3 time range, it is unclear whether significant cross-decoding strength in this time range would replicate). Furthermore, this time period did not show a significant association with post-trial behavior (Fig. 4), making it unlikely that it is a functional process related to behavioral adaptation. It is possible that it reflects overlap in short-latency processing of sensory unexpectedness (visual in the case of surprise in our current task, somatosensory in the case of errors), although further research will likely be necessary to elucidate the nature of significant overlap in processing in this early time postresponse time period.
Most importantly, our results show that action errors are followed by a unique activity that distinguishes it both from standard and surprise trials, starting ∼1 s after error commission. Similar activity was not observed after domain-general surprise as surprise trials could no longer be decoded from standard trials in that same time period. This activity also directedly corresponds to the behavioral results, which showed that only errors were followed by subsequent RT slowing at longer RSIs. Indeed, we directly demonstrated that decoding performance for both action error classifiers (vs standard and vs surprise) in these later time periods was related to PES (Fig. 4). These findings clearly demonstrate distinct error-specific processes in later postevent periods. It is worth noting that although action errors showed positive relationships, the surprise classifier (surprise vs standard) showed a short-lived negative association with PSS (∼1300–1900 ms) right after surprise was no longer differentiable from standard trials. This may indicate that instead of strategic adjustments of subsequent responses following surprise the speeding of RTs may be attributed to the resetting of the current task set, which could have facilitated subsequent processes (Corbetta and Shulman, 2002; Seeley et al., 2007).
To develop some potential insights into the specific nature of the sustained error-related processes that contribute to PES, we examined the contributions of neural activity at different scalp locations to postbehavioral adjustments. In line with the idea that distributed networks were involved for implementing cognitive controls, we found (left) lateral prefrontal, occipital, sensorimotor, and parietal channel contributions for post-error adjustments (e.g., PES and PES-PSS difference). First, the left lateral prefrontal channels (LPFC) were primarily involved in post-error adjustments, starting ∼900 ms to longer post-error periods. This is consistent with previous findings highlighting the role of LPFC in guiding cognitive control following errors, particularly after the action monitoring system involving the anterior cingulate cortex and medial frontal cortex detects errors (Botvinick et al., 2001; Cavanagh et al., 2009; Garavan et al., 2002; Kerns et al., 2004; King et al., 2010; Masina et al., 2019; Ridderinkhof et al., 2004). For instance, Cavanagh et al. (2009) demonstrated that oscillatory synchrony between lateral and medial prefrontal channels predicted PES, indicating the important communication between the action monitoring and cognitive control systems for post-error adaptations (Cavanagh et al., 2009; King et al., 2010; Masina et al., 2019). Furthermore, the LPFC is also suggested to be specifically involved in temporal upregulation of attentional set (Vanderhasselt et al., 2009) as well as in maintaining a representation of task-relevant goals (Magno et al., 2006). Thus, together, our finding suggests that the contribution of the LPFC channel to post-error adaptations is likely to indicate the increased top-down regulation for error-specific processes during postevent periods.
Beyond this early contribution of the LPFC channel, the most consistent and stronger contributions to post-error behaviors came from occipital, sensorimotor, and parietal channels across the later post-error periods. Some evidence suggests there are increases in directional information flow following errors, between occipital and lateral/medial prefrontal channels, revealed in Spectral Granger Causality analyses (Cohen et al., 2009; Cohen and van Gaal, 2013). Occipital channel contributions may indicate the tuning in visual sensory processing following errors, reflecting post-error regulation of selective attention (Norman et al., 2021; Steinhauser and Andersen, 2019; van Driel et al., 2012). In addition, we also found parietal channel contributions in post-error adjustments would indicate the reconfiguration of the task set following errors (Danielmeier et al., 2011; Egner, 2008; Geng and Vossel, 2013; King et al., 2010). Finally, we observed significant sensorimotor channel contributions, which likely reflect the upregulation of motor thresholds (Fischer et al., 2018; King et al., 2010). These findings corroborate the notion that distributed regions are involved in post-error adjustments and that error-specific processes are strategic and involve the fine-tuning of the cognitive system on multiple fronts.
Although many distinct scalp sites were positively related to post-error adjustments, similar but more right-lateralized channels were negatively related to postsurprise processes. Predominantly, right lateral prefrontal and parietal channels were related to these processes, suggesting that task-set reconfigurations following surprising action outcomes contribute to postsurprise speeding at longer intervals. Importantly, in contrast to errors, occipital and sensorimotor channels were not revealed for surprise. It might suggest that surprising outcomes were not necessarily inducing the purported strategic adjustments of motor and stimulus processing during postevent periods.
Beyond the basic science of cognitive control, our new paradigm combined with EEG decoding analysis could have potential clinical implications. For instance, error-specific cognitive control is purportedly pathologically affected in clinical groups such as those with anxiety disorder or obsessive-compulsive disorders. Studies have reported that both groups showed increased ERN (Gehring et al., 2000; Weinberg et al., 2010; for review, see Endrass and Ullsperger, 2014; Saunders and Inzlicht, 2020); however, there are conflicting results regarding PES differences (Rueppel et al., 2022). That is possibly because of the fact that there have not been consistent RSI manipulations among tasks, leading to a differential role of domain-general surprise and error-specific processing across studies. In this regard, we believe that our paradigm and analysis methods can help disentangle such confounds in the future (Fitzgerald et al., 2021; Moran et al., 2015).
Together, we provide evidence here for unique, genuinely error-specific activity in the human brain that is independent of domain-general surprise processing and explains adaptive changes in post-error behavior. Future work could use the experimental paradigm and EEG decoding framework applied here and combine it with simultaneous fMRI (Cichy and Oliva, 2020; Debener et al., 2006) to develop a more detailed understanding of the precise neural dynamics that underpin the error-specific processes identified in the current study.
Footnotes
This work was supported by National Institutes of Health–National Institute of Neurological Disorders and Stroke Grant R01NS102201 to J.R.W. We thank Saara Engineer for assistance with data collection.
The authors declare no competing financial interests.
- Correspondence should be addressed to Jan R. Wessel at jan-wessel{at}uiowa.edu