Abstract
To date it is poorly understood how and when deviance processing interacts with awareness and task relevance. Furthermore, an important issue in the study of consciousness is the prevalent confound of conscious perception with the requirement of reporting it. This study addresses these topics using a no-report inattentional blindness paradigm with a visual oddball sequence of geometrical shapes presented to male and female human participants. Electrophysiological responses were obtained in three physically identical Phases A–C that differed only with respect to the instructions: (A) participants were uninformed about the shapes and attended an unrelated foreground task (inattentional blind), (B) were informed about the shapes but still attended the foreground task, and (C) attended the shapes. Conscious processing of shapes was indexed by the visual awareness negativity but not a P3. Deviance processing was associated with the visual mismatch negativity independently of consciousness and task relevance. The oddball P3, however, only emerged when the stimuli were task relevant, and was absent for consciously perceived but task irrelevant stimuli. The P3 thus does not represent a reliable marker of stimulus awareness. This result pattern supports the view of hierarchical predictive processing, where lower levels display automatic deviance processing, whereas higher levels require attention and task relevance.
SIGNIFICANCE STATEMENT To react to potentially important changes in our environment it is fundamental to detect deviations from regularities of sensory input. It has yet to be understood how awareness and task relevance of this input interact with deviance processing. We investigated the role of awareness in deviance detection while at the same time circumventing the confound of awareness and report by means of a no-report paradigm. Our results suggest that early processes are elicited automatically, whereas, contrary to prominent theories, late processes do not depend on awareness but on task-based attention.
- hierarchical predictive processing
- inattentional blindness
- mismatch negativity
- P3
- visual awareness negativity
Introduction
The detection of deviations from regularities of sensory input is of fundamental importance for reacting to, potentially important, changes in our environment. However, the ways in which awareness and task relevance influence deviance processing are to date poorly understood as both factors are often closely intertwined. Inattentional blindness (IB) coupled with task manipulations can help disentangle these factors (Mack, 2003; Pitts et al., 2012; Shafto and Pitts, 2015). In addition to being highly ecologically valid, the IB design of Pitts et al. (2018) circumvents the need for trial-by-trial subjective reports. The confound of subjective report with conscious perception is a caveat often encountered in consciousness research (Aru et al., 2012; Frässle et al., 2014; Tsuchiya et al., 2015; Pitts et al., 2018). As a result, brain activity related to awareness might be confused with brain activity related to the report (Pitts et al., 2014; Tsuchiya et al., 2015). During no-report paradigms, a negative-going potential at posterior sites indexed stimulus awareness [visual awareness negativity (VAN); Koivisto and Revonsuo, 2010; Pitts et al., 2012; Shafto and Pitts, 2015; Schelonka et al., 2017]. This contrasts with findings postulating late central positivities (P3) as a neural correlate of consciousness (NCC; Dehaene et al., 2006, 2014). However, this issue is still hotly debated and warrants further studies corroborating the evidence of the few existing no-report studies (Boly et al., 2017; Odegaard et al., 2017).
Although the above-mentioned studies demonstrated the viability of IB in disentangling report and awareness, the question of whether (and if so, how and when) complex mental operations like deviance processing can be observed nonconsciously during IB has yet to be addressed. Deviance processing is often probed using an oddball paradigm, in which the expectation of a standard stimulus is violated by an unexpected deviant stimulus. In event-related potentials (ERPs), such regularity violations by deviant stimuli are accompanied by an early negativity termed the mismatch negativity (MMN; Näätänen et al., 2007; Stefanics et al., 2014) and a later positivity (oddball P3; Squires et al., 1975; Snyder and Hillyard, 1976; Conroy and Polich, 2007; Polich, 2007, 2012). ERP components in oddball studies have been linked to the concept of predictive processing (Mars et al., 2008; Lieder et al., 2013; Stefanics et al., 2014, 2018). In this framework, it is postulated that the brain processes information by comparing top-down predictions with incoming data resulting in a prediction error signal being passed on to the next higher level, thus enabling efficient information transmission (Friston, 2010; Clark, 2013). Observing the MMN and oddball P3 under different awareness and task conditions could highlight in what ways these components might represent correlates of a hierarchical comparison process and strengthen the account of predictive processing as a basic principle of information transmission in the brain.
Although there is evidence for a nonconscious MMN, the P3 to deviant stimuli seems to be elicited only during conscious processing (Bekinschtein et al., 2009; Faugeras et al., 2012; Strauss et al., 2015; Jack et al., 2017; but see Silverstein et al., 2015). This outcome accords with the proposal that awareness and the emergence of a P3 are strongly associated (Dehaene et al., 2006, 2014). However, as discussed above, findings from no-report paradigms (Pitts et al., 2012; Shafto and Pitts, 2015; Schelonka et al., 2017) leave open the possibility that oddball effects on the P3 might not only depend on awareness, but on task-based attention. Thus, the current study aimed to elucidate how deviance processing interacts with awareness and task relevance using a roving oddball paradigm and a no-report procedure. We found the VAN to index awareness and, together with the behavioral data, to validate the IB paradigm. Crucially, we observed the MMN independently of awareness and task, whereas the oddball P3 only emerged in participants aware of the stimuli and only if the stimuli were task relevant.
Materials and Methods
Participants.
The sample consisted of 56 participants (39 female) aged from 19 to 33 years (M = 23.54, SD = 3.00). All had normal or corrected-to-normal vision and three were left-handed. Participants volunteered and were compensated with 9 €/h. Before starting, participants were given written instructions on the experimental task and given the opportunity to ask further questions. The study was approved by the local ethics committee and all procedures were performed in accordance with the Helsinki declaration.
Apparatus.
The experiment was run using MATLAB and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997; Kleiner et al., 2007). An iiyama HM903DT monitor at 85 Hz with a resolution of 1280 × 1024 pixels was used for stimulus display. The viewing distance amounted to 60 cm. To respond, participants pressed the space bar and numeric keys of a standard keyboard. A chin rest was used to prevent head movements during the experiment.
Stimulus material and experimental procedure.
Stimuli consisted of a 20 × 20 grid of white lines with a width of 0.45° of visual angle each, spanning 10 × 10° in total and were presented on a black background (Lwhite = 0.22 cd/m2, Lblack = 181.34 cd/m2; for background stimuli; Fig. 1A). Line orientation was chosen at random for each of the 400 lines comprised in the grid, i.e., a random pattern of lines was used for every presentation. Shapes were constructed by orienting lines vertically and horizontally to form a square and two rectangles centrally in the grid using 12 × 12, 8 × 16, and 16 × 8 lines while all other lines were kept random. At all times, a red fixation cross of 0.9° × 0.9° was presented centrally. Concurrently, 12 red dots were presented on 3 circular paths (4 on each circle) with a radius of 2.5, 4.5, and 6.5°, respectively (foreground stimuli; for a stationary image of the dots, see Fig. 1A). The dots, with radii of 0.32, 0.41, and 0.52°, rotated with a constant angular velocity of 1.48 radians/s. On average, every 43 s (jitter: ±0–10 s) a randomly chosen dot slightly decreased in luminance for 500 ms [from L = 26.47 cd/m2 to L = 7.68 cd/m2; (204, 0, 0) to (127.5, 0, 0) in RGB]. The rotation direction changed every 24 s on average from clockwise to counterclockwise and vice versa (jitter: ±0–10 s). Onsets of color and rotation changes were further pseudorandomized in such a way that they never coincided with a shape onset.
In the roving oddball design, stimulus sequences of varying length were presented where the first stimulus is treated as a deviant, while, after several repetitions, it turns into a standard (Lieder et al., 2013; Stefanics et al., 2018). Thus, the oddball comparison involves physically identical stimuli and overall, stimuli are shown an equal amount of times. Shapes were presented in sequences of four to eight shape repetitions and presentation was randomized so that consecutive sequences were never of the same shape (Fig. 1B). Each shape remained 100 ms on the screen followed by a random pattern presented for 700 ms, after which the next shape of the sequence was presented. On average, every ninth sequence contained a shape omission, i.e., the pattern did not contain a rectangle or square, for four consecutive shapes (jitter: ±0–3 sequences). After 42 sequences, there was a mandatory break of at least 10 s, during which participants received performance feedback. In total, 210 sequences were presented during each phase (5 sequence length × 3 shapes × 14 repetitions; resulting in 1260 shape stimuli per phase). The runtime of one phase amounted to 16.8 min (excluding breaks).
For all three experimental phases, the stimulus presentation was physically identical, whereas the task differed. In Phases A and B, participants were instructed to press the space bar whenever they detected a luminance decrease in one of the dots. In Phase C, participants' task was to detect the shape omissions. In Phase A, participants were uninformed about the presence of the background stimuli. The difficulty of the task was designed to elicit IB in about one-half of the subjects, who were thus unaware of the presence of the shapes. In Phase B, the task was held constant, but all participants were informed about the presence of the shapes and were thus aware of them. In Phase C, awareness was held constant, but participants completed a new task, which directed their attention to the shapes. Phase A was followed by either Phase B or Phase C counterbalanced across participants. To accustom participants to the experimental tasks, Phase A and Phase C comprised a practice session in which the task difficulty was gradually increased until the level of the main experiment was reached (in Phase A, the target color difference decreased while only random patterns were presented; in Phase C, the duration of shapes diminished). For an overview of the experimental procedure, see Table 1.
After completing each experimental phase, participants were given a questionnaire, asking whether or not they perceived the shapes and to describe or sketch what they saw as detailed as possible. Then, participants were asked to rate nine different shapes made of line segments (including the 3 shapes shown during the experiment) on how confident they were of having seen the shape (confidence rating) and how often they saw the shape (frequency rating) on a 5-point scale. The awareness questionnaire and ratings relied on Pitts et al. (2012) as displayed in their Appendix. In addition, we included one further question to probe sequence awareness: after the ratings participants were asked whether or not they perceived a regular structure in the shape sequences, i.e., a particular shape was repeated for several times before changing to a new shape.
EEG recording and preprocessing.
A 128-channel BioSemi active electrode system was used to collect electrophysiological data. Electrodes were placed using the equiradial system conforming with BioSemi electrode caps. Furthermore, vertical and horizontal eye movements were recorded with two electrodes attached above and below the left eye (VEOG) and two electrodes attached to the right and left outer canthi (HEOG). Instead of ground and reference, the BioSemi EEG system uses a CMS/DRL feedback loop with two additional electrodes (http://www.biosemi.com/faq/cms&drl.htm). Electrical potentials were recorded with a sampling rate of 512 Hz and impedances were held <20 kΩ. A build-in analog anti-aliasing low-pass filter of 104 Hz was applied before digitization.
Preprocessing of the EEG data was performed using the FieldTrip toolbox (Oostenveld et al., 2011) in MATLAB. Off-line filtering of the continuous data used Butterworth filters with half-power cutoffs if not specified otherwise. Data were band-stop filtered at 49–51 Hz (roll-off: −24 dB/octave) and at harmonic frequencies (up to 199–201 Hz) to minimize line noise. Additionally, a 84–86 Hz band-stop filter accounting for the monitor refresh rate (85 Hz) was applied. A 0.1 Hz high-pass filter (roll-off: −12 dB/octave) removed slow drifts. Then, the EEG signal was segmented into epochs of 200 ms before until 600 ms after stimulus onset. Trials containing eye blinks, muscle artifacts and electrode jumps were manually removed based on visual inspection and bad channels were interpolated. Data were re-referenced from the CMS/DRL to a common average reference. All trials were baseline-adjusted using the average of a prestimulus interval from −200 to 0 ms.
For the oddball contrasts, trials of each subject were averaged separately for deviants and standards. We used the first stimulus of a sequence as deviant and, to control for baseline differences, for the standard we only used stimuli at the same position as the corresponding deviants (Fig. 1C). For example, the deviant after a sequence of four stimuli was compared with the fifth stimulus of a sequence of five repetitions. As the sequence length was chosen randomly, the baseline of both deviant and standard should be equal. This procedure further allowed us to average equal amounts of stimuli per condition. As the eighth stimulus in a sequence was always followed by a change, possibly allowing preparatory activity, we excluded this stimulus and the corresponding deviant from the analysis. For the same reason, deviants following shape omissions were also excluded. This resulted in six waveforms (deviant/standard in 3 phases) per subject. For the awareness contrast, we averaged all standard stimuli in each phase (excluding deviants and the eighth stimulus for the reasons mentioned above), resulting in three waveforms for each subject. Last, grand mean waveforms of the averaged data were computed.
Statistical analysis.
Statistical analysis used a cluster-based permutation test for within-subjects comparisons (Maris and Oostenveld, 2007; Groppe et al., 2011a). To enhance its power, we chose specific time intervals and electrode locations based on previous research. For the analysis of the visual MMN (vMMN), we chose a time interval from 200 ms until 300 ms in posterior electrodes (Stefanics et al., 2014; see Fig. 3). For the P3, we selected fronto-central electrodes and an interval from 300 to 596 ms (Polich, 2007; see Figs. 2, 4). The VAN was tested from 100 to 300 ms using posterior electrodes (Koivisto and Revonsuo, 2010; see Fig. 2). Hypotheses were directional, leading to one-sided cluster-based permutation tests. Clusters were formed by two or more neighboring sensors (in time and space) whenever the t values exceeded the cluster threshold (α = 0.05). The cluster mass, sum(t), was calculated by adding all t values within a cluster. The number of permutations was set to 5000, and the significance value for testing the null hypothesis amounted to α = 0.05. Before the analysis, ERPs were downsampled to 250 Hz and low-pass filtered at 25 Hz (roll-off: −24 dB/octave) to further enhance statistical power (Luck, 2005). To quantify the effect sizes of the significant clusters, we averaged Cohen's d for each electrode and time point. After applying the cluster-based permutation approach, cluster averages from significant clusters were computed and used in the orthogonal planned comparisons. Because the cluster-based permutation for independent t tests is potentially unsuitable if sample size and variance differ between groups (Groppe et al., 2011b), an average amplitude approach in the electrodes and time points described was applied for between-subjects comparisons.
Task performance was quantified as d′ and reaction times of correct responses using the method for paradigms with high event rates introduced by Bendixen and Andersen (2013). Frequency and confidence ratings were obtained for shapes included in the main experiment (hereinafter referred to as “shown”) as well as for shapes not included in the main experiment (“not shown”) and were subtracted from each other. Rating scores, d′ and cluster averages were analyzed using repeated-measures ANOVAs and t tests. Whenever sphericity was violated, the Greenhouse-Geisser correction was applied and corrected p values as well as ε̂ values are reported. Because some of our conclusions rely on null effects, we additionally report Bayes factors (BFs), with BF01 denoting the evidence for the null hypothesis and BF10 the evidence for the alternative hypothesis. We use the conventions from Jeffreys (1961) to interpret the results of our Bayesian analyses. All cluster-based permutations were done using the FieldTrip toolbox (Oostenveld et al., 2011) in MATLAB. Other statistical tests relied on the statistics program R (R Core Team, 2015).
Results
Three participants had to be excluded because of a high number of rejected trials in their EEG data (≥40%) and one because of a wrong monitor setup. With regard to the remaining 52 participants, on average, 17.64% (SD = 7.48%) of trials were excluded from the analysis because of artifacts, and on average, 1.49 (SD = 1.77) electrodes were interpolated.
Behavioral data
Task performance
Task performance quantified by d′ did not differ between phases (dA′ = 2.53, dB′ = 2.83, dC′ = 2.58; F(2,102) = 1.97, p = 0.16, ε̂ = 0.69, BF01 = 2.58), indicating a similar task difficulty in all phases. Further, d′ did not differ between “aware” and “blind” participants in Phase A (daware′ = 2.60, dblind′ = 2.48; t(50) = 0.56, p = 0.58, BF01 = 3.10). Reaction times, however, differed significantly between phases (RTA = 821, RTB = 774, RTC = 1536; F(2,102) = 256.93, p < 0.001, ε̂ = 0.70, BF10 = 3.51 × 1045). In Phase C, reaction times were slower compared with both Phases A and B (all p < 0.001, all BF10 > 5.63 × 1017), probably because of the different task settings, while a marginally significant difference was observed between Phases A and B (t(51) = 1.98, p = 0.053, BF01 = 1.10). Interestingly, in Phase A, aware participants responded significantly more slowly than blind ones (RTaware = 890, RTblind = 771; t(50) = 2.17, p = 0.04, BF10 = 2.51), indicating that either blind participants were more engaged in solving the task or that aware participants were diverted as a consequence of the conscious processing of shapes.
Awareness questionnaire
After completing Phase A, 22 participants indicated awareness of the shapes and 30 reported unawareness. These group assignments were used as the factor awareness in the following analysis. After Phase A, 16 of the 30 blind participants and 10 of the 22 aware participants completed the experiment in the order ABC, whereas all other participants completed the experiment in the order ACB. In Phases B and C, all 52 participants indicated awareness of the shapes. Furthermore, in Phase A, only 6 participants indicated awareness of the sequential structure of the shape presentation, whereas in Phases B and C, 40 and 50 participants noticed the regularity, respectively.
Confidence and frequency ratings
Repeated-measures ANOVA of rating differences (shown − not shown) indicated a significant main effect of phase for both frequency (F(2,102) = 117.96, p < 0.001, BF10 = 3.32 × 1027) and confidence (F(2,102) = 112.32, p < 0.001, ε̂ = 0.87, BF10 = 1.99 × 1026; Table 2). In both Phase B and C participants could significantly better differentiate shown and not shown shapes compared with Phase A (all p < 0.001, all BF10 > 2.78 × 1012). Frequency ratings differed between Phases B and C (t(51) = −2.87, p = 0.005, BF10 = 5.82), whereas no difference was observed for confidence (t(51) = −1.10, p = 0.28, BF01 = 3.76).
Importantly, t tests in the blind subgroup (answer “no” in awareness questionnaire) in Phase A showed no significant difference from zero for frequency (t(29) = 0.07, p = 0.95, BF01 = 5.13) and confidence (t(29) = 0.81, p = 0.43, BF01 = 3.81) indicating that blind participants could not differentiate shown and not shown shapes. On the other hand, the aware subgroup could differentiate between shown and not shown shapes as evidenced by frequency ratings (t(21) = 4.01, p < 0.001, BF10 = 52.87), whereas a tendency in the same direction was observed in the confidence ratings (t(21) = 1.87, p = 0.08, BF01 = 1.02).
EEG data
Electrophysiological measures of awareness
Comparing ERPs elicited by the averaged standard stimuli in Phase A between blind and aware participants revealed a stronger VAN for aware subjects (t(50) = −1.95, p = 0.03, BF10 = 2.51, one-tailed; Fig. 2A), whereas the P3 did not show a significant awareness effect (t(50) = 0.14, p = 0.44, BF01 = 3.20, one-tailed; Fig. 2B).
Furthermore, comparing ERPs elicited by the averaged standards for blind participants in Phase A with Phase B, the cluster-based permutation test for the VAN revealed a significantly stronger negativity in Phase B than in Phase A (maximal cluster: sum(t) = −4106.97, p < 0.001, d = −0.56; Fig. 2C). This VAN effect included the electrodes marked in Figure 2C and lasted from 137 to 300 ms. In contrast, the test did not indicate any awareness-driven differences in the P3 between Phase A and Phase B (maximal cluster: sum(t) = 207.58, p = 0.20; Fig. 2D).
Electrophysiological measures of deviance processing
A vMMN effect was found in all three phases: in Phase A, the cluster-based permutation test revealed a significantly stronger negativity for the deviant than for the standard (maximal cluster: sum(t) = −566.76, p = 0.04, d = −0.32). This vMMN effect included the electrodes marked in Figure 3A, left, and lasted from 200 to 246 ms. Likewise, in Phase B, a vMMN effect was found (maximal cluster: sum(t) = −570.70, p = 0.04, d = −0.29; 215–281 ms; Fig. 3A, middle). Last, in Phase C, a significant vMMN cluster was revealed (maximal cluster: sum(t) = −3586.91, p < 0.001, d = −0.50; 200–300 ms; Fig. 3A, right).
For the oddball P3, the cluster-based permutation test comparing the standard and the deviant ERP did not reveal any significant clusters in Phase A (maximal cluster: sum(t) = 101.95, p = 0.40; Fig. 4A, left). Likewise, in Phase B, no oddball P3 effect was found (maximal cluster: sum(t) = 17.54, p = 0.74; Fig. 4A, middle). However, in Phase C, a significant oddball P3 cluster was found (maximal cluster: sum(t) = 9372.53, p < 0.001, d = 0.48; 300–600 ms; Fig. 4A, right).
Analyses of the vMMN cluster average yielded neither a significant main effect of phase (F(2,102) = 1.70, p = 0.19, BF01 = 2.92; Fig. 3B) nor a difference between blind and aware participants in Phase A (t(50) = 0.03, p = 0.98, BF01 = 3.56; Fig. 3C). Awareness of the sequential structure did not influence vMMN in Phase B (t(50) = 0.47, p = 0.64, BF01 = 2.80). Phase order did not affect the vMMN, neither in Phase B (t(50) = 0.96, p = 0.34, BF01 = 2.46) nor in Phase C (t(50) = −0.40, p = 0.69, BF01 = 3.36). Further testing phase order effects on vMMN separately in blind and aware subgroups did not indicate any significant influences (all p > 0.05, all BF01 < 2.50).
As no significant oddball P3 clusters were found in Phases A and B, cluster averages were extracted based on the significant Phase C effect. Analyses of the oddball P3 cluster average yielded a significant main effect of phase (F(2,102) = 29.56, p < 0.001, BF10 = 2.48 × 109; Fig. 4B), mirroring the results of the cluster-based permutation tests: both Phases A and B significantly differed from Phase C (all p < 0.001, all BF10 > 6.74 × 104), although Phases A and B did not differ (t(51) = 0.28, p = 0.78, BF01 = 6.38). Furthermore, cluster averages did not significantly differ from zero in Phase A (t(51) = −0.54, p = 0.59, BF01 = 5.77) and Phase B (t(51) = −0.89, p = 0.38, BF01 = 4.56).
No significant difference between blind and aware participants in Phase A was observed (t(50) = 0.70, p = 0.48, BF01 = 2.95; Fig. 4C). Awareness of the sequential structure did not influence the oddball P3 in Phase B (t(50) = −0.53, p = 0.61, BF01 = 2.83). Phase order did not affect the oddball P3, neither in Phase B (t(50) = 0.87, p = 0.39, BF01 = 2.63) nor in Phase C (t(50) = −0.77, p = 0.45, BF01 = 2.82). Further testing phase order effects on the oddball P3 separately in blind and aware subgroups did not indicate any significant influences (all p > 0.05, BF01 < 2.86).
Discussion
In the present study, we investigated the influence of awareness and task relevance on neural processing in an oddball paradigm. We showed that the VAN but not the P3 was associated with the general awareness of stimuli. Concerning deviance processing, we found evidence for a vMMN independently of awareness and task: In all three phases, a vMMN was elicited, even among inattentional blind participants in Phase A. However, an oddball P3 was only observed in Phase C, where the oddball stimuli were task relevant. Importantly, BFs indicated substantial evidence for null effects in stimulus discrimination in the blind subgroup and the P3 in Phases A and B. Thus, conscious processing of stimulus sequences does not necessarily evoke late positive components either associated with awareness per se or the oddball quality of stimuli, whereas early negativities are modulated by awareness and deviance.
Although the VAN emerged as a correlate of awareness in our experiment, the P3 did not. This finding is in line with previous work using the IB paradigm (Pitts et al., 2012; Shafto and Pitts, 2015; Schelonka et al., 2017), but stands in contrast to research favoring the P3 as a NCC (Sergent et al., 2005; Dehaene et al., 2006, 2014; Del Cul et al., 2007; Salti et al., 2012). Although studies supporting the latter usually relied on report-based trial selection, in our experiment report was only required after the experimental phases. This finding further corroborates that the P3 in these studies likely reflects task relevance and does not constitute an NCC proper (Tsuchiya et al., 2015; Koivisto et al., 2016). The VAN is believed to represent an index of phenomenal consciousness, that is, the emergence of subjective experiences also referred to as qualia (Koivisto and Revonsuo, 2010; Koivisto et al., 2017). On a physiological level its emergence seems to rely on recurrent processing within parts of the occipitotemporal cortex (Koivisto and Revonsuo, 2010; Lamme, 2010, 2018), however, the link between the VAN and its neurophysiological generators remains an open question.
The main aim of our experiment was to investigate the effects of awareness and task conditions on neural mechanisms of deviance processing. We observed both vMMN and oddball P3 in our experiment. However, they differed in how they interacted with awareness and the task. The vMMN was observable independently of these factors, which is in line with prior findings showing nonconscious effects between deviant and standards during binocular rivalry (Jack et al., 2017). Thus, the current study adds new evidence that the vMMN can also be elicited in this novel and ecologically valid no-report IB paradigm. The topography and timing of the effect was highly similar in all phases. However, upon visual inspection, the vMMN in Phase C seems stronger than in Phases A and B. Although this relation could not be substantiated statistically, a more pronounced vMMN in the task relevant Phase C, where oddball stimuli were fully attended and task relevant is not unlikely in the light of previous research (Wei et al., 2002; Kimura et al., 2010; but see Kuldkepp et al., 2013).
We found evidence for an oddball P3 to deviants only in the phase in which stimuli were task relevant. This contrasts with the findings of Silverstein et al. (2015), who found sustained positive activity in response to deviant compared with standard stimuli during masking. One possible explanation for these diverging findings might lie in the different task settings: whereas participants were distracted by a demanding task in our experiments, in Silverstein's study participants were instructed to attend to the masked stimuli despite not perceiving them. If this was indeed the case, the oddball P3 might depend on task-based attention alone without awareness as a prerequisite. Residual detection effects might represent an alternative explanation. The P3 in oddball designs has been shown to be influenced by various factors like local and global stimulus probability, novelty, motivational significance, salience, and attention devoted to the stimuli (Nieuwenhuis et al., 2005; Polich, 2007). On a physiological level it has been proposed that the oddball P3 is linked to phasic activity of the locus ceruleus-norepinephrine system (LC-NE; Nieuwenhuis et al., 2005). The function of the LC-NE system and therefore of the P3 is seen in a response facilitation to motivationally significant stimuli (Nieuwenhuis et al., 2005). From this viewpoint it is not surprising that we only observed an oddball P3 in Phase C, where shapes were task relevant.
Moreover, the current results could correspond to a proposed hierarchy of predictive processing (Clark, 2013; Stefanics et al., 2014). On lower levels of the processing hierarchy, a violation of expectations can be registered automatically, even for nonconscious stimuli, as indexed by the vMMN. On the contrary, on higher levels, processing as indexed by the oddball P3 might require a certain degree of attention to or relevance of the oddball sequence (Dehaene et al., 2006, 2014). Our findings, however, question the proposal that the P3 to deviant stimuli is a reliable marker of awareness (Bekinschtein et al., 2009; Faugeras et al., 2012; Strauss et al., 2015). Rather, whether or not deviance processing is indexed by a late positivity seems to depend on further conditions, including the attentional focus (Chennu et al., 2013). This notion fits nicely with the observed emergence of the P3 to deviants only if stimuli are attended for purposes of performing a task.
We would like to note that we do not challenge the hypothesis that late positivities are relevant for the quality and depth of conscious processing and are related to better stimulus discrimination and attentional capture. For example, Koivisto et al. (2017) found that, although the detection of stimuli was correlated with the VAN, stimulus identification was indexed by later widespread positivities. Furthermore, the extent of the P3 is linked to better performance and reaction times (Nieuwenhuis et al., 2005). There is no doubt that the P3 and attentional resources devoted to relevant stimuli are often highly correlated and, thus, will also be correlated with the amount, reportability and memory of processing (Lamme, 2003; van Boxtel et al., 2010; Pitts et al., 2018). From this point of view the P3 might also index a global neuronal workspace that functions as a form of access consciousness, including the strengthening of a given percept and inhibition of competing stimuli (Dehaene et al., 2006, 2014).
Last, it is important to point out possible limitations of our study. First, inattentional blindness studies have the advantage of controlling for task and awareness, but the disadvantage that only delayed reports of awareness can be used. Other no-report paradigms do not rely on reports at all, e.g., via eye movements (Frässle et al., 2014), or manipulate the report in a go/no-go design (Koivisto et al., 2016), which allows investigating awareness on a trial-by-trial basis. Furthermore, the delayed report opens the question whether IB participants really experience blindness or rather inattentional amnesia, i.e., perceiving stimuli but swiftly forgetting them (Wolfe, 1999; Lamme, 2006). Although we cannot completely rule out this possibility, one study addressing this issue found that the inability to report stimuli during IB stems from a perceptual deficit, not from memory failure (Ward and Scholl, 2015). Furthermore, our electrophysiological evidence agrees with the behavioral data in which participants unable to differentiate between shown and not shown shapes do indeed lack the VAN. If participants experienced amnesia instead of blindness, no VAN difference would be expected. One might argue that the VAN might thus represent memory consolidation instead of awareness. This, however, is unlikely because the component has been observed in a multitude of experiments where memory does not interfere with awareness assessment (Koivisto and Revonsuo, 2010). Notwithstanding, using a different type of no-report paradigm to substantiate the current findings is worth exploring in future research.
Furthermore, it remains unclear whether an oddball P3 can be elicited in other forms of oddball designs with supposedly stronger deviance effects, as for example in the classical 80/20% design. The roving oddball design properly controls for several confounds of the classical oddball design, but the relative frequency of a rare stimulus compared with a frequent stimulus and the investigation of the dynamics of neural correlates of deviance processing in different designs might change the outcome of studies. We have recently started to investigate these issues in subsequent studies.
Conclusion
In summary, this EEG study used a no-report paradigm to shed light on nonconscious processing of expectations and their violation, while disentangling influences of task relevance and awareness. The VAN but not the P3 indexed conscious stimulus perception, pointing to early processes in visual cortex rather than global and late processes as neural correlates of stimulus awareness. Expectation violation coincided with a vMMN in all experimental conditions, although an oddball P3 was only observed if stimuli were task relevant. This suggests that the P3 is not a reliable marker of awareness of stimulus processing during oddball tasks. However, these results are in line with hierarchical models of predictive processing, showing early automatic deviance processing but later components only for task relevant stimuli.
Footnotes
We thank Karen Vestring and Julia Krömer for their assistance in data collection.
The authors declare no competing financial interests.
- Correspondence should be addressed to Insa Schlossmacher at insa.schlossmacher{at}uni-muenster.de