Abstract
A central question in consciousness theories is whether one is dealing with a dichotomous (“all-or-none”) or a gradual phenomenon. In this 7T fMRI study, we investigated whether dichotomy or gradualness in fact depends on the brain region associated with perceptual awareness reports. Both male and female human subjects performed an emotion discrimination task (fear vs neutral bodies) presented under continuous flash suppression with trial-based perceptual awareness measures. Behaviorally, recognition sensitivity increased linearly with increased stimuli awareness and was at chance level during perceptual unawareness. Physiologically, threat stimuli triggered a slower heart rate than neutral ones during “almost clear” stimulus experience, indicating freezing behavior. Brain results showed that activity in the occipitotemporal, parietal, and frontal regions as well as in the amygdala increased with increased stimulus awareness while early visual areas showed the opposite pattern. The relationship between temporal area activity and perceptual awareness best fitted a gradual model while the activity in frontoparietal areas fitted a dichotomous model. Furthermore, our findings illustrate that specific experimental decisions, such as stimulus type or the approach used to evaluate awareness, play pivotal roles in consciousness studies and warrant careful consideration.
Significance Statement
The neural basis of consciousness is still highly debated as major theoretical and methodological questions on perceptual awareness and its assessment remain unsolved. Most studies have employed a dichotomous measure (yes/no, seen/unseen) to assess awareness and consequently found that consciousness is an “all-or-none” phenomenon. However, finer measures have revealed intermediate levels of awareness. It is still an open question whether affective signal processing, such as for body expressions, presents a gradual or a dichotomous relationship to perceptual awareness. We take up this challenge and show that perceptual awareness is gradual or dichotomous depending on the brain region considered, possibly reflecting different area functionality.
Introduction
The visual processing stream involves a hierarchical organization from early to higher order visual regions and feeds into prefrontal areas linked to perceptual decision making. Different consciousness theories have variously implicated one or another of these areas (Seth and Bayne, 2022). In this regard, the traditional approach to understand the neural basis of perceptual awareness has been to compare perception with and without awareness with a dichotomous measure (i.e., yes/no, seen/unseen responses). The issue with a dichotomous measure is that it may not capture intermediate states of experience and thus may not correctly differentiate genuine forms of nonconscious processing from residual conscious vision (Mazzi et al., 2016). For a full understanding of the neural bases of consciousness, it is important to elucidate how different levels of perceptual awareness differ among each other and lead to conscious perception.
This insight has led to the development of finer measures of perceptual awareness, such as the perceptual awareness scale (PAS), with four different response alternatives: “no experience,” “brief glimpse,” “almost clear experience,” and “clear experience” (Ramsøy and Overgaard, 2004). Recent studies using PAS have provided evidence of intermediate states of perceptual awareness between unseen and completely seen reports and have also reported chance performance during perceptual unawareness in objective forced-choice discrimination tasks (Ramsøy and Overgaard, 2004; Peremen and Lamy, 2014; Lähteenmäki et al., 2015; Lamy et al., 2015; Tagliabue et al., 2016; Lamy et al., 2017; Hesselmann et al., 2018; Lohse and Overgaard, 2019). These findings have therefore instigated debates about nonconscious processing but also sparked theoretical discussions about whether perceptual awareness is either a graded or an “all-or-none” phenomenon. Despite research efforts trying to solve this controversy, a consensus has not yet been achieved as both views are supported by strong empirical evidence (for a review, see Windey and Cleeremans, 2015).
While some important research has been conducted on the relation between emotion and consciousness (Barrett et al., 2007; Tsuchiya and Adolphs, 2007; Tamietto and de Gelder, 2010), emotional stimuli have not been the central focus in the long debates about perception without awareness or the dichotomous versus gradual discussions on consciousness. Indeed, in consciousness theories, such as global workspace, higher order, integrated information or re-entry, and predictive processing theories, the debates mainly concern cognitive processes (Seth and Bayne, 2022). A better understanding of how awareness relates to stimulus perception is especially important for affective stimuli because of the central role of subjective awareness in emotional experience. In this regard, fearful stimuli are considered a particularly strong candidate for nonconscious processing (Whalen et al., 1998; Morris et al., 2001; Vieira et al., 2017) and appear to gain privileged access to awareness in comparison with other emotions (Yang et al., 2007; Gray et al., 2013). Yet, most research on perceptual awareness and affective perception has used facial expressions, and it is not clear to what extent findings from (non)conscious perception of facial expressions generalize to other equally frequent stimuli. Available research on affective processing and awareness has already revealed differences between facial and bodily expressions for the same emotion expression. For example, Zhan et al. (2015) found that angry bodies had shorter suppression times in comparison with other bodily emotions, while angry facial expressions had the longest suppression times. It is still an open question whether affective signals, especially body expressions, are processed under conditions of perceptual unawareness and whether the perception of body expressions presents a gradual or a dichotomous relationship to perceptual awareness.
Here, we used continuous flash suppression (CFS) and 7 T (f)MRI scanning to investigate the processing of threat stimuli at different stages of perceptual awareness. Our methodological choices were motivated by the following considerations. First, CFS was used because it creates a stronger suppression and more stable nonconscious perception than other methods (Yang et al., 2014). Second, PAS was administered on a trial-by-trial basis to differentiate genuine forms unawareness from partial perceptual awareness and to assess whether perceptual awareness is a gradual or a dichotomous phenomenon. Finally, the use of body expressions provided a novel take on the processing of social information beyond facial expressions.
Materials and Methods
Participants and human ethics statement
Fifty-one healthy volunteers were recruited in this study. However, only 17 healthy volunteers (mean age, 20.69 years; age range, 19–29 years; 11 female; all right-handed) met the required criteria (see below, Practice runs) and participated in the fMRI experiment. Participants had normal or corrected-to-normal vision and a medical history without any psychiatric or neurological disorders. The experiment was approved by the Ethical Committee at Maastricht University and was performed in accordance with the Declaration of Helsinki. Participants provided informed written consent before the start of the experiment and received vouchers or credit points after their participation. In addition, participants remained unaware of the aim of the study until the completion of the experiment and were unfamiliar to the CFS paradigm.
Experimental design and procedure
Each participant took part in two scan sessions performed on separate days and in randomized order. In one of the sessions, six functional runs of the main CFS experiment were acquired as well as the anatomical data of the participant (∼2 h15 min). In the other session, resting-state data, the data of a body area localizer and a population receptive field localizer for motion-sensitive early- and mid-level visual cortex were acquired (∼1.5 h). The data of this session was not used for the current research aim and procedures.
Before participation in the scanning sessions, participants underwent a short behavioral experiment (∼30 min) on a separate day to ensure their eligibility for the CFS experiment. First, an eye dominance test was administered to determine the participant's dominant eye. Two practice runs were subsequently administered if the participant showed stable merging during the eye dominance test and did not display a strong suppression. Participants that did not meet these criteria were excluded from further participation in the study and their data were discarded. The same assessment procedure was performed after the practice runs. Therefore, only participants that met these criteria for both the eye dominance and practice runs were contacted for the fMRI sessions.
Eye dominance test
The dichotomous presentation was achieved in the same manner as in the main experiment using a cardboard and prism glasses, although outside the scanner. For this test, 10 different neutral faces (half male) belonging to the Radboud Faces Database (Langner et al., 2010) were selected. The face stimuli (318 × 212 pixels, 5.08° × 3.39° visual angle) were presented to one eye in the center of a rectangular frame (318 × 212 pixels, 5.08° × 3.39° visual angle, 10 pixels wide) while a dynamic colorful mask pattern (318 × 212 pixels, 5.08° × 3.39° visual angle) flashing at 10 Hz, and covering the other entire rectangular frame, was shown to the other eye. Each trial consisted of a gradual ramping up of the face stimulus contrast from 0% to full contrast over 1 s, which was maintained for another second and then followed by the diminishment of the stimuli contrast to 0% over 0.5 s and a 0.5 s blank period. During these 3 s, the contrast of the dynamic colorful mask remained constant. Next, a fixation dot appeared in the screen indicating participants to report whether they saw or did not see a face by pressing one out of two keys (“J” for seen, “K” for unseen). Each stimulus was randomly presented three times to each eye, giving a total of 60 trials. Eye dominance was defined as the eye that perceived the highest amount of seen trials while it was assigned randomly in the cases where the amount of seen trials was equal between both eyes.
Practice runs
Two practice runs were administered with an identical experimental design to the main task of the fMRI session. The only difference was a shorter intertrial interval (ITI) and a shorter interval between the 2 s CFS period and response periods. The aim of these practice runs was to familiarize participants with the PAS as well as to train them to respond as accurate and fast as possible given the short response window (∼1.5 s) of the main experiment.
Main experiment
In the main CFS experiment, the participants' non-dominant eye was presented with a static body posture while a colorful Mondrian mask flickering at 10 Hz was presented to the dominant eye. Dichotomous presentation was accomplished using a cardboard panel and a pair of prism glasses. The cardboard was placed between the mirror attached to the head coil and the screen, dividing it into two halves and ensuring that each eye only perceived half of the screen. The prism glasses (diopter = 6) bent the light in a way that the ipsilateral image was shifted back to the center of each eye (as described in Schurger, 2009). Both the body stimuli and the colorful mask were displayed on a gray background (RGB value = 128, 128, 128) within a black rectangular frame (frame thickness = 10 pixels; frame size = 318 × 352 pixels; 5.08° × 5.62° visual angle) that had a fixation cross at its center, respectively, which facilitated the merging of the two images (Fig. 1).
Schematic view of a trial presentation sequence in the main experiment. After a 1 s fixation period, a 2 s CFS period started with the gradual ramping up of the body stimulus contrast from 0% to full contrast over 1 s, followed by the contrast reduction to 0% within 0.5 and a 0.5 s blank period (see content within frame). The contrast of the dynamic colorful mask was constant throughout the 2 s. However, both the body stimuli and the mask contrasts were determined for each trial using a staircase procedure with 10 steps (body stimuli: 5, 14, 23, 32, 41, 50, 50, 50, 50, 50%; noise: 100, 100, 100, 100, 100, 82, 64, 46, 28, 10%) that depended on the participant's visual experience of the stimulus in the previous trial. After a jittered fixation period (4-6-8 s), participants were required to make two active responses, each within a 1.5 s window: a two-alternative forced-choice task (fear vs neutral) and the rating of their visual experience of the stimulus according to the PAS. The ITI was jittered (4-6-8 s) and the average trial duration was 18 s.
The body stimuli were selected from a large validated stimulus set of still whole-body images (Stienen and de Gelder, 2011) and consisted of a total of 16 distinct images, with eight different actor identities (half females) portraying a fearful or a neutral (opening door) body expression, respectively (318 × 182 pixels, 5.08° × 2.91° visual angle). The facial information was removed to avoid triggering processes specifically related to facial perception. The body postures were presented either to the right or left side of the fixation cross in a randomized order. The colorful Mondrian mask consisted of 600 unique patterns flashing randomly at 10 Hz, which were composed of overlapping small rectangles covering the entire rectangular frame (Fig. 1).
Once the participant reported stable perception of a single rectangle, the experimental run started with a 12 s fixation period. A change in the fixation cross color from black to white indicated the start of each trial and remained white for the whole trial duration. Each trial started with a one-second white fixation period, followed by a 2 s CFS presentation consisting of a gradual increase of the body stimulus contrast over 1 s, followed by the ramp down of the contrast back to 0% within 0.5 s and a 0.5 s blank period. The gradual increase of the stimulus contrast was performed to decrease the likelihood of the body stimulus escaping suppression. The contrast of the colorful Mondrian mask was constant throughout the 2 s CFS presentation within each trial. However, both the contrast of the body stimuli and the Mondrian mask were determined for each trial using a staircase procedure with 10 steps (body stimuli: 5, 14, 23, 32, 41, 50, 50, 50, 50, 50%; noise: 100, 100, 100, 100, 100, 82, 64, 46, 28, 10%) that depended on the participant's visual experience of the body stimulus in the previous trial. If participants reported not seeing anything in the colorful noise, the maximum contrast of the body stimuli increased one step while the contrast of the mask decreased, also by one step. Each run started at step 5 (i.e., 41% contrast for body stimulus and 100% for the noise). This staircase procedure was intended to balance the number of trials per perceptual awareness condition.
The 2 s CFS period was followed by a jittered fixation period (4-6-8 s) after which participants were required to make two responses. The first response required participants to categorize the body stimulus in a two-alternative forced-choice manner (fearful vs neutral) by pressing one of two buttons. The assignment of the two buttons was randomized. Subsequently, participants had to indicate their visual experience of the stimulus according to the PAS by pressing one out of four buttons: “no experience” (PAS1), “brief glimpse” (PAS2), “almost clear experience” (PAS3), and “clear experience” (PAS4). The button assignment was kept constant for this task to facilitate a quick response. Both responses were required even when participants reported not seeing anything in the noise. In those cases, participants were instructed to guess the emotional expression of the stimulus. Both answers had to be given within a 1.5 s window each and always with the right hand. In addition, participants were instructed to keep as still as possible throughout the experiment, to always fixate on the cross and not to blink within the 2 s CFS period. Each response period was followed by a jittered ITI (4-6-8 s), resulting in an average trial duration of 18 s and an average run duration of 10 min approximately. Each run was composed of 32 trials, 16 per emotional condition, with two repetitions for each of the eight body stimulus identities. Therefore, a total of 192 trials were obtained, 96 for each emotional category. One participant only performed four runs due to delays in the scanning. Two participants performed seven runs instead of six. One run of three participants and two runs of another were discarded due to excessive motion.
The experiment was presented in MATLAB R2012a (MathWorks) using Psychtoolbox 3.0.11 (Brainard and Vision, 1997; Pelli and Vision, 1997). The stimuli were back-projected on a translucent screen situated at the end of the scanner bore, behind participants' heads (Panasonic PT-EZ570; screen size = 30 × 18 cm; screen resolution = 1,920 × 1,200 pixels; refresh rate = 60 Hz, visual angle = 17.23° × 10.38°). Participants viewed the screen through a tilted mirror attached to the head coil. The distance between the mirror and the screen was ∼99 cm. Participant responses were recorded using an MR-compatible button box (Current Designs, 30 8-button response device, HHSC-2×4-C).
Behavioral data analysis
Behavioral data were analyzed with SPSS (version 22.0; IBM) and custom code in MATLAB R2020a (MathWorks). First, trials without a response for one or both tasks were excluded from further analyses. Trials in which reaction times deviated >3.5 times the standard deviation from the mean (within run and subject) were also removed. In total, 136 out of 3,239 trials (4.2%) were excluded from further analyses.
Participants' responses in the two-alternative forced-choice task were counted as hits (H), misses (M), correct rejections (CR), and false alarms (FA) according to Signal Detection Theory (SDT) (Tanner and Swets, 1954; Green and Swets, 1966). Hits refer to trials in which fearful bodies were correctly categorized, while misses to those trials in which participants incorrectly categorized fearful bodies as neutral. Correct rejections indicate trials where neutral bodies were correctly categorized whereas false alarms to the trials where neutral stimuli were incorrectly categorized as fearful ones.
To further understand participants' responses, the perceptual sensitivity (d′) and the response criterion or bias (c) were calculated for each PAS level. Sensitivity is commonly calculated by subtracting the z-transformed false alarm rates from the z-transformed hits (Eq. 3) and therefore reflects the distance between the target (fearful body) and noise (neutral body) distribution means, in standard deviation units. Here, a modified form of hit (H′) and false alarm (FA′) rates was used to account for ceiling effects, as proposed by Snodgrass and Corwin (1988; Eqs. 1, 2). Higher sensitivity values indicate higher discriminability of fearful bodies from neutral ones. A value of zero indicates inability to distinguish fearful body expressions from neutral ones. Independent from sensitivity, criterion bias was calculated by multiplying the sum of the z-transformed hit and false alarm by −0.5 (Eq. 4). It reflects the distance between the neutral point (where responses are not biased toward fearful bodies nor neutral ones) and the response criterion, in standard deviation units. Negative response criterion values indicate a bias in reporting the presence of a fearful body over a neutral one (liberal criterion), while positive values show the opposite response pattern (conservative criterion).
Next, to investigate whether perceptual awareness is a gradual or a dichotomous phenomenon, we fitted two linear mixed models to the sensitivity data with different predictor definitions. In the gradual model, the predictors modelled a linear relationship between sensitivity and the PAS levels. In the dichotomous model, the predictor for the PAS1 level was set to zero while the rest of the PAS levels were set to 1, describing an “all-or-none” relationship between recognition sensitivity and perceptual awareness. These two models were performed independently for each participant. To select the model that best represented the recognition sensitivity pattern across PAS levels at the group level, the values corresponding to the Bayesian information criterion (BIC) resulting from each model fitting were analyzed with a paired-sample t test (gradual vs dichotomous). The model with the significantly lower BIC value (which indicates better fit) was selected as the final model.
Finally, the reaction times (RTs) of the emotional recognition task were analyzed using a linear mixed model with SDT (four levels = H, M, FA, CR) as the within-subject factor. The RTs of the perceptual awareness task were also analyzed with a linear mixed model with within-subject factor perceptual awareness (four levels = PAS1, PAS2, PAS3, and PAS4). Both analyses used the Toeplitz covariance matrix for repeated measures.
Eye dominance was deliberately left out as a fixed factor in our study because a prior experiment (N = 30) employing a comparable experimental paradigm showed no statistically significant main effects or interactions. This aligns with consistent findings in related CFS research (e.g., Salomon et al., 2013; Han and Alais, 2018; Zhan et al., 2019). Additionally, the effects of eye dominance extend beyond the scope of our current investigation and would have compromised statistical power due to the smaller sample size.
(f)MRI data acquisition
(f)MRI data were acquired with a 1-transmitter/32-receiver head coil (Nova Medical) in a 7 T Magnetom whole-body scanner (Siemens Medical Systems) located at the Maastricht Brain Imaging Centre (MBIC), the Netherlands. Functional images were obtained using a 2D gradient echo (GE) echo-planar imaging (EPI) sequence [voxel size = 1.2 mm isotropic; no gap; repetition time (TR) = 2,000 ms; echo time (TE) = 21 ms; flip angle (FA) = 75°; in-plane field of view (FoV) = 172.8 × 172.8 mm2; matrix size = 144 × 144; number of slices per volume = 70; multiband acceleration factor = 2; iPAT = 3; phase encoding direction = anterior to posterior; bandwidth = 1,488 Hz/Px; echo spacing = 0.78 ms; number of volumes = 300 (main experimental runs), 440 (body area localizer), 315 (pRF mapping), 330 (resting state)]. The slice positioning of the functional images was performed in a way to include the occipital, parietal, and frontal lobes as well as the amygdala, thus ensuring a good coverage of important areas in body perception. However, limited coverage was obtained for the superior part of the motor cortex, anterior temporal lobe, and orbitofrontal cortex. For distortion correction of the functional images, a short run (5 volumes) was acquired before each experimental run with the same parameters specified above but with opposite phase encoding direction (posterior-to-anterior). Anatomical images were acquired for each participant using a MP2RAGE sequence (voxel size = 0.65 mm isotropic; FoV = 207 × 207 mm2; matrix size = 320 × 320; T1-weighted: TR = 5,000 ms; TE = 2.51 ms; inversion time (IT) 1 = 900 ms; IT2 = 2,750 ms; FA1 = 5°; FA2 = 3°; iPAT = 2; bandwidth = 250 Hz/Px; echo spacing = 7 ms). Dielectric pads covering the occipital and temporal lobes were used for all participants.
(f)MRI data preprocessing
The preprocessing and analysis of the (f)MRI data were performed in BrainVoyager (v22.0; Brain Innovation B.V.) as well as with custom code in MATLAB (vR2020a; The MathWorks). First, functional images underwent top-up distortion correction with the COPE (Correction based on Opposite Phase Encoding) plugin (v1.1.1) in BrainVoyager based on the voxel displacement between the first volume of the functional run and that of the distortion correction run. Subsequently, slice scan time correction was applied to the functional runs using sinc interpolation. Functional images then underwent 3D rigid motion correction with respect to the first volume of each functional run (trilinear/sinc interpolation). Linear trend removal and high-pass temporal filtering were employed to exclude low-frequency drifts using a general linear model (GLM) Fourier basis set with two cycles per time course. In order to reduce the B1 bias field, the anatomical data were background-noise corrected by dividing the UNI image by the T1w image and then masking the resulting ratio image by the INV2 image. In addition, the structural data were corrected for intensity inhomogeneities and upsampled to 0.6 mm isotropic resolution (sinc interpolation; framing cube = 384) to best match the resolution of the functional data. After these steps, each preprocessed functional run was aligned to the first run of the main experiment and normalized to Talairach space. The resulting coregistered images were then spatially smoothed with a Gaussian kernel of a full-width half-maximum of 3 mm. A group-averaged anatomical image was created by averaging the Talairach-normalized anatomical data across participants. All analyses were performed in volume space.
A cortex-based alignment (CBA) procedure was carried out for visualization of group results in surface space. First, the T1-weighted anatomical data of each participant were downsampled to 0.7 mm isotropic (for better software results) and subsequently underwent a DNN-based segmentation procedure in BrainVoyager (strides value slow = 32 × 32 × 32). This approach classified each anatomical voxel into eight possible tissue types, including white matter, gray matter, cerebrospinal fluid, blood vessels, ventricles, subcortical structures, sagittal sinus, and background. With this information, all the individual anatomical UNI datasets were then segmented at the gray–white matter boundary, upsampled to 0.6 mm isotropic and normalized to Talairach space. After this step, manual corrections were performed when necessary, on a slice-by-slice basis. The cortical surfaces were then reconstructed, inflated, smoothed, and mapped onto a high-resolution standard sphere, separately for each hemisphere (vertices = 163,842). A dynamic group averaging approach based on individual curvature information was used to align participants' reconstructed cortical surfaces. After alignment, an averaged folded cortical mesh (N = 17) was created for each hemisphere.
Physiological data acquisition, preprocessing, and noise correction of fMRI data
Cardiac and respiratory measures were acquired using an oximeter (50 Hz) and a pneumatic compression belt (50 Hz) to control for physiological fluctuation effects on the BOLD response (Glover et al., 2000). Low-frequency drifts were removed from the raw cardiac (bandpass, 0.5–8 Hz) and respiratory (low-pass, 2 Hz) data and signal peaks were identified (Elgendi et al., 2013). Physiological noise correction was performed using a modification of the conventional Retrospective Image Correction (RETROICOR) procedure (Glover et al., 2000; Harvey et al., 2008; Hutton et al., 2011). In the current procedure, a cardiac and a respiratory phase were assigned to each functional image using third-order cardiac and fourth-order respiratory harmonics. The respiratory phase not only considered the respiratory timing but also the depth of the breathing (Glover et al., 2000). In addition, a cardiorespiratory interaction (first order) term was defined (Harvey et al., 2008). A total of 20 regressors were created, including 6 cardiac phase regressors, 8 respiratory phase regressors, 4 cardiorespiratory interaction regressors, as well as a filtered heart rate and respiratory rate regressor.
(f)MRI data analysis
Anatomical definition of amygdala and pulvinar
The pulvinar and the amygdala were anatomically defined in each subject using the Chakravarty Atlas (Chakravarty et al., 2006) given their known involvement in nonconscious processing (Tamietto and de Gelder, 2010). The definition of pulvinar covered all different subnuclei of both the left and right hemisphere. Amygdala definition also covered all subnuclei of both the left and right hemisphere and was manually modified according to individual anatomy when necessary.
Definition of regions of interest sensitive to perceptual awareness
Apart from the anatomically defined pulvinar and amygdala, we aimed to identify other brain regions that are influenced by perceptual awareness. For this purpose, a whole-brain fixed-effects GLM was performed for each subject, individually, with the 3 mm-smoothed percent-signal normalized functional data. The GLM included as predictors of interest four predictors corresponding to the perceptual awareness levels (i.e., PAS1–4), a parametric predictor of the mask contrast, a predictor for “no response” trials, and a predictor for each of the two response windows. These predictors were convolved with a two-gamma hemodynamic response function. In addition, six motion predictors and 20 physiological predictors (see previous section) were included in the design matrix as nuisance predictors. For each subject, a beta map for each perceptual awareness condition was obtained and entered into a group repeated-measures ANOVA in BrainVoyager. One subject was excluded from this analysis due to a missing condition (PAS4). The resulting t-map showing the main effect of PAS at the group level was corrected for multiple comparisons using a cluster-threshold procedure based on Monte Carlo simulations (initial p-value = 0.001; α-level = 0.05). The resulting clusters were then defined as regions of interest (ROIs) in each subject for subsequent analyses.
Analysis of the defined ROI data
The activity of both the anatomically defined (hypothesis-driven) and the functionally defined (data-driven) ROIs were analyzed to investigate their role in the processing of body expressions under different degrees of perceptual awareness. For this end, a fixed-effects GLM similar to the one described above was performed for each subject with the 3 mm-smoothed functional data. However, the predictors corresponding to the perceptual awareness levels were now separated according to the emotional category of the stimulus (i.e., N1, N2, N3, N4, F1, F2, F3, F4; N = neutral; F = fear). For each subject, the β values corresponding to these eight main conditions were extracted from each ROI (functionally and anatomically defined) and entered into a linear mixed model analysis in SPSS. This analysis was performed for each ROI separately and included two within-subject factors: emotion (two levels = neutral, fear) and perceptual awareness (four levels = PAS1, PAS2, PAS3, PAS4). All analyses used the unstructured covariance matrix for repeated measures. Multiple comparisons were corrected within each ROI with the Sidak method and across ROIs with the Benjamini–Hochberg false discovery rate (BHFDR) method. To examine whether ROI activity was consistently above or below baseline, a one-sample t test against 0 was performed for each of the eight experimental conditions within each ROI (FDR correction at q < 0.05).
As with the sensitivity data, we conducted further analyses to investigate whether brain activity showed a gradual or a dichotomous relationship to perceptual awareness. Four linear mixed models (two models [gradual and dichotomous] by two emotions [neutral and fear]) were fitted into the data of each ROI and subject, respectively. The resulting BIC values were entered into a repeated-measures ANOVA with within-subjects factor model (two levels = gradual and dichotomous) and emotion (two levels = neutral and fear). In the cases where there was a significant effect of model but not a significant model*emotion interaction, a paired t test was performed between the coefficient estimates of the neutral and fearful models to assess how different the model slopes and intercepts were across emotions. In the cases where there was a significant model*emotion interaction, a different model was selected for each emotion. When no significant effect of emotion and model were found, as well as no significant interaction, two model fittings were performed (gradual and dichotomous) after averaging the ROI data across emotions. Subsequently, a paired-sample t test was performed with the resulting BIC values from each model fitting. The BHFDR method was used to correct for multiple comparisons across ROIs and the Sidak method to correct them within each ROI.
Cardiac data analysis
For each trial, the systolic peaks corresponding to the 2 s CFS period were identified. Peaks beyond a biologically feasible range were rejected (i.e., beats per minute <35 or >180) as well as outliers that were 2.5 standard deviations from the mean. Subsequently, the mean heart rate (beeps per minute) was obtained by averaging the time differences between consecutive peaks. These average estimates were baseline corrected with respect to the mean heart rate corresponding to the 1 s preceding each CFS period. The resulting values were entered into a linear mixed model procedure with within-subject factors emotion (two levels = anger and fear) and perceptual awareness (four levels = PAS1, PAS2, PAS3, and PAS4). The compound symmetry covariance matrix for repeated measures was used as well as the Sidak method to correct for multiple comparisons. Two outliers (single data points within the whole sample) were removed based on their standardized residuals, resulting in a model with significantly better fit.
Results
Behavioral results
Sensitivity
The analysis of recognition sensitivity showed a significant main effect of perceptual awareness (F(3,41.76) = 37.13; p < 0.001), indicating a significantly higher sensitivity during “brief glimpse” (PAS2, M = 0.38; SE = 0.08), “almost clear” (PAS3, M = 1.32; SE = 0.13), and “clear” (PAS4, M = 1.36; SE = 0.18) experience conditions than that during “no experience” (PAS1, M = −0.03; SE = 0.03). Sensitivity was also significantly lower during “brief glimpse” than for “clear” and “almost clear” experience (Fig. 2A). Sensitivity values differed from the chance level in PAS2 to PAS4 (p < 0.001), but not in PAS1 (p = 0.352). Additional analyses were conducted to examine whether a gradual or dichotomous model provided a better fit to the data, as determined by lower BIC values. The results indicated that the relationship between perceptual sensitivity and perceptual awareness was significantly better described by a gradual model (M = 1.05; SE = 1.61) than that by a dichotomous one (M = 6.61, SE = 1.13; t(16) = −3.69, p = 0.002).
Overview of behavioral results. A, Estimated marginal means of sensitivity values separated by PAS levels. Shadowed area indicates standard error from the mean. B, Estimated marginal means of criterion bias values separated by PAS levels. Shadowed area indicates standard error from the mean. C, Average proportion of responses separated by PAS and SDT measures (i.e., hit, miss, false alarm, correct rejection). It is important to note that the distribution of response patterns varies across different PAS levels since the classification relies on participants' subjective reports. The sum of these proportions across all PAS levels and signal detection measures is 1. Hit = correctly categorized fearful stimuli; Miss = fearful stimuli categorized as neutral; False alarm = neutral stimuli categorized as fearful; Correct rejection = correctly categorized neutral stimuli. Notes: Black asterisks denote significant differences between PAS levels. Rhombi denote significant difference from zero; */◆;, p < 0.05; **/◆;◆;, p < 0.01; ***/◆;◆;◆;, p < 0.001. Abbreviations: PAS, perceptual awareness scale; PAS1, “no experience”; PAS2, “brief glimpse”; PAS3, “almost clear experience”; PAS4, “clear experience.”
Criterion bias
The criterion bias analysis showed a significant main effect of perceptual awareness (F(3,23.17) = 26.01; p < 0.001) with less conservative emotional categorizations for “clear” (M = −0.92; SE = 0.05) and “almost clear” (M = −0.88; SE = 0.05) experiences than “no experience” (M = −0.31; SE = 0.05) or “brief glimpse” (M = −0.46; SE = 0.05; Fig. 2B). Criterion bias values did not differ significantly between “clear” and “almost clear” experiences as well as between “no experience” and “brief glimpse”. Criterion bias scores were different from zero at all levels of the PAS (p < 0.001).
Reaction times
The analysis of RTs of the emotional recognition task did not show differences between different response types (F(3,27.62) = 0.78; p = 0.513). The analysis of RTs of the perceptual awareness task showed a main effect of perceptual awareness (F(3,21.77) = 5.29; p = 0.007), indicating significant faster responses for “no experience” (M = 0.48; SE = 0.04) than those for “brief glimpse” (M = 0.56; SE = 0.04) and marginally faster than those for “clear” ratings (M = 0.58; SE = 0.04).
Brain results
Regions sensitive to perceptual awareness
A group ANOVA with within-subjects factor perceptual awareness (i.e., PAS1–4) was performed to localize the areas sensitive to different degrees of perceptual awareness. ROIs showing a main effect of perceptual awareness were found bilaterally in the lateral occipitotemporal cortex (LOTC), fusiform gyrus, inferior temporal gyrus (ITG), as well as right amygdala and left precuneus, occipital cortex, intraparietal sulcus (IPS), inferior frontal cortex (IFC), superior temporal gyrus (STG), and posterior superior temporal sulcus (pSTS; Fig. 3; see Table 1 for more details on ROI location, size, and statistical values).
Areas showing a main effect of perceptual awareness at the group level. These ROIs resulted from the repeated-measures ANOVA (N = 16) with within-subject factor perceptual awareness (four levels = PAS1–4; cluster size corrected with Monte Carlo simulation, α level = 0.05; initial p = 0.001; numbers of iterations = 5,000). Abbreviations: FG, fusiform gyrus; IFC, inferior frontal cortex; IPS, intraparietal sulcus; ITG, inferior temporal gyrus; LOTC, lateral occipitotemporal cortex; OG, occipital gyrus; preCS, precentral sulcus; pSTS, posterior superior temporal sulcus; STG, superior temporal gyrus. See Table 1 for more details regarding ROI localization and size.
Information details of the clusters resulting from the group ANOVA with within-subject factor perceptual awareness scale (four levels = PAS1–4; cluster size corrected with Monte Carlo simulation; α level = 0.05; initial p = 0.001; numbers of iterations = 5,000)
Effect of emotion and perceptual awareness on PAS-sensitive ROI activity
To further understand their involvement in perceptual awareness, we then analyzed the β values of these ROIs with a linear mixed-model analysis with within-subject factors emotion (two levels = neutral and fear) and perceptual awareness (four levels = PAS1–4). All the ROIs showed a significant main effect of PAS but no significant main effect of emotion or interaction effect (see Fig. 4 for ROI activity visualization; see Extended Data Table 4-1 for further statistical details). Subsequently, different pairwise comparisons were performed to understand differences in activity levels between nonconscious and conscious perception (PAS1 vs PAS4; PAS1 vs PAS3), between nonconscious and threshold vision (PAS1 vs PAS2), and between threshold and clearer degrees of conscious perception (PAS2 vs PAS4; PAS2 vs PAS3) and to assess whether there was a difference between levels of conscious perceptual awareness (PAS3 vs PSA4) in the different ROIs.
Brain responses across perceptual awareness levels separately for each stimuli type. Brain responses represent %-BOLD signal changes of the ROIs showing a main effect of PAS as well as the anatomically defined amygdala and pulvinar. Shadowed area indicates standard error from the mean. Notes: Black asterisks denote significant differences between PAS levels; * = pcorrected < 0.05; ** = pcorrected < 0.01; *** = pcorrected < 0.001; † = puncorrected < 0.05. Circles denote activity level significant difference from baseline at each PAS level for neutral (blue) and fearful (red) conditions. Abbreviations: aIPS, anterior intraparietal sulcus; def., definition; FG, fusiform gyrus; Fourth OG, fourth occipital gyrus; IFC, inferior frontal cortex; ITG, inferior temporal gyrus; LOTC, lateral occipitotemporal cortex; mIPS, medial intraparietal sulcus; PAS, perceptual awareness scale; pIPS, posterior intraparietal sulcus; preCS, precentral sulcus; pSTS, posterior superior temporal sulcus; SOG, superior occipital gyrus; STG, superior temporal gyrus; [l], left hemisphere; [r], right hemisphere. See Table 2 and Extended Data Tables 4-1–4-3 for further statistical information.
Table 4-1
Results of the linear mixed model analysis with within-subject factors Emotion and PAS, separately for each ROI. Download Table 4-1, DOCX file.
Table 4-2
Results of comparing ROI activity against baseline for each Emotion x PAS condition. Download Table 4-2, DOCX file.
Table 4-3
Analyses on the gradual vs. dichotomous model fitting of neural data Download Table 4-3, DOCX file.
Significantly higher activity for PAS4 than PAS1 was observed in all ROIs with the exception of the left superior occipital gyrus (SOG), left STG, and two areas in the left fourth occipital gyrus (Fig. 4). Overall, similar results were observed for the PAS3 and PAS1 comparison. The ROIs in the left fourth occipital gyrus, left STG and SOG showed the opposite pattern, with higher activity for PAS1 than PAS3 and PAS4 (Fig. 4), although only significantly for the former two ROIs. Significantly higher activity for PAS4 in comparison with PAS2 was found in bilateral LOTC and ITG, right amygdala, and left pSTS, although not surviving multiple-comparisons correction. Pairwise comparisons revealed that the activity for PAS3 and PAS4 conditions was not significantly different in any of the defined ROIs. Most of the ROIs showed a significantly different activity level between PAS2 and PAS1 conditions, except the bilateral ITG, left IPS, and left LOTC.
Overall, most of the defined ROIs showed an activity level significantly different from baseline across the eight emotion*perceptual awareness conditions (see Extended Data Table 4-2 for further statistical details; see Fig. 4 for ROI activity visualization). An exception was found in the right amygdala and bilateral ITG, where the activity level was not significantly different from zero at PAS1 for both emotions, and at PAS2 for fear. In addition, the activity in the left precuneus did not differ from zero at PAS1 for fearful body expressions. The activity in the left STG did not differ from zero at any of the emotion*perceptual awareness conditions.
To further investigate whether perceptual awareness is a gradual or a dichotomous phenomenon, two linear mixed models corresponding to each phenomenon were fit to the data. The activity pattern in the right LOTC was significantly better described by a gradual model, regardless of the emotion of the stimuli (see Extended Data Table 4-3 for further statistical details; see Table 2 for results on model slope and intercept comparisons). The responses in the left medial IPS and IFC were significantly best described by a dichotomous model, also with no differences across emotions. Yet, emotion specificity was found in some areas. The activity pattern in the SOG was different for the neutral and fearful body expressions across all PAS levels (i.e., significant interaction effect). For neutral bodies, SOG activity was significantly best described by a gradual model while in the case of fearful bodies, the activity was better represented by a dichotomous model, although it did not reach significance in the latter case. The responses of the left anterior IPS also showed an emotion*perceptual awareness interaction effect, with the activity elicited by neutral bodies better described by a dichotomous model while a gradual model better described the activity elicited by fearful body expressions. However, pairwise comparisons were not significant in both cases. Other ROIs did not show a significant preference for either model.
Analysis of model slopes and intercepts of neural data
Effect of emotion and perceptual awareness in anatomically defined amygdala and pulvinar
The analysis of amygdala betas yielded a significant main effect of perceptual awareness (F(3,14.39) = 8.60; p = 0.002), showing that amygdala activity was significantly lower during “no experience” (M = 0.08; SE = 0.09) than during “brief glimpse” (M = 0.31; SE = 0.06), “almost clear” (M = 0.40; SE = 0.07), and “clear experience” (M = 0.58; SE = 0.13; Fig. 4). This analysis also yielded a significant main effect of emotion (F(1,13.23) = 6.67; p = 0.023), indicating a higher amygdala activity for neutral (M = 0.38; SE = 0.07) than fearful body expressions (M = 0.30; SE = 0.07). One-sample t tests revealed that amygdala activity differed from baseline at PAS2 to PAS4 (p < 0.001) but not at PAS1 (i.e., “no experience”), for both emotions. Amygdala activity did not show a significant preference for either the gradual or the dichotomous model.
The analysis of the activity in pulvinar did not show significant main effects for emotion or perceptual awareness nor a significant emotion*perceptual awareness interaction (Fig. 4). However, pulvinar responses were significantly different from zero in all PAS levels for both emotions (p < 0.005). As for the amygdala, pulvinar did not show a significant preference for either the gradual or the dichotomous model.
Heart rate
The analysis of heart rate responses yielded a marginal significant effect of perceptual awareness (F(3,106.30) = 2.68; p = 0.051) and a significant emotion*perceptual awareness interaction (F(3,106.03) = 2.75; p = 0.047), indicating that at PAS3, heart rate in response to fearful bodies (M = −3.59; SE = 0.62) was slower than to neutral bodies (M = −1.91; SE = 0.63). For fearful expressions, heart rate was marginally significantly higher during “no experience” (M = −1.54; SE = 0.63) and “clear” experience (M = −1.50; SE = 0.63) than during “almost clear” experience (M = −3.59; SE = 0.62; Fig. 5).
Heart rate across perceptual awareness levels and emotional stimuli categories. Heart rate (beats per minute) was baseline corrected and averaged over the 2 s CFS period. Shadowed area indicates standard error from the mean. Abbreviations: *p < 0.05; HR, heart rate; PAS, perceptual awareness scale.
Discussion
This ultrahigh field fMRI study investigated the perception of threat body stimuli in healthy participants using CFS in combination with PAS. Behaviorally, we found a gradual relationship between recognition sensitivity and perceptual awareness and no evidence of perceptual discrimination without perceptual awareness. Heart rate was slower for fearful than neutral bodies during almost clear stimulus perception, indicating freezing behavior. At the brain level, the activity in occipitotemporal, parietal, and frontal regions as well as in amygdala increased with increased stimulus awareness, while the activity in early visual areas showed the opposite pattern. The relationship between temporal cortex activity and perceptual awareness was better characterized by a gradual model while the activity in frontoparietal areas by a dichotomous model, suggesting different roles in conscious processing. Interestingly, no evidence of nonconscious body expression processing was found in the amygdala as well as no increased activation for threat stimuli, in contrast to some findings in the literature (Tamietto and de Gelder, 2010).
Behavioral evidence for the gradual account but not for nonconscious processing
The behavioral results revealed a continuum of intermediate states of perceptual awareness (Fig. 2C) as well as a linear increase in recognition sensitivity with increased perceptual awareness (Fig. 2A), replicating our earlier study (Poyo Solanas et al., 2022). Support for the gradual view has been found for the perception of low-level features such as color or shape (Ramsøy and Overgaard, 2004; Overgaard et al., 2006; Wierzchoń; et al., 2014; Windey et al., 2014; Lähteenmäki et al., 2015) but also for high-level object and semantic (e.g., emotion) perception (Lähteenmäki et al., 2015; Lohse and Overgaard, 2019; Poyo Solanas et al., 2022).
In contrast with previous work reporting nonconscious processing of emotional information in healthy participants (Watanabe and Haruno, 2015; Khalid and Ansorge, 2017; Vieira et al., 2017), we found no behavioral evidence for body expression discrimination during perceptual unawareness (Fig. 2A). One possible reason for these divergent results may be that most studies reporting nonconscious affective processing used facial expressions (Tamietto and de Gelder, 2010) and emotional faces and bodies may be processed differently during CFS (Zhan et al., 2015). Another possible reason may be due to methodological differences regarding the assessment of perceptual awareness. Earlier studies mostly relied on dichotomous measures, which may not be adequate for correctly differentiating true perceptual unawareness from partial/degraded states of perceptual awareness (Mazzi et al., 2016). In agreement with our findings, growing evidence fails to find evidence for emotion processing outside conscious awareness when using finer scales of perceptual awareness in combination with objective force-choice discrimination tasks to assess performance (Ramsøy and Overgaard, 2004; Peremen and Lamy, 2014; Lähteenmäki et al., 2015; Lamy et al., 2015; Tagliabue et al., 2016; Lamy et al., 2017; Hesselmann et al., 2018; Lohse and Overgaard, 2019; Poyo Solanas et al., 2022) as well as when controlling for other methodological confounds (Hedger et al., 2015; Rajananda et al., 2020).
Different brain areas involved in gradual and “all-or none” perceptual awareness
Several areas spanning early visual as well as temporal, parietal, and frontal regions were sensitive to perceptual awareness. Activity increased with increased stimulus sensitivity in high-level visual object areas known to be involved in object recognition (Downing et al., 2001; Peelen and Downing, 2005), mainly the LOTC (overlapping with the extrastriate body area), the fusiform gyrus (overlapping with the fusiform body area), and the inferior temporal cortex (Fig. 3). Previous studies have shown ventral area activity effects in response to participants' confidence in recognizing an object (Bar et al., 2001) and to stimulus visibility (Pessoa et al., 2006; Zhan et al., 2018). These studies and the current findings thus indicate that the level of perceptual awareness and the magnitude of activation of high-level visual areas, including body-selective ones, are tightly linked. This also supports the notion that degrees of perceptual (object) awareness correspond to degrees of stimulus recognition.
Furthermore, the current results show an effect of perceptual awareness along the IPS. This region is part of the dorsal attention network and responds to stimulus salience, direction of attention and saccades (Corbetta and Shulman, 2011). Our previous study used a similar experimental design but combined with pupillometry and showed that participants maintained central fixation in 95% of the trials (Poyo Solanas et al., 2022), which makes eye movements an unlikely explanation for the perceptual awareness modulations found in IPS. In addition, transcranial magnetic stimulation (TMS) over IPS has also shown to cause perceptual vanishing of visual stimuli (Kanai et al., 2008; Brascamp et al., 2018), suggesting that this area may indeed mediate perception of stimuli entering consciousness. Our results also suggest an important role of the IFC in perceptual awareness (Fig. 3), in line with previous work (Meenan and Miller, 1994; Windmann et al., 2006; Sterzer and Kleinschmidt, 2007; Zaretskaya et al., 2010; Brascamp et al., 2018). Importantly, a recent study combining neuroimaging methods with TMS argued for a causal role of the IFC in conscious experience, showing that this area may play a role in changes in conscious experience by continually monitoring conscious representations and comparing them to available sensory data (Weilnhammer et al., 2021).
Taken together, the current findings are consistent with previous research findings suggesting that the frontoparietal and temporal cortex constitute a corticocortical network involved in perceptual stimulus awareness (Logothetis and Schall, 1989; Leopold and Logothetis, 1996; Sheinberg and Logothetis, 1997; Kreiman et al., 2002; Panagiotaropoulos et al., 2012). On the one hand, the gradual relationship observed between perceptual awareness and temporal cortex activity (Fig. 4) suggests that this area may encode subjective stimulus perception rather than merely physical stimuli properties. On the other hand, the dichotomous activity pattern observed in anterior and medial parts of the IPS as well as in IFC suggests that these areas may be crucial in mediating stimuli entering into consciousness. This fits the current findings of a significantly lower activity related to the “no experience” reports (when no perceptual switches occurred) in comparison with other perceptual awareness levels (when perceptual switches occurred) as well as a nonsignificant activity difference between the PAS levels in which perceptual switches occurred (i.e., PAS2–4; Fig. 4).
Perceptual awareness and the amygdala
Our results do not show evidence of body expression processing without subjective awareness in amygdala. There are three main factors that may explain this negative result when viewed against the literature: the measure of awareness, the specific task and the specific stimuli used. First, previous studies often failed to formally evaluate participant's perception. Earlier backward masking studies often determined target discrimination based on subject debriefing after the experiment (Whalen et al., 1998; Sheline et al., 2001). However, when assessing participants' behavioral performance on a trial-by-trial basis and with SDT measures, a difference in amygdala activation between fearful and neutral stimuli was only found for consciously perceived stimuli (Pessoa et al., 2006; Hoffmann et al., 2012; Hoffmann et al., 2015). Another explanation mentioned earlier could be that previous studies did not account for different degrees of perceptual awareness, and therefore amygdala activation during reported unawareness might have been confounded with residual awareness. Finally, concerning the type of stimuli, previous masking (Morris et al., 1998; Whalen et al., 1998) and binocular rivalry studies (Pasley et al., 2004; Williams et al., 2004) reporting amygdala responses to nonconsciously perceived emotional stimuli used stimuli other than body expressions, usually facial expressions. Evidence for amygdala involvement in body expressions processing is still limited and not consistent, with some studies reporting its involvement (Hadjikhani and de Gelder, 2003; van de Riet et al., 2009; Sinke et al., 2010; Kret et al., 2011; Poyo Solanas et al., 2020) while others failed to show it (Zhan et al., 2018, 2021; Seinfeld et al., 2021).
Affective processing and perceptual awareness
In line with freezing behavior, fearful bodies triggered a slower heart rate compared with neutral ones for “almost clear” stimulus experience (Fig. 5; Roelofs, 2017; Mello et al., 2022). During almost clear experience of the stimulus, stimulus visibility may have been enough to recognize a body (or parts of it) but not sufficient to be completely certain of the emotional expression. This perceptual ambiguity may explain why freezing behavior was most pronounced during an “almost clear” experience of the stimulus. However, it is important to interpret these results with caution given the relatively small sample size.
At the brain level, the activity in the areas showing an effect of perceptual awareness was not influenced by emotion. Previous studies showing emotion influences in frontoparietal areas used facial expressions (Whalen et al., 1998), reported an area location that did not correspond to the ones in the current study (IFC, IPS; Vuilleumier et al., 2002; Amting et al., 2010), or included large parts of the frontoparietal cortices (Kiss and Eimer, 2008), which may explain the current results. In the case of the amygdala, higher activity was found for neutral body expressions than fearful ones, in disagreement with previous literature involving this region with the processing of threatening signals (Adolphs, 1999). Although participants were able to correctly distinguish neutral bodies from fearful ones when reporting higher stimulus visibility (during almost clear and clear trials), it could be that the action represented by the neutral body posture (opening door) was more ambiguous than the fearful expression without more contextual information. This interpretation agrees with previous studies suggesting this area as key in signaling and resolving ambiguity (Whalen et al., 1998; Adolphs, 2002; de Gelder et al., 2014; Hortensius et al., 2017; Poyo Solanas et al., 2018).
Data and Materials Availability
The data used in the current study are publicly available at doi: 10.34894/1YZZSQ. It should be noted that this data became available only after publication, and neither the reviewers requested nor were provided with the data during the review process.
Footnotes
This work was supported by the European Research Council (ERC) Synergy grant (Grant Agreement 856495; Relevance), by the Future and Emerging Technologies (FET) Proactive Program H2020-EU.1.2.2 (Grant Agreement 824160; EnTimeMent), by the Industrial Leadership Program H2020-EU.1.2.2 (Grant Agreement 825079; MindSpaces), by the Horizon-CL4-2021-Human-01-21 (Grant Agreement: 101070278; Re-Silence), and by the Horizon 2020 Program H2020-FETPROACT-2020-2 (Grant Agreement 101017884; GuestXR). We thank J. Eck for the help with physiological noise correction of fMRI data, M. J. Vaessen for the assistance during the piloting of this study, and V. Smekal for comments on an earlier draft of the introduction section.
The authors declare no competing financial interests.
M.Z.’s present address: Cognitive Neuroimaging Unit, CEA DRF/I2BM, INSERM, Université Paris-Sud, Université Paris-Saclay, NeuroSpin Center, France.
- Correspondence should be addressed to Beatrice de Gelder at b.degelder{at}maastrichtuniversity.nl.
