Abstract
The brain has the extraordinary capacity to construct predictive models of the environment by internalizing statistical regularities in the sensory inputs. The resulting sensory expectations shape how we perceive and react to the world; at the neural level, this relates to decreased neural responses to expected than unexpected stimuli (“expectation suppression”). Crucially, expectations may need revision as context changes. However, existing research has often neglected this issue. Further, it is unclear whether contextual revisions apply selectively to expectations relevant to the task at hand, hence serving adaptive behavior. The present fMRI study examined how contextual visual expectations spread throughout the cortical hierarchy as we update our beliefs. We created a volatile environment: two alternating contexts contained different sequences of object images, thereby producing context-dependent expectations that needed revision when the context changed. Human participants of both sexes attended a training session before scanning to learn the contextual sequences. The fMRI experiment then tested for the emergence of contextual expectation suppression in two separate tasks, respectively, with task-relevant and task-irrelevant expectations. Effects of contextual expectation emerged progressively across the cortical hierarchy as participants attuned themselves to the context: expectation suppression appeared first in the insula, inferior frontal gyrus, and posterior parietal cortex, followed by the ventral visual stream, up to early visual cortex. This applied selectively to task-relevant expectations. Together, the present results suggest that an insular and frontoparietal executive control network may guide the flexible deployment of contextual sensory expectations for adaptive behavior in our complex and dynamic world.
SIGNIFICANCE STATEMENT The world is structured by statistical regularities, which we use to predict the future. This is often accompanied by suppressed neural responses to expected compared with unexpected events (“expectation suppression”). Crucially, the world is also highly volatile and context-dependent: expected events may become unexpected when the context changes, thus raising the crucial need for belief updating. However, this issue has generally been neglected. By setting up a volatile environment, we show that expectation suppression emerges first in executive control regions, followed by relevant sensory areas, only when observers use their expectations to optimize behavior. This provides surprising yet clear evidence on how the brain controls the updating of sensory expectations for adaptive behavior in our ever-changing world.
Introduction
Extracting regularities from our highly structured environment shapes how we perceive and react to the world (de Lange et al., 2018; Sherman et al., 2020). Sensory expectations let us predict future events and thus react faster and more accurately to expected than unexpected sensory inputs (Kim et al., 2009; Bertels et al., 2012). At the neural level, sensory expectations often generate a decreased neural response to expected than unexpected stimuli, a phenomenon known as “expectation suppression” (Alink et al., 2010; Kaposvari et al., 2018; Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). Following predictive coding theories (Rao and Ballard, 1999; Friston, 2005, 2010), expectation suppression automatically arises in cortical networks that process the sensory information at hand. For example, predictions about visual object identity produce expectation suppression throughout the ventral visual stream (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). Notably, these previous studies show expectation suppression also in a set of downstream regions: insula, inferior frontal gyrus, and posterior parietal cortex. However, whether and how their involvement diverges from the one of upstream sensory regions have yet to be addressed, raising questions on the origin of expectation effects in the brain.
Crucially, sensory expectations may need revision as context changes. For instance, whether you are in The Netherlands or in the United Kingdom, you need to pay attention to the same information to drive safely, such as the behavior of other drivers. However, how events unfold in time depends on the context: entering a roundabout in The Netherlands, you expect vehicles to circulate counterclockwise and stop on your right, while in the United Kingdom you expect the opposite. As a result, you adapt your behavior depending on the context. Formally, this process is known as hierarchical structure learning (Collins and Frank, 2013; Gershman, 2017; Collins, 2018): agents organize their state space (e.g., drivers behavior) into smaller subspaces conditioned on context (e.g., The Netherlands or United Kingdom) and consequently update their action policy as context changes. While hierarchical structure learning has been extensively investigated during reinforcement learning (Collins, 2018), it has generally been neglected in relation to sensory expectations. Importantly, evaluating their update as context changes may also clarify the role of downstream brain regions. Indeed, a frontoparietal executive control network (encompassing IFG and posterior parietal cortex) is responsible for the selection of task-relevant sensory contingencies in a multidimensional environment, where different contingencies coexist (Niv et al., 2015; Leong et al., 2017). Similarly, this network may also guide the selection of contextual sensory expectations in a volatile environment, where different contingencies appear depending on context changes. Further, such a top-down gating mechanism (Niv et al., 2015; Leong et al., 2017) raises the question of whether contextual updating takes place automatically or rather selectively when expectations are task-relevant, hence serving adaptive behavior.
The present study examined how sensory expectations spread throughout the cortical hierarchy as participants update their contextual beliefs. We created a volatile environment where two alternating contexts contained different event sequences, thereby producing context-dependent expectations that needed revision when the context changed (Gershman, 2017; Collins, 2018). Participants attended a training session before scanning to learn contextual temporal associations among pairs of object images (Richter et al., 2018). After successful structure learning, corroborated by supplementary behavioral testing, the fMRI experiment tested for the update of contextual expectation suppression under two separate task demands, respectively, with task-relevant and task-irrelevant expectations.
In brief, behavioral and neural effects of contextual expectation emerged progressively within a context, accompanied by distinct neural profiles across the cortical hierarchy: expectation suppression appeared first in the insula and frontoparietal regions, followed by the ventral visual stream, up to early visual cortex. This applied selectively to task-relevant expectations. Together, these results suggest that an insular and frontoparietal executive control network may guide the flexible deployment of contextual sensory expectations for adaptive behavior in our complex and ever-changing world.
Materials and Methods
Data and code availability
All materials, data, and code that are relevant to replicate the current findings are publicly available at: https://doi.org/10.34973/7tce-8s22.
Participants
The study included the main fMRI experiment and two supplementary behavioral experiments performed via online testing. For the fMRI experiment, a total of 41 volunteers were recruited from the Radboud University research participation system. Following a priori defined criteria (see Exclusion criteria: fMRI), 39 participants underwent MRI scanning and 34 of them were included in the analysis and results (9 males; mean age 26, range 20-40 years). This final number of included participants was based on a priori power analysis (G*Power 3.1) (Faul et al., 2009), computing the required sample size to achieve a power of 0.8 to detect a medium effect size of Cohen's d = 0.5, at α = 0.05, for a two-tailed paired t test.
For the two supplementary behavioral experiments, two independent pools of participants were recruited from Academic Prolific (www.prolific.co). In a structure learning experiment, participants underwent similar tasks to those used in the main fMRI experiment. In an incidental exposure experiment, we presented the same contextual contingencies unbeknownst to participants, who performed a cover task. In total, we recruited 121 volunteers for the structure learning experiment and 103 volunteers for the incidental exposure experiment. Following a priori defined criteria (see Exclusion criteria: behavioral), 100 participants were included in the analysis and results of the structure learning experiment (37 males; mean age 29, range 21-40 years) and the incidental exposure experiment (36 males; mean age 30, range 19-40 years). This final number of included participants was based on a priori power analysis, computing the required sample size to achieve a power of 0.8 to detect a small effect size of Cohen's d = 0.3, at α = 0.05, for a two-tailed paired t test. The larger sample size was also motivated by the notion that online studies may produce noisier results than laboratory studies.
All volunteers were right-handed and reported normal or corrected-to-normal vision and no history of neurologic or psychiatric conditions. They provided written informed consent and received financial reimbursement for their participation. The study followed institutional guidelines of the local ethics committee (CMO region Arnhem-Nijmegen, The Netherlands).
Stimuli
The stimulus material consisted of a pool of 56 full color images of everyday objects derived from a previous study (Brady et al., 2008). For each participant, we randomly sampled a set of 6 stimuli, thereby eliminating potential effects induced by individual image features at the group level. Objects were either electronic (consisting of electronic components and/or requiring electricity to function) or nonelectronic. Stimuli spanned ∼5° × 5° visual angle and were presented on a mid-gray background in the center of the screen.
Experimental design
We used identical designs for the main fMRI experiment and the two supplementary behavioral experiments. In each experimental trial, participants were exposed to two object images presented in quick succession: a leading image was followed by a trailing image (each image duration: 500 ms; no interstimulus interval [ISI]). Each stimulus set comprised 6 object images: 2 serving as leading objects and 4 serving as trailing objects (Fig. 1A). The expectation manipulation consisted of a repeated pairing of images: each leading image predicted the identity of its two paired trailing images with a 40% chance, respectively (expected condition); any of the two other trailing images occurred with 10% chance (unexpected condition). Each trailing image was only (un)expected depending on which leading image preceded it (i.e., P(trailing | leading) = 0.4 for expected condition; P(trailing | leading) = 0.1 for unexpected condition). Thus, each trailing image served both as an expected and unexpected image depending on the leading image. In addition, trial order was pseudo-randomized, with different pairs in consecutive trials. In sum, any difference between expected and unexpected trials cannot be explained in terms of individual stimulus frequency or familiarity, adaptation, or trial history. Crucially, we created a volatile environment by means of two contexts: the transition probability matrix governing image associations in Context 1 was reversed in Context 2. In other words, each context contained a different set of image sequences, hence creating context-dependent expectations about how events (here, image presentations) would unfold in time. The two contexts were presented over separate blocks in alternating order (Fig. 1B), and the beginning of each context block was explicitly signaled by a preliminary context cue (see Experimental procedure: fMRI). Hence, each context represented a period over which we showed a specific set of image sequences. Alternating the two contexts rendered the environment volatile and raised the need for belief updating when the context changed. Since we aimed to evaluate how contextual expectation effects emerged as a function of context start, we split each context block into separate consecutive bins. Each context block was characterized by a 3:1 ratio of expected to unexpected trials (32 trials in total: 24 expected, 8 unexpected). Given this ratio and the constraint of 8 unexpected trials per context block, we created a total of 8 bins per block, with each bin containing 3 expected trials and 1 unexpected trial based on their order of appearance since context start. In other words, we assigned to Bin 1 the first three trials of the expected condition and the first trial of the unexpected condition, and so forth. This way, we compared expected and unexpected conditions within each bin, and we then evaluated how such comparison evolved across bins, since the start of the context. In summary, collapsing across context type, the study conformed to a 2 (expectation: expected/unexpected) × 8 (bin since context start) repeated-measures design. Further, to create a general statistical framework across behavioral and fMRI analyses and optimize statistical power for the whole-brain fMRI analysis, we split each context block into two halves (early bins 1:4/late bins 5:8, see Statistical analysis: general overview).
General experimental design and procedure. A, Image transition matrix determining image pairs. Two leading images (L1-L2) and four trailing images (T1-T4) were randomly sampled for each participant and session. The expectation manipulation consisted of a repeated pairing of images: each leading image predicted the identity of its two paired trailing images with 40% reliability, respectively (expected condition in orange); any other trailing image occurred with 10% reliability (unexpected condition in purple). Further, we created two contexts: the set of image associations presented in Context 1 (blue) were reversed in Context 2 (green). Thus, the image pairs that were expected in one context became unexpected in the other context and vice versa. B, General run structure across tasks. The two contexts (C1 and C2) were presented over blocks of 32 trials (24 expected, 8 unexpected). Each run contained four context blocks (two per type) in alternating order. To evaluate how contextual expectation effects emerged as a function of context start, we created 8 bins for each context block. Each context bin contained 3 expected and 1 unexpected trial. In summary, collapsing across context type, the study conformed to a 2 (expectation: expected/unexpected) × 8 (bin since context start) repeated-measures design. C, General trial structure across tasks. At the beginning of a context block, a 2 s context cue (i.e., color of the fixation point) signaled context type. Within a context block, each trial presented a leading image followed by a trailing image (each image duration: 500 ms; no ISI). Within the ITI, participants provided their response, depending on the task (see Experimental procedure: fMRI).
Experimental procedure: fMRI
The fMRI experiment consisted of two separate sessions per participant on different days. In each session, we sampled a new set of stimuli.
Preliminary screening session: day 1
The aim of this session was to preselect participants who showed robust structure learning. In a behavioral cubicle, participants performed two tasks. First, they briefly familiarized themselves with the stimuli. In each trial, an object image was presented for 3500 ms in the center of the screen and participants categorized the object as electronic or nonelectronic via two-alternative forced choice (keys counterbalanced across participants). Then, written feedback indicated the correct response (i.e., “electronic” or “nonelectronic”) for 1500 ms, followed by an intertrial interval (ITI) of 500 ms (6 stimuli × 2 trials/stimulus = 12 trials in total). Second, participants performed a pair recognition task, which evaluated their ability to learn contextual associations among pairs of images. Participants were explicitly instructed to discover which image associations were more likely in each context, while maintaining their gaze on a fixation bull's-eye (outer circle 0.7° visual angle) that was superimposed on the images in the center of the screen. At the beginning of each context block (Fig. 1C), participants were informed of the context type by a 2 s cue (i.e., color of the fixation point). Within a context block, each trial presented a leading image followed by a trailing image (each image duration: 500 ms; no ISI). Within a 2 s response window since trailing image onset, participants reported whether the trailing image was expected to appear given the leading image via two-alternative forced choice (keys counterbalanced across participants). Then, written feedback revealed the correct response (i.e., “expected” or “unexpected”) for 1500 ms, followed by a 500 ms ITI. Each run comprised four context blocks (2 blocks per context type presented in alternating order). Participants completed three runs, each lasting ∼10 min ([24 expected trials + 8 unexpected trials] × 2 context types × 2 blocks × 3 runs = 384 trials in total). Contingent on their performance (see Exclusion criteria: fMRI), they were invited back for the fMRI session on a different day.
fMRI session: day 2
Only participants who passed the preliminary screening session participated in the subsequent fMRI session. In a behavioral cubicle outside the scanner, participants performed the same two tasks that they tried during the screening session. First, they briefly familiarized themselves with a new set of stimuli. Second, using this new set of stimuli, they performed the pair recognition task to learn contextual associations among pairs of images. Third, they moved into the MRI scanner, where they were exposed to the same contexts and image pairs. There, they performed two main tasks (i.e., categorization and oddball tasks), in which predictions were task-relevant and task-irrelevant, respectively.
In a categorization task, participants reported whether the leading and trailing images belonged to the same category (electronic or nonelectronic) via two-alternative forced choice (keys counterbalanced across participants). Here, predictions about trailing image identity were task-relevant: knowing which trailing images most likely followed each leading image facilitated the categorization process. In particular, if participants used their predictive knowledge, we expected them to be faster for expected relative to unexpected trailing images (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). In line with this prediction, participants were instructed to respond as accurately and fast as possible. Importantly, across the set of 6 images, object category was counterbalanced with respect to image status (leading/trailing), such that participants could not anticipate the correct categorization response solely based on the leading image. Furthermore, object category was also counterbalanced with respect to the expectation condition. Hence, differences in response times (RTs) would not arise by mere response adjustments costs, but instead reflect perceptual surprise to unexpected trailing objects. Although essential for the categorization task, the above counterbalancing was kept throughout all tasks for consistency. At the beginning of each context block, participants were informed of the context type by a 2 s cue (i.e., color of the fixation point). Within a context block, each trial presented a leading image followed by a trailing image (each image duration: 500 ms; no ISI). Participants' categorization responses were collected between the onset of the trailing image and the subsequent ITI (2500- 9500 ms, randomly sampled from a truncated exponential distribution and serving as implicit baseline). We later limited the behavioral analyses to trials without response outliers (i.e., | RT | > 3 SDs above the group mean). Each run comprised 4 context blocks (2 blocks per context type presented in alternating order). Participants completed 2 runs, each lasting ∼12 min ([24 expected trials + 8 unexpected trials] × 2 context types × 2 blocks × 2 runs = 256 trials in total).
In a separate oddball task, participants pressed a button whenever an image was presented upside down (oddballs: 12.5% of total trials). Oddballs' appearance was counterbalanced with respect to image status (leading/trailing) and category (electronic/nonelectronic). Hence, stimulus predictability was uncorrelated with oddballs' appearance; in other words, predictions about trailing image identity were task-irrelevant. Otherwise, we used the same trial structure, context blocks, number of trials, and runs used for the categorization task ([24 expected trials + 8 unexpected trials] × 2 context types × 2 blocks × 2 runs = 256 trials in total). Across participants, we counterbalanced the order of categorization and oddball tasks. Participants provided a response in each trial for the categorization task, while they responded occasionally (i.e., at the appearance of an upside-down image) for the oddball task. Hence, comparing these two tasks inherently conflates task relevance and response preparation/execution.
After the two main tasks, participants underwent a functional localizer to define individual object-selective brain regions. Participants were exposed to (1) the same object images as shown during the previous tasks and (2) a globally phase-scrambled version of each image (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). In a block design, we flashed each image at 2 Hz (300 ms on, 200 ms off) for a period of 11 s. To monitor participants' vigilance, we asked them to press a button whenever an image (object or phase-scrambled object) dimmed in brightness. We inserted 1 baseline block every 6 image blocks. Baseline blocks had variable durations (5.5, 8, or 10.5 s) presented in counterbalanced order to optimize design efficiency for the estimation of the contrast object > scrambled object (Henson, 2006). Participants completed 1 run lasting ∼14 min (6 images × 2 conditions × 6 blocks = 72 image blocks + 12 baseline blocks = 84 blocks in total).
In all tasks, trial order was pseudo-randomized (localizer: no consecutive trials with same images; all other tasks: no consecutive trials with same image pairs) and response mapping was counterbalanced across participants. Before each task, participants familiarized themselves with the procedure via one preliminary practice run. After scanning, we evaluated whether participants retained the knowledge acquired during the prescanning pair recognition task. To this end, we ran a post-scanning pair recognition task that used the same design and procedure as during the prescanning task, but without corrective feedback and with equal conditional probabilities between image pairs to avoid learning at this final test stage. Participants completed 1 run lasting ∼3 min ([16 expected trials + 16 unexpected trials] × 2 context types = 64 trials in total).
Experimental procedure: behavioral
The two supplementary behavioral experiments were performed by two independent pools of participants. Across the two experiments, we used the same tasks that were used in the fMRI experiment, with the same trial structure (except a fixed ITI = 1500 ms), same context blocks, number of trials, and runs. Before starting each task, participants familiarized themselves with the stimuli and procedure via one short practice run. In each experiment, trial order was pseudo-randomized (no consecutive trials with same image pairs) and response mapping was counterbalanced across participants.
Structure learning experiment
The aim of this experiment was to verify robust structure learning at the group level. This was an important premise for the fMRI experiment, where we instead preselected participants based on the presence of structure learning. In other words, this experiment verified that learning in our experimental setting holds true at the population level, where there is no preselection. First, participants performed the same pair recognition task that was presented before scanning in the fMRI experiment; thus, they were explicitly instructed to learn which image associations were more likely in each context. Second, participants performed the categorization task. If they used their predictive knowledge, we expected their categorization response to be faster in trials with expected relative to unexpected trailing images. Finally, we evaluated whether participants retained the knowledge acquired during the initial pair recognition task. To this end, we run the same pair recognition task that was used post-scanning in the fMRI experiment.
Incidental exposure experiment
The aim of this experiment was to investigate the presence of incidental, automatic learning in our experimental setting, akin to statistical learning (Saffran et al., 1996; Turk-Browne et al., 2010; Aslin, 2017; Batterink and Paller, 2019; Batterink et al., 2019). If incidental exposure to the contextual regularities triggered automatic learning, the same may have applied during the scanning period in the fMRI experiment. In this case, we would not know whether the fMRI results reflected, at least partially, automatic learning of the contextual contingencies rather than their selective retrial and usage. First, participants performed the categorization task without being informed about the presence of statistical regularities among image pairs, nor about the two different contexts (signaled by the color of the fixation point as in the other experiments). As a cover story, participants were told that color changes of the fixation point served as a vigilance task (i.e., press a dedicated button whenever a color change occurs). This also ensured that participates still paid attention to the color changes. Incidental learning was probed via comparison of RTs for expected relative to unexpected trailing images. Second, we evaluated whether participants developed any knowledge of the contextual regularities among leading and trailing images. To this end, we run the same pair recognition task that was used after scanning in the fMRI experiment.
Experimental setup: fMRI
The experiment was presented via Presentation 20.2 (Neurobehavioral Systems, www.neurobs.com; RRID:SCR_002521) on a Windows 7 machine. In the behavioral cubicle outside the scanner, participants sat in front of the computer monitor at a viewing distance of 50 cm and gave responses via a keypad with their right hand. In the MRI scanner, images were displayed via a BOLDscreen 32 (SKU M0135; 1920 × 1080 pixels resolution; 60 Hz frame rate), and they were visible to the participants via a mirror mounted on the MR head coil (visual field of ∼33° horizontal × ∼19° vertical visual angle at a viewing distance of ∼109 cm). Participants gave responses via an MR-compatible keypad (Current Designs HHSC-2 × 4-C) with their right hand.
Experimental setup: behavioral
The two supplementary experiments were presented via Gorilla Experiment Builder (www.gorilla.sc) (Anwyl-Irvine et al., 2020). In order to collect precisely timed responses via keyboard presses, we restricted participation to individuals with a computer, as opposed to phones and tablets. Participants were instructed to sit in front of the computer monitor at a viewing distance of 50 cm and to provide responses with their right hand using specific keyboard keys. Specific a priori exclusion criteria were used to control for proper task execution (see Exclusion criteria: behavioral).
fMRI data acquisition
A 3T Siemens Skyra MR scanner was used to acquire both a T1-weighted MP-RAGE anatomic image (TR/TE = 2300/3.03 ms, flip angle = 8°, FOV = 256 mm × 225 mm, voxel size = 1 mm isotropic, 192 sagittal slices acquired in sequential ascending direction) and T2*-weighted EPI with BOLD contrast (multiband factor of 4, TR/TE = 1500/33.4 ms, flip angle = 75°, BW = 1850 Hz/Px, A/P phase encoding direction, FOV = 192 × 192 × 114 mm2, voxel size = 2 mm isotropic, 68 axial slices acquired in interleaved ascending direction). For each participant, we acquired 425 volumes × 2 runs for the categorization and oddball tasks, respectively, and 606 volumes × 1 run for the localizer.
Statistical analysis: general overview
Using RTs and voxel-wise BOLD responses, we investigated how contextual expectation effects emerged within a context. To create a general statistical framework across behavioral and fMRI analyses and optimize statistical power for the whole-brain analysis, we split each context block into two halves (bins 1:4/bins 5:8). Complementarily, in the ROI analysis, we additionally characterized the emergence of expectation suppression across the 8 individual context bins. In the following sections, we provide a detailed description of the behavioral and fMRI analyses. Across both, after testing for normality assumptions, we performed frequentist parametric analyses and we followed up on nonsignificant results by running the correspondent Bayesian tests (see Bayesian analysis). Effect sizes were calculated in terms of partial η-squared (
Behavioral analysis
For both the fMRI experiment and the two supplementary behavioral experiments, we limited the analyses to trials without missed responses (i.e., no answer within the RT window), premature responses (i.e., RTs < 100 ms), or response outliers (i.e., | RT | > 3 SDs above the group mean). Only few trials (across participants' mean ± SEM) were discarded in the fMRI experiment (4.4 ± 0.1%), in the supplementary structure learning experiment (5.2 ± 0.1%), and in the supplementary incidental exposure experiment (5.5 ± 0.1%). In the following, we describe the behavioral analysis of the three different tasks used across the fMRI and behavioral experiments.
Pair recognition task
Analysis of participants' responses evaluated whether they possessed robust knowledge of the contextual regularities. We applied the same analysis rationale to the prescanning and post-scanning pair recognition tasks using signal detection theory (Wickens, 2002). For each participant, “yes” responses to expected trailing images were labeled as hits, while “yes” responses to unexpected trailing images were labeled as false alarms. It follows that:
Categorization task
Analysis of participants' RTs evaluated the behavioral effects of contextual expectation on categorization performance as a function of context start. On each trial, participants reported whether the leading and trailing images belonged to the same category (electronic or nonelectronic). If participants relied on learned contextual regularities to provide their categorization responses, we expected them to be significantly faster in trials with expected relative to unexpected trailing images (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). For each participant, median RTs of correct categorization trials were entered into a 2 (expectation: expected/unexpected) × 2 (context half: bins 1:4/bins 5:8) repeated-measures ANOVA. We report two-tailed p values for the two main effects and the interaction, followed by the planned two-tailed paired t tests evaluating the expectation effect (expected/unexpected) within each context half (bins 1:4/bins 5:8). Further, in the text, we report across participants' absolute RT differences in ms (ΔRT).
Oddball task
Analysis of participants' responses evaluated vigilance during scanning, to reject individuals with subpar performance (see Exclusion criteria: fMRI). For each participant, we calculated proportion of hits (i.e., press button after an oddball image, within the ITI) and, complementarily, proportion of omissions (i.e., 1 – proportion of hits).
fMRI analysis
Preprocessing
MRI data were converted to nifti and additional meta data provided, compliant with Brain Imaging Data Structure 1.4.0 (Gorgolewski et al., 2016) (https://bids.neuroimaging.io; RRID:SCR_016124) via BIDScoin 3.6.3 (https://bidscoin.readthedocs.io/). Data quality was subsequently evaluated using MRIQC 0.15.1 (Esteban et al., 2017) (https://mriqc.readthedocs.io). In particular, we quantified instantaneous head motion during scanning in terms of % frame-wise displacement above the critical threshold (default value: 0.2 mm) relative to the full run time series. Data from participants who did not show excessive head motion (see Exclusion criteria: fMRI) were included in the remaining analysis steps. Data were preprocessed using fMRIPrep 20.2.3 (Esteban et al., 2019) (www.fmriprep.org; RRID:SCR_016216). Briefly, the fMRIPrep pipeline included brain extraction, motion correction, unwarping, and normalization into MNI standard space (only for the whole-brain analysis); finally, FSL FEAT 6.00 (Smith et al., 2004) (www.fmrib.ox.ac.uk/fsl; RRID:SCR_002823) was used for spatial smoothing (Gaussian kernel of 5 mm FWHM). The time series of all voxels were high-pass filtered to 1/128 Hz. The first 5 volumes of each functional sequence were discarded to allow for signal stabilization.
GLM specification
Voxel-wise GLMs were fit to each participant's data using FSL FEAT 6.00. The main fMRI tasks (i.e., categorization and oddball) were modeled in an event-related fashion with regressors entered into the design matrix after convolving each unit impulse (duration: 1 s, i.e., image pair duration) with a double gamma HRF. In addition to modeling the 16 conditions in our 2 (expectation: expected/unexpected) × 8 (bin since context start) repeated-measures design, the statistical model included the following nuisance regressors: 6 motion regressors to account for residual motion artifacts; 3 additional nuisance regressors to account for the influence of head motion on the BOLD signal (frame-wise displacement, signal variations in while matter and CSF); context cues (color change of fixation point at the beginning of each context block); for oddball task only: target trials (upside-down images); first temporal derivative of all regressors. Condition-specific effects (i.e., parameter estimates for the HRF regressors) were estimated for each participant according to the GLM. Data were combined across runs using FSL's fixed effect analysis.
ROI analysis
Based on previous work (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022), we preselected early visual cortex (EVC) and object-selective lateral occipital complex (LOC) along the ventral visual stream and superior parietal lobule (SPL), inferior frontal gyrus (IFG), and insula (INS) as downstream regions. Within each ROI (see ROI definition), the 16 parameter estimates in our 2 × 8 design were extracted separately from the whole-brain maps. Per participant, the mean parameter estimate of each experimental condition within the ROI was calculated and divided by 100 to yield an approximation of mean percentage signal change relative to baseline. The resulting values were entered into a 2 (expectation: expected/unexpected) × 2 (context half: bins 1:4/bins 5:8) repeated-measures ANOVA, mirroring the RT analysis. We report two-tailed p values for the two main effects and the interaction, followed by the planned two-tailed paired t tests evaluating the expectation effect (expected/unexpected) within each context half (bins 1:4/bins 5:8).
Complementarily, we tested for the emergence of expectation suppression over the course of the 8 individual context bins. For each participant and ROI, we first calculated the expectation suppression profile across the context block by taking the unexpected – expected difference of parameter estimates for each of the 8 bins; then, we fitted a linear regression to the resulting expectation suppression profile. A positive regression slope would indicate an increase of expectation suppression along the context block. This was tested for each ROI by subjecting the slope parameters across subjects to a two-tailed one-sample t test, comparing the obtained slopes against zero. Crucially, if downstream executive control regions gated access to the relevant contextual expectations (Niv et al., 2015; Leong et al., 2017), they would show expectation suppression in response to the current set of expected images before sensory regions. We thus characterized when we first saw expectation suppression in each ROI by computing the bin value corresponding to the first zero-crossing of the expectation suppression profile (hereafter termed “suppression point”). Using a jackknife approach (Miller et al., 1998), a suppression point was computed for each of the n grand average expectation suppression profiles derived from a subsample of n – 1 of the n individual participants (i.e., in a leave-one-out approach, each participant was omitted from one of the subsample grand averages). If an expectation suppression profile (and hence, suppression point) is consistent over participants, then the average value will not change substantially depending on which participant is left out. This analysis approach does not depend on linearity assumptions for the expectation suppression profiles, thus affording a more precise suppression point estimation. First, for each ROI, we tested whether the respective suppression point was significantly different from zero via two-tailed one-sample t tests. Second, to test for pairwise differences of suppression points between sensory and downstream ROIs, we contrasted the jackknife-estimated suppression points of EVC and LOC, respectively, with those of SPL, IFG, and INS via planned two-tailed paired t tests. Before testing for statistical significance, corrected t values were calculated as follows (Ulrich and Miller, 2001):
ROI definition
ROIs were defined bilaterally in each participant's native space using a combination of anatomic and functional constraints (p < 0.05 at the cluster level Gaussian-random-field [GRF]-corrected within the entire brain, with a cluster-forming uncorrected threshold of p < 0.001). EVC and LOC were defined via the functional localizer, which was modeled in a blocked fashion with regressors entered into the design matrix after convolving each block with a double gamma HRF. In addition to modeling the 2 experimental conditions (object/scrambled object), the statistical model included the following nuisance regressors: 9 motion regressors to account for residual motion artifacts (6 motion directions, frame-wise displacement, signal variations in while matter and CSF); target trials (dimmed images). For EVC, the contrast [object + scrambled object] > baseline was constrained to anatomic V1 derived from Freesurfer 6.0.1 cortical surface reconstruction (Fischl, 2012) (recon-all; https://surfer.nmr.mgh.harvard.edu/; RRID:SCR_001847). For LOC, the contrast object > scrambled object was constrained to anatomic LOC derived from Freesurfer's Desikan-Killiany cortical atlas (Desikan et al., 2006). The downstream areas (SPL, IFG, and INS) were functionally defined via the group-level contrast “unexpected > expected” derived from a previous study (i.e., categorization task in Richter and de Lange, 2019). The contrast was inverse-normalized to each participant's native space and constrained to anatomic SPL, IFG, and INS derived from Freesurfer's Desikan-Killiany cortical atlas (Desikan et al., 2006). Finally, all ROIs were constrained to the 300 most active voxels. In order to verify that results did not depend on this arbitrary number of voxels, we repeated all ROI analyses with masks ranging from 100 to 400 voxels in steps of 100 voxels (1562-6250 mm3), in line with previous work (Richter and de Lange, 2019).
Whole-brain analysis
We supplemented the ROI analysis with a whole-brain analysis in MNI standard space. For each participant, we evaluated the main effect of expectation (i.e., unexpected > expected and vice versa), the main effect of context half (i.e., bins 5:8 > 1:4 and vice versa), and crucially, their interaction (i.e., [[unexpected > expected] bins 5:8] > [[unexpected > expected] bins 1:4] and vice versa). In other words, the interaction effect evaluated whether expectation suppression (i.e., unexpected > expected) was stronger in the first or second context half. We further assessed expectation suppression separately in the first half (i.e., [unexpected > expected] bins 1:4) and second half (i.e., [unexpected > expected] bins 5:8). Data were combined across runs using FSL's fixed effect analysis. FSL's mixed effect model FLAME 1 was used for the across-participants analysis, which allowed inferences at the population level. We report activations at p < 0.05 at the cluster level GRF-corrected for multiple comparisons within the entire brain, with a cluster-forming uncorrected threshold of p < 0.001 (Friston et al., 1994; Woo et al., 2014).
Bayesian analysis
To assess whether any nonsignificant results constituted a likely absence of an effect, or rather indicated a lack of statistical power to detect possible differences, we additionally ran the corresponding Bayesian tests. Briefly, all Bayesian analyses were performed using the JZS Bayes factor (Rouder et al., 2009) implemented in JASP 0.11.1 (Love et al., 2019) (www.jasp-stats.org; RRID:SCR_015823) using default settings (Cauchy prior width of 0.707). Interpretations of the resulting Bayes factors quantifying evidence for no effect (B01) were based on established classification (Lee and Wagenmakers, 2014).
Exclusion criteria: fMRI
We excluded participants from all analyses based on three a priori defined criteria. First, participants were excluded after the preliminary screening session (see Experimental procedure: fMRI) if performance during the pair recognition task indicated (1) slack or automatized behavior, as indexed by excessively fast responses (mean RT < 400 ms) or excessively biased responses (|bias| > 1), or (2) lack of robust knowledge of the contextual regularities (d′ < 1). These exclusion criteria are based on a preliminary pilot study (N = 30). We excluded 2 participants because of lack of robust contextual knowledge. Second, participants were excluded after the fMRI session if performance was subpar during the oddball task (i.e., omissions 3 SDs above the group mean) or during the categorization task (proportion of correct responses < 0.85). These exclusion criteria are based on a preliminary pilot study (N = 15). We excluded 1 participant because of subpar performance in the categorization task. Third, we excluded participants who showed excessive head motion during scanning, as indicated by a total % frame-wise displacement (relative motion) 3 SDs above the group mean. We excluded 4 participants based on this criterion.
Exclusion criteria: behavioral
We excluded participants from all analyses based on two a priori defined criteria. First, participants were excluded from the structure learning experiment if performance during the pair recognition task indicated slack or automatized behavior, as indexed by excessively fast responses (mean RT < 400 ms) or excessively biased responses (|bias| > 1). Based on this criterion, 13 participants were excluded from the structure learning experiment. Second, participants were excluded if performance was subpar during the categorization task (proportion of correct responses < 0.85). Based on this criterion, 8 participants were excluded from the structure learning experiment and 3 participants were excluded from the incidental exposure experiment.
Results
Successful development and maintenance of contextual expectations
Participants attended a training session before scanning to learn contextual temporal associations among pairs of object images in a pair recognition task. To confirm that they successfully learned the contextual associations before entering the MRI scanner, we assessed their ability to discriminate expected from unexpected image pairs in each context using signal detection theory. Indeed, d′ in the last run of the prescanning pair recognition task was significantly above the threshold value of 1, reflecting robust knowledge of the contextual regularities at the group level (d′ = 3.29 ± 0.13, mean ± SEM; t(33) = 17.77, p < 0.001, dz = 3.05). Likewise, d′ in the post-scanning pair recognition task was significantly above threshold (d′ = 2.89 ± 0.14, mean ± SEM; t(33) = 13.48, p < 0.001, dz = 2.31), indicating that participants were able to correctly recall the same contextual associations at the end of the fMRI study, and thus maintained this knowledge during the scanning period. We found consistent results in the supplementary structure learning experiment, where we did not apply any preselection criteria for participant inclusion: d′ in the last run of the prescanning pair recognition task was significantly above threshold (d′ = 1.86 ± 0.11, mean ± SEM; t(99) = 7.80, p < 0.001, dz = 0.78). Similarly, participants were also able to correctly retrieve the same contextual associations at the end of the experiment: d′ in the post-scanning pair recognition task was significantly above threshold (d′ = 1.21 ± 0.14, mean ± SEM; t(99) = 8.50, p < 0.001, dz = 0.85). These results confirmed that structure learning in our experimental setting holds true at the population level.
In sum, participants' performance in the pair recognition task confirmed the successful encoding, storage, and retrieval of contextual expectations, which represented an important premise for all subsequent analyses.
Behavioral effects of contextual expectation develop along the context
Having established robust structure learning, we turned to the behavioral analysis of the categorization task inside the scanner. We anticipated participants to be highly correct at reporting whether the leading and trailing images belonged to the same category (electronic/nonelectronic), having preselected them based on accurate performance (see Exclusion criteria: fMRI). Indeed, the proportion of correct responses was close to ceiling in all experiments (fMRI: 0.95 ± 0.01; supplementary structure learning: 0.95 ± 0.01; supplementary incidental exposure: 0.96 ± 0.01, mean ± SEM). We then concentrated on the RT analysis to assess the behavioral effects of contextual expectation as a function of context start (Fig. 2A; Table 1). Participants were significantly faster in trials with expected than unexpected trailing images (i.e., main effect of expectation, ΔRT ± SEM = 18.79 ± 6.51, F(1,33) = 14.40, p < 0.001,
Results of expectation × context half RT analysis in the categorization taska
Behavioral effects of contextual expectation in the categorization task. Across participants' mean (±SEM) RTs across the three experiments: A, in the fMRI experiment; B, in the supplementary structure learning experiment; and C, in the supplementary incidental exposure experiment. Top row plots data as function of expectation (expected, orange; unexpected, purple) and bin (1-8) since the start of a context block. Bottom row plots the same data as a function of expectation and context half (first half, bins 1:4/second half, bins 5:8). After structure learning (A,B), participants responded faster in expected relative to unexpected trials in the second context half; such effect was not present during incidental exposure (C). *p < 0.05. **p < 0.01. ***p < 0.001.
To corroborate the present findings and address potential alternative interpretations of the results, we turned to the analysis of the two supplementary behavioral experiments. We found consistent RT results in the structure learning experiment, where we did not apply any preselection criteria for participant inclusion (Fig. 2B; Table 1). A clear expectation effect on RTs was evident only later in the context block (i.e., expectation × context half interaction, F(1,99) = 16.98, p < 0.001,
Overall, the RT analysis showed that, after successful structure learning, participants relied on learned contextual regularities to provide their categorization responses. However, applying this knowledge was not immediate after the start of a context; conversely, behavioral consequences of contextual expectation emerged progressively along the context.
Neural effects of contextual expectation develop along the context
After successful structure learning, corroborated by supplementary behavioral testing, the fMRI experiment tested for contextual expectation effects at the neural level using voxel-wise BOLD responses. Based on previous work (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022), we ran an ROI analysis selecting EVC and object-selective LOC along the ventral visual stream and SPL, IFG, and INS as downstream regions. In accordance with the RT results, neural effects of contextual expectation developed over the course of the context block in the categorization task (Fig. 3A). In particular, we did not observe any reliable differences between expected and unexpected trials (i.e., main effect of expectation), nor between the first and second halves of the context block (i.e., main effect of context half); instead, we observed a significant interaction between them, reflecting an increase of expectation suppression over the course of the context block. This held true in all the preselected ROIs but EVC, although results appeared qualitatively consistent across all ROIs (Fig. 3B; Table 2). Further, the planned paired t tests evaluating the expectation effect within each context half revealed a significantly larger BOLD response to expected than unexpected image pairs in the first context half in LOC (t(33) = 2.56, p = 0.015, dz = 0.44), SPL (t(33) = 2.31, p = 0.027, dz = 0.40), and IFG (t(33) = 2.23, p = 0.032, dz = 0.38), while results remained inconclusive in EVC (t(33) = 1.81, p = 0.079, dz = 0.31, B01 = 1.27) and INS (t(33) = 1.35, p = 0.186, dz = 0.23, B01 = 2.38). By contrast, the qualitatively larger BOLD response to unexpected than expected image pairs in the second context half did not reach significance in any ROI (all p > 0.100, B01 < 1.80). Here, it is worth noting that expected and unexpected pairs were indeed reversed when the context changed: the pairs that were expected in the previous context became unexpected in the current context. Hence, considering progressive expectation effects in terms of RTs, it is conceivable that what we find in the first context half is indeed a carryover suppression effect for the expected pairs of the previous block. Over the course of the current block, as participants attuned themselves to the context, BOLD responses to the current expected pairs then became progressively suppressed relative to the current unexpected pairs.
Results of expectation × context half ROI analysis in the categorization taska
Neural effects of contextual expectation in the categorization task. A, Across participants' mean (±SEM) % signal change relative to baseline for each ROI as function of expectation (expected, orange; unexpected, purple) and bin (1-8) since the start of a context block. B, Same data plotted as a function of expectation and context half (first half, bins 1:4/second half, bins 5:8). C, Expectation suppression (i.e., unexpected > expected) in the second context half (bins 5:8) relative to the first context half (bins 1:4), overlaid on the MNI152 2 mm template. Colors represent the parameter estimates: red-yellow clusters represent expectation suppression; opacity represents the associated z statistics. Black contours outline statistically significant clusters (p < 0.05 cluster-corrected). Expectation suppression emerged in the second context half in three widespread networks: ventral visual stream (EVC, early visual cortex; LOC, lateral occipital complex), frontoparietal executive control network (SPL, superior parietal lobule; IFG, inferior frontal gyrus), and salience network (INS, insula; ACC, anterior cingulate cortex). a.u., Arbitrary units. *p < 0.05. **p < 0.01.
The whole-brain analysis replicated no reliable BOLD response differences between expected and unexpected trials (i.e., main effect of expectation), nor between the first and second halves of the context block (i.e., main effect of context half); instead, we observed an increase of expectation suppression over the course of the context block (i.e., expectation × context half interaction; Fig. 3C; Table 3). Cortical areas showing more expectation suppression in the second relative to the first context half included object-selective LOC and fusiform gyrus, SPL and intraparietal sulcus, IFG, INS, and anterior cingulate cortex (Fig. 3C, black contours). The same contrast also showed the activation of EVC at p < 0.001 uncorrected (Fig. 3C, red-shaded areas). Whole-brain results were in line with the ROI analysis also when assessing the expectation effect (i.e., unexpected > expected and vice versa) separately in the first and second context halves. We found a significantly larger BOLD response to expected than unexpected image pairs in the first context half in consistent brain regions to the ones mentioned above, while no significant differences between expected and unexpected trials were present in the second context half (Table 3).
Results of expectation × context half whole-brain analysis in the categorization taska
Combined, the ROI and whole-brain analyses pointed to an increase of expectation suppression for contextually expected pairs within the current context. To characterize this more directly, we complementarily tested for the emergence of expectation suppression (i.e., unexpected > expected) over the course of the 8 individual context bins in all ROIs (Fig. 4A). After fitting a linear regression to the expectation suppression profile per participant and ROI, the across-participants' estimated regression slope (b, mean ± SEM) was significantly >0 for all ROIs: EVC (b = 3.98 ± 1.88, t(33) = 2.12, p = 0.042, dz = 0.36), LOC (b = 4.36 ± 1.65, t(33) = 2.65, p = 0.012, dz = 0.45), SPL (b = 4.15 ± 1.90, t(33) = 2.18, p = 0.036, dz = 0.37), IFG (b = 6.52 ± 2.28, t(33) = 2.86, p = 0.007, dz = 0.49), and INS (b = 4.15 ± 1.65, t(33) = 2.51, p = 0.017, dz = 0.43). These results confirmed an increase of expectation suppression along the context block in all ROIs. To ensure that the present results are not dependent on the arbitrarily selected mask size, we repeated all ROI analyses for mask sizes between 100 and 400 voxels (see ROI definition): the direction and statistical significance of all effects were identical across all ROI sizes.
Expectation suppression profile across the cortical hierarchy in the categorization task. A, Across participants' mean (±SEM) expectation suppression profile for each ROI. The regression line in black is derived from the across participants' mean linear regression parameters (i.e., slope and intercept). The estimated regression slopes (b) were significantly >0 for all ROIs, indicating an increase of expectation suppression over the course of the context block (see across participants' mean b and p value in each subplot). B, Suppression point for each expectation suppression profile (colored dot with black contour), showing that expectation suppression emerged at different moments across the cortical hierarchy: first in the insula (INS), followed by inferior frontal gyrus (IFG), superior parietal lobule (SPL), lateral occipital complex (LOC) and finally early visual cortex (EVC). C, The suppression points in INS and IFG appeared significantly earlier than the suppression points in EVC and LOC. a.u., Arbitrary units.
In summary, in line with the RT results, the ROI and whole-brain results suggested that neural effects of contextual expectation emerged progressively within a context and reflected activity in sensory and downstream regions involved in visual object prediction (Manahova et al., 2018; Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022).
Distinct temporal profiles of contextual expectation suppression across the cortical hierarchy
Finally, we characterized when expectation suppression first emerged for each ROI by computing the bin value corresponding to the first zero-crossing of the expectation suppression profile (hereafter termed “suppression point”). The suppression point represented when BOLD responses to expected image pairs first became suppressed relative to unexpected ones over the course of a context block. First, the across-participants' suppression point (s, mean ± SEM) was significantly >0 for all ROIs: EVC (s = 5.64 ± 0.02, t(33) = 8.65, p < 0.001, dz = 1.48), LOC (s = 5.39 ± 0.02, t(33) = 8.70, p < 0.001, dz = 1.49), SPL (s = 4.81 ± 0.05, t(33) = 2.78, p = 0.009, dz = 0.48), IFG (s = 3.50 ± 0.02, t(33) = 4.71, p < 0.001, dz = 0.81), and INS (s = 2.85 ± 0.02, t(33) = 3.99, p < 0.001, dz = 0.68). In line with the ROI and whole-brain results described above, these results are compatible with a consistent carryover effect of expectation suppression for the expected pairs of the previous block, which subsided as the system attuned itself to the current context. Crucially, as shown in Figure 4B, different suppression points appeared across the cortical hierarchy: first in INS, followed by the remaining downstream regions (IFG, SPL), and finally in the ventral visual stream (LOC, EVC). In particular, the suppression points in INS and IFG appeared significantly earlier than the suppression points in EVC and LOC (Fig. 4C), as indicated by the respective planned paired t tests (Table 4). Importantly, we confirmed that these results are not dependent on the mask size of the ROIs: the direction and statistical significance of all effects were identical across all mask sizes between 100 and 400 voxels. In sum, distinct profiles of contextual expectation suppression emerged across the cortical hierarchy, with surprise signals first being evident in the insula and in frontoparietal regions, and later arising in sensory regions. These results suggest that downstream regions may guide the selection of appropriate sensory expectations in a volatile environment, where different contingencies appear depending on context.
Results of suppression point analysis in the categorization taska
No activity modulation for task-irrelevant contextual expectations
A top-down gating mechanism for the selection of contextual expectations may critically depend on task relevance. In the categorization task, predictions were indeed task-relevant: knowing which trailing images most likely followed each leading image served the categorization process. We addressed whether we found the same neural effects of contextual expectation in a separate oddball task where expectations were instead task-irrelevant: oddballs were fully counterbalanced with respect to image status (leading/trailing) and category (electronic/nonelectronic); thus, object predictions did not serve oddball detection.
We anticipated participants to be highly correct at detecting the oddballs since we preselected them based on accurate performance (see Exclusion criteria: fMRI). Indeed, the across-participants' proportion of correct responses was close to ceiling (0.96 ± 0.01, mean ± SEM), which confirmed high vigilance during the oddball task. Crucially, here we did not find any expectation suppression throughout the brain, in sharp contrast with the results of the categorization task (Fig. 5A). In particular, while there was no significant interaction between expectation and context half in any ROI, we observed a significant main effect of expectation in LOC and SPL, with qualitatively similar profiles across ROIs (Fig. 5B; Table 5). Furthermore, the whole-brain analysis did not show any significant brain activations for the main effect of expectation, the main effect of context half, and the interaction between them (Fig. 5C).
Results of expectation × context half ROI analysis in the oddball taska
Neural effects of contextual expectation in the oddball task. A, Across participants' mean (±SEM) % signal change relative to baseline for each ROI as function of expectation (expected, orange; unexpected, purple) and bin (1-8) since the start of a context block. B, Same data plotted as a function of expectation and context half (first half, bins 1:4/second half, bins 5:8). C, Expectation suppression (i.e., unexpected > expected) in the second context half (bins 5:8) relative to the first context half (bins 1:4), overlaid on the MNI152 2 mm template. Colors represent the parameter estimates: red-yellow clusters represent expectation suppression; opacity represents the associated z statistics. We did not observe expectation suppression in either the ROI analysis or at the whole-brain level. a.u., Arbitrary units. *p < 0.05.
Complementarily, we tested for the emergence of expectation suppression over the course of the 8 individual context bins in all ROIs. After fitting a linear regression to the expectation suppression profile per participant and ROI, the across-participants' estimated regression slope (b, mean ± SEM) was not significantly different from zero for any ROIs, with all Bayesian results confirming moderate evidence for the absence of an effect: EVC (b = −1.03 ± 1.93, t(33) = −0.53, p = 0.596, dz = −0.09, B01 = 4.76), LOC (b = 0.09 ± 1.68, t(33) = 0.05, p = 0.959, dz = 0.01, B01 = 5.56), SPL (b = −0.04 ± 1.59, t(33) = −0.02, p = 0.982, dz = −0.01, B01 = 5.56), IFG (b = −1.81 ± 1.99, t(33) = −0.91, p = 0.368, dz = −0.16, B01 = 3.70), and INS (b = −0.82 ± 1.47, t(33) = −0.56, p = 0.579, dz = −0.10, B01 = 4.76). Once again, the direction and statistical significance of all effects were identical across all mask sizes between 100 and 400 voxels.
Collectively, the ROI and whole-brain results showed evidence against expectation suppression throughout the brain in the case of task-irrelevant contextual expectations, suggesting that task relevance may represent a boundary condition for the top-down gating of contextual expectations.
Discussion
The present study examined the behavioral and neural effects of contextual sensory expectation as observers updated their beliefs in a volatile environment. After developing visual object predictions conditioned on context, participants showed progressively faster responses to expected than unexpected stimuli within the context, suggesting that they used their predictive knowledge to optimize task performance. This was accompanied by progressively suppressed BOLD responses for unexpected compared with expected trials (i.e., expectation suppression) across three widespread systems: in early visual cortex, lateral occipital complex, and fusiform gyrus, which belong to the object-selective ventral visual stream; in superior parietal lobule, intraparietal sulcus, and inferior frontal gyrus, which are part of the frontoparietal network for top-down attention gating; and in the insula and anterior cingulate cortex, which belong to the salience network, a control hub for surprise detection and cognitive control. Importantly, we found distinct neural profiles across the cortical hierarchy: expectation suppression appeared first in the insula and frontoparietal regions, followed by the ventral visual stream. Finally, this neural pattern pertained selectively to task-relevant predictions. Collectively, the present study suggests that an insular and frontoparietal executive control network may gate access to task-relevant contextual expectations to guide behavior in a complex and dynamic environment, where changing contexts determine the need for belief updating.
Encoding and retrieval of contextual sensory expectations
To investigate how sensory expectations are deployed conditioned on context, we created a volatile environment where two alternating contexts contained different sequences of events, thereby creating context-dependent expectations (Collins, 2018). Across the fMRI experiment and a supplementary structure learning experiment, participants successfully encoded and retrieved contextual regularities after they were instructed to discover structure conditioned on context. These findings support and extend previous work on structure learning (Collins and Frank, 2013; Gershman, 2017; Collins, 2018) by showing that, also outside the realm of reinforcement learning, organizing knowledge by context and focusing selectively on the task-relevant sensory contingencies scaffolds fast and robust learning inside a changing environment (Wilson and Niv, 2012). In striking contrast, in an incidental exposure experiment matched in terms of duration and environmental structure, participants did not show any evidence of learning, either in terms of implicit RT facilitation or in terms of explicit recognition. Thus, the incidental and automatic extraction of regularities, known as statistical learning (Saffran et al., 1996; Turk-Browne et al., 2010; Batterink and Paller, 2019; Batterink et al., 2019), is at least slowed down and thus not manifest in a hierarchically structured and volatile environment like the one used in the present study (for complementary evidence during statistical learning of distractor suppression, see also de Waard et al., 2022). Such evidence has theoretical implications for statistical learning, which is typically described as a rapid mechanism able to extract environmental regularities in a matter of few exposures (Aslin, 2017; Saffran and Kirkham, 2018; Sherman et al., 2020). Traditional statistical learning experiments present participants with fixed environments that contain simple regularities, such as chunks of stimuli linked by deterministic dependencies (Saffran et al., 1996; Turk-Browne et al., 2009; Henin et al., 2021), which arguably represent an oversimplification of real-life conditions. Increasing structure complexity, here in the form of probabilistic, context-dependent, and volatile regularities, clearly hinders the effectiveness of statistical learning and hence highlights a possible boundary condition that deserves further investigation, as recently discussed (Conway, 2020).
Updating sensory expectations in a volatile environment
Having established robust structure learning in the fMRI experiment, we investigated its behavioral and neural consequences using RTs and voxel-wise BOLD responses. Both measures showed that contextual expectation effects emerged progressively in a categorization task where predictions were task-relevant. This pattern of results may look surprising, given that a preliminary explicit cue was fully informative of the upcoming context identity and its associated sensory contingencies. Crucially, our volatile environment required the inhibition of irrelevant contextual contingencies when the context changed, therefore generating switching costs at the beginning of a new context period (Monsell, 2003; Koch et al., 2010; Collins, 2017). This was substantiated by a consistent carryover effect of expectation suppression for the expected pairs of the previous block, which subsided as the system attuned itself to the current context.
Brain areas showing progressive contextual expectation suppression within a context comprised sensory and downstream regions involved in visual object prediction (Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). Crucially, distinct neural profiles appeared across the cortical hierarchy. An insular and frontoparietal system first reacted to expectation violations. On the one hand, the insula is a central node of the salience network, a control hub that marks relevant events in conflict with current states (Menon and Uddin, 2010; Uddin, 2015). In particular, it reflects surprise both during value-based decision-making and during sensory processing (Fazeli and Büchel, 2018; Loued-Khenissi et al., 2020). On the other hand, the posterior parietal cortex (SPL/IPS in particular) and the inferior frontal gyrus are part of the frontoparietal network for attention (re)orienting (Corbetta and Shulman, 2002; Shomstein and Behrmann, 2006; Corbetta et al., 2008; Scolari et al., 2015). Further, the inferior frontal gyrus is thought to be involved in switching among “task sets” by inhibiting irrelevant task representations when the context changes (Aron et al., 2004; Sakai, 2008; Hyafil et al., 2009). Together, these two systems are known to enable adaptive behavior by marking salient events and directing processing resources toward them (Shulman et al., 2009; Menon and Uddin, 2010). As previously suggested (Meyer and Olson, 2011; den Ouden et al., 2012; Richter et al., 2018), a stronger neural response to unexpected than expected stimuli fits this computational goal because unexpected events may instruct the need to update expectations in a volatile environment.
Origin of contextual sensory expectations in the brain
The presence of distinct neural profiles across the cortical hierarchy, with expectation suppression first evident in the insula and frontoparietal regions, followed by the ventral visual stream, offers insight into how expectation effects may arise in the brain. The neural mechanisms implementing expectation effects on perception likely depend on the specific type of environmental regularities (de Lange et al., 2018). On the one hand, expectations deriving from stable features of the physical environment can become encoded in the tuning properties of our sensory cortices, as a result of long-term phylogenetic plasticity (Girshick et al., 2011; Teufel and Fletcher, 2020). Further, expectations deriving from long-term semantic memory support visual recognition under high perceptual uncertainty via top-down signals from medial prefrontal to early visual cortex (Summerfield et al., 2006; Summerfield and Egner, 2009). On the other hand, contextual expectations arising from arbitrary and volatile contingencies may either emerge from local short-term plasticity (Reber, 2013; Conway, 2020) or require the involvement of downstream executive control regions that monitor the environment for surprising events (Shulman et al., 2009; Menon and Uddin, 2010; Uddin, 2015; Conway, 2020). Previous work is compatible with both hypotheses, as it used extensive behavioral training before neuroimaging that could have generated synaptic plasticity in sensory areas (Meyer and Olson, 2011; Kaposvari et al., 2018; Richter et al., 2018; Richter and de Lange, 2019; He et al., 2022). Conversely, the present study used a very short training period right before scanning. This experimental feature, coupled with the empirical observation that expectation suppression first appeared in downstream executive control areas, suggests that contextual expectation effects may arise in these regions and be subsequently fed back to functionally relevant sensory regions (here, the object-selective ventral visual stream). However, another account may be viable (Alink and Blank, 2021): contextually unexpected (surprising) stimuli may automatically attract attention, hence involving downstream executive control regions that monitor the environment for salient events. As a result of automatic attentional allocation, the processing of surprising stimuli may be enhanced in sensory areas (Desimone and Duncan, 1995; Itti and Baldi, 2009). From this perspective, the expectation effect first arising in downstream areas may reflect attention-driven “surprise enhancement” (Feuerriegel et al., 2021a,b) rather than prediction-driven “expectation suppression” (Richter et al., 2018; Richter and de Lange, 2019). Critically, we found expectation effects only for task-relevant predictions, which could be used to optimize task execution; no such evidence was found in a separate task where predictions were task-irrelevant. This questions the attention-driven account, which postulates automatic attentional capture for surprising events regardless of the task at hand. Hence, while future studies may further address the issue (Alink and Blank, 2021), here we suggest that an insular and frontoparietal executive control system, sensitive to surprising events in conflict with current task-relevant predictions (Shulman et al., 2009; Menon and Uddin, 2010; Uddin, 2015; Conway, 2020), may guide the selection of appropriate contextual sensory expectations in a volatile environment, where different contingencies appear depending on changing contexts. Similarly, a frontoparietal network encompassing inferior frontal gyrus and posterior parietal cortex is responsible for the selection of task-relevant sensory expectations in a multidimensional environment, where different contingencies coexist (Niv et al., 2015; Leong et al., 2017). Collectively, the top-down gating of task-relevant sensory expectations from downstream executive control areas to sensory areas may enable adaptive behavior in our complex and dynamic world.
Footnotes
This work was supported by EC Horizon 2020 Program, ERC Consolidator Grant “Surprise” 101000942 to F.P.d.L. We thank Yamil Vidal and Jakub Szewczyk for helpful comments on a previous version of the manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Ambra Ferrari at ambra.ferrari{at}donders.ru.nl
