Abstract
Temporal expectation is the ability to construct predictions regarding the timing of events, based on previously experienced temporal regularities of different types. For example, cue-based expectations are constructed when a cue validly indicates when a target is expected to occur. However, in the absence of such cues, expectations can be constructed based on contextual temporal information, including the onset distribution of the event and recent prior experiences, both providing implicit probabilistic information regarding the timing of the event. It was previously suggested that cue-based temporal expectation is exerted via synchronization of spatially specific neural activity at a predictable time of a target, within receptive fields corresponding to the expected location of the target. Here, we tested whether the same theoretical model holds for contextual temporal effects. Participants (n = 40, 25 females) performed a speeded spatial-cuing detection task with two-thirds valid spatial cues. The hazard-rate function of the target was modulated by varying the foreperiod—the interval between the spatial cue and the target—among trials and was manipulated between groups by changing the interval distribution. Reaction times were analyzed using both frequentist and Bayesian generalized linear mixed models, accounting for hazard and sequential effects. Results showed that the effects of contextual temporal structures on reaction times were independent of spatial attention. This suggests that the spatiotemporal mechanisms, thought to account for cue-based expectation, cannot explain other sources of temporal expectations. We conclude that expectations based on contextual structures have different characteristics than cue-based temporal expectation, suggesting reliance on distinct neural mechanisms.
SIGNIFICANCE STATEMENT Temporal expectation is the ability to predict an event onset based on temporal regularities. A neurophysiological model suggested that temporal expectation relies on the synchronization of spatially specific neurons whose receptive fields represent the attended location. This model predicts that temporal expectation would be evident solely within the locus of spatial attention. Existing evidence supported this model for expectation based on associations between a temporal cue and a target, but here we show that it cannot account for temporal expectation that is based on contextual information, that is, the distribution of intervals and recent priors. These findings reveal the existence of different predictive mechanisms for cued and contextual temporal predictions, with the former depending on spatial attention and the latter nonspatially specific.
- FP-RT slope
- hazard-rate function
- reaction time
- sequential effect
- temporal attention
- variable foreperiod effect
Introduction
Temporal expectation is the ability to construct predictions regarding the timing of events, based on temporal regularities. These regularities come in multiple forms, including contextual information, when statistical inferences regarding distributions of events and recent priors are used to predict timings of future events, and cued associations, when events are preceded by informative temporal cues (Coull and Nobre, 1998). Expectations of all these sources were associated with enhanced motor and perceptual performance (Niemi and Näätänen, 1981; Nobre and van Ede, 2018).
Despite abundant evidence on behavioral effects of temporal expectation, relatively little is known regarding its neurophysiological correlates. One framework suggests that temporal expectation results from neural synchronization at expected target time. It was suggested synchronized neuronal populations are spatially specific, that their receptive fields correspond to the expected target location (Rohenkohl et al., 2014; Nobre and van Ede, 2018). According to this view, temporal and spatial expectations are tightly linked as temporal expectation is restricted to the locus of spatial attention; to gain from knowing when a target will occur, one must know where it would occur. However, evidence for this spatiotemporal framework is limited to studies that manipulated cue-based temporal expectation (Doherty et al., 2005; Rohenkohl et al., 2014; Seibold et al., 2020). It remains unknown whether the same spatiotemporal mechanism accounts for temporal expectation based on other sources of regularities, such as those based on the overall distribution of intervals, and recent previous trials.
The distribution of intervals is linked to a frequently observed finding—the variable-foreperiod effect (VFE). When participants are presented with a warning signal (WS) followed by a target after a varying time interval (foreperiod; FP), and the foreperiods are randomly sampled from a uniform distribution, performance for targets appearing after long foreperiods is enhanced relative to short ones (Niemi and Näätänen, 1981). According to one interpretation, this effect depends on the conditional probability of the event, or the likelihood of an event to occur, given that it has yet to occur (Vallesi and Shallice, 2007; Nobre and van Ede, 2018). Conditional probabilities can be described as a function of time (hazard-rate function), which monotonically increases when foreperiods are uniformly distributed; but other distributions could lead to different hazard-rate functions (Luce, 1986). Another interpretation explains VFE as the result of memory activation. The WS acts as a memory retrieval cue, which activates traces of prior trials and results in reaction time (RT) modulations based on the frequencies of previous foreperiods (Los et al., 2017, 2021). Regardless of its interpretation, VFE reflects how the distribution of prior foreperiods modulates temporal expectation.
Another source of contextual information are previous trials. When a target appears following a foreperiod that is shorter than that of the previous trial, performance is reduced, relative to trials that were preceded by an identical foreperiod. This sequential effect (SE) is asymmetrical; performance remains unchanged when a target appears following a foreperiod that is longer than that of the previous trial (Bertelson, 1961; Possamai et al., 1973). There are various accounts for the SE, attributing this effect to a dual process of arousal and controlled monitoring (Vallesi and Shallice, 2007; Capizzi et al., 2015), to trace conditioning (Los et al., 2001) or, more recently, to retrieval of memory traces (Los et al., 2014, 2017, 2021). Regardless of their interpretations, VFE and SE both demonstrate how manipulating contextual priors changes the behavioral outcome of temporal expectations.
There is surprisingly little evidence regarding the interaction of spatial attention and contextual temporal expectation effects. Only one study examined a related question, focusing on the link between SE and inhibition of return, an exogenous attention effect (Los, 2004), but no study to date manipulated endogenous attention to examine its link with contextual temporal expectation. Here, we investigated this link by simultaneously manipulating attention and the distribution of foreperiods. In each trial, participants were presented with a spatial cue that was either congruent, incongruent, or neutral in respect to the location of the target that appeared after a varying foreperiod. The distribution of the foreperiods varied between participants to create two different hazard-rate functions. Finding that VFE and SE are independent of spatial attention would indicate that the spatiotemporal framework suggested to account for temporal expectation does not account for these processes; finding otherwise would support the link between cue-based and contextual temporal expectations.
Materials and Methods
Participants
A total of 40 participants were included in this study, 20 in the uniform distribution group (12 females, 2 left-handed, mean age 25.35 ± 3.5 SDs) and 20 in the inverse-U-shaped distribution group (13 females, one left-handed; mean age 24.55 ± 4.0 SD). There were no exclusions of participants. Sample size was determined according to a pilot study consisting of 20 participants. A power simulation based on this pilot study showed that with only n = 8, these three effects are observed with
Stimuli
The fixation object consisted of a dot (0.075° radius) within a ring (0.15° radius), embedded within a diamond shape (0.4 × 0.4°). The edges of the diamond changed color from black to white, cuing attention to the left side (two left edges became white) or right side (two right edges became white) of fixation object, or remaining neutral in respect to target location (all four edges became white; Fig. 1). The target was a black asterisk (0.4 × 0.4°) presented at 4° eccentricity to the right or left of fixation object. A 1000 Hz pure tone was sounded for 60 ms as negative feedback following errors. Fixation object and target were presented on a midgray background.
Experimental design
Participants were seated in a dimly lit room, with a computer monitor placed 100 cm in front of them (24 inch LCD ASUS VG248QE, 1920 × 1080 pixels resolution, 120 Hz refresh rate, midgray luminance measured to be 110 cd/m2). During the experiment, participants rested their heads on a chinrest. MATLAB R2015a (MathWorks) was used to code and control the experiment, with stimuli displayed using Psychophysics Toolbox version 3 (Brainard, 1997). Gaze position was monitored binocularly using the EyeLink 1000 Plus infrared video-oculographic desktop-mounted system (SR Research) throughout the experiment, at a sampling rate of 1000 Hz. This system has 0.01° spatial resolution and an average accuracy of 0.25–0.5° when a chinrest is used, according to the manufacturer. A nine-point calibration of the eye-tracker was performed before each block and whenever necessary.
Each trial started with a central black fixation object, presented until an online gaze-contingent procedure verified 1000 ms of stable fixation (gaze was placed within a radius of 1.5° of screen center). Following this, the edges of the fixation object changed color for 200 ms to represent a spatial informative or uninformative cue. After a varying foreperiod (500/900/1300/1700/2100 ms) the target was briefly (33 ms) presented at 4° to the left or right of center, with target being congruent to a spatially informative cue direction in 50% of trials (valid condition), incongruent in 25% of trials (invalid condition), or neutral with respect to a spatially uninformative cue in the remaining 25% of trials (uninformative condition). Participants were requested to press a key with their dominant hand as quickly as possible, and after no longer than 1000 ms, on target detection. Between groups, participants were presented with the five foreperiods in either a uniform distribution (20% probability for each foreperiod) or an inverse-U-shaped distribution (a ratio of 1:2:3:2:1 among the five foreperiods, leading to trial percentages of ∼11, 22, 33, 22, and 11%, respectively). These prior distributions resulted in different time-dependent conditional probabilities, that is, different hazard-rate functions, as depicted in Figure 2. The manipulation of hazard rate was required to differentiate its effect from other foreperiod effects related to the WS, such as arousal (Steinborn and Langner, 2012; Weinbach and Henik, 2012). The different distributions were examined in separate participant groups to avoid carry-over effects of distribution learning (Mattiesing et al., 2017). Fixation was monitored throughout the foreperiod, using an online gaze-contingent procedure, and trials that included
Participants of the uniform distribution group performed 10 blocks of 160 trials each, divided into two sessions of ∼1.25 h each. Participants of the inverse-U-shaped distribution group performed 18 blocks of 144 trials each, divided into three sessions of ∼1.25 h each. This number of repetitions guaranteed that we have a minimum of 50 trials in all conditions and for all foreperiods in each of the two distributions, and a large enough number of trials to conduct a sequential analysis on pairs of consecutive trials. A short break was given after each block. Feedback on performance in each block was provided at the end of each experimental block and included mean RT and number of error trials (including both missed trials or premature responses). Starting from the second experimental block, participants were also presented with a message that encouraged them to perform faster if the mean RT of the current block fell below their global mean RT of the entire session. A practice block of 10 trials with random conditions was administered at the beginning of each session.
Statistical analysis
A negligible number of trials with no response (error trials;
The RTs of the remaining trials were modeled using a generalized linear mixed model (GLMM), assuming a gamma family of responses with an identity link (see explanation below; Baayen and Milin, 2010; Lo and Andrews, 2015). Unlike ANOVA, GLMM is suited for non-normally distributed variables, like the positively skewed RT distribution, while also allowing to model trial-level covariates, thus increasing the power of the analysis (Baayen and Milin, 2010). Hierarchical models are also well suited for unevenly distributed trial numbers among conditions, as is the case with the inverse-U-shaped distribution and the relation to the previous trial in the current study, by weighting the population-level mean according to the number of samples included in the subject-level means for each condition. An assumption of this analysis is that the RTs follow gamma distribution. Gamma distributions are suited to describe continuous responses that are zero bounded and have a unimodal and rightward-skewed distribution (e.g., RTs). We further assumed that the predictors are linearly related to the predicted RT, thus an identity link was used (i.e., no transformation was made on the value produced by the predictors; Lo and Andrews, 2015).
The following fixed effects were modeled: (1) linear and quadratic terms for foreperiod duration to model the VFE; (2) cue (valid/invalid/uninformative) to model the effect of spatial attention; (3) the FP-distribution (uniform/inverse-U-shaped) to model the effect of the hazard-rate function; (4) linear and quadratic terms for the relation to previous trial to model SE, calculated as the difference between the current trial foreperiod and the previous trial foreperiod, so that positive values indicate the previous trial was shorter than the current trial and vice versa for negative values; and (5) the interaction terms between foreperiod duration, cue, and FP-distribution and between relation to previous trial, cue, and FP-distribution. For the purpose of examining the relation between spatial attention and each of the temporal structures, we assumed no interaction between the relation to previous trial and foreperiod duration, for example, we assumed that the cost in performance for a current trial of 500 ms and previous trial of 900 ms equals the cost of a 900 and 1300 ms pair of trials. It is acknowledged that this assumption is a simplification and does not strictly hold in all cases (Possamai et al., 1973). To reduce computational complexity, all continuous factors were z-scaled. To allow the computation of the relation to previous trial, the first trial of each session for each participant was discarded from analysis (total of 100 trials). Treatment contrasts coding scheme was used for cue, with the uninformative condition set as the reference level, and sum contrasts coding scheme was used for FP-distribution. Statistical significance for main effects and interactions was determined via a likelihood ratio (LR) test against a reduced nested model excluding the fixed term (i.e., type II sum of squares). Statistical significance for parameter coefficients was determined according to the Wald z test (Fox, 2016).
In addition to the fixed effects, we considered the z-scaled current trial number (i.e., the running trial identifier for the given session) as a covariate to capture effects of fatigue and training along the experiment (Baayen and Milin, 2010). Because the different experimental groups may have experienced different fatigue or training effects, we additionally considered the interaction between FP-Distribution and trial number. Covariates were added to the model if the extended model converged and was found to significantly improve fit (p < 0.05) in an LR test against the model without the covariate (Bates et al., 2015a).
The random effect structure of the model was selected according to the model that was found to be most parsimonious with the data, that is, the fullest model that the data permits while still converging with no singular estimates (Bates et al., 2015a), to balance between type I error and statistical power (Matuschek et al., 2017). To do this, we followed a procedure suggested by Bates et al. (2015a). We started with the simplest model, a random intercept-by-subject-only model, and then added random slopes to it, first, random slopes for fixed terms by subject and their correlation parameters, and then random interaction slopes by subject. In each iteration, we examined whether the new model converged and then used likelihood ratio test (using
To provide support for null results (p < 0.05), we additionally modeled the data using a Bayesian GLMM with weakly informative priors (Gelman et al., 2017) on the fixed and random effects of the model (
Analyses were performed in R version 4.0.3 using RStudio version 1.3.959 (https://www.r-project.org/). Frequentist modeling was performed using the lme4 (Bates et al., 2015b) package, Bayesian modeling was performed using the brms package (Bürkner, 2017), and additional model diagnostics were performed using the performance package (Lüdecke et al., 2021). An R markdown file describing all the model fitting steps and diagnostic checks on the final model is available at the OSF repository for the project (see below, Data availability).
Data Availability
The datasets generated by this study and an R markdown file that reproduces all the reported modeling, statistical analyses, and graphs in the article are uploaded to the Open Science Foundation repository and are available at https://osf.io/25gzj.
Results
RTs were modeled using a GLMM with FP-distribution (uniform/inverse-U-shaped) as a between-subject fixed term and FP-duration (continuous), relation to previous trial (continuous), and cue (valid/invalid/uninformative) as within-subject fixed terms, as well as the full interaction terms between FP-duration, FP-distribution, and cue, and between relation to previous trial, FP-distribution, and cue. Trial number and the interaction between trial number and FP-distribution were added as covariates, and we allowed for a random intercept and a random slope for the linear term of FP-duration and cue by participant.
Effects of foreperiod and spatial attention
Results showed that the VFE, the decrease in RT as foreperiod increases, changed with distribution, for each of the cues (Fig. 3). We observed a significant main effect for FP-duration (
Effects of the distribution shape
The between-group variable of FP-distribution (uniform/inverse-U-shaped) was used to assess the involvement of expectations based on the prior and posterior foreperiod distribution on the VFE and the relation of this effect to spatial attention. Findings showed no main group effect of FP-distribution on RT (
Sequential effects
To test for the existence of SEs, we calculated the difference between the FP-duration of one trial and the FP-duration of the previous trial (
We next tested whether these effects were modulated by spatial attention by examining the interaction between them and cue. Results showed no significant interaction between relation to previous trial and cue (
Model estimates for all fixed factors described are depicted in Figure 5. Model estimates for covariates and additional model information can be found online in the project OSF repository (see above, Data availability).
Bayesian modeling
Our results indicated that there was no evidence for a three-way interaction among cue, FP-distribution, and FP-duration, as well as no three-way interaction among cue, FP-distribution, and relation to previous trial. To examine whether the evidence supports these null results, we constructed a Bayesian GLMM using the same model terms. Model estimates closely resembled the coefficients found in the frequentists model. We compared the resulting Bayesian model with two nested models, each lacking the corresponding three-way interaction term. Results showed large support for the null model lacking the FP duration three-way interaction term compared with the full model (mean
Discussion
In this study we examined whether the spatiotemporal model that was proposed to account for cue-based temporal expectation also carries for temporal expectation based on contextual information, that is, the VFE and SE. As expected, findings showed the VFE with RT decreasing as the foreperiod increases. This VFE changed according to foreperiod distribution. In addition, we found the expected asymmetrical SE, slower RTs for trials in which the foreperiod was longer than their previous trial, and no opposite effect for trials in which the foreperiod was shorter than the previous trial. Critically, all these effects were unaffected by spatial attention. Similar modulations of expectations were found in both attended and unattended spatial locations. This indicates that temporal expectations based on contextual information—the VFE and SE—are independent of spatial attention.
The spatiotemporal model of temporal expectation
Doherty et al. (2005) were the first to demonstrate an interaction between cue-based temporal and spatial attention in early visual event-related potentials (ERPs) components. They presented participants with moving objects that disappeared behind an occluder and reappeared in an expected or unexpected location and/or time. Participants were requested to indicate whether a target was presented on the reappearing object. Findings showed that when a target appeared at an expected location, the early visual P1 component was increased relative to an unexpected location, with enhanced effect when the target also appeared at the expected time. However, when a target appeared at the expected time but not the expected location, there was no enhancement relative to a neutral condition, suggesting that early perceptual benefits of temporal attention depend on the allocation of spatial attention. This spatiotemporal synergism was not found in later ERP components, such as the P3, considered to be less affected by perceptual processes and more by response requirements, and not in RTs.
In a later study by Rohenkohl et al. (2014), symbolic spatial and/or temporal cues predicted with 80% validity the time and location of a grating-patch target, for which participants were requested to perform a nonspeeded orientation discrimination task. Findings showed that valid temporal cues improved both RT and perceptual sensitivity relative to invalid cues but that this effect was limited to trials where spatial attention was focused at the location of the target. These findings provided, again, evidence for a strong synergistic interaction between temporal and spatial expectations in a discrimination task. Consistently, evidence by Seibold et al. (2020) showed that temporal attention boosts the effect of spatial attention on early ERP components in a visual search task.
This evidence of a tight link between spatial attention and cue-based temporal expectation led Nobre and van Ede (2018) to propose their spatiotemporal neurophysiological model, according to which the interaction between spatial and temporal processes stems from time-specific synchronization of spatially specific neural populations at the attended retinotopic receptive fields. These neurons, coding the attended location and relevant features, acquire a temporal structure from repeated exposure to the temporal cues, which affects them but not populations outside the receptive field (Nobre and van Ede, 2018). This model was developed based on evidence on cue-based expectation and was never before examined for other sources of temporal expectations. The present evidence indicates that expectation effects based on contextual information (VFE and SE) do not depend on spatial attention, suggesting that these forms of expectation cannot be explained by this spatiotemporal mechanism.
This further suggests that cue-based temporal expectation and temporal expectation which are driven by contextual information and are often described as two manifestations of the same expectation process, likely rely on distinct neural mechanisms. This evidence is consistent with studies that dissociated VFE and cue-based temporal expectation and found that these two sources of expectations share some, but not all, of their underlying brain networks (Lima et al., 2011; Coull et al., 2016; Amit et al., 2019). More generally, this conclusion is compatible with the increasing recognition that there is no single unified expectation mechanism but that distinct sources of temporal expectations facilitate performance via distinct neural mechanisms (van Ede et al., 2020).
Spatiotemporal synergism and cue-based expectations
It is important to note, however, that evidence regarding the dependency, or lack thereof, of cue-based temporal expectation on spatial attention is ambivalent. In addition to the supporting evidence described above, a few studies provided evidence challenging this interaction. For example, one study investigated the combined influence of temporal, spatial, and feature-based attention and found no synergetic effects between spatial and temporal attention (Rolke et al., 2016). Another study (MacKay and Juola, 2007) used a visual search task in a rapid stimulus visual presentation stream of letters and showed that both spatial and temporal types of cues were effective on their own and that their combined effect was additive, indicating there was no interaction between temporal and spatial attention. In a later study, Weinbach et al. (2015) used a spatiotemporal cuing paradigm and showed that temporal cuing improves RT even when coupled with an invalid spatial cue. Moreover, there was no interaction between the effect of the temporal and the spatial cues, indicating that enhancement resulting from temporal attention was not affected by spatial attention. The authors noted that the discrepancy between their findings and previous findings may have stemmed from differences in task demands, and whereas most previous studies used demanding perceptual-discrimination tasks, Weinbach et al. (2015) used a speeded-RT detection task. In the present study, we have also used a speeded detection task and therefore cannot rule out the possibility that the effect we have observed is specific to speeded easy tasks. However, we note that all previous studies that manipulated the foreperiod distributions to study VFE have either used very easy tasks (Trillenberg et al., 2000; Cravo et al., 2011; Mattiesing et al., 2017; Grabenhorst et al., 2019; Los et al., 2021) or have not reported accuracy rates (Herbst et al., 2018), with the exception of one study that has measured the effect of foreperiod distribution on perceptual speed (Vangkilde et al., 2013). This raises the hypothesis that the VFE depends on a speeded response to an easy task. If this is the case, it would constitute another inherent dissociation between cue-based and contextual temporal expectation. However, with the present data in our hands, we cannot confirm or rule out this hypothesis or determine the link between task difficulty and independency of spatial attention.
Temporal attention and temporal expectation
The apparent discrepancies among different findings on spatiotemporal dependency could be accounted for by the dissociation between attention and expectation processes. According to one view, described in Summerfield and Egner (2009), expectation reflects the narrowing down of the probability space of possibilities constructed according to prior knowledge, whereas attention is the selection of specific, goal-relevant information that should be prioritized. Both attention and expectation coexist and are often entangled; for example, cuing to the left visual field increases our expectation of encountering a target at that location and induces a shift of attention that prioritizes information on that particular visual space. Tailored experimental designs can dissociate attention and expectation, as was demonstrated in visual spatial attention and feature attention studies (Summerfield and Egner, 2009, 2016; Kok et al., 2012).
Similar to spatial cues, temporal cuing paradigms often create a symbolic association between a certain cue and a specific target onset time. Thus, the onset of the cue induces an attentional shift that prioritizes information processing around the cued time interval. In addition, in these designs, the repeated exposure to target onset after a cue changes the probability space and induces temporal expectation, which is independent of attention according to the definition described above (Summerfield and Egner, 2009). Therefore, according to this view, in these designs, temporal attention often coincides with temporal expectation, although specific experimental designs can dissociate these functions (Denison et al., 2019, 2021). Contextual information narrows down the probability space and therefore can be viewed as a source of temporal expectation. This narrowing down of the probability space can be based on exposure to recent previous trials or on the activation of a memory trace of prior, even not so recent experiences (Mattiesing et al., 2017). In a series of studies, Los et al. (2001, 2014, 2017, 2021) presented the multiple trace theory of temporal preparation, suggesting that the basis for the VFE is not conditional probabilities but the activation of memory traces of previous trials. Furthermore, Los et al. (2021) showed that different cues can be associated with memory traces of different temporal distributions, thereby modifying temporal preparation according to cue identity. Arguably, regardless of whether the probability space is gathered from very recent experiences or based on a memory trace associated with a cue, both the VFE and SE could be the result of redefining a probability space rather than focusing attention on a specific time point.
We hypothesize that this proposed dissociation between expectation and attention could account for the discrepancies among previous studies on the spatiotemporal dependency, with temporal attention being spatially specific and temporal expectation remaining independent of the spatial locus of attention. This, in turn, could explain the results observed here, that is, because the hazard rate and sequential manipulations affect only temporal expectation and not attention, their manifestations were free of spatial constraints.
Conclusions
We conclude that the benefits of contextual temporal expectation are not spatially specific but rather reflect a general nonspecific enhancement that is not accompanied by shifts of attention. Furthermore, we suggest that the spatiotemporal neurophysiological model proposed by Nobre and van Ede (2018) to explain cue-based expectation cannot account for the VFE and SE and their link with temporal expectation. Future studies are encouraged to examine the dissociation between different mechanisms of temporal expectation and to refine the terminology to reflect this dissociation.
Footnotes
This work was supported by Israel Science Foundation Grant 1960/19 to S.Y-G. We thank Danielle Allon and Keren Nistor for assistance in running the experiment.
The authors declare no competing financial interests.
- Correspondence should be addressed to Noam Tal-Perry at noamtalperry{at}gmail.com