Abstract
Gibbon's scalar expectancy theory assumes three processing stages in time estimation: a collating level in which event durations are automatically tracked, a counting level that reads out the time-tracking system, and a comparing level in which event durations are matched to abstract temporal references. Pöppel's theory, however, postulates a dual system for perception of durations below and above 2 s. By testing the neurophysiological plausibility of Gibbon's proposal using functional magnetic resonance imaging, we validate a three-staged model of time estimation and further show that the collating process is duplicated. Although the motor system automatically tracks durations below 2 s, mesial brain regions of the so-called “default mode network” keep track of longer events. Our results further support unique counting and comparing systems, involving prefrontal and parietal cortices in collators' readout, and the temporal cortex in contextual time estimation. These findings provide a coherent neuroanatomical framework for two theories of time perception.
Introduction
Perception of time is an innate trait of cognition, almost a sense, for which however there is no dedicated sensory cortex. All sensory events have a temporal dimension, and many neuronal tissues have intrinsic temporal properties (e.g., rhythmic activity), which could serve to track this dimension. Yet, how the brain codes the duration of sensory events and whether it exploits its own temporal properties to sense time is a complex and unresolved question.
Neuroanatomically, time perception emerges from neural activity in distributed brain systems. Although lesion data show that the executive (Harrington et al., 1998) and motor (Ivry and Spencer, 2004) systems are necessary for explicit time estimation, functional neuroimaging further involves sensory associative regions (Rao et al., 2001; Lewis and Miall, 2003a; Coull et al., 2004). Time perception thus presumably results from interactions across three neural systems: (1) a core timer (e.g., the motor system) (Meck et al., 2008), (2) an executive system that extracts stimulus duration (Lewis and Miall, 2006), and (3) an associative system that contextualizes durations (Spetch and Rusak, 1992).
Among many models of time perception (Wittmann, 2009), the scalar expectancy theory (SET) (Gibbon, 1977) is particularly interesting from a neurophysiological perspective as it comprises three levels of processing that potentially match established functional neuroanatomy. According to SET, an intrinsic pacemaking mechanism produces a series of pulses that are “collated” (i.e., concatenated in larger time bins). These bins are subsequently “counted” to estimate durations. Finally, a comparing system connects the output of the counting process with memorized time representations to estimate durations. Unlike other cognitive models of prospective time estimation (Wittmann, 2009), in which every step depends on explicit attention demands (Block, 1990; Zakay and Block, 1997), SET assumes that the collating system can automatically be triggered by external stimuli and tracks their temporal dimension whatever the task performed. In this framework, only the counting and comparing processes work on explicit time measurement demands. SET additionally implies that the comparing stage is insensitive to stimulus absolute duration but essentially detects whether a stimulus is longer or shorter than a reference (i.e., processes relative durations) (Fig. 1).
Pöppel (1997), however, postulated a dual system for perceiving durations shorter and longer than ∼2 s. Although this theory has received plenty of experimental support (Elbert et al., 1991; Rammsayer, 1999; Kagerer et al., 2002; Ulbrich et al., 2007), it has not yet been explored using functional magnetic resonance imaging (fMRI). To determine how SET relates to the neurofunctional scheme described above, and how it potentially interacts with a dual timescale system, we used an fMRI experimental paradigm adapted from Coull et al. (2004).
We probed the three levels of Gibbon's model using a hierarchical fMRI experimental design (Fig. 1). Subjects performed either an explicit time estimation or a color decision task on visual stimuli of varying durations. We reason that those regions showing stimulus duration dependency regardless of task correspond to the collating system. Conversely, those regions showing stimulus duration dependency only when time estimation is explicitly required are involved in the counting process. Finally, we assume that the comparing process correspond to those brain regions that are sensitive to stimulus relative durations regardless of the absolute timescale. To decorrelate absolute and relative stimulus durations and address the dual timescale hypothesis, we used a broad range of durations encompassing the theoretical boundary between the two time perception systems (i.e., spanning below and above 2 s).
Materials and Methods
Subjects.
Seventeen healthy volunteers (two males; two left-handed; aged 20–34 years; mean, 25.4 years; written informed consent) with normal vision, and no history of neurological or psychiatric illness, participated in the fMRI experiment.
Stimuli and protocol.
Subjects were asked to compare either the duration or the average color (task, experimental factor 1) between two disks that were sequentially (interstimulus interval ranging between 450 and 550 ms) presented at the center of a screen, for a variable duration, ranging in 11 steps from the subsecond to the multisecond timescale (100, 400, 500, 900 ms, 1.1, 1.2, 2, 2.9, 4.7, 5.2, 8.4 s). These durations (experimental factor 2) were selected to match task difficulty across four different time windows (see also next paragraph). To further match difficulty across tasks, disk color alternated continuously every 33 ms between three of five different shades (blue, lavender, purple, pink, red). Depending on the choice of shades, this flickering resulted in an overall categorical perception of the disk as blue, purple, or red. Subjects were instructed to indicate which of the two disks was presented for a longer duration (time condition), or which one was bluer (color condition) (Coull et al., 2004, 2008; Livesey et al., 2007). Presentation of two consecutive disks corresponded to one trial (Fig. 2a). Responses were given with the right index (first disk longer or bluer), middle (both disks equal), or ring finger (second disk longer or bluer) as soon as a black screen appeared after each trial (for 2–4.7 s).
To probe the influence of relative stimulus durations, we introduced a contextual factor (experimental factor 3). Stimuli were clustered in four 2.5 min blocks, spanning different timescales. In one block, three different stimulus durations belonging to the same “working” timescale were randomly paired. Within each block, stimuli could thus implicitly be categorized as short, intermediate, and long (Fig. 2b). This entailed the following trials (short/short, long/long, intermediate/intermediate, short/long, intermediate/long, short/intermediate, and vice versa). We designed the 11 stimuli to match, across working timescales, the duration increment between the shortest and the intermediate stimulus and that between the intermediate and the longest stimulus.
The four blocks were randomly presented twice in two 24 min sessions, once preceded by a “time” instruction and once by a “color” instruction presented as a written sentence on the screen (“Which disk is bluer?” or “Which disk lasts longer?”). There was a 25 s pause between each block. As stimulus duration varied but block length was fixed to control for the contextual factor across blocks, the number of trials varied across blocks (from block 1 to 4, trials ranged from 40, 25, 15, to 10 per block) (Fig. 2b). The two dimensions of the stimulus (duration and color) were counterbalanced so that any of the three duration ranges could be paired with any of the three color categories.
Before scanning, participants were familiarized with the experimental setting and instructed to refrain from intentionally counting to estimate stimulus duration. A post hoc debriefing ensured they performed the task without inner counting.
Data acquisition.
A total of 710 image volumes was acquired using a standard echoplanar imaging sequence with an echo time of 30 ms, repetition time of 2000 ms, and a voxel size of 3 × 3 × 3 mm3 (1 mm gap; 33 slices, which covered the whole brain) on a Siemens Allegra 3T magnetic resonance scanner with a standard head coil.
Data analysis.
We assessed behavior during scanning using a repeated-measure ANOVA with two factors, block and task. We also assessed a possible effect of stimulus duration on performance using Pearson's regression in the time conditions trials in which identical stimuli were paired (short/short, long/long, etc.).
Standard preprocessing [realignment, normalization to the echoplanar imaging Montreal Neurological Institute (MNI) template, and smoothing with an 8 mm Gaussian kernel] and statistical analyses were performed using SPM5 (http://www.fil.ion.ucl.ac.uk/spm). We assessed the three levels of cognitive time estimation (collating, counting, comparing) using three distinct event-related models. All stimuli were modeled on the basis of their respective onsets and durations. Instruction, motor response, and the six realignment parameters with their respective first derivative were accounted for in each model. The baseline consisted of 25 s pause between blocks. Statistics were computed using a general linear model (global normalization scaling option). Effects of interest (see below) were computed within subjects by contrasting the parameter estimates associated with each condition, and across subjects in a second-level analysis, using t tests.
To assess the collating system, we built a parametric regressor based on stimulus duration (parametric modulation option) separately for each task (time and color). According to the tested model (Fig. 1), the collating process is task and behavior independent, and thus essentially varies along factor 2 (stimulus absolute duration). Therefore, trials with incorrect responses were included in this analysis. Regions associated with the collating process were probed by parametric increase and decrease with stimulus duration, in both time and color tasks. Results are reported in Table 1 for each task with a cluster level correction for multiple comparisons (p < 0.05; cluster extent, >120) (Friston et al., 1996). We illustrate this result by a conjunction analysis of both tasks (as implemented in SPM5) using the same cluster level correction for multiple comparisons, overlaid on the main effect of time estimation (T > C), which permits to visualize slight task modulations within the collating system. The main effect of time estimation (T > C) corresponds to the nonparametric contrasts between time and color tasks (p < 0.05, corrected for multiple comparisons; cluster extent, >120). We further investigated the parametric behavior of the collating process (i.e., shape and slope of stimulus duration dependency) by carrying out a separate analysis in which we modeled the 11 durations separately for each task and plotted the β-values.
To examine time counting and comparing systems, we separated correct from incorrect responses and modeled out errors. The counting system corresponded to the stimulus absolute duration-by-task interaction in correct trials only (interaction of factor 1 and factor 2; t test, p < 0.001, uncorrected; cluster extent, >12; masked by T > C at p < 0.01). We assessed the comparing system, by pooling across blocks but separately for each task, short, intermediate, and long stimuli. We then computed the stimulus relative duration-by-task interaction (interaction of factor 1 and factor 3) (e.g., short > long durations, in the time > color condition). We report results at p < 0.001, uncorrected (cluster extent, >12; masked by T > C at p < 0.01). This analysis therefore probed those regions that respond to stimulus relative durations regardless of their absolute duration.
Finally, we examined incorrect responses to explore duration misestimation. We compared responses to stimuli associated with behavioral errors with responses to identical stimuli, but correctly perceived. We reasoned that an incorrectly compared stimulus pair corresponded either to the overestimation of the shorter stimulus, or to the underestimation of the longer one. Because we were unable to dissociate the two alternatives, we split the stimulus pairs associated with incorrect responses into two regressors probing responses associated with underestimation and overestimation, respectively. We further controlled for correct estimation of one of the two stimuli by building “control” regressors, using stimuli that were associated with correct responses matching all other dimensions of the stimuli [i.e., stimulus duration, order of presentation (sample or probe), difficulty, and number of repetition]. Residual events were modeled out. We then analyzed the interaction between the underestimated, the overestimated stimuli, and their correctly estimated control stimuli for each subject, and we report results at cluster level correction for multiple comparisons (p < 0.05; cluster extent, >50).
Results
Behavioral data
Subjects performed the experiment with a mean accuracy of 85% (SD = 7) for the time task and 83% (SD = 8) for the color task (Fig. 3a). There was no significant accuracy difference across tasks (F(1,16) = 2.71; p > 0.1), but in a block-by-block analysis the color task appeared more difficult in block 4 than the time task (color 72% vs time 83%; p = 0.004). Block 4 included longer durations hence fewer stimuli, implying that fewer errors yielded a steep drop in performance. Within time condition trials in which the same stimulus duration was presented twice, we observed a negative correlation between accuracy and stimulus duration (r = −0.704; p < 0.000) (Fig. 3b), but only a trend for a dependence between reaction time and stimulus duration (r = −0.165; p = 0.037) (Fig. 3c), meaning that subjects became less accurate but not slower when stimulus duration increased.
Functional data
We hypothesized that regions related to collating should modulate their activity as a function of stimulus duration independent of the task performed. Regions in which activity increased with stimulus duration in both time and color tasks included mesial prefrontal, posterior cingulate (PCC), and lateral parietal cortices (Fig. 4, green clusters; Table 1). Regions in which activity decreased included bilateral precentral gyrus, anterior insulae, pre-supplementary motor area (pre-SMA)/SMA, basal ganglia, thalamus, mesencephalon, cerebellum, and the right middle frontal gyrus (Fig. 4, blue clusters). A post hoc analysis (illustrated for the left SMA and for the right PCC in Fig. 4) showed that activity varied nonlinearly (logarithmically) with stimulus duration in both systems, in accordance with the Weber–Fechner law of sensory perception (Weber, 1850; Fechner, 1860).
Whereas color estimation activated bilateral occipital regions including visual area V4 (McKeefry and Zeki, 1997) and the left inferior frontal gyrus (IFG) (Table 1), time estimation (T > C) involved bilateral dorsolateral prefrontal cortex (DLPFC), PCC, posterior superior temporal gyri (STG)/angular gyri, pre-SMA/SMA, anterior insulae, dorsal striata, mesencephalon, cerebellar hemispheres, the right frontal operculum, the right middle temporal gyrus (MTG), and the left posterior superior temporal sulcus (STS) (Figs. 4, 5, gray clusters).
The counting system postulated by Gibbon was probed by a parametric modulation with stimulus duration only in the task that required explicit time estimation, but not in the color condition. The counting system was restricted to those regions in which activity also decreased with stimulus duration, namely, bilateral posterior STG/angular gyrus and DLPFC, the right MTG and frontal operculum, and the left posterior STS (Fig. 5, orange clusters). We did not detect brain regions in which activity increased with stimulus duration (interaction of parametric and task effect, T > C).
The comparing system was probed by sorting stimuli according to their relative duration in a block, rather than according to their absolute duration. We could not detect regions that were more strongly activated for longer than for shorter relative stimulus durations (interaction with T > C). The right MTG and left posterior STS were significantly more activated for shorter than for longer relative stimulus durations (interaction with T > C) (Fig. 5, yellow clusters).
Finally, the effect of stimulus duration misestimation, analyzed separately by comparing responses to stimuli associated with behavioral errors with responses to identical stimuli, but correctly perceived, was associated with activation of the mesial prefrontal cortex (underestimation > control, t(16) = 3.853, p < 0.001; overestimation > control, t(16) = 4.583, p < 0.000). Neural activity in the anterior and posterior cingulate cortices and bilateral inferior temporal gyrus increased during underestimation and decreased during overestimation (Fig. 6, green clusters; Table 2). The pre-SMA showed the opposite behavior: its activity was lower for duration underestimation than for correct estimation (Fig. 6, blue cluster) and higher for overestimation.
Discussion
Two collating systems
Against the view that time perception only occurs under attention demands (Block, 1990), Gibbon postulated the existence of a collating mechanism that automatically tracks durations. This system should modulate its activity depending on the amount of elementary time pulses added up during the time period delimited by a sensory stimulus. We found two systems in which neural activity followed stimulus duration regardless of the task performed. Neural activity decreased with stimulus duration in the motor system, whereas the so-called “default mode network” (DMN) (Fox et al., 2005) was less deactivated when duration increased. These complementary systems support a dual timescale mechanism for perceiving durations shorter and longer than ∼2 s, as previously posited and tested by many authors (Fraisse, 1984; Elbert et al., 1991; Pöppel, 1997; Rammsayer, 1999; Wittmann, 1999; Kagerer et al., 2002; Mauk and Buonomano, 2004; Buhusi and Meck, 2005; Ulbrich et al., 2007). The current results being limited to durations spanning from 100 ms to 8.4 s, we do not exclude other mechanisms for durations <100 ms or in the minutes range (van Wassenhove, 2009).
Time collating by the motor system
The motor system appeared a key component of the collating process (Stevens et al., 2007) as it automatically tracked durations within the timescale at which it operates to coordinate movements (tens to hundreds of milliseconds) (Brooks, 1986) but was inefficient above 2 s. Accuracy of the motor system in the subsecond time range may be related to fast (30–40 Hz) oscillatory activity of the motor cortex (Funk and Epstein, 2004; Giraud et al., 2007), which could be used to track durations of brief events (Pöppel, 1997). In this context, motor thalamo-cortical-striatal cycles (Buhusi and Meck, 2005) could also be exploited, and serve as pulses that are subsequently collated. The motor system has previously been implicated in discriminating short durations (Jueptner et al., 1995; Rao et al., 2001; Lewis and Miall, 2003b). Here, we additionally show that its involvement decreased exponentially with stimulus duration. This effect was paralleled by a loss of time estimation accuracy presumably reflecting progressive disengagement of the motor system when durations exceeded its optimal working range.
Our findings corroborate previous claims that the pre-SMA/SMA is a core time measurement region (Ferrandez et al., 2003; Coull et al., 2004; Pouthas et al., 2005; Macar et al., 2006), as it was the only motor region to be related to time perception errors. Misestimation of short durations within the operational range of the motor system (<2 s) could arise from alteration of the counting process. As we observed that the time task modulated (nonsignificant result) the neural activity in the SMA, errors could originate in attention fluctuations affecting the reading of this automatic system. Alternatively, overestimation could arise from timescale shifts occurring within the collating motor system. Patients with Parkinson's disease or a history of drug addiction, both affecting the dopamine system, have altered time perception (Pastor et al., 1992; Wittmann et al., 2007; Merchant et al., 2008). It is thus conceivable that instabilities of the core motor neurochemistry might induce errors at the collating level and subsequently affect estimation of short durations.
The default mode network and time perception
Our results point to the involvement of the DMN in continuous tracking of long duration stimuli, at the seconds level. The DMN underpins task-independent self-referenced cognition, memory (Greicius et al., 2004; Sperling et al., 2009), and is usually deactivated during sensory stimulation (Buckner et al., 2008). That the DMN was increasingly mobilized for long durations (i.e., despite increasing task demand) (Singh and Fawcett, 2008; Wu et al., 2009), speaks to its specific involvement in perception of durations above 2 s. This finding is interesting in the context of the theoretical distinction between prospective and retrospective time estimation (Zakay and Block, 2004). Although attention is required to prospectively assess duration, it is always possible to retrospectively estimate the duration of an event, even if time was not the focus of attention. It is thus assumed that retrospective timing more strongly relies on memory and less on attention than prospective timing (Ornstein, 1970; Noulhiane et al., 2007). Our results support this view by showing the involvement of a memory system (the DMN) in automatic time tracking, for durations above 2 s. Our results thus imply that the DMN, as automatic collating system for long durations, could participate in both prospective and retrospective time estimation.
Another argument for DMN contribution to long-duration perception stems from time perception errors. Although all errors were paralleled by enhanced activity in the mesial prefrontal cortex, usually related to response error or decision uncertainty (Ridderinkhof et al., 2004), underestimation was associated with neural activity in three regions of the DMN. Time perception errors could theoretically arise from perturbations of any of the three stages of Gibbon's model. If the collating systems were perfectly scaled, our results would be paradoxical as more activity in the DMN would be associated with a longer duration (and vice versa for the pre-SMA) (Figs. 4, 6). Therefore, the most plausible interpretation of our findings is that errors are attributable to slight scale shifts of the collating systems, which may go in either direction (i.e., toward time expansion or time compression).
The DMN has never been related to time perception, probably because it is activated during baseline conditions that are usually subtracted out in functional neuroimaging studies (Raichle et al., 2001; Buckner et al., 2008) or during tasks that do not require focal attention (Hahn et al., 2007). That lesion studies do not reveal specific time impairment for mesial wall lesions is not incompatible with its role in time estimation (Devinsky et al., 1995; Derouesné, 2003; Yücel et al., 2003). Lesions to this large network are rare and induce global apathy and incapacity to organize actions, rather than specific cognitive impairments. Estimation of long time intervals is necessary to plan high-level behavior, and its impairment a plausible silent clinical correlate of DMN lesions.
Following the logic of Pöppel (1997) that cortical rhythms reflect integration properties, the multisecond timescale in which the DMN was found to operate is compatible with its oscillatory properties, around 10 Hz (alpha rhythm) (Jann et al., 2009), which should be more adapted to track longer durations than fast gamma rhythms in the motor cortex (30–40 Hz). Gamma motor rhythms correspond to temporal integration windows of ∼30 ms, potentially relevant to subsecond timing, whereas alpha rhythm corresponds to a 100 ms integration window likely useful to suprasecond timing. We estimated the boundary (or overlap) between the two complementary collating systems to lie around 2 s, even though it cannot be precisely inferred from the present data.
Time counting system
The counting system encompassed the right frontal operculum, previously associated with rhythmic timing and sequencing functions (Schubotz et al., 2000; Ferrandez et al., 2003; Tregellas et al., 2006; Livesey et al., 2007). It also involved bilateral DLPFC, previously associated with time estimation (Smith et al., 2003) and response selection (Rao et al., 2001). In Gibbon's framework, these prefrontal regions should index the collators' output for explicit stimulus duration readout (e.g., when time evaluation is required). The counting system further included bilateral posterior STG/angular gyri, previously implicated in time estimation both by lesion (Harrington et al., 1998) and functional neuroimaging studies (Lewis and Miall, 2003b; Coull et al., 2004; Livesey et al., 2007). The latter indicate a role for these regions in attention to time (Rubia and Smith, 2004) and duration encoding (Rao et al., 2001). The posterior STG corresponds to a buffer that transiently stores sounds with their temporal structure (Hickok and Poeppel, 2007). Finally, the counting system was associated with bilateral activations of the middle temporal cortex (right MTG and left STS), auditory association regions also found in previous studies of time perception (Larsson et al., 1996; Coull et al., 2000, 2004; Rao et al., 2001; Lewis and Miall, 2003b).
It may seem surprising to observe consistent activations of dorsal and ventral auditory cortical systems (Hickok and Poeppel, 2007) in a visual task that did not explicitly target auditory processing. Yet, the perceived duration of all physical events heavily relies on temporal integration, a critical feature of the auditory sense. Durations may thus be best represented by the auditory system as it is equipped to maintain representations belonging to longer streams (posterior STG/angular region), and hosts representations that retain the temporal dimension of stimuli. Functional neuroimaging and behavioral (Guttman et al., 2005) studies converge to suggest that the auditory association system plays a role in stimulus duration representation destined for additional abstract processing.
We did not find regions in which neural activity increased with stimulus duration. Our results thus indicate that a unique counter reads out both collating outputs. This counting system could possibly constitute a means for unifying time perception across distinct timescales.
Time-comparing system
The time-comparing system only involved the right MTG and the left posterior STS. These auditory associative regions were more active for durations that were categorized as shorter, regardless of the absolute timescale. Rao et al. (2001) previously showed that the posterior temporal cortex encodes duration. Within Gibbon's hierarchical framework, the comparing system represents an abstract level of time representation that is independent of absolute stimulus duration on a linear timescale. Accordingly, the comparing system only partly overlapped the counting system and extended toward more associative regions. Ventral temporal regions close to this comparing system were activated during underestimation of stimulus durations, suggesting that the temporal cortex is also involved in temporal distortions (Ivry and Schlerf, 2008), a hypothesis that is supported by time distortions in temporal lobe epilepsies (Panayiotopoulos, 2005). Yet, as time underestimation was also associated with activations in mesial wall regions of the DMN, the latter might also participate in the detection of temporal distortions occurring in the long duration collating system.
Summary
This study confirms existing results on functional neuroanatomy of time perception (Rao et al., 2001; Lewis and Miall, 2003a; Coull et al., 2004), while assigning previously identified systems to specific computational steps. We propose a dual collating process involving the motor system and the DMN for durations below and above 2 s, respectively, a widespread counting system that encompasses frontal executive and auditory association regions, and a comparing system limited to regions of the temporal semantic system (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). By showing distinct automatic time-tracking systems within a three-staged model, our data unify several, seemingly unrelated, views on time perception.
Footnotes
A.-L.G. is funded by the Centre Nationale de la Recherche Scientifique. We are grateful to Marine Ardoint, Floris De Lange, Christopher Summerfield, Julie Grèzes, Etienne Koechlin, Andreas Kleinschmidt, Luc H. Arnal, and Iris Trinkler for their comments on this paper.
- Correspondence should be addressed to Benjamin Morillon, Laboratoire de Neurosciences Cognitives, Ecole Normale Supérieure, 29 rue d'Ulm, 75005 Paris, France. bmorillon{at}ens.fr