Abstract
It has been proposed that numerical and temporal information are processed by partially overlapping magnitude systems. Interactions across different magnitude domains could occur both at the level of perception and decision-making. However, their neural correlates have been elusive. Here, using functional magnetic resonance imaging in humans, we show that the right intraparietal cortex (IPC) and inferior frontal gyrus (IFG) are jointly activated by duration and numerosity discrimination tasks, with a congruency effect in the right IFG. To determine whether the IPC and the IFG are involved in response conflict (or facilitation) or modulation of subjective passage of time by numerical information, we examined their functional roles using transcranial magnetic stimulation (TMS) and two different numerosity–time interaction tasks: duration discrimination and time reproduction tasks. Our results show that TMS of the right IFG impairs categorical duration discrimination, whereas that of the right IPC modulates the degree of influence of numerosity on time perception and impairs precise time estimation. These results indicate that the right IFG is specifically involved at the categorical decision stage, whereas bleeding of numerosity information on perception of time occurs within the IPC. Together, our findings suggest a two-stage model of numerosity–time interactions whereby the interaction at the perceptual level occurs within the parietal region and the interaction at categorical decisions takes place in the prefrontal cortex.
Introduction
Judgments of duration are prone to the influence of seemingly irrelevant numerical magnitude information in the stimulus (Dormal et al., 2006; Xuan et al., 2007; Oliveri et al., 2008). Typically, participants made more errors in relative duration judgments of two successively presented intervals when the changes of task-irrelevant numerical magnitudes were incongruent with the changes in the durations of the two stimuli (e.g., increase in duration but decrease in the number of dots) than when the changes in duration and other magnitude dimensions were congruent (e.g., increase in both duration and number of dots). Although this congruency effect was interpreted as evidence of interference of numerical magnitude with duration processing, it remains unclear whether these congruency effects reflect genuine perceptual changes of duration or biasing categorical judgments.
Based on the “A Theory Of Magnitude (ATOM)” (Walsh, 2003), the numerosity–time interaction was interpreted as a reflection of shared neural representations for time and numerosity (Xuan et al., 2007). The ATOM posits that various dimensions of magnitude information, such as space, time, and quantity, are encoded by “common neural metrics” in the parietal cortex, and thus, multiple inputs of magnitude information may interact with each other. The idea that the parietal cortex supports common representations for different magnitude dimensions is supported by several functional magnetic resonance imaging (fMRI) studies that examined the interaction between number and size, size and brightness, and number and brightness (Pinel et al., 2004; Cohen Kadosh et al., 2008).
However, the previous studies only focused on magnitude dimensions other than time, and it remains unknown whether the representation of time is also shared by such a generic magnitude system in the human brain. Moreover, it has not been established whether the influence of non-temporal magnitude on duration judgments indeed reflects changes in perceived duration. The numerosity–time interaction has been demonstrated only in comparison tasks so far (Dormal et al., 2006; Xuan et al., 2007; Oliveri et al., 2008; but see Chang et al., 2011). However, it is possible that the congruency effect could be attributed entirely to response error (or facilitation) as a result of conflicting (or congruent) more-versus-less categorical information inherent in comparison tasks (Ivry and Schlerf, 2008) rather than to distortion of subjective passage of time.
The goal of the present study was to determine the brain regions that mediate the numerosity–time interaction. To do so, we first determined brain regions that were involved in both numerical and temporal processing and then tested whether those regions exhibited a congruency effect (fMRI, experiment 1). Next, we examined the functional roles of those regions using transcranial magnetic stimulation (TMS). We used two different numerosity–time interaction tasks: (1) a duration discrimination task comparable with the first fMRI experiment (experiment 2); and (2) a duration reproduction task that minimized the involvement of categorical magnitude judgment (experiment 3). As such, the duration reproduction task was conceived as reflecting changes in perceived duration rather than response conflict (or facilitation). Finally, we tested the functional role of the parietal cortex in a numerical processing task (experiment 4).
Materials and Methods
Experiment 1: numerosity–time interaction in duration discrimination task (fMRI)
Subjects.
Twenty-six healthy volunteers (12 males and 14 females, 19–30 years old) participated in the fMRI study. All subjects were right handed according to the Edinburgh handedness inventory (Oldfield, 1971). None of the subjects had a history of neurological or psychiatric illness. All participants gave written consent. This experiment was approved by the National Institute for Physiological Sciences ethics committee.
Task and stimuli.
The tasks were either (1) to discriminate which one of two dot arrays was presented for a longer time (duration discrimination task) or (2) to discriminate which one of two dot arrays contained a larger number of dots (numerosity discrimination task). In a control task (20% of all trials), subjects were asked to press a button with either the right index or the middle finger, according to the instruction presented at the beginning of the trials. No feedback was given to the participants.
Within a trial, two dot arrays were sequentially presented (Fig. 1A). We manipulated combinations of an increase or decrease in stimulus duration and in number of dots. Trials in which the length of stimulus duration and numerosity changed in the same direction (duration–numerosity; increase–increase or decrease–decrease) were defined as congruent trials, and those that changed in the opposite directions (increase–decrease or decrease–increase) were defined as incongruent trials (Fig. 1B). The degree of change in the task-relevant dimension (i.e., the longer/shorter duration ratio in the duration discrimination trials and the larger/smaller numerosity ratio in the numerosity discrimination trials) was constant throughout the sessions, whereas that in the task-irrelevant dimension (i.e., the larger/smaller numerosity ratio in the duration discrimination trials and the longer/shorter duration ratio in the numerosity discrimination trials) was varied. This manipulation was made based on the idea that the congruency effect should be enhanced when the ratio in the task-irrelevant dimension was large. Therefore, the experiment was designed as a 2 × 2 × 2 within-subject factorial design with factors task (duration task/numerosity task), congruency (congruent/incongruent), and distance (i.e., the degree of change in the task-irrelevant dimension; close/far).
In the duration discrimination task, we used three pairs of duration: 460–538, 560–655, and 660–772 ms (longer/shorter = 1.17). In the duration discrimination task, the task-irrelevant numerosity pairs for the close condition were 20–25, 31–39, 49–61, and 76–95 (larger/smaller = 1.25) and for the far condition were 20–49, 39–95, 25–61, and 31–76 (larger/smaller = 2.44). Similarly, in the numerosity discrimination task, we used three pairs of numerosity: 30–38, 44–55, and 58–73 (larger/smaller = 1.25). The task-irrelevant duration pairs for the close condition were 330–386, 452–529, 618–724, and 847–990 ms (longer/shorter = 1.17) and for the far condition were 330–618, 529–990, 386–724, and 452–847 ms (longer/shorter = 1.87). The average magnitudes of numerosity and duration were balanced between the tasks to control the physical characteristics (duration task, average duration of 607.5 and average numerosity of 49.5; numerosity task, average duration of 609.5 and average numerosity of 49.7).
Stimuli were dot arrays presented on a black background. The dot arrays were generated by an automated program that was developed by Piazza et al. (2004). The shape of each dot was a circle, and its size was equal across the stimuli. Each pattern of the dot arrays was presented just once for a subject, including practice runs. The dot arrays were presented in an invisible circle with a radius of 5°. To maximize the statistical power while retaining psychological validity, the sequences of the stimulus events were optimized using the genetic algorithm developed by Wager and Nichols (2003).
Each trial was followed by a fixation cross that was presented for 500, 1000, 1500, 2000, or 2500 ms and an instruction cue (600 ms) that indicated whether the following trial was a discrimination or control trial. Instruction cues (written in a Chinese character) for discrimination trials were “time” or “numerosity,” which referred to the duration or numerosity discrimination task, and for the control trials were “index” or “middle,” which indicated the responding finger the subjects had to press in the response period. Regardless of the control or discrimination trial, two panels of dot arrays were sequentially presented at the center of the screen after the fixation cross for 1400 ms, with an interval of 400 ms, and then the red fixation cross was shown for 1000 ms to indicate the response period. Intertrial interval was 8000 ms. Subjects performed the duration and numerosity discrimination task in separate fMRI runs. Each run consisted of 60 trials, and two runs for each task were performed. Before starting each task, subjects performed a practice run of 16 trials to get used to the task procedure and experimental environment.
The subject's right index and middle fingers were placed on the response buttons. The assignments of the buttons were explained before the fMRI scanning. Subjects were asked to respond as quickly and accurately as possible in the discrimination tasks and to press the button within the response period in the control trials. Subjects were also instructed to fixate on the center of the screen throughout the session. They were asked to ignore the task-irrelevant dimension during the discrimination trials and both dimensions during the control trials. The order of the tasks and the assignment of buttons were randomized and counterbalanced between subjects.
Behavioral data analyses.
Accuracy of the behavioral performance during the scanning was individually computed for each condition. Differences of these values were then statistically tested using a three-way repeated-measures ANOVA. Reaction times were not analyzed, because reaction times in a duration discrimination task are difficult to interpret. In a duration discrimination task, stimulus duration could theoretically be judged even before the end of the second stimulus interval when the stimulus has already lasted longer than the length of the first stimulus. However, in our experiment, the response cue was presented only after the end of the whole trial. Therefore, we used accuracy only as a dependent variable, as in the previous study (Xuan et al., 2007).
fMRI data acquisition and analyses.
Visual stimuli were projected onto a half-transparent screen by a liquid crystal display projector. The screen was visible through a mirror mounted on the head coil. We confirmed that the subjects were able to see the screen at the center of their view. Presentation software (Neurobehavioral System) was used to run the stimuli.
A time course series of 249 volumes per session was acquired using ascending T2*-weighted gradient-echo echo-planar imaging sequences using a 3 tesla MR imager (Allegra; Siemens). Each volume consisted of 34 oblique slices, each 3.5 mm in thickness with 0.56 mm slice gap, to cover the entire cerebral and cerebellar cortex. The time interval between two successive acquisitions of the same slice was 2000 ms, with a flip angle of 80° and a 30 ms echo time. The field of view was 192 × 192 mm. The digital in-plane resolution was 64 × 64 pixels, with a pixel dimension of 3.0 × 3.0 mm. The head motion was minimized by taping and placing towels between the subject's head and the head coil. High-resolution whole-brain MR images were also obtained using a T1-weighted three-dimensional (3D) magnetization-prepared rapid acquisition gradient-echo sequence (voxel size, 0.9 × 0.9 × 1.0 mm).
The first five volumes of each fMRI session were discarded because of unsteady magnetization, and the remaining 244 volumes per session (a total of 976 volumes per subject) were used for the analysis. The data were analyzed using statistical parametric mapping (SPM8; Wellcome Trust Centre for Neuroimaging, London, UK) implemented in MATLAB (MathWorks; Friston et al., 1995). After realignment and slice timing correction, all of the images were coregistered to the high-resolution 3D T1-weighted MRI images. The parameters for affine and nonlinear transformation into a template of T1-weighted images that was already fitted to a standard stereotaxic space [Montreal Neurological Institute (MNI) template] were estimated with the high-resolution 3D T1-weighted MR images using the least-square mean. The parameters were applied to the coregistered fMRI data. The anatomically normalized fMRI data were filtered using a Gaussian kernel of 8 mm full-width at half-maximum in the x-, y-, and z-axes.
For individual statistical analysis, the signal time course of each subject was modeled with hemodynamic response functions, high-pass filtering (128 s), and session effects. The regressors were set at the onsets of the instruction, dot arrays, and button response periods independently. The presentations of dot arrays comprised five independent regressors for congruent-close, congruent-far, incongruent-close, incongruent-far, and control conditions. The duration of these regressors were set at the length between the onset of the first and the offset of the second dot array. In total, seven regressors were set for each run. The individual task-related activity was evaluated using a general linear model (Friston et al., 1995). The resulting set of voxel values for each comparison constituted a statistical parametric map (SPM) of the t statistic [SPM{t}]. Global mean scaling was applied to minimize the effect of noise caused by a drift of blood oxygen level-dependent signal.
The summary data for each individual were incorporated into the second-level analysis using a random-effects model to make inferences at population level. The weighted sum of the parameter estimates in the individual analysis constituted “contrast” images, which were used for the group analysis (Friston et al., 1999). The contrast images for each condition were obtained via individual analyses representing the task-related increment of MR signals. The contrast image for the control condition was subtracted from each of the contrast images for congruent-close, congruent-far, incongruent-close, and incongruent-far conditions in advance. For these contrast images, a 2 (task) × 2 (congruency) × 2 (distance) repeated-measures three-way full factorial analysis was performed for every voxel within the brain, to obtain population inferences. The resulting set of voxel values for each contrast constituted an SPM of the t statistic [SPM{t}].
Experiment 2: numerosity–time interaction in duration discrimination task (TMS)
Subjects.
Ten healthy volunteers (three males and seven females, 20–30 years old) completed a TMS experiment. All participants gave written consent. This experiment was approved by the University College London ethics committee.
Task and stimuli.
Subjects performed a duration discrimination task, before and after the offline TMS. The task was essentially the same as that in the first fMRI experiment.
In this experiment, we used three pairs of durations [450–540, 600–720, and 750–900 ms (longer/shorter = 1.20)] and four pairs of numerical magnitudes (1–4, 7–10, 1–7, and 4–10). The dot arrays were black on a large gray disk of ∼6°, presented against a black background. The dot arrays were generated by an automated program (Piazza et al., 2004). The shape of each dot was a circle, and its size was equal across the stimuli. Spatial positions of the dots were randomized. Each pattern of the dot arrays was presented just once for each subject, including the practice trials.
Two panels of dot arrays were sequentially presented, with an interval of 400 ms, and followed by a black fixation cross that indicated 2000 ms of response period. The fixation cross disappeared when the response was submitted. A dot array for the next trial was presented 1500 ms after the response period (see Fig. 4A).
Subjects were asked to indicate which one of the two dot arrays was presented for a longer period of time, as quickly and accurately as possible, by pressing with their right index finger key J or with their middle finger key K. Key J denoted that “the first array was presented longer,” and key K denoted that “the second array was presented longer.” Subjects were also instructed to fixate on the center of the screen throughout the sessions. They were asked to ignore the task-irrelevant dimension, such as the number of dots or the spatial positions of dots, and to avoid using the counting strategy for the duration discrimination.
Stimuli were presented at the center of the cathode ray tube (CRT) monitor running at 100 Hz. Subjects put their chin on the chin rest positioned at a distance of ∼60 cm from the CRT monitor. The computer keyboard was placed between the CRT monitor and the chin rest. Psychtoolbox (http://psychtoolbox.org) implemented on MATLAB software (MathWorks) was used to present the stimuli.
TMS experimental procedure.
Subjects performed 192 trials of the duration discrimination task before (pre) and after (post) the offline continuous theta-burst stimulation (cTBS). Pre- and post-TMS tests were preceded by six practice trials. The same subjects were tested three times on different days; on each day, they received cTBS over the right intraparietal cortex (IPC), right inferior frontal gyrus (IFG), or vertex (control).
TMS pulses were delivered at 40% of stimulator output using a Magstim Super Rapid Stimulator (Magstim) via a 70 mm figure-of-eight coil. TBS consisted of 50 Hz stimulation in bursts every 200 ms and was delivered continuously for 40 s (Huang et al., 2005).
The right IPC [MNI coordinates: (50, −32, 50)] and the right IFG (48, 6, 26) were selected as the stimulation sites (see Fig. 4B) based on the preceding fMRI study (experiment 1) in which these regions were jointly activated in response to both time and numerosity processing (see Fig. 3A,B; Tables 3, 4). The vertex was used as a control site. The right IPC and the right IFG were localized for each subject by converting the coordinates in the standard MNI space into the position in the subject's structural MR image. A linear (affine) transformation matrix was computed by transforming subject's structural MRI scans to a standard template using FSL (Functional MRI of the Brain Software Library) software [FLIRT (FMRIB's Linear Image Registration Tool)]. Subsequently, the inverse of the transformation was used to obtain the positions of the right IPC and right IFG in a subject's MRI space. The target position in the subject's structural MRI scans was used to guide TMS coil position using the Brainsight system (Rogue Research). The handle of the TMS coil pointed posteriorly. The position of the vertex was localized as the position of Cz in the international 10/20 system.
Data analyses.
From all the collected data, trials in which response latencies were <100 ms were excluded from the analysis because these were likely to be outliers driven by motor errors or lack of attention attributable to fatigue. By applying this criteria for each subject, an average of 2.0% (range of 0.0–13.5%) of data were excluded from the analysis. Changes of the overall accuracy and size of the congruency effect (the difference in accuracy between the congruent and incongruent conditions) between pre- and post-TMS were individually computed for each stimulation site. These values were used as the input data in the statistical analysis. One-sample two-tailed t test was used to test statistical significance.
Experiment 3: numerosity–time interaction in duration reproduction task (TMS)
Subjects.
In total, 58 healthy volunteers participated in two pilot behavioral tests, one with digits and one with dot arrays, and in a TMS experiment. Data of two volunteers who participated in the pilot behavioral test with digits were excluded from the analysis because they showed high error rates (>20%) (57.8 and 21.0%). After the initial data analysis, data of one volunteer was further excluded because a partial correlation coefficient between stimulus duration and reproduced duration showed a negative value, which raised a possibility that this participant did not follow our instruction. Therefore, the data of 28 volunteers (11 males and 17 females, 18–35 years old) for the pilot behavioral experiment with digits, of 16 volunteers (six males and 10 females, 19–32 years old) for the pilot behavioral experiment with dot arrays, and of 14 volunteers (six males and eight females, 19–36 years old) for the TMS were used. Three of the volunteers participated in both pilot behavioral (digits) and TMS studies. All participants gave written consent. All experiments were approved by the University College London ethics committee.
Task and stimuli.
The task was to reproduce the duration of the stimulus (digit or dot arrays) by holding down the space bar after the stimulus disappeared (see Fig. 6A). No feedback was given to the participants.
The visual stimuli were seven-segment font digits or dot arrays. The digits were white and the dot arrays were black on a large gray disk of ∼6°, presented against black background. The numerical magnitude of digits (2, 5, 6, or 9) or number of dots (1, 4, 7, or 10) and the stimulus duration (500, 600, 700, 800, and 900 ms) were manipulated (see Fig. 6B). The dot arrays were generated by an automated program (Piazza et al., 2004). The shape of the dots was a circle, and the size was the same across the stimuli. Spatial positions of the dots were randomized. Each trial was initiated by the participant's key press of the space bar, and the next stimulus appeared 2 s later. All combinations of the numerosity and duration were used with equal probabilities. The experimental setup was the same as in experiment 2.
First, we tested whether the numerosity information influences time estimation at a perceptual level using a duration reproduction task, without TMS. Each experiment consisted of two sessions, the right-hand and left-hand sessions, and each session consisted of two blocks. Each session was preceded by 30 practice trials. The order of the hands was counterbalanced between participants. In each session, subjects completed 200 trials; in total, 400 trials per subject were analyzed.
TMS experimental procedure.
The TMS was conducted applying the visual dot arrays used for the behavioral pilot experiment described above. Subjects performed 300 trials of a reproduction task before and after the offline cTBS. Pre- and post-TMS tests were preceded by 20 practice trials. The same subjects were tested three times on different days; on each day, they received cTBS over the right IPC, right IFG, or vertex (control). The parameters of TMS, coordinates of the target regions (see Fig. 4B), and methods for coil navigations were the same as in experiment 2.
Data analyses.
From all the collected data, trials in which the response latencies were shorter than 100 ms or the reproduced durations were longer or shorter than ±2 SDs from the averaged reproduced duration for each stimulus duration condition (500, 600, 700, 800, and 900 ms) were excluded from the analysis, because these were likely to be outliers driven by motor errors or lack of attention attributable to fatigue. By applying these criteria for each individual, an average of 6.7% (range of 3.0–13.8%) and 7.4% (range of 3.3–15.5%) of all data in the behavioral pilot experiments with digits and dot arrays were excluded from the analysis. In the TMS experiment, an average of 4.7% (range of 2.7–8.0%) of all data were excluded.
The goal of the reproduction tasks was to determine to what extent the perceived passage of time was influenced by numerical information. However, because we intermixed different stimulus durations, we needed to partial out the contribution of the physical stimulus duration and isolate the contribution of numerical information to the reproduced duration. For this purpose, we constructed a two-factor model of reproduced duration in which factors of stimulus duration and numerical magnitude contributed to the reproduced duration independently. More specifically, we computed nonparametric partial correlations between the reproduced duration and stimulus duration and between the reproduced duration and numerical magnitude. In this analysis, the three variables (i.e., the stimulus duration, numerical magnitude, and reproduced duration) were converted to rank data. Nonparametric tests were used because they allowed us to capture monotonic increase (or decrease) even if the relationship was nonlinear. Moreover, nonparametric tests are generally more robust across various distributions than parametric tests. We therefore used nonparametric partial correlations to estimate independent effects of stimulus duration and numerical magnitude by treating variables of non-interest (stimulus duration for calculating the partial correlation between numerical magnitude and reproduced duration, and numerical magnitude for calculating the partial correlation between stimulus duration and reproduced duration) as covariates. A significant effect of stimulus duration would indicate that the observers adjusted their response duration according to the physical duration of the stimulus. More importantly for our study, a significant effect of numerosity would indicate that reproduced durations were systematically influenced by numerical magnitude.
Changes of the correlation coefficients between pre- and post-TMS were individually computed for each stimulation site. These values were used as the input data in the statistical analysis. One-sample two-tailed t test was used to test statistical significance.
Experiment 4: functional role of IPC in numerical processing (TMS)
Subjects.
Sixteen healthy volunteers (seven males and nine females, 20–30 years old) completed a TMS experiment. All participants gave written consent. This experiment was approved by the University College London ethics committee.
Task and stimuli.
The task was to judge which one of two visual dot arrays contained a greater number of dots. No feedback was given to the participants.
Two dot arrays were simultaneously presented to the left and right visual fields (see Fig. 8). Each consisted of black dots shown on a large gray disk of ∼4.5 ° presented against black background. We used eight pairs of numerical magnitudes: 2–4, 3–5, 6–8, 7–9, 2–6, 3–7, 4–8, and 5–9. The dot arrays were generated by an automated program (Piazza et al., 2004). The shape of the dots was a circle, and the size of the dots was the same for all the dots within an array. The dot size in one of two simultaneously presented dot arrays was four times as large as that in the other. This manipulation was included to prevent the total area of dots in an array from providing a cue for the correct answer. The spatial positions of dots in the dot arrays were randomized.
Subjects were asked to indicate which one of the two dot arrays contained a greater number of dots, as quickly and accurately as possible, by pressing with their right index finger key J or with their middle finger key K. Key J was assigned to the dot array presented on the left, and key K was assigned to the one presented in the right visual field. Subjects were instructed to fixate at the center of the screen throughout the session. They were asked to ignore the task-irrelevant dimension, such as size of dots or spatial positions of dots. The dot arrays were visible until the subject submitted a response. The next dot arrays were presented 1.5 s after the response period of 2 s. The experimental setup was the same as in experiments 2 and 3.
Procedure of TMS experiment.
Subjects performed 128 trials of the numerosity discrimination task before and after the offline cTBS. The pre- and post-TMS tests were preceded by six practice trials. The same subjects were tested twice on different days; on each day, they received cTBS over either the right IPC or vertex (control). The parameters of TMS, the coordinates of the target regions, and the methods for coil navigations were the same as in experiment 2.
Data analyses.
The same criteria were used for excluding outliers as in experiment 2. By applying these criteria for each individual, an average of 0.3% (range of 0.0–7.0%) of the data were excluded from the analysis. We computed an efficiency score [accuracy divided by reaction time (seconds)], which has been commonly used in several previous studies (Machizawa and Driver, 2011) to take into account both accuracy and reaction time that may show the speed–accuracy tradeoff. Changes of the efficiency score between pre- and post-TMS were individually computed for each stimulation site. These values were used as the input data in the statistical analysis. One-sample one-tailed t test was used to test statistical significance.
Results
Experiment 1: numerosity–time interaction in duration discrimination task (fMRI)
In the fMRI experiment, we determined the neural basis of numerical and temporal processing and the congruency effect between these stimulus dimensions using experimental stimuli similar to those used by Xuan et al. (2007). Subjects were asked to judge “which stimulus lasted longer” in the duration task and “which stimulus contained a greater number of dots” in the numerosity task (Fig. 1A). In the congruent trials, the duration and the numerosity changed in the same direction between the two dimensions (duration–numerosity; increase–increase or decrease–decrease), whereas in the incongruent trials, they changed in the opposite directions (increase–decrease or decrease–increase) (Fig. 1B). We also manipulated the degree of change in the task-irrelevant dimension. The differences in the behavioral performance and brain activity were compared between the congruent and incongruent conditions.
Stimulus sequence and experimental conditions of the fMRI experiment (experiment 1). A, The stimulus sequence. Each trial followed an instruction cue, which indicated the task that subjects had to perform to successive presentations of a pair of dot array stimuli. Subjects were asked to respond by pressing a button, before the red fixation returned to white. Numerical magnitude of the dot arrays (Num A and Num B) and duration of stimulus presentations (Dur A and Dur B) were varied. B, Examples of congruent and incongruent conditions. A condition that duration and numerosity changed in the same direction was defined as congruent and in the opposite direction was defined as incongruent condition.
Behavioral results
Behavioral performance measured during the fMRI scanning confirmed the influence of numerical magnitude on temporal processing (Fig. 2). The analysis of accuracy with 2 (task) × 2 (congruency) × 2 (distance) repeated-measures ANOVA showed significantly better performance in the numerosity than duration discrimination task (main effect of task, F(1,25) = 151.615, p < 0.001, η2 = 0.527) and in the congruent than incongruent condition (main effect of congruency, F(1,25) = 5.662, p < 0.05, η2 = 0.012). The congruency effect was larger in the duration task than in the numerosity task (task × congruency, F(1,25) = 25.015, p < 0.001, η2 = 0.047). The main effect of distance and other interactions were not significant (p > 0.05) (main effect of distance, η2 = 0.003; task × distance, η2 = 0.003; congruency × distance, η2 = 0.002; and task × congruency × distance, η2 < 0.001).
Behavioral performances in the fMRI scanner. Bars represent mean accuracy (percentage) for each condition. “Duration” and “Numerosity” represent duration and numerosity tasks, and “Cong” and “Incong” represent congruent and incongruent conditions. Error bars denote SEM. ***p < 0.001.
The post hoc analysis showed that the congruency effect was significant in the duration task (F(1,50) = 26.478, p < 0.001, r = 0.59), replicating the findings of the previous study (Xuan et al., 2007). This result indicates that duration tended to be judged to be longer when the concurrently presented numerosity was larger. However, a significant congruency effect was not observed for the numerosity task (F(1,50) = 2.743, p > 0.05, r = 0.23), likely because of the very high overall accuracy for this task (>90% on average).
fMRI results
The fMRI data showed that the duration and numerosity discrimination tasks involved widespread regions across the brain with much overlap between the two tasks. The duration task and the numerosity task activated a network of brain regions classically associated with temporal (Table 1) and numerical (Table 2) processing. To find common regions activated by the two tasks, we performed a conjunction analysis. Joint activations were predominantly found in the right hemisphere, including bilateral occipital regions, a right IFG cluster extending into the rolandic operculum, insula, and putamen, and an IPC cluster extending to the central gyrus (Fig. 3A; for full details, see Table 3).
Activated clusters in main effect of duration task (p < 0.001, uncorrected at voxel level; p < 0.05, familywise error-corrected at cluster level)
Activated clusters in main effect of numerosity task (p < 0.001, uncorrected at voxel level; p < 0.05, familywise error-corrected at cluster level)
Joint activations for duration and numerosity processing and the congruence effect. A, Joint activations. Both duration and numerosity tasks activated fronto-parieto-occipital regions predominantly in the right hemisphere (Tables 1⇑–3). The result was illustrated at a statistical threshold (p < 0.001, uncorrected at voxel level). B, Congruency effect in the right IFG. Within the areas that showed joint activation between duration and numerosity tasks, only the right IFG was highlighted by the “congruent–incongruent” contrast (Table 4). The result was illustrated at a statistical threshold (p < 0.001, uncorrected at voxel level). C, A plot of mean effect size across activated voxels in the right IFG. “Duration” and “Numerosity” represent duration and numerosity tasks, and “Cong” and “Incong” represent congruent and incongruent conditions. Error bars denote SEM. *p < 0.05 and **p < 0.01.
Activated clusters in “Duration AND Numerosity” contrast (p < 0.001, uncorrected at voxel level)
Within these regions, only the activation level of the right IFG was sensitive to the congruency effect (Fig. 3B; Table 4). The activation of this region was higher in the congruent than incongruent condition. No brain regions were highlighted by the opposite (incongruent > congruent) and interaction (task × congruency) contrasts. We further analyzed the activation level of the right IFG for different experimental subconditions by extracting the average effect size from the voxels in the right IFG (Fig. 3C). To do so, we examined the activation level using a three-way repeated-measures ANOVA with factors of task (duration/numerosity), congruency (congruent/incongruent), and distance (close/far). All three factors had a significant effect on the activation level. As expected from the initial analysis, we found a significant main effect of congruency (F(1,25) = 22.251, p < 0.001, η2 = 0.039). The activation level of the right IFG was higher in the congruent than incongruent condition. Also, we found an overall difference in the activation level depending on the task (main effect of task, F(1,25) = 7.018, p < 0.05, η2 = 0.154). The duration task resulted in greater activation than the numerosity task. The activation level of the right IFG also depended on the degree of change in the task-irrelevant dimension (main effect of distance, F(1,25) = 11.857, p < 0.01, η2 = 0.013). It was greater in the conditions with a large change (the far condition) than in the conditions with a small change (the close condition).
Activated clusters in “Congruent − Incongruent” contrast masked by the “Duration AND Numerosity” contrast (p < 0.001, uncorrected at voxel level)
Moreover, the analysis on the interaction between congruency and distance showed that the congruency effect was modulated by the degree of change in the task-irrelevant dimension (F(1,25) = 5.210, p < 0.05, η2 = 0.007). Other interactions did not reach statistical significance (p > 0.05) (task × congruency, η2 = 0.005; task × distance, η2 = 0.002; task × congruency × distance, η2 < 0.001). Finally, separate two-way repeated-measures ANOVAs for each task showed that the activation in the right IFG was sensitive to congruency regardless of the task performed (main effect of congruency for the duration task, F(1,25) = 6.451, p < 0.05, η2 = 0.071; and for the numerosity task, F(1,25) = 15.182, p = 0.001, η2 = 0.199). These results together indicate a close relationship between the activation level of the right IFG and the congruency effect.
Experiment 2: numerosity–time interaction in duration discrimination task (TMS)
Having established the brain regions that are activated both for time and numerosity processing in experiment 1, we next investigated causal involvement of these brain regions in the numerosity–time interaction using TMS. We targeted the right IFG and right IPC (Fig. 4B). The right IFG was selected based on our fMRI finding that the activation in this region was sensitive to the congruency effect in both numerosity and time tasks. Although the right IPC did not show differential activation levels between congruent and incongruent conditions, it was chosen based on the current literature suggesting a strong association of the IPC as a locus of numerosity–time interaction (Bueti and Walsh, 2009).
Stimulus sequence of a duration discrimination task and stimulation sites of TMS in experiment 2. A, The stimulus sequence for the duration discrimination task. Subjects compared durations of sequentially presented two dot arrays and judged which one of two dot arrays (Num A or Num B) presented for a longer time (Dur A or Dur B). B, The stimulation sites of TMS. The cTBS was applied over the right IPC (left figure) and the right IFG (right figure). Figures represent coronal sections of these regions (right IPC, y = −32; right IFG, y = 6). Red blobs on the left figure correspond to the jointly activated regions by the duration and numerosity tasks (see also Fig. 3A; Table 3) and that on the right figure correspond to the region that exhibited the congruency effect in experiment 1 (see also Fig. 3B; Table 4). These brain regions were stimulated also in experiment 3. The IPC was stimulated in experiment 4.
TMS results
The causal involvement of the brain regions were assessed by comparing the overall accuracy and size of the congruency effects (difference in accuracy between congruent and incongruent conditions) in the numerosity–time interaction with the duration discrimination task before and after the offline cTBS.
First, we confirmed that there was a congruency effect in all six sessions [2 (pre- and post-TMS) × 3 stimulation sites (IPC, IFG, and vertex)]; participants showed higher accuracy in the congruent than incongruent condition (mean accuracy for the congruent condition was 72.8% and for the incongruent condition was 55.4%). The overall accuracy dropped when cTBS was applied over the right IFG (t(9) = 2.767, p < 0.05, Cohen's d = −0.875) (Fig. 5A), whereas cTBS over the right IPC and the control site did not show any effect on the overall accuracy (p > 0.05) (Cohen's d = −0.316 and −0.221, respectively). Conversely, the size of the congruency effect was modulated by cTBS over the right IPC (t(9) = 2.262, p = 0.05, Cohen's d = 0.715) (Fig. 5B). We found cTBS effects neither over the IFG nor over the control site on the size of the congruency effect (p > 0.05, Cohen's d = 0.122 and −0.040, respectively). Different patterns of the effect of cTBS over the IPC and IFG indicate that these brain regions may play different roles in the numerosity–time interaction.
Results of the TMS experiment with a duration discrimination task (Experiment 2). A, Differences in overall accuracy between pre- and post-TMS for each stimulus locations. Bars represent the mean values of the overall accuracy ([Post − Pre]) TMS. B, Differences in size of congruency effect for each stimulus location. Bars represent mean values of the differences in size of congruency effect between pre- and post-TMS. Error bars denote SEM. *p < 0.05.
Experiment 3: numerosity–time interaction in duration reproduction task (TMS)
Reproduction task without TMS
Experiment 2 showed that the cTBS over the right IPC and right IFG exhibited different patterns of impairments of the task performance, suggesting that the right IPC and IFG may play different roles in the numerosity–time interaction. However, on the basis of these results, one cannot determine the specific roles of these regions in the interaction. Specifically, it remains unclear whether these effects reflect simple response errors (or facilitation) because of conflicting (or congruent) more-versus-less categorical decision inherent in comparative tasks (Ivry and Schlerf, 2008) or distortion of subjective passage of time attributable to task-irrelevant magnitude information. To avoid ambiguity as to the stage at which numerical information interfered with the duration information, we first sought to establish that numerical information influences duration estimation in a time reproduction task. The time reproduction task was used to minimize the involvement of categorical magnitude judgments. Participants were asked to estimate the duration of visual stimuli with numerical information (digits and dot arrays) and to reproduce the duration by holding down a spacebar (Fig. 6A,B). We aimed to evaluate the degree of the contribution of the stimulus duration and numerical magnitude to reproduced duration independently and the influence of TMS on these factors. For this purpose, we used a partial correlation analysis to evaluate the effects of stimulus duration and numerosity as independent factors that contributed to the reproduced duration, which was the only dependent variable in this experiment.
Stimulus sequences of a time reproduction task used in experiment 3. A, The stimulus sequences of numerosity–time interaction with a time reproduction task. A digit (left) or a dot array (right) was presented for 500, 600, 700, 800, or 900 ms, and subjects reproduced the duration after the stimulus has disappeared. B, Digits and dot arrays used in the experiments. Four levels of numerical magnitudes in the form of digits (2, 5, 6, or 9 with 7-segment font) or dot arrays (1, 4, 7, or 10 dots) were presented. The TMS experiment was performed using the dot stimuli.
The results showed highly significant effects of the stimulus duration both when the numerosity information was presented by digits (mean ± SD, 0.517 ± 0.210, t(27) = 13.012, p < 0.001, Cohen's d = 2.459) or by dot arrays (mean ± SD, 0.464 ± 0.256, t(15) = 7.241, p < 0.001, Cohen's d = 1.810), indicating that the participants adjusted the response durations according to changes in the actual stimulation duration. Critically, an increase in numerical magnitude presented both in the form of digits (mean ± SD, 0.031 ± 0.049, t(27) = 3.415, p < 0.01, Cohen's d = 0.645) and dot arrays (mean ± SD, 0.099 ± 0.104, t(15) = 3.810, p < 0.01, Cohen's d = 0.952) increased the reproduced duration. The finding that individual differences in the partial correlation coefficient of the effect of stimulus duration and of the effect of numerosity were not correlated when the numerosity information was presented by digits (Spearman's correlation, r = 0.169, p > 0.05) or by dot arrays (r = 0.006, p > 0.05) confirmed that the effect of duration and numerosity were independent. In other words, these two factors had additive effects.
To investigate whether the degree of the effect of numerosity and the effect of stimulus duration were different across different levels of stimulus duration and numerical magnitude, we computed the partial correlations at different levels of each independent variable. A one-way repeated-measures ANOVA on the effect of stimulus duration across different numerical magnitudes showed that the effect was not different across numerical values when the numerosity information was presented by digits (F(3,81) = 1.959, p > 0.05, η2 = 0.068) or by dot arrays (F(3,45) = 2.479, p > 0.05, η2 = 0.143). The effect of numerosity had also a similar impact across different stimulus durations when the numerosity information was presented by digits (F(4,108) = 0.329, p > 0.05, η2 = 0.012) or by dot arrays (F(4,60) = 1.945, p > 0.05, η2 = 0.115).
These results together indicate that numerosity information interacts with temporal processing even when categorical decisions are not required and support the notion that subjective passage of time is influenced by numerical magnitude at a perceptual level in the duration reproduction task.
TMS results
Having established the numerosity–time interaction in the reproduction task, we next investigated the neural locus in which numerosity information influences time estimation using TMS. The causal involvement of the right IFG and right IPC was assessed by comparing the effects of numerosity and stimulus duration on reproduced duration before and after offline cTBS.
The effect of numerosity on reproduced duration was enhanced by disruption of the right IPC (t(13) = 4.681, p < 0.001, Cohen's d = 1.251) (Fig. 7, top). Moreover, the disruption of the right IPC also impaired reliable reproduction of duration, which was observed as a reduced effect of stimulus duration on reproduced duration (t(13) = 2.865, p < 0.05, Cohen's d = −0.766) (Fig. 7, bottom). Conversely, disruption of the right IFG did not have any influences on the effect of stimulus duration (p > 0.05, Cohen's d = −0.457) or numerosity (p > 0.05, Cohen's d = 0.262). Neither did we find any effect of cTBS over the control site (p > 0.05, Cohen's d = −0.084 for the effect of stimulus duration and Cohen's d = 0.506 for the effect of numerosity).
Results of the TMS experiment with a time reproduction task (experiment 3). Bars represent mean values of partial correlation coefficients ([Post − Pre]) TMS for the effect of numerosity (top) and stimulus duration (bottom), when the right IPC (left), right IFG (middle), or vertex (right) was stimulated. Error bars denote SEM. *p < 0.05 and ***p < 0.001.
Experiment 4: functional role of IPC in numerical processing (TMS)
In experiment 3, we showed that cTBS over the right IPC paradoxically resulted in enhancement of the effect of numerosity on perceived duration (Fig. 7, top). This result suggests that disruption of the right IPC might lead to facilitation of automatic numerical processing. Alternatively, disruption of the right IPC might have increased the interference from task-irrelevant dimension by diminishing the ability to focus on the task-relevant dimension. The effect of the right IPC disruption on numerosity processing was assessed by comparing the efficiency score in a numerical comparison task before and after offline cTBS (Fig. 8).
Stimulus sequence of a numerosity comparison task used in experiment 4. Two dot arrays were simultaneously presented on the right and left visual fields. Subjects were asked to respond which one of two dot arrays contained larger number of dots.
TMS results
The efficiency score increased by disruption of the right IPC [mean ± SD of (post − pre) efficiency score, 4.253 ± 9.409, t(15) = 1.808, p < 0.05, Cohen's d = 0.452]. We did not find influences of cTBS over the control site [mean ± SD of (post − pre) efficiency score, −0.472 ± 3.856, t(15) = −0.489, p > 0.05, Cohen's d = −0.122]. These results indicate that the cTBS over the right IPC facilitated numerical processing, which is in line with our interpretation that the enhanced numerosity effect on reproduced duration in experiment 3 is attributable to greater automatic numerical processing after the cTBS over the right IPC.
Discussion
In the present study, we investigated the neural substrates that mediate the interaction between numerosity and time. We first performed an fMRI experiment and showed that the right IFG, right IPC, and bilateral occipital regions exhibit joint activation for numerical and duration processing in comparison tasks. The right IFG activity was modulated by congruency between these two dimensions of magnitude. Then, we conducted three TMS experiments to investigate the causality and specific roles of these regions in the numerosity–time interaction using duration discrimination and reproduction tasks. The results showed that disruption of the right IFG impaired duration discrimination, whereas disruption of the right IPC modulated the degree of the influence of numerosity on time perception and also impaired time estimation accuracy. Together, these results suggest that, although the right IFG plays a role in the interaction at a decision stage, mixing of magnitude information at the perceptual level takes place in the right IPC.
Congruency effect in the right IFG
Our results suggest that activity of the right IFG reflects congruency effect at a decision stage, because only this region, among all the regions commonly activated by time and numerosity tasks, showed a neuronal congruency effect (experiment 1) and because TMS over the right IFG influenced the task performance only in the duration comparison task (experiments 2 and 3). The sensitivity of the right IFG to more-versus-less categorical information receives additional support from previous neuroimaging studies demonstrating representations of categorical information in regions similar to our right IFG cluster (Jiang et al., 2007; Cromer et al., 2010). Furthermore, a recent study in nonhuman primates revealed that the prefrontal cortex (PFC) shows category-based activity in a duration discrimination task (Genovesio et al., 2009). The authors found that the neurons in the PFC showed differential activity predominantly based on stimulus order (i.e., whether the first or second stimulus had lasted longer) rather than the degree of difference in stimulus duration. Importantly, neurons in the same area had similar properties for relative spatial distance, suggesting that neurons in the PFC are involved in the comparison of both time and space (Genovesio et al., 2011). The sensitivity of the right IFG to congruency observed in our fMRI experiment may reflect joint coding of numerosity and time for the categorical decision rather than changes in perceived duration.
The activation of the right IFG was higher in the congruent than incongruent condition. This could be explained by assuming that the response of the IFG was enhanced when the task-irrelevant magnitude dimension matched the categorical decision to be made regarding the main task. In a previous study, Roy et al. (2010) showed that neurons in the PFC represent competing categories. This kind of neuronal activity was modulated not only by task-relevant but also by competing task-irrelevant categorical information (Roy et al., 2010). Therefore, the higher activation in the congruent condition found in the present study may have resulted from facilitation of the categorical representation of time (i.e., “short” versus “long”) by the task-irrelevant numerical category (“few” versus “many”). This view is in line with the idea proposed by Ivry and Schlerf (2008) that the mixing of different magnitude dimensions is produced by incidental activation of the temporal concept “short” and “long” by the numerical concepts “few” and “many.”
One might think that the congruency effect found in the right IFG reflects response inhibition that could be recruited resulting from the need for suppressing task-irrelevant information. However, this interpretation is unlikely because previous studies showed that the right IFG is activated more strongly under a conflicting than nonconflicting condition (Kaufmann et al., 2005), a finding that is opposite to our fMRI results. This discrepancy arises presumably from the fact that response inhibition involves a more anterior part of the right IFG [MNI coordinates: (46, 26, 12)] (Jahfari et al., 2011) than the region found in our experiment [(54, 12, 28); Table 4]. Therefore, the activation of the right IFG observed in our study is more compatible with the interpretation based on the congruency effect.
Another potential possibility is that the IFG activation could reflect differences in working memory load between congruent and incongruent conditions. However, this alternative is an unlikely one because, in our design, we equated all stimulus parameters (two dot arrays and two stimulus durations) between the congruent and incongruent conditions. The only factor that differed between these conditions was the congruency of increase or decrease in numerosity and time.
TMS effect on time estimation
Our TMS experiment on a time reproduction task (experiment 3) showed that cTBS over the right IPC impaired the estimation of time. This is consistent with previous TMS and lesion studies showing that the IPC is crucial for temporal processing (Harrington et al., 1998; Bueti et al., 2008).
Many neuroimaging studies of temporal processing reported activations not only in the IPC but also in the PFC (Rao et al., 2001; Basso et al., 2003; Pouthas et al., 2005). Our TMS study showed that the disruption of the right IFG impaired task performance in duration discrimination (experiment 2), but in contrast, it had no effect on time estimation in the duration reproduction task (experiment 3). One possible reason for this difference is that the role of the PFC is more relevant in the discrimination of relative duration than in the encoding of a single duration. This is in line with a previous report showing that the right PFC activation emerged during the comparison of two stimulus intervals but not during the encoding of a single duration (Rao et al., 2001).
Enhancement of numerosity effect on time estimation
In the present study, one might expect that cTBS over the right IPC would disrupt numerical processing and thus reduce the size of the congruency effect in duration discrimination (experiment 2) and the effect of numerosity on time estimation (experiment 3). However, our results showed that the size of the congruency effect was enhanced (Fig. 5B) and the subjects reproduced longer durations more systematically with increasing numerical magnitude (Fig. 7, top). Although this finding may appear to conflict with the notion that the IPC processes numerical information, a number of recent studies on interhemispheric interactions in the IPC offer a possible interpretation of our findings. Namely, cTBS over the right IPC may release the left IPC from strong interhemispheric inhibition and thereby facilitate the numerosity processing in the left IPC and its interference with temporal processing. Several TMS studies showed significant or similar trends of facilitation in numerical processing after TMS over the right IPC (Andres et al., 2005; Cappelletti et al., 2007; Dormal et al., 2008). Our complementary TMS experiment (experiment 4) also supports this view.
Previous studies interpreted the facilitation of numerical processing by TMS as evidence that numerosity processing is regulated by inhibitory control from homologous areas in the contralateral hemisphere (Cappelletti et al., 2007). A recent trifocal TMS study showed that the right, but not left, human IPC exerts a strong inhibition over the contralateral homologous area, which is mediated by direct transcallosal projections (Koch et al., 2011). The functional consequences of the interhemispheric inhibition were demonstrated by showing that TMS over the right IPC caused overactivation of the left IPC (Heinen et al., 2011). Finally, it should be noted that, although previous TMS studies reported that disruption of the left IPC impairs numerical processing (Cappelletti et al., 2007; Dormal et al., 2008), disruption of bilateral IPC was necessary to completely impair numerical processing (Andres et al., 2005). These studies suggest that bilateral representation of numerosity may play a complementary role for representing numerical magnitude. Together, these studies support the idea that disruption of the right IPC results in facilitation of automatic numerical processing that results in an enhancement of the numerosity effect on temporal processing.
Future direction
Our study showed common neural representations of numerosity and time in two different brain regions. However, the mechanisms of the interaction at the neuronal level remain to be clarified in future studies. There are two potential mechanisms of interaction. One possibility is that the interaction occurs within a population of neurons that encode both numerosity and time information. An alternative possibility is that numerosity and time are represented by separate population of neurons, but they interact with each other via their tight connections. Future studies applying the single-unit recording technique could provide insight to this issue.
Conclusions
Our current study demonstrates that the right IFG is involved in the congruency effect between numerosity and time during the performance of a categorical decision task; however, this region was not causally relevant for the interaction in a time reproduction task. Instead, the right IPC was the locus of the interaction during time estimation, in line with the ATOM. Together, our results suggest that the two different brain regions share representations of numerosity and time at two different levels: the right PFC mediates categorical, decision-based representation, whereas the right IPC represents their lower perceptual properties. These two levels of shared representation mediate the interaction between numerosity and time.
Footnotes
This study was partly supported by the Excellent Young Researchers Overseas Visit Program (M.J.H.), Research Fellowship for Young Scientist (M.J.H.), Grant-in-Aid for Scientific Research (C) Grant 20500361 (H.C.T.) and (S) Grant 21220005 (N.S.), Scientific Research on Innovative Areas Grant 22101007 (H.C.T., N.S.), Challenging Exploratory Research Grant 23650224 (H.C.T.) from the Japan Society for the Promotion of Science (JSPS), Researcher Exchange Program between the Academy of Finland (AF) and JSPS (M.J.H.), Brain Research at Aalto University and University of Helsinki postdoctoral program (M.J.H.), AF (National Centers of Excellence Program 2006-2011) (S.C.), the aivoAALTO project of the Aalto University (S.C.), United Kingdom Medical Research Council (V.W.), and the Royal Society (V.W.).
- Correspondence should be addressed to Dr. Masamichi J. Hayashi, Neuroscience Unit, Institute of Biomedicine, Physiology, P.O. Box 63, University of Helsinki, Helsinki 00014, Finland. mjhgml{at}gmail.com
This article is freely available online through the J Neurosci Open Choice option.
