In the human brain, a well known frontoparietal circuit, including lateral prefrontal cortex (LPFC), presupplementary motor area/anterior cingulate cortex (pre-SMA/ACC), and both the superior and inferior parietal cortex, is involved in cognitive control. One proposal is that the frontoparietal cortex holds a flexible description of attended or task-relevant information, biasing processing in favor of this information in many different parts of the brain. Here, we separate frontoparietal coding of attended information from its active use in behavior. In two experiments, subjects watch a stream of visual stimuli in a fixed location. In the first experiment, there is no task to perform; in the second, decisions are orthogonal to the occurrence of new stimulus events. Even in these simple circumstances, we find that attended stimulus changes give extensive activation of LPFC, pre-SMA/ACC and parietal cortex, whereas unattended changes do not. Even without behavior to control, these classical “control” regions are active in simple update of attended information.
Many experiments show the importance of the frontoparietal cortex in cognitive control. In the frontal lobe, for example, the lateral prefrontal cortex (LPFC) and the anterior cingulate cortex (ACC) have been linked to control functions such as preparatory set and signaling cognitive conflict (MacDonald et al., 2000; Sakai and Passingham, 2003). Frontoparietal activity is strongly associated with shifts of attention between locations (Corbetta et al., 2000) or sensory modalities (Shomstein and Yantis, 2004). Similar frontoparietal activity is associated with many different kinds of tasks (Cabeza and Nyberg, 2000; Duncan and Owen, 2000), suggesting shared dependence on similar control mechanisms.
How is such control exerted? One proposal is that, in different contexts, the frontoparietal cortex holds a focused description of attended or task-relevant information. This frontoparietal representation acts as a source of bias in other brain systems, supporting related information processing (Desimone and Duncan, 1995; Dehaene et al., 1998; Miller and Cohen, 2001). In the behaving monkey, single-cell recording shows that many cells in the LPFC code task-relevant information (Sakagami and Niki, 1994; Freedman et al., 2001). Included is coding of relevant stimulus categorizations, rules, rewards, working-memory contents, etc. (Duncan, 2001; Miller and Cohen, 2001). A broad variety of task-relevant responses has also been described in the medial frontal and parietal cortex (Procyk et al., 2000; Caselli et al., 2004).
An interesting implication is that classical “control” regions might be activated by much simpler cognitive events than those typically used in functional magnetic resonance imaging (fMRI) studies. Even with no decisions to be made, a change to attended information should produce update in the frontoparietal representation. In the monkey, indeed, frontal and parietal cells may show sensory responses even in passive viewing of new stimuli (Colby and Goldberg, 1999; Nelissen et al., 2005). To test this hypothesis in the human brain, we studied responses to attended visual events with task-based decision-making reduced to a minimum.
In experiment 1, subjects directed attention to a fixed location in the visual field. They simply watched a series of words flashing on and off, with no decisions or responses required. Each word was repeated several times before unpredictably changing to a new word. Using fMRI, we studied responses to these periodic stimulus changes, comparing them to equivalent changes at unattended locations. Despite its appeal as the simplest possible task situation, passive viewing leaves open the possibility of uncontrolled cognitive activity (e.g., subvocal reading) associated with stimulus changes. In experiment 2, subjects again directed attention to one part of the visual field. This time, stimuli were novel shapes, and subjects watched for occasional targets (brief flickers). Although a specific task set was imposed, this was orthogonal to the events of interest (object changes), which themselves required no decisions. Again we compared brain responses to object changes at attended and unattended locations. Our results show that, even with no behavioral implications, a periodic change to attended information is sufficient to activate classical “control” regions of the human brain.
Materials and Methods
Experiment 1 involved 15 subjects (seven males and eight females; mean age, 23 years), with one additional subject excluded for a suspected failure to attend as instructed. In the scanner, the instruction was simply to watch events at a fixed spatial location, with no other requirement mentioned or implied. A display containing two four-letter words, one white and one black (see Fig. 1a), flashed repeatedly on and off (cycle 100 ms on, 200 ms off). To encourage central fixation, both words were centered on the screen, the black one printed horizontally and the white one vertically. Throughout the scanning run, subjects were not required to make any responses. Seven subjects were told simply to watch the black words flashing, whereas eight watched white words. To assist subjects in holding attention at the correct location, black and white words, along with a surrounding pattern of black and white Xs, always flashed on and off simultaneously, reducing the salience of new-word transients (Rensink et al., 1997). As viewed in the scanner, the whole display subtended 3.9° horizontally and vertically.
Although continuous from the point of view of the subject, for the purposes of the experimental design the sequence was divided into successive 1500 ms (5 cycle) “trials.” At the start of each trial, one of three things happened: either the black word changed to a new word, then continuing to flash on and off as before, the white word changed, or there were no changes. Word changes never occurred at any other point in the sequence. The whole experiment was conducted in a single scanning run, comprising 150 trials (50 changes to the black word, 50 changes to the white word, 50 with no change, in random order). Once a word had been changed it was not used again; thus, a hundred and two different words were used in total, 51 presented in black and 51 in white. Words allocated to different colors did not differ on concreteness or frequency (both F values < 1).
At the end of the scanning run, memory for attended and unattended words was assessed with a surprise recognition test. The test comprised 40 word pairs, each pair having one “old” word (20 shown previously in the attended color, 20 in the unattended color) and one “new” foil. Foils matched “old” words on concreteness and frequency (both F values < 1). For each pair, the subject indicated which word had been presented previously.
Experiment 2 involved 13 subjects (four males and nine females; mean age, 24 years), with three additional subjects excluded for poor target detection (see below). Subjects viewed a display comprised of complex nonsense shapes similar to those in Figure 2a. As viewed in the scanner, each shape subtended ∼1.1° of visual angle vertically and horizontally, with the display as a whole subtending 3.4° vertically and horizontally. The green shapes always occupied positions left and right of fixation, and the red shapes, above and below. The shapes in a given color were always mirror reflections of each other. The shapes were hand-drawn with a commercial graphics software package. There was no restriction regarding the shapes other than that the portion of the shape closest to the center of the display should be a straight line and that they should not resemble real objects or familiar patterns. This time, each trial consisted of a single 1000 ms on period followed by a 500 ms off period. All figures and colored lines flashed on and off simultaneously. On each trial, either the green shapes changed from those on the previous trial (60 occurrences), or the red shapes changed (60 occurrences), or there were no changes (60 occurrences), these events occurring in random order. Once a shape had been changed it was not reused, maintaining stimulus novelty throughout the sequence.
In this experiment, the task again was to maintain central fixation, watching either the green (five subjects) or the red (eight subjects) shapes for brief target events. The target event was a slight flickering of one of the two attended shapes. Subjects responded to each detected target with an immediate key-press made with the right hand. Flickering was created by interposing a 20 ms blank frame at one of three time points (255 ms, 490 ms, or 745 ms after the start of a trial) during the stimulus on period. Flickering occurred on one-third of all trials, equally divided between attended-change, unattended-change and no-change categories. Half of these flickers (one-sixth of all trials) occurred on an attended shape, and half on an unattended shape, but only attended flickers required a response. Although this experiment imposed a specific task set, this was orthogonal to the events of primary interest (object changes), which we examined only on no-flicker trials.
Image acquisition and analysis.
Subjects were scanned on a 3T Bruker (Ettlingen, Germany) Medspec scanner outfitted with a head coil. Functional volumes were acquired with an echoplanar imaging sequence (repetition time, 1102 ms; echo time, 30 ms; field of view, 25 × 25 cm; flip angle, 65.5). Each volume comprised 21 slices, aligned with the anterior commissure–posterior commissure line, covering the whole brain except for the extreme anterior portion of the inferior temporal lobe and the inferior aspect of the cerebellum. Slice thickness was 4 mm, interslice distance 1 mm, and in-plane resolution 3.91 × 3.91 mm.
fMRI data were processed and analyzed using SPM 99 (experiment 1) and SPM 2 (experiment 2) (Wellcome Department of Imaging Neuroscience, London, UK). Before analysis, all images were subjected to slice-time correction, with the first slice in each volume taken as a reference. Images were then corrected for motion artifacts, undistorted (Cusack et al., 2003), and normalized to MNI (Montreal Neurological Institute) space. Finally, the normalized images were spatially smoothed with an 8 mm full-width half-maximum Gaussian kernel.
For experiment 1, there were two levels of statistical analysis. First, contrasts of interest for each subject were computed using the general linear model, fitting the activity of each voxel with a combination of functions obtained by convolving the synthetic hemodynamic response function with the time series of events. For this purpose, each trial was considered a single event of 1500 ms duration. All three trial types (attended change, unattended change, no change) were explicitly modeled and entered into contrasts of attended change minus unattended change, attended change minus no change, and unattended change minus no change. Low-frequency noise was removed with a high-pass filter. Second, for each contrast, individual subjects' data were combined and subjected to a random effects analysis. All reported peaks passed a whole-brain false detection rate (Genovese et al., 2002) corrected threshold of p < 0.05.
To allow for a direct comparison between experiments, experiment 2 was analyzed with both a whole-brain approach and regions of interest (ROIs) based on experiment 1 data. Spherical ROIs (10 mm radius cortical, 5 mm radius subcortical) were defined around averaged left and right activation peaks from experiment 1 (see Table 1). If an area was only activated unilaterally in experiment 1, its coordinates were mirrored for the opposite hemisphere. The ROI analysis was performed with in-house software (http://marsbar.sourceforge.net/). Raw contrast values for a given contrast were extracted from each ROI of each subject and these were entered into a one-sample t test against zero. For the primary contrast of attended-change, unattended-change and no-change events, only trials with no flicker (either attended or unattended) were included. In a subsidiary analysis of flicker targets, attended flickers (whether detected or undetected) were compared with equivalent no-flicker controls.
For fMRI data, our main contrast compared brain responses to attended versus unattended word changes. Regions with significantly stronger responses to attended changes are shown in Figure 1b. For such a simple mental event, attention to a new visual input produced an extraordinarily broad pattern of brain activity. Especially in the left hemisphere, attended events produced stronger activity in large parts of the occipital and inferior temporal cortex. Such results agree with many previous demonstrations of attentional modulation in higher-level visual areas (Desimone and Duncan, 1995). These visual activations, however, were accompanied by highly significant activity in many other parts of the brain (Table 1). In the left hemisphere, there was strong activity in the superior temporal cortex, extending back to the temporoparietal junction (TPJ). More dorsally, there was conspicuous bilateral activity along the intraparietal sulcus (IPS). Bilateral activity was also seen in a large posterior area of the lateral frontal lobe, including parts of the posterior prefrontal, ventrolateral prefrontal (VLPFC), and premotor cortex; and on the medial frontal surface, including the presupplementary motor area/anterior cingulate cortex (pre-SMA/ACC). Not visible in the figure were accompanying subcortical activations, in the thalamus and basal ganglia (Table 1). There were no brain regions with significantly stronger responses to unattended than to attended changes.
Subsidiary contrasts separately compared attended and unattended word changes to no-change events. The contrast between attended change and no change gave results similar to those of attended change versus unattended change. Correspondingly, the comparison of unattended change with no change showed no significant differences in any part of the brain. Again, the results confirm the success of our attentional manipulation and show that in this experiment, unattended word changes produced little detectable brain activity.
The surprise recognition memory test, administered after scanning was complete, also confirmed the success of the attentional manipulation. In two-alternative forced-choice, the mean proportion correct was 0.70 for attended words, but only 0.55 for unattended words (contrast of attended and unattended scores, p < 0.005, t test).
For Experiment 2, the primary data analysis concerned response to new events (shape changes), measured on trials without flickers or responses. The whole-brain contrast of attended minus unattended shape changes (Fig. 2b) suggested a pattern of activity closely similar to that observed in experiment 1. For direct comparison, our primary analysis focused on ROIs defined from experiment 1 data (see Materials and Methods). The results are shown in Table 2. Significantly greater response for attended changes than for unattended changes was seen in 13 of 18 ROIs, with borderline effects in an additional three subcortical ROIs. The only ROIs with little suggestion of a response were the left and right premotor cortex. Except for these regions, perhaps active in experiment 1 because of implicit word naming, the results replicate the entire pattern of cortical and subcortical activity associated with attended stimulus changes. As before, there was no such activity in a contrast between unattended and no-change events.
Our frontoparietal activations associated with new attended events are closely similar to those involved in many kinds of decision-making and response (Duncan and Owen, 2000; Miller and Cohen, 2001). In a subsidiary analysis, we examined responses to flicker targets (detection rate, 0.82, with <1 false alarm/subject). Significant responses in 15/18 of the previously defined ROIs, with borderline effects in the remainder, show substantial overlap between frontoparietal activity in registering new events and detecting specific, task-defined targets (Table 2). In a whole-brain analysis for this contrast, a prominent additional feature was activity of the left sensorimotor cortex. On both sides of the brain, a strong, continuous region of activity linked thalamus, basal ganglia, and insula.
Our results show strong frontoparietal activity associated with attended stimulus changes. In line with single unit data, our results point to a selective representation or model of currently attended information, constructed in extensive regions of the frontal and parietal cortex (Dehaene et al., 1998). As proposed in biased competition theories (Desimone and Duncan, 1995; Miller and Cohen, 2001), this model may serve as an important source of cognitive control, biasing processing to task-relevant information in many parts of the brain. Even when control processes are minimized, however, with no task-based decision making, and attention held at a fixed stimulus location, periodic update of this model is associated with strong response in classical “control” regions of the LPFC, pre-SMA/ACC, and parietal cortex.
Model update will certainly include a variety of processes. As suggested by our memory data, attended events were consciously detected, and encoded to produce some lasting memory record. Indeed, our comparison of attended and unattended events resembles several recent fMRI studies in which attention (Beck et al., 2001; Marois et al., 2004), masking (Dehaene et al., 2001), and other manipulations have been used to contrast the brain's response to seen and unseen visual stimuli. Often, consciously seen stimuli produce extensive frontoparietal activity, whereas unseen stimuli do not. In these previous experiments, there is no separation of perception and task-based decision-making. Consciously seen stimuli require some response such as naming (Dehaene et al., 2001), attention movement (Beck et al., 2001), or target detection and identification (Beck et al., 2001; Marois et al., 2004). Obviously such responses are not required by unseen stimuli. In our experiments, even attended stimuli required no task-based decisions, separating update of attended information from behavioral control. Our contrast between attended and unattended events provides strong support for a link between conscious perception and frontoparietal models underlying behavioral control.
Of course, perceptual update is only one way to activate the frontoparietal system. As we have said, a closely similar pattern of brain activity is produced by many kinds of cognitive control operations, and by increased task difficulty in many different contexts (Cabeza and Nyberg, 2000; Duncan and Owen, 2000). As shown by experiment 2, one strong stimulus for frontoparietal activity is a task-relevant target. The result is reminiscent of a large behavioral literature showing strong attentional demands associated with target events (Ostry et al., 1976; Duncan, 1980). Plausibly, these attentional demands reflect strong frontoparietal coding of a sought-for stimulus with immediate implications for behavior (Posner et al., 1988). Again, supporting monkey data show strong LPFC responses to attended target stimuli (Everling et al., 2002).
In a study somewhat similar to ours, Downar et al. (2000) presented occasional stimulus events in visual, auditory, and tactile modalities. Although subjects had no task to perform, still each event was associated with substantial frontoparietal activity. In that study, however, frontoparietal activity could have reflected exogenous drawing of attention to the stimulated modality and/or location. Attentional movements are one of the core control functions usually ascribed to frontoparietal systems (Posner et al., 1984; Corbetta et al., 2000). To minimize this kind of attentional switch, we fixed attention through the whole experiment on a single stimulus location. Although switches within the attended region (e.g., attention movements between letters in experiment 1) cannot be completely discounted, we would regard this explanation as unlikely. It is well known, for example, that short words are read by parallel analysis across the whole letter string, not attentional movement between letters (New et al., 2006).
In our experiments, stimulus changes occurred in the context of more frequent stimulus repetitions. A related procedure has been used in many event-related potential studies, showing a strong late positivity (P3) generated by novel or task-relevant stimuli (Soltani and Knight, 2000). In a typical P3 procedure, subjects monitor a series of repeated stimuli for an occasional mismatching target. Strong P3 responses are produced both by targets, and by other novel stimuli that break the sequence (Soltani and Knight, 2000). Again, this context requires more active decision-making than ours. If a subject monitors for an occasional target that breaks a sequence of nontarget repetitions, it seems likely that an active “no go” decision will be required even for other novel stimuli that break the sequence but do not require a response. This said, it also seems likely that our fMRI results are closely related to the P3. Although the generators of the P3 are complex, they may include many of the regions identified in our data (Bledowski et al., 2004). The P3 has been related to attentional demand (Sergent et al., 2005), perceptual awareness (Sergent et al., 2005), and perceptual update (Donchin and Coles, 1988). Plausibly, our response to object change may reflect a similar neural process, in our case without the task-based decision making that typical P3 experiments require.
A related question concerns what aspect of stimulus change is most important in producing model update and frontoparietal response. In our experiments, a change to object identity was somewhat less frequent than a repeat. Is the object change itself, or its relative unpredictability, the more important part of the change event? As monkey data show, many different kinds of attended information can be represented in the frontal and parietal cortex. Plausibly, this representation can concern both stimulus identity and higher-order patterns of stimulus repetition and change.
In our experiments, no significant activity was associated with the contrast of unattended change minus no change. In other studies, fMRI shows some evidence of responses to unseen stimuli (e.g., masked words) (Dehaene et al., 2001). Typically, such responses are substantially weaker than responses to seen stimuli (Dehaene et al., 2001). Quite possibly, our experiment missed some weak response of this sort.
Certainly, the model of attended information built up in frontoparietal cortex is important in complex cognitive control, decision-making, and response selection. At the same time, our results show substantial activity associated with a much simpler mental event. Even with no task to carry out and no behavior to control, an attended stimulus change produces strong activity in classical frontoparietal “control” regions. This activity is produced not by complex decision-making, but by simple update of an attended visual representation.
This work was supported by the Medical Research Council. Scanning was performed at the University of Cambridge Wolfson Brain Imaging Centre.
- Correspondence should be addressed to either of the following: John Duncan, Medical Research Council Cognition and Brain Sciences Unit, 15 Chaucer Road, Cambridge CB2 2EF, UK, ; or Nicholas Hon, Department of Psychology, Faculty of Arts and Social Sciences, National University of Singapore, 11 Law Link, Singapore 117570,