Abstract
Posterior parietal cortex has been traditionally associated with perceptual attention and sensory-motor processing, but recent studies also indicate a potential role in episodic memory retrieval. Here, we developed a new paradigm to isolate top-down attention-related activity directed to either memory or perceptual information. We demonstrated a robust topographic separation in human posterior parietal cortex associated with searching for task-relevant information in episodic memory or in the environment. Control analyses confirmed that this difference was not dependent on differences in sensory stimulation or eye movements across tasks. Notably, we observed in memory- and perception-related regions a mechanism of reciprocal dynamic competition that was related to behavioral performance. These results provide the first evidence for a double dissociation between parietal networks involved in top-down attention to memory and the environment and support the idea of neural competition between perception and memory.
Introduction
The posterior parietal cortex is classically associated with visuospatial and sensorimotor functions, based on neurophysiological (Colby et al., 1996; Snyder et al., 1997) neuroimaging (Kastner and Ungerleider, 2001; Corbetta and Shulman, 2002) and lesion studies (Vallar and Perani, 1987; Mesulam, 1999). Nevertheless, recent neuroimaging studies (for review, see Wagner et al., 2005; Cabeza et al., 2008; Vilberg and Rugg, 2008) have reported the involvement of the left posterior parietal cortex during tasks requiring episodic memory retrieval, the conscious recollection of events from one's personal past (Tulving, 1985; Wheeler et al., 1997). Other studies have identified parietal regions involved in orienting visuospatial attention on the basis of short-term (Kuo et al., 2009) and long-term memory (Summerfield et al., 2006) representations. The function of parietal activity during episodic memory retrieval is controversial, particularly since parietal lesions, compared with lesions of the medial temporal lobe (Cohen and Eichenbaum, 1993), do not result in a severe loss of memory function (but see Berryhill et al., 2007).
Several recent papers have proposed that left parietal activations during memory retrieval partly reflect the overlap of parietal attentional functions for memory and perception (Wagner et al., 2005; Cabeza, 2008; Cabeza et al., 2008; Ciaramelli et al., 2008; Vilberg and Rugg, 2008). In contrast, theories that relate episodic memory retrieval to activation of the default network, which is generally deactivated by attention to the environment, suggest that parietal memory and perceptual functions are anatomically segregated (Buckner et al., 2008). Surprisingly, a comparison of attention and episodic memory domains has only been performed using across-experiment meta-analytic techniques (Hutchinson et al., 2009). Virtually no experimental data from the same group of subjects has been reported concerning the overlap between parietal regions involved in the voluntary allocation of attention to episodic memory or to visuospatial information.
Here, we compared search for information in the environment and in episodic memory by adapting procedures that have been widely used to separate the processing stages underlying visual search or monitoring (Shulman et al., 2003, 2007; Silver et al., 2007) and working memory (e.g., encoding, maintenance, and retrieval) (Courtney et al., 1997; D'Esposito et al., 1999; Todd and Marois, 2004). Specifically, we measured the BOLD signals that varied with the duration of perceptual and memory search, where each was manipulated over trials. We use the phrase memory search or memory retrieval as a proxy for a broad range of processes that involve the “retrieval of episodic information guided by the behavioral goal, the organization and evaluation of the retrieved information, and the accumulation of evidence that pointed to a particular response” (Moscovitch and Winocur, 1995; Badre and Wagner, 2007; Mecklinger, 2010). These memory search processes, which likely involved working memory as well as long-term memory, were separated from subsequent processes associated with the preparation and execution of the motor response, which were named “memory detection” for consistency with the perceptual paradigm. To obtain the necessary, large variation of memory retrieval times, subjects were asked true/false questions concerning two television episodes that were viewed 1 d before the imaging session. Perceptual search latency was varied by controlling the onset of target objects in similar audiovisual material. Using this approach, we tested whether parietal regions involved in environmentally and memory directed attention were overlapping, independent or competing, with a particular emphasis on the left hemisphere, since retrieval effects are commonly left lateralized (Wagner et al., 2005). We found that memory and perceptual search-related regions of the left parietal lobe were not only anatomically segregated, but were organized in a functional competition that was related to behavioral outcome.
Materials and Methods
Subjects
Nineteen healthy right handed subjects (mean age = 26.8 years, range 23–32 years) gave informed consent in accordance with guidelines set by the Human Studies Committee of Washington University. Each participant performed two tasks involving audiovisual material on different days: an episodic memory search and a perceptual search task. The order of tasks was counterbalanced across subjects. Exclusion criteria were insufficient knowledge of the English language and familiarity with the audiovisual material used.
Procedure
Episodic memory search task.
During the encoding session subjects watched two episodes from an English language television sitcom (see supplemental Methods, available at www.jneurosci.org as supplemental material). Subjects performed the fMRI retrieval session ∼24 h after the encoding session. On each trial (Fig. 1a), a sentence addressing memory for details and events across the two episodes was presented for 4 s at the center of the screen (e.g., “Richard mentioned his problem with alcohol before his intimacy problem”). The sentence was followed by a retrieval period in which a black display with a white central fixation cross was presented. Subjects were asked to maintain fixation on the cross for the entire run except while they were reading the sentences. Subjects were instructed to read the sentence, wait until it disappeared and then take the time they needed, up to 15 s, to retrieve the specific information and provide a “Yes”/“No” judgment about the accuracy of the sentence. Judgments were made using four buttons on the response key and the following category-key mapping: left middle finger = Yes, high confidence, left-index = Yes, low confidence, right index = No, low confidence, right middle = No, high confidence. Following the subject's response, the fixation cross turned red, indicating the onset of a variable intertrial interval (4.1 s, 6.2 s, 8.3 s) that preceded the next sentence. During catch trials, the sentence was immediately followed by the black display containing the red fixation cross. Subjects were periodically reminded to avoid retrieving episodic information following catch trials. While they did not report having problems with this instruction, it is possible that some retrieval activity occurred following the end of the catch trial. Five runs of 20 trials pertaining to the episode encoded first were presented, followed by five runs pertaining to the episode encoded second. Occasionally, while subjects were answering questions concerning one episode, a question was presented concerning the other episode. We refer to these questions as “invalid questions.” However, the BOLD responses to these questions were not the focus of the present paper and, although they were specified in the general linear model, they will not be discussed further.
Perceptual search task.
On each trial (Fig. 1b), a sentence instructing subjects to search for a specific target that could appear at any time and location in a subsequent video-clip was presented at the center of the screen for 4 s (e.g., “Can you detect a man standing on the street wearing red pants?”). Next, a video clip was presented at the center of a black display for 12 s, followed by a variable intertrial interval (∼4.1 s, ∼6.2 s, ∼8.3 s) in which a central red fixation cross was presented. Clips, extracted from four movies, covered ∼8.7 degrees of visual angle on the horizontal axis and were presented with the associated soundtrack. A central white fixation cross was superimposed over the clip. Three kinds of clips were presented: clips containing the target (“target”), clips containing an oddball target (“oddball”) and clips not containing any target (“non-target”). While standard targets were either specific objects or characters, the oddball target consisted of a transient (500 ms), visual perturbation superimposed over the video clip (see supplemental Methods, Study materials section, available at www.jneurosci.org as supplemental material). Like in the memory task, subjects were asked to maintain fixation on the center cross except when they were reading a sentence. Subjects were instructed to read the sentence and search for the specific visual target in the upcoming video clip. When either the target or the oddball target was detected, subjects had to press one of the two Yes keys, depending on their confidence, as quickly as possible. If a target was not detected, subjects had to press one of the two No buttons, depending on their confidence, at the end of the clip. Response confidence was rated with the same category-key mapping used for the memory task. Subjects knew that targets could appear only once during the clip and they were instructed to passively watch the end of the clip after a target had been detected. Responses were categorized in four classes: “Hit,” if one of the Yes buttons was pressed during either a target or an oddball trial, within a time window starting at target onset and ending 1.5 s after target offset; “Miss,” if one of the No buttons was pressed at the end of either a target or an oddball trial; Correct Rejection (CR), if one of the No buttons was pressed at the end of a non-target trial; False Alarm (FA) if one of the Yes buttons was pressed outside the time window during a target or oddball trial, or during a non-target trial. A total of 10 runs, each containing 25 trials, were conducted.
The scanning sessions for the perception and memory tasks were conducted in different sessions. As a result, subjects did not have to constantly switch between tasks, allowing a consistent task set to be adopted (e.g., a constant retrieval mode during the episodic memory task). In addition, by running the two tasks in different sessions, a large number of memory trials could be collected that nonetheless involved a roughly constant encoding-retrieval interval. An alternative solution to the second problem was to mix the two tasks in each of two sessions and show one TV episode before each session. However, subjects might have encoded the second episode differently if they had already experienced the memory task in the scanner in the first session as part of a mixed design. Moreover, memory retrieval in the scanner for the second episode would have suffered unequal interference from the first episode. Finally, in a mixed design it would not have been possible to present “invalid” trials during the memory task in the first session, since invalid trials could only be presented if subjects had viewed both episodes (although as noted above, the results for these trials are not presented here). The Discussion considers potential effects on the results of blocking the memory and perception tasks.
fMRI apparatus
Stimuli were presented with an Intel Core-2-duo laptop running Microsoft Windows XP. E-Prime2 Professional (Psychology Software Tools) software was used for stimulus presentation and response collection. Visual stimuli were projected to the head of the bore of the scanner via a liquid crystal display (LCD) projector (Sharp LCD C20X) and viewed with a mirror attached to the head coil. Auditory stimuli associated with the video-clips were delivered using a pneumatic headset designed to minimize interference from scanner noise. A magnet-compatible fiber optic key-press device recorded the subject's responses. Eye position was measured with an ISCAN ETL-200 system.
Imaging methods and preprocessing of BOLD images
Images were acquired with a Siemens Allegra 3T scanner. Structural images were obtained during the first scanning session using a sagittal MPRAGE T1-weighted sequence (TR = 1810 ms, TE = 3.93 ms, flip angle = 12°, TI = 1200 ms, voxel size = 1 × 1 × 1.25 mm) and a T2-weighted spin-echo sequence (TR = 3800 ms, TE = 90 ms, flip angle = 90°). Blood oxygenation level-dependent (BOLD) contrast functional images were acquired with a gradient echo echoplanar sequence (TR = 2064 ms, TE = 25 ms, flip angle = 90°, 32 contiguous 4 mm axial slices, 4 × 4 mm in-plane resolution). For the memory task, a variable number of frames was acquired on each run, depending on subject's performance (range = 124–197 frames, ∼4–7 min). During the perceptual task, a total of 256 frames (∼9 min) were acquired on each run.
The preprocessing and statistical analysis of fMRI data were performed using in-house software. The first four frames of each BOLD run were discarded from the analysis. Preprocessing included motion correction, within and between runs, slice scan-time correction and whole brain normalization, applied to each run. The whole brain normalization corrected for changes in overall image intensity between BOLD runs. Importantly, since the same normalization factor was uniformly applied to all MR frames within a BOLD run, the normalization could not result in artifactual deactivations (Aguirre et al., 1998). Functional images were resampled at a voxel size of 3 × 3 × 3 mm and warped into a standardized atlas space (Talairach and Tournoux, 1988).
Linear modeling
Data were analyzed using two general linear models (GLMs). The overall approach has been described and validated in several previous studies (Shulman et al., 1999, 2003, 2007; Ollinger et al., 2001a,b). Supplemental Figure 1 (available at www.jneurosci.org as supplemental material) illustrates the linear modeling method that was used for the analysis of the perceptual and the memory task.
The “process” regression model.
The aim of the first regression model was to separately estimate the BOLD signal for the different task processes that temporally overlapped in the course of a trial. For example, in the perceptual task, BOLD signals in a particular cortical region could be affected by sentence reading (Sentence), by visual and auditory stimulation (Display), by search for a target (Search), by target detection and response (Detection), or by a combination of these processes. Analogously, in the memory task, BOLD signals in a particular region could be affected by sentence reading (Sentence), by search in memory (Search), by responding to the question (Detection), or by a combination of these processes. A multiple parameter regression model was created that specified the effects of the various task processes on the observed BOLD time course. The model assumed that the observed BOLD response on each trial was the sum of the hemodynamic responses that were generated by the above processes, and was used to identify the voxels that were activated or deactivated by each process. The assumed response for each process was generated by convolving a function representing the duration of the process (rectangle functions for sustained processes, delta functions for transient processes) with a standard hemodynamic response function (Boynton et al., 1996). The experiment was specifically designed to enable the separation of the component processes by the process model. The sentence-reading phase was separated using a “catch-trial” technique (Shulman et al., 1999; Ollinger et al., 2001a,b), in which on a random 20% of the trials, the trial ended following the sentence-reading phase. This technique has been validated and widely used to separate the BOLD signals to a cue from the signals to subsequent events. The display, search, and detection processes were separated by arranging for them to have very different temporal profiles, which decorrelated the corresponding regressors in the linear model. While the display duration was always 12 s, the search duration could vary over trials from 1 to 12 s. Similarly, while the display and search components were sustained processes that always began at display onset, the detection process was a transient event that randomly occurred at very different times following display onset.
For the perceptual task, a sustained “Sentence” parameter modeled the 4 s reading time. A sustained “Display” parameter modeled the 12 s of audio-visual stimulation presented on each trial. Six versions of the “Search” parameter specified different search outcomes [“Search-Hit-HC” (high confidence), “Search-Hit-LC” (low confidence), “Search-Miss,” “Search-FA,” “Search-CR,” “Search-No-Response”]. The duration of the search process varied across trials depending upon the target onset and the subject's response. Transient BOLD responses related to the detection-response phase of the trial were estimated with separate parameters (see supplemental Methods, available at www.jneurosci.org as supplemental material). For the memory task, two sustained Sentence parameters modeled the 4 s reading time for valid or invalid questions. Four Search parameters [Search-Corr-HC (correct high confidence), Search-Corr-LC (correct low confidence), Search-Inc-HC (incorrect high confidence), Search-Inc-LC (incorrect low confidence)] separately coded for searches with different outcomes while a fifth parameter modeled trials in which subjects did not press any key (Search-No-response). Transient BOLD responses related to the detection-response phase of the trial were estimated with separate parameters (see supplemental Methods, available at www.jneurosci.org as supplemental material).
The “frame-by-frame” regression model.
While the previous model estimated the selective contribution of each process to the observed BOLD response, it involved assumed response functions. To examine the overall time course of BOLD activity for different types of trials (e.g., “early hit” trials versus “late miss” trials), which reflects the sum of the BOLD signal for the processes operative on those trials, a second GLM was created that made no assumption about the shape of the hemodynamic response. This frame-by-frame model provided an unbiased estimate of the time course for each trial type (Ollinger et al., 2001a,b), generating separate delta function regressors for each MR frame up to ∼30 s after trial onset. For the perceptual task, we estimated the time course for 13 types of trials: sentence reading, three Hit-target, three Hit-oddball, and three Miss types of trials, depending on the interval of target presentation (early, middle, late), and CR, FA and trials in which subjects did not press any key. For the memory task, we estimated 8 types of trials: sentence reading, three correct and three incorrect response trials (early, middle, late), and trials in which subjects did not press any key. For both tasks, the sentence only trials (catch trials) allowed us to separately estimate the time course for sentence reading so that the BOLD responses for sentence reading were not included in the modeled BOLD time courses for the task events that followed the sentence (“within-trial” model; see supplemental Methods, available at www.jneurosci.org as supplemental material) (Shulman et al., 1999; Ollinger et al., 2001a,b). We also created an additional version of the frame-by-frame GLM, aligning trials on sentence onset (the “between-trial” model; see supplemental Methods, available at www.jneurosci.org as supplemental material).
fMRI statistical analysis
Using the process regression model, voxelwise z-maps were computed for each parameter in the model (e.g., the search parameter for high confidence hits). Single subject and group statistical maps were Monte Carlo corrected over the brain for multiple comparisons (cluster size = 17, z = 3, corresponding to p < 0.05, corrected). Group voxelwise statistical maps were obtained in which subject was treated as a random effect. One-sample voxelwise t tests were conducted to determine voxels in which a parameter was significantly different from zero, while paired t tests were used to test for significant differences between parameters.
Regions of interest (ROIs) were created using a peak-search algorithm that identified peaks in the uncorrected z-map and consolidated foci closer than 12 mm by coordinate averaging. Spherical ROIs of 9 mm radius were formed, centered on the consolidated foci, that excluded voxels not contained in the multiple-comparison corrected z-map. Time courses of BOLD activity for early, middle and late Hit trials in the perceptual task and for early, middle and late correct trials in the memory task were obtained using the frame-by-frame regression model. One subject was excluded from this analysis since she did not provide answers in the shortest interval.
For the analysis of the relation between BOLD search-related activity and behavioral performance, all search parameters were collapsed in each task (with the exception of the false alarms in the perceptual task). One subject was excluded from the analysis since he did not provide a sufficient number (<10) of low confidence responses in the memory task. We then selected ROIs from the map of the contrast between all memory search parameters versus all perceptual search parameters. Two-tailed paired t tests were performed to determine whether the Search-Hit-HC parameter differed significantly from the Search-Miss parameter (perceptual task) and whether the Search-Corr-HC parameter differed from the Search-Corr-LC parameter (memory task). For the time course analysis of the catch sentence trials, a two-way ANOVA [task (memory, perceptual) × time (11 frames)] was performed to assess across task differences while a one-way ANOVA [time (11 frames)] was performed to assess the significant difference of each time course from zero.
Results
Subjects performed two tasks involving either perceptual or memory search. In the memory task, subjects judged the accuracy of sentences describing specific details about one of two movies that had been previously viewed, while maintaining central fixation (Fig. 1a). In this task, the source of search duration variability was the subject's decision time, reflecting the time needed to retrieve the critical information and answer the question. In the perceptual task, visually presented sentences instructed participants to search for a visual target that could be presented at any time during the upcoming 12 s video clip, while maintaining central fixation (Fig. 1b). We manipulated search time by varying the onset of the target, allowing us to separate the neural signals associated with search from signals related to the sensory display, whose duration was constant across trials, and from the signals that occurred at or after target presentation, such as those related to detection and response execution (see Materials and Methods) (Shulman et al., 2003). Similar audiovisual material was used in both tasks to minimize the possibility that differences in hemispheric lateralization across tasks were related to stimulus differences rather than to the search domain (Cabeza, 2008). Several controls discussed below addressed remaining differences between the two tasks, such as the presence of visual stimulation during perceptual search but not memory search.
Behavior
Both tasks were difficult, with an overall level of accuracy of 0.73 ± 0.1 and 0.69 ± 0.1 for the memory and the perceptual task, respectively. This difference was not statistically significant (t(18) = 2.10, p = 0.50, two-tailed paired t test). Accuracy for the perceptual task was calculated as the average of target (0.62 ± 0.2) and non-target (0.91 ± 0.1) trials, while oddball trials (0.94 ± 0.1) were not considered for the present analysis (see Materials and Methods).
Accuracy was much greater for high confidence than low confidence judgments (Fig. 1c): 0.81 ± 0.2 for high and 0.63 ± 0.2 for low confidence trials in the memory task (t(18) = 7.40, p < 0.001, two-tailed paired t test) and 0.79 ± 0.2 for high and 0.39 ± 0.2 for low confidence trials in the perceptual task (t(18) = 10.72, p < 0.001). Subjects gave fewer low-confidence responses in the perceptual task than in the memory task (28% and 42% of the total number of responses, respectively; t(18) = 2.93, p < 0.01, two-tailed paired t test).
A crucial analysis focused on reaction time (RT) distributions for the two tasks, since an adequate variability in RT was required to isolate the search process. It was particularly important to demonstrate this variability for the memory paradigm, in which retrieval duration was not controlled by the presentation of the target as in the perceptual task. Figure 1d shows that the paradigm successfully varied the duration of memory retrieval. Because very few responses occurred in the 12 to 15 s interval and performance at two time points was at chance, trials in this interval were excluded from the subsequent fMRI analysis. The remaining trials were divided into three bins (early: 0–4 s, middle: 4–8 s, late: 8–12 s) as in the perceptual paradigm. The difference in the detailed shape of the two RT distributions for correct trials, shown in Figure 1e, is due to the fact that in the perceptual paradigm targets were placed within a window centered on each 4 s interval to maximize the signal-to-noise ratio in the fMRI analysis, while retrieval times were more continuously distributed. For each task we calculated performance corresponding to the three 4 s bins (supplemental Fig. 2, available at www.jneurosci.org as supplemental material). Memory trials showed a progressive decrease in accuracy (0.82 ± 0.01, 0.73 ± 0.02 and 0.67 ± 0.02, respectively) (F(2,34) = 85.25, p < 0.001, one-way ANOVA). Accuracy on target-present trials in the perceptual task increased from the early to middle bin followed by a drop at the late bin [early (0–4 s) = 0.55 ± 0.02, middle (4–8 s) = 0.68 ± 0.02, late (8–12 s) = 0.64 ± 0.02; F(2,36) = 23.72, p < 0.001, one-way ANOVA], possibly reflecting an increase in the hazard function over the trial and a greater probability on late-target trials of committing a false alarm before target presentation.
Overall, both tasks showed a large spread of reaction times and high levels of difficulty.
fMRI
Because the behavioral results showed a strong effect of confidence on performance in both tasks, only correct high confidence trials were analyzed in most fMRI analyses, therefore emphasizing activations related to optimal performance. BOLD data were analyzed using two types of general linear models (GLM). A GLM that used an assumed hemodynamic response function provided separate voxelwise maps of the BOLD activity associated with each process within a trial (e.g., in the perceptual task: sentence reading, visual stimulation, search, detection), similar to previous decompositions of search and working memory tasks (Courtney et al., 1997; D'Esposito et al., 1999; Shulman et al., 2003). The time course of BOLD activity was determined using a GLM that did not assume a specific shape for the hemodynamic response function (see supplemental Fig. 1, available at www.jneurosci.org as supplemental material and supplemental material for more information on the linear decomposition of different task processes).
The distribution of memory and perceptual search-related activity in the parietal lobe
The search component of the memory and perceptual tasks activated distinct regions of the left parietal lobe (Fig. 2; supplemental Tables 1 and 2, available at www.jneurosci.org as supplemental material; see supplemental Fig. 3, available at www.jneurosci.org as supplemental material for right hemisphere). Memory search-related activations (Fig. 2a, red/yellow) were observed in the angular gyrus (AG), extending rostrally toward the supramarginal gyrus (SMG) and dorsally toward the lateral bank of the intraparietal sulcus (IPS), and in regions of precuneus (PreCu) and posterior cingulate cortex (PCC), extending ventrally toward retrosplenial cortex. Regions of search-related deactivation (green/blue colors) were identified in parietal operculum and in the ventral bank of IPS. Conversely, perceptual search-related activations (Fig. 2b) were observed along the ventral and the medial banks of posterior IPS and in regions of the superior parietal lobule (SPL). These parietal perceptual search regions were very similar to those that have been identified in a large number of studies concerned with attention to spatial location and attention to visual features, both in the presence and absence of visual stimulation (Kastner et al., 1999; Shulman et al., 1999; Corbetta et al., 2000; Hopfinger et al., 2000; Yantis et al., 2002; Serences and Boynton, 2007). Search-related deactivations were located bilaterally in SMG, and in AG and PCC regions that fall within the default network (Shulman et al., 1997; Raichle et al., 2001).
Additional analyses showed there was little or no overlap in the significant activations for the search parameter in the two tasks (Fig. 2c). A tiny sliver of overlap (yellow color) was present along the IPS, with a second sliver in dorsal precuneus. Even less overlap was found in the right parietal lobe (supplemental Fig. 3c, available at www.jneurosci.org as supplemental material). The minimal overlap in the group-averaged image also held for representative individual subjects (Fig. 2d). The degree of overlap over all subjects was quantitatively determined within the borders of Brodmann's area 7 (BA7) (defined from the surface borders of the PALS Atlas, Caret 5.5 software (Van Essen, 2005) that contained the bulk of adjacent activation in the two conditions. For each subject, the percentage of overlapping and nonoverlapping voxels was calculated among all the voxels that were significantly activated in either of the two tasks. These percentages were then averaged over subjects. On average, only 13.4% of the voxels activated by the two tasks overlapped in BA7. Finally, the consistency across individuals of the anatomical location of these overlapping voxels was then determined. A frequency map was created (Fig. 2e) in which each voxel was given a color according to the number of individuals who showed an overlap of significant search-related activity in the two tasks. Regions of overlap along the lateral bank of IPS were not spatially consistent across subjects, although a somewhat more consistent region was found in dorsal precuneus [peak voxel = 8 of 19 subjects (x = −7, y = −60, z = 57)]. However, while similar precuneus activations have been observed in studies of attentional switching (Yantis et al., 2002), studies of episodic memory retrieval have generally reported activity in more ventral regions of the medial parietal lobe (Vilberg and Rugg, 2008).
To summarize, memory and perceptual search-related (top-down) processes activated distinct regions of parietal cortex, both at group and individual levels of analysis.
Perceptual search-related activity does not reflect the mere presence of sensory stimulation.
The two search tasks differed in the amount of sensory stimulation, since in the perceptual task a video-clip was presented for 12 s while in the memory tasks subjects were instructed to fixate a white cross that was centered on a black screen. We conducted two controls for this problem. First, the design of the perceptual search experiment allowed us to separate activations related to the presence of the display from activations related to searching the display. The key design feature was that the duration of the display was the same on every trial (12 s) while the duration of search varied over a wide range across trials (1–12 s). This large difference in the temporal profiles of the search and display processes decorrelated the corresponding regressors (see also supplemental Fig. 1, available at www.jneurosci.org as supplemental material). Any component of the BOLD activation that depended on the duration of search did not reflect the presence of the physical display, but reflected attention to the display and/or the modulation of display-evoked activity by attention. Conversely, any component of the BOLD activation that was independent of search duration reflected the sensory stimulation from the physical display. A single region might show neither, one, or both types of BOLD components.
In parietal cortex, the voxelwise map for the sensory stimulation or display parameter (Fig. 3a) was almost coextensive with the map for the search parameter (Fig. 3b), but in early visual cortex only the display parameter was significant. The overlap in parietal cortex replicated an earlier study using a similar methodology (Shulman et al., 2003) and was expected based on the previous literature, since many studies have demonstrated that parietal regions show both attentional modulations and responses to passive visual stimulation (Bushnell et al., 1981; Corbetta et al., 1993; Colby et al., 1996; Swisher et al., 2007). A recent mapping study, for example, that used passive visual stimulation demonstrated five separate topographic maps along the full extent of IPS (Swisher et al., 2007).
The time course of the BOLD signal in parietal cortex (e.g., pIPS, Fig. 3c) was consistent with the presence of signals related to sensory- and attention-related activity. On an early target trial, the BOLD signal fell off sharply relative to a middle or late target trial, reflecting the search component of the BOLD signal. Any regions whose activity was solely dependent on sensory stimulation would not have shown this marked fall-off. However, the signal did not immediately return to baseline, reflecting a sustained component of the BOLD signal that was related to the presence of the display. Therefore, the time course closely fit the profiles that were expected based on the significance of both the display and search parameters in the assumed response model. This impression was confirmed by both the excellent overall fit (adjusted R2 = 0.96) of the model and the significant contribution of both Display (β = 0.22) and Search (β = 0.40) parameters. The presence of both sensory responses and attentional modulations in parietal cortex contrasted sharply with regions in early visual cortex, which showed only display activity (Fig. 3b). Correspondingly, the time course of activity on early, middle and late target trials was highly similar, with almost no fall-off of activity on early-target trials (Fig. 3c, early visual cortex), as expected from the assumed response model (overall model fit, R2 = 0.99; Display β = 1.23; Search β = 0.12).
The strong contrast between the occipital and parietal time courses illustrates that the observed parietal activity was not simply caused by the presence of the display but reflected a strong top-down search component. Later in Results, we present a second control for sensory activity in which activity in the perceptual and memory tasks was compared under conditions that held sensory stimulation constant.
Functional properties of memory and perceptual search-related regions
A direct contrast comparing search-related responses in the two tasks showed that regions in AG, lateral IPS and PCC-PreCu were significantly more activated during memory than perceptual search, while medial-posterior IPS and SPL were significantly more activated during perceptual than memory search (Fig. 4a; supplemental Table 3, available at www.jneurosci.org as supplemental material). This double dissociation rules out any explanation based on overall differences in task difficulty or arousal.
The analysis of the signal time course in the perception- and memory-search regions revealed a task-dependent competitive mechanism between the two systems. BOLD time courses were extracted from two regions showing a perceptual search preference (Fig. 4b, regions 1, 2) and four regions showing a memory search preference (Fig. 4b, regions 3–6). Left parietal perceptual regions showed a sustained positive response during visual search. The time of peak activation and offset of the response shifted predictably with the presentation of the target, consistent with the idea that these regions were involved in actively searching the display until the target was detected. In contrast, during the memory task these parietal perceptual areas were deactivated at the onset of memory search, with the duration of deactivation increasing with retrieval time (Shulman et al., 2003). The BOLD signal rose above baseline only at the end of the memory search period, before the motor response. A search-dependent, BOLD signal suppression in these areas was also observed when aligning trials on the onset of the sentence reading phase (between-trial model, see supplemental Methods, available at www.jneurosci.org as supplemental material) without separating the sentence reading from the search phase of the memory task (supplemental Fig. 4a,b, available at www.jneurosci.org as supplemental material). In particular, the BOLD signal fell off more steeply on memory-retrieval trials than on catch trials, consistent with an active suppression of the BOLD signal during memory-retrieval, and this suppression was greater for longer memory-retrieval durations.
In contrast, regions more activated by memory than perception search showed the opposite pattern: a sustained activation during memory trials and a sustained deactivation during perceptual trials. Therefore, search-related activations for one task were mirrored by search-related deactivations for the other task.
Behavioral significance of memory and perceptual search-related signals
The competitive interaction between perceptual and memory search-related regions was quantitatively assessed in relation to behavioral performance. Both “enhancement” effects (larger BOLD activations associated with better performance) and “suppression” effects (larger BOLD deactivations associated with better performance) were observed, suggesting that the degree of competition was related to the efficiency with which each task was performed.
To avoid any bias associated with region selection, ROIs were created from a voxelwise z-map (Fig. 5a) contrasting memory and perceptual search-related activations in which search processes followed by different outcomes were collapsed in each task (see Materials and Methods). The magnitudes of the search parameters for trials with different behavioral outcomes were compared within a task (perception or memory), not across tasks. Confidence was used as the critical behavioral variable for the memory task, since a strong effect on accuracy was observed between low and high confidence judgments (Fig. 1c). On average, each subject contributed 70 ± 14 correct-high confidence and 41 ± 17 correct-low confidence trials. Because of the very small number of low confidence hit trials, the difference between hits (high confidence) and misses was used as the behavioral variable in the perceptual task (Shulman et al., 2007). On average, each subject contributed 64 ± 17 (mean ± SD) Hit-high confidence and 35 ± 11 Miss trials. Separate ANOVAs were conducted for the enhancement and suppression effects for each task for memory and perception regions, with Region (e.g., SPL vs pIPS for perception regions) and Condition (high vs low confidence correct for memory, and hits vs misses for perception) as factors.
During the perception task (Fig. 5b, blue bars), perception-search regions (SPL, pIPS) showed a significant enhancement effect (F(1,17) = 6.02, p = 0.025), with no interaction with Region (F(1,17) = 2.18, p > 0.1), while memory-search regions (AG-SMG, AG, AG-latIPS) showed a significant suppression effect (F(1,17) = 7.31, p = 0.015), with a marginal interaction with Region (F(2,34) = 2.86, p = 0.07) (see Fig. 5b for separate t tests on each region). During the memory task (Fig. 5b, red bars), memory-search regions showed a significant enhancement effect during the memory task (F(1,17) = 5.96, p = 0.026, with no interaction with Region (F(2,34) = 0.33), while perception-search regions showed a marginally significant suppression effect (F(1,17) = 4.27, p = 0.054), with a significant interaction of the suppression effect with Region (F(1,17) = 5.58, p = 0.03).
These findings indicate that search-related BOLD signals in parietal perception-search and memory-search regions were related to performance on both tasks, and support the hypothesis that these regions were not independently modulated but showed a competitive interaction.
Differences in perception- and memory-task activations during sentence reading
If these two sets of regions in posterior parietal cortex were indeed specialized for searching in the environment or memory, then a similar topographic segregation might be observed during the sentence reading phase of the tasks. For example, during the memory task subjects might have retrieved relevant information from episodic memory as they read or finished reading the sentence, particularly since the 4 s duration of the sentence phase was chosen based on pilot work that established the maximum reading time across sentences and subjects. Importantly, during the sentence reading phase of the task, the sensory stimulation in the memory and perception tasks was equated, allowing activations from each task to be compared with sensory stimulation controlled. Figure 6 shows the BOLD response during sentence reading for the memory (Fig. 6a) and the perceptual task (Fig. 6b). In parietal perceptual search regions, significant voxelwise activity during sentence reading was observed in both memory and perception tasks, which was expected since in both tasks subjects attended to the sentence display (although in regional analyses, the magnitude of activity varied between tasks, as shown below). However, in memory search regions, voxelwise activity from sentence reading was not only significantly stronger during the memory than perception tasks (Fig. 6c), but was not even evident during the perception task.
The time courses of BOLD activity during sentence reading were extracted from regions showing significantly greater activity during the search phase of the perception (regions 1–2) or memory (regions 3–6) tasks (i.e., same regions as in Fig. 4). Importantly, these time courses were obtained from trials in which the cue sentence was not followed by the search task (catch trials, between-trial model, see supplemental Methods, available at www.jneurosci.org as supplemental material). A two-way ANOVA [task (memory, perceptual) × time (11 frames)] on the sentence reading time courses yielded a significant task by time difference in all regions. The SPL (F(10,170) = 3.38; p < 0.0001) and pIPS (F(10,170) = 4.24; p < 0.0001) ROIs showed higher activity for perceptual compared with memory sentence reading. The signal enhancement during the perception task may have reflected preparatory feature-based attention for the object described in the sentence or preparatory spatial-based attention to the area of the display containing the video clip. Conversely, the left parietal regions significantly more activated by memory than perceptual search were also more activated by sentence reading in the memory than perception tasks (AG: F(10,170) = 16.78; p < 0.0001; AG-latIPS: F(10,170) = 4.53; p < 0.0001; PCC-PreCu: F(10,170) = 32.37; p < 0.0001; PCC: F(10,170) = 15.82; p < 0.0001). Finally, consistent with the search results, the BOLD signal in memory regions (regions 3,5,6) went significantly below baseline in the perception task [AG: F(10,170) = 3.38, p < 0.0001; PCC-PreCu: F(10,170) = 3.68, p < 0.0001; PCC: F(10,170) = 3.39, p < 0.0001; one-way ANOVA (time)].
Therefore a double dissociation between the perception and memory tasks was demonstrated in parietal cortex (i.e., SPL and pIPS, perception > memory; AG, AG-latIPS, PCC-PreCu, PCC, memory > perception) under conditions in which sensory stimulation for the two tasks was equated. Interestingly, these results also show that parietal memory search regions were not activated by lexical, syntactic, or semantic processes, since these regions were not activated during the sentence-reading phase of the perception task (Fig. 6d). Finally, the results indicate that memory retrieval likely began during the 4 s sentence-reading phase of the memory task since regions that were strongly activated during the memory retrieval phase of the task were also significantly activated during the memory task by sentence reading (for further evidence, see supplemental Fig. 4c–f, available at www.jneurosci.org as supplemental material, which shows the time course of the BOLD signal over an entire memory trial, i.e., from the onset of the sentence).
No differences in eye movements during the perception and memory tasks
We checked whether the greater activation of dorsal posterior parietal regions during the perception task could have reflected a greater number of eye movements compared to the memory task. In fact, these regions are part of the dorsal attention network which is known to be activated by both covert and overt shifts of attention (Corbetta, 1998). Sixteen of the 19 subjects had eye movement records of sufficient quality in both tasks in at least half of the scanning runs. The SD of the eye movement position (supplemental Fig. 5a, left bars, available at www.jneurosci.org as supplemental material) and the amount of time the eyes were positioned outside of a 1° radius window (supplemental Fig. 5b, left bars, available at www.jneurosci.org as supplemental material) were the same during the sentence reading phase of the memory and perception tasks. Similarly, the analogous quantities were also the same during the search phase of the two tasks (supplemental Fig. 5a,b, right bars, available at www.jneurosci.org as supplemental material). A two-way ANOVA [task (memory, perceptual) × phase (sentence, search)] yielded only a significant main effect of phase, for both the measure of SD (F(1,15) = 12.60, p < 0.005) and time outside 1° threshold (F(1,15) = 8.44, p < 0.05). Similar results were obtained with a 2° window except that the overall time spent outside the window was smaller. The figures show the analysis of the eye movements along the x-axis since the x-axis was the most relevant axis for the sentence-reading phase of the task and the y-axis data were noisier. However, no differences between memory and perceptual search were observed when the analysis combined the x-axis and y-axis. Therefore, the observed differences between the memory and perception tasks were not due to differences in eye movements.
Discussion
We investigated the BOLD activity in the human parietal lobe associated with searching for task-relevant information in episodic memory or in the environment to test the anatomical and functional overlap of top-down attentional networks across domains. The results showed that adjacent yet clearly distinct regions of the parietal lobe were activated in a “push-pull” manner by top-down, search-related processes in episodic memory and perception. Again, we use the term memory search to indicate an ensemble of processes, including the retrieval of episodic information guided by the behavioral goal the organization and evaluation of the retrieval information, and the accumulation of evidence that pointed to a particular response (Moscovitch and Winocur, 1995; Badre and Wagner, 2007; Mecklinger, 2010). Moreover, depending on the sentence query, part or all of this process may be repeated several times in a dynamic fashion. By memory detection we instead refer to the subsequent processes related to preparation and execution of the motor response.
Memory-related activity was observed in left lateral parietal cortex, primarily angular gyrus and lateral IPS, and in midline structures, primarily precuneus, posterior cingulate and retrosplenial cortex (Fig. 2a). These regions overlap with regions of the default mode network (Shulman et al., 1997; Raichle et al., 2001; Buckner et al., 2008), and are similar to those showing old/new differences in item recognition studies (cf. Wagner et al., 2005; their Fig. 1), encompassing both dorsal and ventral parietal regions that have been associated with effects of familiarity and recollection, respectively (Vilberg and Rugg, 2008). A very similar pattern was observed in the right hemisphere (supplemental Fig. 3a, available at www.jneurosci.org as supplemental material), although, qualitatively, memory-related activations were generally more extensive in the left hemisphere.
Memory-search regions were adjacent to but anatomically distinct from regions activated by perceptual search, which were located in medial IPS and SPL, between the lateral and medial clusters of memory-related activations (Fig. 2c; supplemental Fig. 3c, available at www.jneurosci.org as supplemental material). These regions have been consistently reported in studies of top-down attention to spatial location, visual features, and objects and correspond to the dorsal frontoparietal attention network. Consistent with this result, search-related activations during the perceptual task were also observed in frontal eye field (FEF; data not shown), the second major component of the dorsal attention network (Kastner and Ungerleider, 2001; Corbetta and Shulman, 2002).
No consistent overlap between search-related regions for the memory and perception tasks was observed along IPS, either in group or in single-subject analyses. This result is consistent with the topographic segregation of dorsal attention and default mode network in studies of functional connectivity at rest (Fox et al., 2005). The overall pattern of results agrees with a recent meta-analysis of the memory and attention literatures, which reported a similar dissociation between the neural correlates of top-down attention in the visuospatial and the episodic memory domain (Hutchinson et al., 2009), the difference being particularly evident in the lateral/medial axis.
Methodological controls
The segregation of regions activated by memory and perceptual search did not reflect passive sensory responses in the perception task. First, a double dissociation in the activity of search-related regions was observed for perception and memory tasks during the sentence-reading phase of the tasks, in which sensory stimulation was equated. Second, activity in parietal search regions during the search phase of the perception task reflected attentional modulations rather than pure sensory activity, since this activity strongly depended on the duration of perceptual search under conditions in which sensory stimulation was held constant. Consistent with this result, these perceptual search activations occurred in regions (medial IPS and SPL) that have shown attentional modulations in many previous studies, both in the presence and absence of sensory stimulation, as well as responses to passive sensory stimulation (Kastner et al., 1999; Shulman et al., 1999; Corbetta et al., 2000; Hopfinger et al., 2000; Yantis et al., 2002; Serences and Boynton, 2007). Importantly, memory search regions were not activated by the sentence-reading phase of the perception task (in fact, some regions were deactivated), indicating that they were not involved more generally in semantic processing, but were specifically activated by tasks involving episodic memory.
Differences in the activations for each task were also not caused by differences in the pattern of eye movements, which were very similar across tasks during the sentence reading and search phases. Finally, activation differences were not due to the presence, during the memory but not perception task, of trials in which the order of events or the number of events was queried. As shown in Results, similar activations in the memory task were obtained when the analysis was repeated using only trials concerning single events (supplemental Fig. 6, available at www.jneurosci.org as supplemental material), although we acknowledge that this analysis had less power due to the use of fewer trials.
The memory and perception tasks were conducted in different sessions. A positive feature of this design is that subjects did not have to constantly switch between tasks, allowing a consistent task set to be adopted (e.g., a constant retrieval mode during the episodic memory task). This design also enabled a large number of memory trials to be collected with a roughly constant encoding-retrieval interval. However, there were several potential limitations associated with comparing BOLD activity across tasks performed in different sessions. First, the signal-to-noise of the two sessions might have differed due to variability in the scanner, the degree of subject movement, or the subject's overall arousal. However, while these factors might have produced an overall decrease in the activation for one task relative to the other task across all regions, they could not explain the critical result reported in the paper; namely the double dissociation in the activity of different parietal regions across tasks. Second, the results were not due to effects of task order since order was counterbalanced across participants. Finally, consistent differences in the locus of activations between tasks were not an artifact of inaccuracies in across-session registration. First, our alignment procedure put scans that were collected on different days into the same space with sub-voxel precision (∼0.2 mm at the cortical surface). Second, differences in across-session registration would not have produced consistent changes in activation loci between tasks (e.g., memory task activations in AG, and perceptual task activations in IPS/SPL). Instead, registration errors would have produced the opposite result, i.e., they would have reduced the consistency of differences in the locus of activation between tasks, particularly since task order was counterbalanced. Therefore, we strongly believe that the current results cannot be attributed to artifacts from testing each task in a different session.
Task-relevant stimuli such as targets activate a ventral attention network comprising right temporoparietal junction and ventral frontal cortex, along with the dorsal attention network (Corbetta and Shulman, 2002). During perceptual search of non-target items, before target onset and detection, the ventral attention network is deactivated (Shulman et al., 2003). Some theories of episodic memory retrieval have suggested that a similar ventral network is activated by the retrieval of salient or vivid memories, even if task-irrelevant (Cabeza et al., 2008). Although the present experiment was not designed to test these theories and the results do not bear on their validity, it has to be noted that vivid memories may have been retrieved at unpredictable times during the memory task. However, these memories would not have replaced the need for sustained goal-directed memory search based on the specific information indicated by the sentence cue. Moreover, vivid memories would not necessarily have occurred at a constant time relative to sentence onset, while top-down search was initiated by the sentence cue and was sustained over the interval before the subject's response. Therefore, the present results do not simply reflect retrieval of vivid memories.
Episodic memory and perceptual search networks dynamically compete
A key finding of the present study is that the parietal networks activated by memory and perceptual search, while anatomically segregated, showed a pattern of dynamic competition at the functional level. Regions that were more activated by one task were deactivated by the other task and these deactivations varied with the duration of search. The deactivation of parietal memory search regions during perceptual search is consistent with suggestions that the default system plays an important role in episodic memory (Buckner et al., 2005, 2008), since deactivation during goal-driven behavior to environmental stimuli is a defining feature of the default system (Shulman et al., 1997; Raichle et al., 2001). However, we are not aware of previous studies that have shown the converse relationship, in which attending to memory deactivated parietal regions that were activated by attention to the environment (i.e., the dorsal attention network). Importantly, both signal enhancement of the task-preferred network as well as signal suppression of the opposite network were functionally significant, i.e., were related to behavioral performance.
These results could reflect competitive interactions between perceptual and memory mechanisms, in line with standard assumptions about the neural and functional correlates of BOLD signal activations and deactivations (i.e., more activation → more task involvement, more deactivation → less task involvement) (see Logothetis, 2008, for caveats). Greater deactivation of memory regions might have produced better performance on the perception task by reducing interference from irrelevant memory representations. Conversely, the search-related deactivations observed in posterior IPS during memory search may have prevented irrelevant perceptual information from disrupting memory retrieval. In other words, dynamic competition could reflect a task-specific suppression that reduces crosstalk between networks. This interpretation is consistent with the idea that successful performance is associated with the activation of task-relevant regions and the simultaneous deactivation of task-irrelevant ones (Hester et al., 2004; Weissman et al., 2006). A recent study showed that across subjects the strength of the negative correlation between the default system and task-positive networks was significantly related to performance on a flanker task (Kelly et al., 2008).
An alternative interpretation reflects a different assumption concerning BOLD signal deactivations. For example, a region of deactivation may correspond to an overall decrease in local field potential that may coexist with more selective responses (Hayden et al., 2009) or a change in directional interaction of a subpopulation of neurons within the region. Instead of indicating disengagement from the current task, they may indicate more selective, task-relevant responses. For instance, Shulman and colleagues (Shulman et al., 2003) reported during a demanding visual identification task both search-related activations in dorsal parietal regions, similar to those recruited in this study, as well as search-related deactivations in temporoparietal junction (TPJ), the core region of a ventral attention network for stimulus-driven reorienting (Corbetta et al., 2008). The degree of search-related deactivation in TPJ predicted the successful identification of subsequently presented targets (Shulman et al., 2007). The authors suggested that top-down signals (activations) from dorsal parietal and other regions filtered the input to TPJ so that reorienting only occurred to a restricted set of task-relevant stimuli. This filtering deactivated TPJ and enabled more selective, task-relevant responses. According to this view, activated and deactivated regions may both play critical roles within the same task (see also Mayer et al., 2010). We think that the latter hypothesis is unlikely in the present case, since regions of the same network would have to reverse their functional role according to the nature of the information (mnemonic, perceptual) being processed. Recent computational studies suggest that large-scale system dynamics based on competitive interactions enable functional networks to be quickly activated by small modulations of sensory or internal signals (Deco et al., 2009). Interestingly, dorsal attention and default networks have been reported to be in competitive interaction not only during task performance, as in this study, but also at rest (Fox et al., 2005). This relationship between task and rest dynamics indicates a potentially fundamental, and yet unknown, link between ongoing cortical states and their recruitment during attention to memory and the environment.
Footnotes
-
This work was supported by the J.S. McDonnell Foundation and National Institute of Mental Health Grant R01 MH-71920-06. We are grateful to Abraham Z. Snyder, Annalisa Tosoni, and Gian Luca Romani for helpful discussions, Daniel L. Pope for technical help, and Mark McAvoy for guidance with statistical analyses.
- Correspondence should be addressed to Carlo Sestieri, Washington University School of Medicine, 4525 Scott Avenue, Department of Neurology Campus Box 8225, Saint Louis, MO 63110. c.sestieri{at}unich.it