Higher order cognition depends on the ability to recall information from memory and hold it in mind to guide future behavior. To specify the neural mechanisms underlying these processes, we used event-related functional magnetic resonance imaging to compare brain activity during the performance of a visual associative memory task and a visual working memory task. Activity within category-selective subregions of inferior temporal cortex reflected the type of information that was actively maintained during both the associative memory and working memory tasks. In addition, activity in the anterior prefrontal cortex and hippocampus was specifically enhanced during associative memory retrieval. These data are consistent with the view that the active maintenance of visual information is supported by activation of object representations in inferior temporal cortex, but that goal-directed associative memory retrieval additionally depends on top-down signals from the anterior prefrontal cortex and medial temporal lobes.
- long term
- short term
- event related
The ability to retrieve information from memory and hold it in mind for future use is critical for goal-directed behavior. Through studies of visual associative long-term memory (LTM) in nonhuman primates, researchers have learned a great deal about the neural mechanisms of LTM retrieval and the working memory (WM) maintenance processes that keep retrieved information in the active state. For example, results from Miyashita and colleagues suggest that visual WM maintenance and visual associative LTM retrieval require sustained activity within inferior temporal areas such as area TE (Miyashita and Chang, 1988; Sakai and Miyashita, 1991), and that visual associative LTM retrieval additionally requires feedback signals from regions in the prefrontal cortex (PFC) and medial temporal lobes (MTL) to inferior temporal cortex (Eacott and Gaffan, 1992; Gutnikov et al., 1997; Hasegawa et al., 1998; Tomita et al., 1999).
Here, we used event-related functional magnetic resonance imaging (fMRI) to characterize and compare the neural mechanisms of visual associative LTM retrieval and WM maintenance in humans. Fifteen volunteers were trained on a set of faces, houses, and face– house pairs (Fig. 1) and scanned during performance of two memory tasks with these stimuli. On delayed paired associate (DPA) trials, subjects were shown a face or house that was learned previously in a face– house pair, and they were required to recall and maintan the associated object across a delay. On delayed matching-to-sample (DMS) trials, subjects were shown a previously learned face or house, and they were required to maintain a mental image of the presented stimulus across a delay. The primary difference between DMS and DPA trials was that DMS trials required active maintenance of a face or house, whereas DPA trials required retrieval and maintenance of a face– house association.
Previous neurophysiological studies have reported sustained activity in object-selective inferior temporal neurons during visual associative LTM retrieval and WM maintenance (Miyashita and Chang, 1988; Sakai and Miyashita, 1991). Although fMRI cannot resolve differences in object selectivity at the single-neuron level, fMRI studies have reliably identified inferior temporal subregions that selectively respond to categories of objects, such as the fusiform face area (FFA) for faces and the parahippocampal place area (PPA) for buildings (Aguirre et al., 1998b; Haxby et al., 2001; Malach et al., 2002; Spiridon and Kanwisher, 2002). We hypothesized that delay-period activity in these regions during WM maintenance and associative recall would reflect the type of stimulus that was active in memory. That is, we predicted sustained FFA activity increases when faces were recalled on DPA trials and when faces were maintained on DMS trials, whereas we predicted sustained PPA activity increases when houses were recalled on DPA trials and when houses were maintained on DMS trials.
In addition, we performed map-wise analyses to specify the roles of PFC and MTL regions in WM maintenance and associative retrieval. We reasoned that regions that generally contribute to WM maintenance should exhibit sustained delay-period activity on DPA and DMS trials, whereas any PFC or MTL regions that disproportionately support associative LTM retrieval should exhibit enhanced cue-period activation on DPA trials (i.e., when the association is recalled).
Materials and Methods
Participants. Fifteen neurologically intact, right-handed, native English speakers (seven females) 18-25 years of age participated in the study. These volunteers were recruited from the University of California at Davis student community and were financially compensated for their participation. Because of technical difficulties, behavioral data were not recorded for one subject. Therefore, we report analyses of behavioral data from the remaining 14 subjects and fMRI data from all 15 subjects.
Procedure. A total of 16 grayscale houses and 16 grayscale faces was used as stimuli in this experiment. The subjects were not familiar with these stimuli before the experiment. The experimental design is schematically depicted in Figure 1. Immediately before scanning, subjects were given a training session to learn these stimuli. During training, subjects saw eight single houses and eight single faces (each presented randomly to the left or right of fixation) and eight face– house pairs (with each item randomly shown in the left or right position). Subjects were instructed to intentionally learn these stimuli and the face– house pairings in anticipation of a test. After four exposures to each face, house, and face– house pair, subjects were given item recognition memory tests on each of these stimuli (i.e., the single faces and houses and the constituent items in each face– house pair). Next, subjects were shown a page with a random sequence of faces and houses and were asked to recall which face was shown with which house. Once subjects reached criterion-level performance on the item recognition and pair memory tests (95% accuracy), subjects proceeded to the scanning phase.
During the scanning phase, subjects performed DMS and DPA trials in alternating scanning runs (Fig. 1). Each run was preceded by a verbal instruction to remind subjects which task (DMS or DPA) was to be performed. As described below, the familiarity of each object and overall stimulus and task properties were closely matched across DMS and DPA trials. Each DMS and DPA trial lasted 9 sec and was followed by a 12 sec intertrial interval (ITI). On each DMS trial, a cue stimulus (one of the learned faces or houses that was not associated with another item) was shown for 1 sec, and subjects were instructed to rehearse this item across a 7 sec delay. Next, a probe stimulus (either a matching item or non-matching studied item from the same category) was shown for 1 sec, and subjects were instructed to indicate with a button press whether it matched the cue item. Match-nonmatch decisions were made by pressing one of two buttons on a magnet-compatible response device. On each DPA trial, a cue stimulus (one of the items that was learned in a face– house pair) was shown for 1 sec, and subjects were instructed to use this cue to recall the associated item and to rehearse the associate during the 7 sec delay. Next, a probe stimulus (either the associate of the cue or an item from the same category that was paired with a different item) was shown for 1 sec, and subjects were instructed to indicate with a button press whether it was associated with the cue item during the study phase. Across the experiment, subjects completed a total of 72 DMS and 72 DPA trials.
In addition to the DPA and DMS tasks, subjects performed a visuomotor response task (used to derive an estimated hemodynamic response function) (Aguirre et al., 1998c) and one to two runs of either a passive viewing or a one-back task to identify face- and scene-sensitive regions of the inferior temporal cortex (Kanwisher et al., 1997; Epstein and Kanwisher, 1998).
MRI acquisition and processing. MRI data were collected on a 1.5 T GE Signa scanner at the University of California at Davis Research Imaging Center. Functional MRI was performed with a gradient echo-planar imaging sequence (repetition time, 2000; echo time, 40; field of view, 240; 64 × 64 matrix), with each volume consisting of 24 5-mm slices. Coplanar and high-resolution T1-weighted images also were acquired within the same session. fMRI data preprocessing was performed with statistical parametric mapping (SPM99) software for all subjects. Analyses of data from inferior temporal regions of interest (ROI) were performed on native-space data to maximize the ability to discriminate category-specific areas of inferior temporal cortex (Aguirre et al., 1998a). For these ROI analyses, native-space images were sinc interpolated in time to correct for between-slice timing differences in image acquisition and realigned using a six-parameter, rigid-body, transformation algorithm. For map-wise statistical analyses, images were sinc interpolated in time to correct for between-slice timing differences in image acquisition, realigned using a six-parameter, rigid-body, transformation algorithm, spatially normalized to the template from the International Consortium for Brain Mapping Project (Cocosco et al., 1997), resliced into 3.5 mm isotropic voxels, and spatially smoothed with an 8 mm full width at half maximum Gaussian filter.
Data analysis. Activation during each phase of each trial was assessed using multiple regression (Courtney et al., 1997; Postle et al., 2000; Rowe et al., 2000; Ranganath and D'Esposito, 2001). Separate covariates modeled activation associated with the cue, delay, and probe periods of DPA and DMS trials relative to baseline activity during the ITI (see below). Additional nuisance covariates included the global signal (orthogonalized with respect to the design matrix) (Desjardins et al., 2001), an intercept, and trial-specific baseline shifts. The convolution matrix included a time-domain representation of the 1/f power structure (Aguirre et al., 1997; Zarahn et al., 1997) and filters to remove frequencies >0.25 Hz and <0.02 Hz.
Data from the visuomotor response task were not available for two subjects, and in these cases, covariates modeling blood oxygenation level-dependent (BOLD) responses for each subject during DPA and DMS trials were derived by convolving the vector of expected neural activity with the “canonical” hemodynamic response function (HRF) included in SPM99. For each of the other 13 subjects, covariates modeling BOLD responses during each task phase were derived by convolving the vector of expected neural activity with a subject-specific HRF estimated from responses in the central sulcus during the visuomotor response task (Postle et al., 2000; Ranganath and D'Esposito, 2001). Plots of these HRFs are available on-line as supplemental material (available at www.jneurosci.org).
Subject-specific HRFs were used, rather than a single canonical HRF, because previous work has demonstrated substantial inter-subject variability in the shape of the HRF (Aguirre et al., 1998c). One potential concern with using an HRF derived from a subject's motor cortex is that there may be inter-regional differences in the shape of the HRF within a given subject (Miezin et al., 2000). However, recent findings suggest that, within a subject, HRFs from different regions exhibit similar waveform characteristics, and that these waveforms can differ remarkably from canonical HRFs (Handwerker et al., 2004). Thus, using an empirically derived HRF enhanced the sensitivity, specificity, and validity of our inferences regarding activation during each task phase.
For analyses of results in inferior temporal cortex, PPA and FFA ROI were defined by first anatomically masking the data using individually defined boundaries for the collateral sulcus and fusiform gyrus. Within these anatomical boundaries, the FFA and PPA were functionally defined on the basis of data from the FFA-PPA localizer scans using criteria derived from previous studies by Kanwisher and colleagues (Kanwisher et al., 1998; Epstein et al., 1999; O'Craven and Kanwisher, 2000). The FFA was defined by identifying the most anterior cluster of voxels in the right fusiform gyrus, showing statistically significant activation in a contrast of faces against scenes (p < 0.05; corrected for multiple comparisons within the fusiform region). Likewise, the PPA was defined by identifying the most anterior cluster of voxels in the left and right collateral sulcus that showed statistically significant activations in the scenes minus faces contrast (p < 0.05, corrected for multiple comparisons within the parahippocampal-collateral sulcus region). A figure that shows the locations of the PPA and FFA for a representative subject is available on-line as supplemental material (available at www.jneurosci.org). Time series data during DPA and DMS trials were averaged across all voxels within these ROI and used as the dependent measure for single-subject multiple regression analyses. Linear combinations of parameter estimates derived from these analyses were used as dependent measures for group analyses.
For map-wise statistical analyses, results from single-subject analyses of spatially normalized data were entered into second-level t tests treating subjects as a random variable. In these analyses, images of parameter estimates for each contrast of interest (i.e., linear combinations of parameter estimates from the regression analyses described above) were then entered into a second-level group analysis, a one-sample t test, in which the mean value across the group for each voxel was tested against zero. Significant regions of activation were identified using a two-tailed threshold of p < 0.001 (uncorrected) and a minimum cluster size of at least eight contiguous voxels. Corresponding statistical maps were overlaid on T1-weighted images using the MRIcro software package (Rorden and Brett, 2000).
Overall, accuracy was high in both DPA (M = 93.3%; SD = 3.47%) and DMS (M = 97.7%; SD = 4.49%) trials. An ANOVA comparing accuracy across trial type (DPA vs DMS) and probe type (match vs nonmatch) showed that accuracy was slightly, but significantly, higher in DMS trials than in DPA trials (F(1,13) = 7.83; p < 0.05). In addition, reaction times (RTs) were slightly, but significantly, faster in DMS (M = 1998.6 msec; SD = 186.7) than in DPA (M = 2049.6 msec; SD = 206.2) trials (F(1,13) = 6.35; p < 0.05).
fMRI results: inferior temporal ROI
In these analyses, we tested the hypothesis that maintenance of perceived objects and maintenance of recalled objects occurs through sustained activation of object representations in inferior temporal cortex. By this reasoning, one would predict that active maintenance or recall of houses and faces would be likely to activate the FFA and PPA, respectively. Using results from an independent localizer scan (see Materials and Methods), we were able to identify FFA and PPA ROI for 12 of the 15 subjects. As shown in Figure 2, BOLD responses in each of these ROI were averaged for DPA and DMS trials according to whether a house or face was the cue stimulus. Based on the reasoning described above, we hypothesized that delay-period activity in the FFA and PPA on DPA and DMS trials should reflect the type of stimulus that is currently active in memory. Thus, we hypothesized that FFA activity during the delay period should be enhanced on trials when faces were recalled or maintained, whereas PPA activity during the delay period should be enhanced on trials when houses were recalled or maintained.
On DPA trials, we hypothesized that delay-period activity should reflect the type of stimulus that was being recalled in anticipation of the probe decision, rather than the type of stimulus that was actually presented during the cue phase. Specifically, we predicted that FFA delay-period activity would be larger when a house stimulus was used as a cue to recall an associated face than when a face stimulus was used as a cue to recall an associated house, and that the opposite pattern of results would be observed within the PPA. To test this hypothesis, we separately extracted parameter estimates indexing cue- and delay-period activation in each ROI on DPA trials when a face was used as a cue to recall an associated house and on DPA trials when a house was used to recall an associated face. These parameter estimates, shown in Figure 3, were entered into repeated measures ANOVAs evaluating the effects of cue-type (house vs face) and trial period (cue vs delay). These analyses revealed significant cue type by trial period interactions in both the FFA (F(1,11) = 13.37; p < 0.001) and the PPA (F(1,11) = 34.28; p < 0.001). This interaction confirmed that the direction of the activation difference between face-cue and house-cue trials significantly changed between the cue and delay phases. Follow-up analyses were performed on parameter estimates indexing delay-specific activation to determine whether activation in each ROI reliably differed according to cue type. As shown in Figure 3A, FFA delay activity was significantly greater during house-cue trials than during face-cue trials [t(11) = 3.86; p < 0.005], whereas PPA delay activity was greater during face-cue trials than during house-cue trials [t(11) = 3.61; p < 0.005]. The present findings confirmed that delay-period activation on DPA trials reflected the type of information that was actively recalled, rather than the type of information that was presented during the cue phase.
To more stringently test the hypothesis that inferior temporal delay-period activity on DPA trials reflected the type of information that was recalled in anticipation of the upcoming test probe, we directly compared the magnitudes of PPA and FFA delay-period activation in these trials as a function of the type of information that was recalled. Specifically, we conducted a region (PPA vs FFA) by cue-type (house vs face) ANOVA on parameter estimates indexing delay-period activation on DPA trials. Consistent with our hypothesis, we observed a significant region by cue-type interaction (F(1,11) = 21.13; p < 0.001). Follow-up analyses confirmed that delay-period activation was significantly higher in the FFA than in the PPA in house-cue trials [t(11) = 2.27; p < 0.05], whereas activation was higher in the PPA than in the FFA in face-cue trials [t(11) = 2.55; p < 0.05]. These findings demonstrate that the topography of inferior temporal activity during the delay period of DPA trials reflected the type of information that was recalled in anticipation of the upcoming test probe, rather than the type of information that was presented during the cue period.
As described above, we hypothesized also that WM maintenance would be associated with sustained activation of stimulus representations in inferior temporal cortex. Accordingly, we predicted that on DMS trials, FFA delay-period activity would be enhanced when a face stimulus was encoded and maintained, whereas PPA delay-period activity would be enhanced when a house was encoded and maintained. To test this hypothesis, parameter estimates indexing cue- and delay-period activation in each ROI on DMS trials were entered into repeated measures ANOVAs, evaluating the effects of cue-type (house vs face) and trial period (cue vs delay). This analysis revealed significant main effects of cue type in both the FFA (F(1,11) = 18.70; p < 0.001) and in the PPA (F(1,11) = 24.07; p < 0.001), demonstrating that across both the cue and delay periods, activation was enhanced for the preferred stimulus type of each region (Fig. 3B). In addition, a significant main effect of trial period was observed for the PPA (F(1,11) = 6.11; p < 0.05), showing that overall activation was greater during the cue than during the delay. To verify that delay-specific activity reflected the type of information that was actively maintained, follow-up analyses were performed on parameter estimates indexing delay-specific activation in each ROI. As shown in Figure 3B, delay-period activity in the FFA was significantly greater during face-cue trials than during house-cue trials [t(11) = 3.61; p < 0.005], but PPA activity was not significantly greater during house-cue trials than during face-cue trials [t(11) = 1.57; p = 0.14].
To more stringently test whether inferior temporal delay-period activity in DMS trials reflected the type of information that was actively maintained, we conducted a region (PPA vs FFA) by cue-type (house vs face) ANOVA and parameter estimates indexing delay-period activation in DMS trials. Results from this analysis revealed a significant region by cue-type interaction (F(1,11) = 12.77; p < 0.001). Again, this crossover interaction demonstrates that the topography of activity during the delay period of WM trials reflected the type of information that was actively maintained. However, it should be noted that the magnitudes of the simple effects (i.e., the FFA-PPA activation difference for face-cue trials and for house-cue trials) did not reach statistical significance [face-cue trials: t(11) = 1.705, p = 0.116; house-cue trials: t(11) = -1.39, p = 0.192].
To verify the robustness of the effects described above, we examined individual variability in delay-period activity during DPA and DMS trials. As shown in the scatter plot in Figure 4A, although the magnitude of category specificity during the delay varied somewhat across subjects, the effects were robust and not driven by any single subject. We next investigated whether some of the inter-individual variability in category-specific delay activity might reflect meaningful variability in the ability to retain a vivid, object-specific representation across the delay. We reasoned that if this was the case, the degree of category selectivity during the delay period of DPA trials should be positively correlated with the degree of category selectivity observed during the delay period of DMS trials. In other words, one might expect subjects who showed more selective activity in the FFA when recalling faces on DPA trials to also show more selective FFA activity when maintaining faces on DMS trials, and subjects who showed more selective activity in the PPA when recalling houses to show more selective PPA activity when maintaining houses.
To test this prediction, we computed indices of category selectivity for delay-period activity on DPA and DMS trials for each subject. These “mnemonic category selectivity” indices were derived by first calculating differences in delay-period activity when faces were recalled relative to when houses were recalled in each ROI (i.e., DPA trials with house cues minus DPA trials with face cues) and differences in delay-period activity when faces were actively maintained relative to when houses were actively maintained. Within each ROI (FFA and PPA), these values were then z-transformed using the mean effect pooled across subjects and trial types. As shown in the scatter plot in Figure 4B, there was a consistent relationship between mnemonic category selectivity during the delay period of DPA trials and category selectivity during the delay period of DMS trials. Within the FFA, the correlation between DPA and DMS category selectivity was 0.536, and within the PPA, the correlation was 0.516. These category selectivity values were not reliably correlated with the magnitudes of HRFs derived for each subject, suggesting that the correlations in mnemonic category selectivity observed across ROI reflected inter-individual variability in cognitive-neural processing rather than variability in overall hemodynamic responsiveness. The remarkable consistency of the two findings across the two ROI suggests that some of the inter-subject variance in category-specific delay-period activity reflected reliable variability in the ability to maintain an object-specific representation across the delay.
fMRI results: map-wise analyses
In these analyses, we sought to determine to what extent similar versus different regions mediated associative LTM retrieval and WM maintenance. Our first map-wise analyses identified regions that exhibited differential activation during the cue, delay, and probe periods of DMS and DPA trials. Based on the hypothesis that signals from the PFC and MTL support reactivation of associative codes, we hypothesized that these areas would exhibit enhanced activity during the cue or delay periods of DPA trials relative to DMS trials. Consistent with this prediction, results revealed several areas that showed greater activity during DPA than during DMS trials, including the right anterior PFC [at or near Brodmann's area (BA) 10/46] and left posterior hippocampus as well as bilateral regions in the thalamus, caudate nucleus, cerebellum, and visual cortex (Fig. 5) (a complete listing of activation foci identified in this contrast is available on-line as supplemental material; available at www.jneurosci.org). Map-wise analyses did not reveal any areas that exhibited increases in activity during the delay period of DPA trials relative to DMS trials, suggesting that the signals that were differentially related to associative recall primarily occurred during the cue period. No areas showed significantly greater activation during the cue period of DMS trials relative to DPA trials. During the delay period, the only areas showing enhanced activity during DMS trials relative to DPA trials were in the left and right central sulcus and the right middle occipital gyrus.
We next sought to identify regions that were active during maintenance of perceived objects and maintenance of recalled associations. Thus, we conducted a separate “conjunction analysis” to identify regions that exhibited sustained increases in activity during the delay periods of both DMS and DPA trials (Cabeza et al., 2002; Ranganath et al., 2003a). For this analysis, random-effect statistical parametric maps (SPMs) were generated testing for significant delay-period activation in DPA and DMS trials. Each of these SPMs was thresholded at p < 0.01, and voxels surviving this threshold for both DMS and DPA trials were identified as exhibiting significant delay-period activity across both trial types at a joint probability threshold of p < 10-4.
Based on the idea that lateral PFC regions support associative retrieval through their more general role in WM maintenance processes, we hypothesized that lateral PFC regions would be identified in this analysis. Consistent with this hypothesis, results of this analysis revealed substantial areas of overlap in delay-period activation across DPA and DMS trials within lateral PFC. These regions included areas in the left ventrolateral (BA 44 and 47) and dorsolateral (BA 46) PFC (Fig. 6A) as well as in the frontal eye fields, presupplementary motor area, parietal cortex, and visual cortical areas (a detailed listing of coordinates corresponding to the areas identified in this analysis is available from the authors by request) that have been previously implicated in visual WM maintenance (Schumacher et al., 1996; Courtney et al., 1997; Petit et al., 1998; Pessoa et al., 2002; Ranganath et al., 2003a).
In the present study, we used event-related fMRI to characterize and compare the neural mechanisms of visual associative LTM retrieval and WM maintenance. Results showed that during both associative LTM retrieval and WM maintenance, activity in category-specific inferior temporal subregions reflected the type of object that was currently active in memory. Furthermore, we identified the anterior PFC and hippocampus as sources of signals that guided reactivation of object representations during associative retrieval. We expand on these results and their implications below.
Inferior temporal activity reflects active information in memory
As noted above, previous studies have shown that inferior temporal neurons exhibit sustained activity during active maintenance of their preferred objects (Fuster and Jervey, 1981, 1982; Miyashita and Chang, 1988; Miller et al., 1993; Nakamura and Kubota, 1995), and that these neurons can respond to an initially nonpreferred object through repeated association with a preferred object (Miyashita, 1988; Yakovlev et al., 1998). This phenomenon was directly elicited by Sakai and Miyashita (1991), who identified a class of object-selective “pair recall” neurons that exhibited heightened delay-period activity when the associate of the preferred object was presented as a cue stimulus (Naya et al., 1996, 2001, 2003a, b). Results from these studies suggest that the neural representation of the associate was activated in anticipation of the upcoming test probe (i.e., “prospective coding”) (Rainer et al., 1999).
Our results showed that population level inferior temporal activity during the delay periods of DPA and DMS trials reflected the type of information that was relevant for the upcoming memory decision (Figs. 2, 3, 4). On DPA trials, delay-period activity in the FFA and PPA reflected prospective recall of the paired associate rather than maintenance of the previously presented object. On DMS trials, however, the relative topography of delay-period activity reflected the type of information that was actively maintained. This finding accords with results from another study in which we found that, independent of perceptual stimulation, the FFA and PPA each exhibited greater encoding- and maintenance-related activity when its favored stimulus was relevant to the upcoming test probe (Ranganath et al., 2004).
Neuroimaging studies of visual imagery have also observed FFA activity during extended periods of face imagery (Ishai et al., 2000, 2002; O'Craven and Kanwisher, 2000), suggesting that both visual imagery and visual memory processes might depend on “top-down” activation of inferior temporal object representations. Based on findings suggesting that individuals reliably differ with respect to specific imagery abilities (Kosslyn et al., 1984, 2004), we investigated whether such individual differences might be expressed through the degree of category-specific activity observed during DPA and DMS trials. Results demonstrated that subjects exhibited reliable individual differences in the ability to elicit category-specific inferior temporal activity in both types of trials (Fig. 4). This finding suggests that inter-individual variability in FFA and PPA recruitment during maintenance of faces and buildings reflected differences in the ability to retain an object-specific representation. Additional research will be necessary to determine how this ability might relate to individual differences in specific kinds of imagery abilities (Kosslyn et al., 1984, 2002, 2004).
Top-down signals for WM maintenance and LTM retrieval
Results from previous lesion and neurophysiological studies in monkeys strongly suggest that the PFC and MTL are sources of top-down signals that reactivate object representations in inferior temporal cortical areas, such as area TE (Miyashita and Hayashi, 2000; Rainer and Ranganath, 2002). For example, lesions that disconnect the PFC from inferior temporal cortex have been shown to disrupt visual associative recall (Eacott and Gaffan, 1992; Gutnikov et al., 1997; Hasegawa et al., 1998; Tomita et al., 1999), and neurons in lateral PFC have shown object-selective activity that reflects reactivation of associations (Rainer et al., 1999).
Although previous physiological work did not distinguish between MTL (perirhinal cortex) and inferior temporal (area TE) contributions to associative recall, more recent work has shown that neural correlates of associative recall within the perirhinal cortex precede those observed in inferior temporal area TE (Naya et al., 2001). Furthermore, lesions of perirhinal and entorhinal cortex disrupted neural correlates of associative recall in inferior temporal regions (Miyashita et al., 1996). These findings suggest that the MTL plays a critical role in reactivating inferior temporal object representations during associative recall.
The present results expand on these findings by demonstrating complementary neural mechanisms that support associative LTM. On one hand, posterior dorsolateral and ventrolateral PFC (BAs 9, 44, 45, and 47) were robustly recruited during the cue and delay periods of both DMS and DPA trials. In contrast, anterior PFC (BA 10/46) and the posterior hippocampus were disproportionately active during the cue period of DPA trials relative to DMS trials. Thus, our results suggest that the anterior PFC and hippocampus contribute more specifically to associative LTM retrieval, whereas lateral PFC subregions contribute to WM processes that support maintenance of information recalled from LTM.
Before considering the functional significance of this finding, it is useful to consider some factors that might have confounded comparisons between DPA and DMS trials. For example, one potential confound is that DPA trials were more difficult than DMS trials (as indexed by RT and accuracy). However, several factors suggest that our findings of differential activation between DPA and DMS trials cannot simply be attributed to factors inherent in making a more difficult memory decision. For example, if the observed differences were related to simply making a more difficult judgment, such differences would be apparent during the probe phase of DPA trials. Instead, anterior PFC and hippocampal responses differentiated between DPA and DMS trials during the cue period, but similar responses were observed during the probe phase (Fig. 6). Furthermore, individual differences in relative task difficulty (i.e., the RT or accuracy difference between DPA and DMS trials) were not reliably correlated with the cue-period activity difference (DPADMS) in anterior PFC or hippocampus. These findings accord well with previous findings demonstrating that anterior PFC and hippocampal activity can be associated with LTM retrieval processes independent of task difficulty (Ranganath and Paller, 1999; Ranganath et al., 2000; Cabeza et al., 2002).
Another potential explanation for activity differences between DPA and DMS trials is that they might have reflected differences in memory load. That is, subjects only maintained the cue item on DMS trials, but they might have maintained both the cue item and its associate on DPA trials. However, studies manipulating visual working memory load generally have not reported load sensitivity within anterior PFC or hippocampus (Jha and McCarthy, 2000; Druzgal and D'Esposito, 2003; Linden et al., 2003). Using a previously published dataset in which memory load for face stimuli was parametrically manipulated (Druzgal and D'Esposito, 2003), we directly investigated whether anterior PFC or hippocampal activity might be sensitive to memory load effects. Within the anterior PFC and hippocampal regions identified in this study, there were no reliable trends for load sensitivity in the Druzgal and D'Esposito dataset (t(10) < 1). Thus, it is likely that activity differences between the two conditions reflected a qualitative difference in processing rather than simply the amount of maintenance processing that was engaged.
Although differences between the two trial types could not be reduced to simple differences in task difficulty or memory load, it is likely that DPA trials required additional processing beyond what was engaged on DMS trials. One salient difference was that DPA trials required retrieval of the relationship between the cue object and its associate. Results from studies of human and nonhuman primates suggest that the hippocampus may be critical for forming and retrieving memories for arbitrary relationships (Petrides, 1985; Kroll et al., 1996; Henke et al., 1999; Stark and Squire, 2001; Cansino et al., 2002; Brasted et al., 2003; Davachi et al., 2003; Duzel et al., 2003; Ranganath et al., 2003b; Wirth et al., 2003; Bunge et al., 2004; Preston et al., 2004). In contrast, the recruitment of anterior PFC may reflect the fact that subjects were required to use the recalled relationship between the shown cue item and its associate to “switch” the item that was to be actively maintained (recall that on DPA trials, inferior temporal activation initially reflected the cue item but then switched to the associate during the delay). This explanation fits with recent functional characterizations of the anterior PFC that emphasize its role in using higher-order task representations to direct the selection and maintenance of relevant information in working memory (Koechlin et al., 1999; Braver et al., 2003; Bunge et al., 2003; Sakai and Passingham, 2003; Ramnani and Owen, 2004).
In conclusion, results showed that activation of object representations, either through WM maintenance or associative LTM retrieval, was supported by sustained activity within category-specific inferior temporal subregions and within lateral PFC subregions. Furthermore, the hippocampus and anterior PFC were disproportionately recruited during associative LTM retrieval, suggesting that these regions may direct activation of associative memory codes to guide behavior (Miyashita and Hayashi, 2000).
This research was supported by grants from the McDonnell-Pew Program in Cognitive Neuroscience, National Institutes of Health (AG05863, AG15793, AG09253, MH63901, NS40813), University of California at Davis Alzheimer's Disease Center, and University of California at Davis Academic Senate. We thank Sarah Trefethen and Susan Landau for assistance and Silvia Bunge, Clayton Curtis, Adam Gazzaley, Wendy Suzuki, Ewa Wojciulik, and two anonymous reviewers for helpful suggestions on previous versions of this manuscript.
Correspondence should be addressed to Dr. Charan Ranganath, University of California at Davis Center for Neuroscience, 1544 Newton Court, Davis, CA 95616. E-mail:.
Copyright © 2004 Society for Neuroscience 0270-6474/04/243917-09$15.00/0