Abstract
Humans encounter and form memories for multiple types of experiences that differ in content, novelty, and memorability. Critical for understanding memory is determining (1) how the brain supports the encoding of events with differing content and (2) whether neural regions that are sensitive to novelty also influence whether stimuli will be subsequently remembered. This event-related functional magnetic resonance imaging (fMRI) study crossed content (picture/word), novelty (novel/repeated), and subsequent memory (remembered/forgotten) to examine prefrontal and temporal lobe contributions to encoding. Results revealed three patterns of encoding-related activation in anatomically connected inferior prefrontal and lateral temporal structures that appeared to vary depending on whether visuospatial/visuo-object, phonological/lexical, or semantic attributes were processed. Event content also modulated medial temporal lobe activity; word encoding predominately activated the left hemisphere, whereas picture encoding activated both hemispheres. Critically, in prefrontal and temporal regions that were modulated by novelty, the magnitude of encoding activation also predicted whether an event would be subsequently remembered. These results suggest that (1) regions that demonstrate a sensitivity to novelty may actively support encoding processes that impact subsequent explicit memory and (2) multiple content-dependent prefrontal–temporal circuits support event encoding. The similarities between prefrontal and lateral temporal encoding responses raise the possibility that prefrontal modulation of posterior cortical representations is central to encoding.
- declarative memory
- explicit memory
- fMRI
- neuroimaging
- human memory
- prefrontal cortex
- medial temporal lobe
- parahippocampal gyrus
- hippocampus
Recent neuroimaging studies have consistently demonstrated that the prefrontal cortices and medial temporal lobes (MTLs) are active during episodic encoding (Nyberg et al., 1996; Lepage et al., 1998; Buckner et al., 1999; Schacter and Wagner, 1999; Wagner, 1999). One factor affecting the nature of prefrontal encoding may be the content of the to-be-encoded information. Encoding verbal information preferentially activates the left inferior prefrontal cortex (LIPC), whereas encoding nonverbal information preferentially activates the right inferior prefrontal cortex (RIPC) (Kelley et al., 1998; Wagner et al., 1998a). Dissociations within the LIPC suggest that semantic and phonological feature processing activates the anterior LIPC [Brodmann's area (BA) 45/47] and the posterior LIPC (BA 44/6), respectively (Buckner et al., 1995; Fiez, 1997; Poldrack et al., 1999).
The content of experiences also appears to affect the nature of MTL encoding-related activation. Word encoding predominately activates the left MTL, scene and object encoding activate the MTL bilaterally, and unfamiliar face encoding predominately activates the right MTL (Stern et al., 1996; Gabrieli et al., 1997; Martin et al., 1997; Brewer et al., 1998; Kelley et al., 1998; Wagner et al., 1998b).
As in inferior prefrontal cortex (Raichle et al., 1994; Demb et al., 1995; Gabrieli et al., 1996; Wagner et al., 1997; Schacter and Buckner, 1998), the relative novelty of the experience modulates the magnitude of MTL activation. Activation is greater when stimuli are viewed for the first time relative to when they have been seen multiple times (Stern et al., 1996; Gabrieli et al., 1997). Interpretations of these differential MTL responses across relative novelty include that they reflect responses to novelty detection or stimulus priming. However, recent functional magnetic resonance imaging (fMRI) findings indicate that the magnitude of MTL and prefrontal activation during the encoding of an experience predicts subsequent memory for that experience, even when the novelty of the subsequently remembered and forgotten stimuli is held constant (Brewer et al., 1998; Fernandez et al., 1998; Wagner et al., 1998b).
Although previous reports suggest that the encoding of different stimulus features depends on distinct MTL and prefrontal regions, critical questions remain. First, do these regions interact with additional posterior brain structures representing the specific features being encoded into memory? Second, to the extent that novelty-related activity may reveal regions contributing to encoding (Knight, 1996; Tulving et al., 1996), do regions demonstrating a differential response during the encoding of novel stimuli also predict subsequent memory? To address these questions, an event-related fMRI study of the encoding of pictures and words was conducted. Stimulus type (picture/word) was crossed with novelty (novel/repeated) and subsequent memory (remembered/forgotten). Novelty has been defined previously in a number of ways (Martin et al., 1997). Here novel encoding consisted of studying items not encountered previously in the experimental context, whereas repeated encoding consisted of repeatedly studying the same two pictures or words during the experiment [similar to Stern et al. (1996); Gabrieli et al. (1997)]. As in previous studies, novelty was held constant when considering regions that predict subsequent memory (Brewer et al., 1998; Fernandez et al., 1998;Wagner et al., 1998b).
MATERIALS AND METHODS
Subjects. Nine right-handed native English-speaking volunteers participated in this study (seven men; two women; age range, 18–34 years). All subjects had normal or corrected-to-normal vision. Informed consent was obtained in a manner approved by the Human Studies Committee of Massachusetts General Hospital.
Tasks. Stimuli were complex color pictures of indoor or outdoor scenes, four- to eight-letter words naming indoor or outdoor objects, and fixation crosses. There were five trial types: novel pictures, repeated pictures, novel words, repeated words, and fixation. Each of 10 functional scans consisted of 100 stimulus presentations (20 presentations of each of the five trial types), for a total of 1000 stimulus presentations per subject. Participants saw a unique picture or word during each novel stimulus trial. In the repeated stimulus trials, the subjects saw one of two repeating pictures or one of two repeating words (each of these exemplars was seen 100 times across the 10 functional scanning runs). A new trial occurred every 2 sec. The five trial types were pseudorandomly intermixed with counterbalancing such that each trial type followed every other trial type equally often. For picture and word trials, subjects performed an incidental encoding task in which they made one of two left-handed key press responses to indicate whether the item represented an indoor or outdoor entity. Before the first functional run, participants made indoor or outdoor decisions for each of the repeated picture and word stimuli a total of 10 times during two practice sequences to ensure that these stimuli were familiar and that the participants understood task requirements. This design results in the novel and the repeated conditions for each stimulus class (picture/word) differing in situational novelty (novel vs repeated) as well as in the number of unique items contributing to the condition (200 vs 2).
A surprise recognition memory test was administered outside of the scanner 15–20 min after the scanning session. During this test, a random sequence of 400 pictures and 400 words was presented; half were the novel items seen during the functional encoding scans, and half were new, unstudied items. Subjects indicated whether they recognized the test item as having been studied as well as their confidence by manually responding “high-confidence studied,” “low-confidence studied,” or “new.”
Data acquisition. Anatomical and functional data were acquired on a 3.0T GE Signa scanner with an ANMR upgrade. Structural data were acquired using a T1-weighted rf-spoiled GRASS sequence (60 sagittal slices; 2.8 mm slice thickness). Functional data were acquired using a whole-brain echoplanar T2*-weighted gradient echo sequence (TR = 2 sec; TE = 30 msec; flip angle = 60°; 16 AC-PC axial slices with a 1 mm skip between slices; 3.125 × 3.125 × 7 mm resolution; 108 images per slice). The duration of each functional run was 3 min and 44 sec. Ten runs were acquired per subject.
Data analysis. Only trials in which the subjects made the correct indoor or outdoor designation (96.9%) were analyzed. Functional data were selectively averaged within subjects (data from one run for one subject were not included because of scanner malfunction). Data were then transformed into Talairach space (Talairach and Tournoux, 1988) and averaged across subjects. The procedures for selective averaging and statistical map generation are described elsewhere (Dale and Buckner, 1997; Buckner et al., 1998). Briefly, voxel-based statistical activation maps were constructed on the basis of the differences between trial types using a tstatistic. Clusters of five or more voxels exceeding a statistical threshold of p < 0.001 were considered significant foci of activation.
Repeated-measures mixed-effect ANOVAs, which treated subjects as a random effect, were conducted to further examine the effects of novelty, stimulus type, hemisphere, and subsequent memory on the percent signal change relative to the fixation baseline in frontal, lateral temporal, and medial temporal regions of interest (ROIs). ROIs were identified using an automated algorithm that identified all contiguous voxels within 10 mm of a peak activation (Buckner et al., 1998; Wagner et al., 1998b). ROIs were identified from the novel stimulus comparison (all novel > all repeated) if possible when effects of novelty were examined because this contrast is unbiased with respect to examining the effects of stimulus type (picture/word), hemisphere (left/right), and subsequent memory (high-confidence remembered/forgotten) (Fig.1a–e). Left lateral temporal and left anterior fusiform ROIs were defined from the novel word encoding (novel > repeated words) and word versus picture encoding (novel word > novel picture) comparisons, respectively, because these contrasts revealed that these regions are separable (Fig.1f,g). ROIs were also derived from a picture subsequent memory comparison (high-confidence remembered > forgotten picture trials) to explore subsequent memory effects in the word condition and the relation of these effects to novelty effects. The peak of the hemodynamic response (typically occurring between 4 and 8 sec after stimulus onset) was identified for each ROI in which the effects of novelty, stimulus type, and hemisphere were examined. The peak was identified when collapsing across subjects and trial types, and the percent signal change at the peak for each trial type (relative to the fixation baseline) was determined on a subject-by-subject basis and submitted to a mixed-effect ANOVA (the significance criterion wasp < 0.05, Bonferroni corrected). For subsequent memory ROI analyses, this procedure was modified slightly because only a modest number of word trials were subsequently forgotten (see Results), resulting in more variability in the peak of the hemodynamic response for this condition across subjects. Thus, for ROI-based investigations of subsequent memory, the summed percent signal change across the time window corresponding to 4–8 sec after stimulus onset was determined for each trial type and subjected to a mixed-effect ANOVA.
Inferior prefrontal, fusiform, and left lateral temporal encoding responses. Columns A–E, Activity during novel stimulus (novel > repeated pictures + words), novel picture (novel > repeated), picture versus word (novel picture > novel word), novel word (novel > repeated), and word versus picture (novel word > novel picture) encoding, respectively. Regions of interest were defined from the novel stimulus (a–e), novel word (f), and word versus picture (g) comparisons. a, Anterior LIPC: −37, 25, 3; BA 45/47. b, Posterior LIPC: −34, 6, 34; BA 6/44. c, Posterior RIPC: 43, 3, 37; BA 6/44. d, Left posterior fusiform: −28, −55, −9; BA 37. e, Right posterior fusiform: 34, −49, −12; BA 37. f, Left lateral temporal cortex: −56, −43, 3; BA 21/22. g, Left anterior fusiform: −40, −43, −6; BA 37. L, Left; R, right.
RESULTS
Behavioral results
Classification accuracy on the indoor or outdoor discrimination during scanning was similar for novel pictures (95.2%) and words (95.5%; F < 1). However, participants took longer to make the indoor or outdoor judgment for novel words (878 msec) than for novel pictures [811 msec; F(1,8) = 8.11; p < 0.05]. Behavioral performance on the postscan memory test revealed that corrected recognition memory (hits − false alarms) was better for the words (58.2%) than for the pictures [30.7%; F(1,8) = 27.58;p < 0.05]. Using participants' responses on this postscan memory test, the encoding trials were then sorted on the basis of subsequent memory into those that were later remembered with high confidence and those that were later forgotten. Participants remembered an average of 70 pictures and 139 words with high confidence and forgot an average of 78 pictures and 40 words out of 200 words and 200 pictures. The reaction times to make the indoor or outdoor decision did not differ between high-confidence remembered (797 msec) and forgotten (816 msec) pictures (F < 1) or between high-confidence remembered (846 msec) and forgotten (837 msec) words (F< 1). Thus, any observed neural subsequent memory effects likely do not reflect differences in stimulus processing time during encoding.
Novelty analyses
A series of voxel-based analyses was conducted on the fMRI encoding data that consisted of the following comparisons: (1) novel stimulus encoding (novel pictures and words > repeated pictures and words), (2) novel picture encoding (novel > repeated), and (3) novel word encoding (novel > repeated). Repeated-measures mixed-effect ANOVAs were then conducted to examine further the effects of novelty, stimulus type, and hemisphere on the percent signal change in frontal, lateral temporal, and medial temporal ROIs.
The voxel-based analyses revealed effects of novelty in multiple neural regions. Novel stimulus encoding (collapsed across words and pictures) was associated with activation in the anterior and ventral extent of the left inferior prefrontal cortex (anterior LIPC) and bilateral activation in several regions, including the posterior and dorsal extent of the left and right inferior prefrontal cortex (posterior LIPC and RIPC, extending into the corresponding premotor cortex), dorsolateral prefrontal cortex (DLPC), posterior hippocampus, posterior parahippocampal gyrus, fusiform gyrus, lingual gyrus, posterior cingulate, and intraparietal sulcus. Novel picture encoding was associated with activation in these same regions, with the exception of the anterior LIPC. Novel word encoding was associated with activation in the anterior LIPC, posterior LIPC and RIPC, bilateral DLPC, bilateral supplementary motor area, bilateral posterior hippocampus, bilateral posterior parahippocampal gyrus, left lateral temporal cortex, bilateral fusiform gyrus, bilateral posterior cingulate, left intraparietal sulcus, and right cerebellum (a complete list of Talairach coordinates is available on request).
Hypothesis-driven ROI analyses were conducted on prefrontal, lateral temporal, and medial temporal regions observed to be associated with novel stimulus encoding in the voxel-based comparison. These analyses revealed three encoding-related patterns of activity in inferior prefrontal and lateral temporal regions: the posterior RIPC and bilateral posterior fusiform demonstrated picture preferential encoding effects, the posterior LIPC and left anterior fusiform demonstrated word preferential encoding effects, and the anterior LIPC and left lateral temporal cortex demonstrated word specific encoding effects (see Table 1). Content sensitive encoding activity was also observed in the medial temporal lobe.
Summary of the prefrontal-lateral temporal encoding patterns
Picture and word preferential encoding patterns: prefrontal–fusiform cortices
Figure 1 illustrates the posterior LIPC, posterior RIPC, and fusiform regions that were engaged during novel stimulus encoding, and Figure 2 reveals the percent signal change time courses from these regions. Both the posterior LIPC and posterior RIPC (Figs. 1, 2, regions b, c) demonstrated significant novelty responses [LIPC,F(1,8) = 16.51; RIPC,F(1,8) = 16.80] and novelty effects for both pictures [LIPC, F(1,8) = 27.78; RIPC, F(1,8) = 68.53] and words [LIPC, F(1,8) = 73.77; RIPC,F(1,8) = 16.30]. There were trends for a main effect of stimulus type [LIPC,F(1,8) = 9.07; p < 0.02 uncorrected; RIPC, F(1,8) = 6.60;p < 0.04 uncorrected], with words eliciting a greater response in the posterior LIPC and pictures eliciting a greater response in the posterior RIPC. There also were trends for a stimulus × novelty interaction in both the posterior LIPC [F(1,8) = 5.51; p < 0.05 uncorrected] and posterior RIPC [F(1,8) = 9.00; p < 0.02 uncorrected], revealing a greater novelty effect for words in the posterior LIPC and a greater novelty effect for pictures in the posterior RIPC.
Percent signal change time courses from the inferior prefrontal, fusiform, and left lateral temporal regions of interest shown in Figure 1 and labeled a–g. The posterior RIPC (c) and bilateral posterior fusiform regions (d, e) demonstrated picture preferential encoding responses, the posterior LIPC (b) and left anterior fusiform (g) demonstrated word preferential encoding responses, and the anterior LIPC (a) and left lateral temporal cortex (f) demonstrated word specific encoding responses.
To examine the effect of hemisphere, an additional analysis including hemisphere as a factor was conducted on the posterior LIPC and RIPC regions of interest. A hemisphere × stimulus interaction [F(1,8) = 18.51] revealed that although activation in the posterior LIPC was greater for words than for pictures [F(1,8) = 13.92], activation in the posterior RIPC was greater for pictures than for words [F(1,8) = 5.54]. Moreover, the posterior LIPC demonstrated a greater response than did the posterior RIPC during word encoding [F(1,8) = 19.87], whereas both regions were similarly active during picture encoding [F(1,8) = 2.65; NS]. The hemisphere × stimulus × novelty interaction was significant [F(1,8) = 12.46]. Comparing the difference in the percent signal change between novel and repeated pictures and words indicated a greater word novelty response in the posterior LIPC [F(1,8) = 8.26] and a trend for a greater picture novelty response in the posterior RIPC [F(1,8) = 4.43;p < 0.07].
Three regions in the fusiform cortex were also influenced by novelty and stimulus type. Bilateral posterior fusiform regions (Figs. 1, 2,regions d, e) revealed novelty responses [left, F(1,8) = 31.86; right, F(1,8) = 26.44], with greater responses to novel stimuli being observed both for pictures [left,F(1,8) = 70.76; right,F(1,8) = 60.83] and words [left,F(1,8) = 5.83; right,F(1,8) = 4.42; p < 0.07]. Both regions demonstrated a main effect of stimulus type [left, F(1,8) = 18.65; right,F(1,8) = 48.72]. As was observed in the posterior RIPC, both the left and right posterior fusiform demonstrated greater activation during picture relative to word encoding. In addition, a stimulus × novelty interaction was observed in both posterior fusiform regions [left,F(1,8) = 17.99; right,F(1,8) = 16.22], which reflected a greater novelty response for pictures than for words in both hemispheres. When including hemisphere as a factor in the analysis, a trend for a hemisphere × stimulus × novelty interaction was observed [F(1,8) = 5.67;p < 0.05 uncorrected]. Comparing the difference in the percent signal change between novel and repeated pictures and words indicated that although the word novelty response was similar in left and right posterior fusiform regions [F(1,8) = 1.01; NS], there was a greater picture novelty response in the right relative to the left posterior fusiform cortex [F(1,8) = 19.05].
In contrast to posterior fusiform regions, a left anterior fusiform region demonstrated a pattern similar to that observed in the posterior LIPC (Figs. 1, 2, region g). Specifically, this region revealed a main effect of stimulus type [F(1,8) = 11.67] such that activation in this region was greater during word relative to picture encoding. Moreover, there was an effect of novelty [F(1,8) = 30.16], and the stimulus × novelty interaction was not reliable [F(1,8) = 1.21; NS], indicating that novelty effects were observed for both pictures and words.
Word specific encoding pattern: prefrontal–lateral temporal cortices
A third pattern of activation was observed in the anterior LIPC and left lateral temporal cortex (Figs. 1, 2,regions a, f). Both regions revealed a main effect of stimulus type [anterior LIPC,F(1,8) = 18.68; lateral temporal cortex, F(1,8) = 22.45], with activation being greater during word relative to picture encoding. Moreover, there was a greater response during novel relative to repeated stimuli [anterior LIPC,F(1,8) = 56.89; lateral temporal cortex, F(1,8) = 7.11;p < 0.03 uncorrected]. Significant stimulus × novelty interactions [anterior LIPC,F(1,8) = 13.96; lateral temporal cortex, F(1,8) = 12.30] revealed novelty effects selective for words [anterior LIPC,F(1,8) = 32.77; lateral temporal,F(1,8) = 13.92] with reliable novelty effects not being observed for pictures [anterior LIPC,F < 1; lateral temporal,F(1,8) = 1.51; NS]. As Figure 2illustrates, activation in these regions was not above baseline for the picture trials revealing that these regions selectively responded during word encoding.
Content sensitive encoding activity: medial temporal lobe
Figure 3 illustrates medial temporal lobe regions that demonstrated novelty responses. Both picture and word novelty responses were observed bilaterally in the posterior hippocampal region and parahippocampal and fusiform gyri. Voxels demonstrating a word novelty response overlapped with those revealing a picture novelty response. ROI analyses of bilateral parahippocampal/fusiform regions demonstrated a significant novelty response in both regions [left,F(1,8) = 27.30; right,F(1,8) = 22.81]. The main effect of stimulus type was significant in both hemispheres [left,F(1,8) = 92.27; right,F(1,8) = 64.18], with both regions demonstrating a greater response during picture encoding. The left ROI failed to reveal a reliable stimulus × novelty interaction [F(1,8) = 3.01; NS], whereas there was a trend for an interaction in the right ROI [F(1,8) = 10.46; p < 0.02 uncorrected]. These effects reflect the presence of a picture novelty effect in both hemispheres [left,F(1,8) = 29.28; right,F(1,8) = 38.66] and a word novelty effect only in the left hemisphere [left,F(1,8) = 8.75; right,F(1,8) = 2.71; NS]. When hemisphere was included as a factor in the analysis, there was a trend for a region × stimulus interaction [F(1,8) = 4.74; p < 0.07 uncorrected]. Although picture encoding did not differentially activate the two hemispheres (F < 1), word encoding resulted in more activation in the left than in the right parahippocampal/ fusiform cortex [F(1,8) = 14.42].
Parahippocampal/fusiform regions engaged in novel stimulus encoding. Columns A–C represent novel stimulus, novel picture, and novel word comparisons, respectively. To compare the percent signal change responses to novel pictures and novel words in similar regions in both hemispheres, we used the coordinates from the left parahippocampal/ fusiform ROI in the novel stimulus condition to grow a region in the right hemisphere. a(−21, −40, −9; BA 35/36/37) and b (21, −40, −9; BA 35/36/37) reflect the percent signal change time courses from these ROIs. Picture and word encoding activated overlapping regions of the posterior MTL. Novel pictures, solid white line withwhite squares; repeated pictures, dashed white line with white squares; novel words,solid yellow line with yellow squares; repeated words, dashed yellow line with yellow squares.
Subsequent memory analyses
Having identified regions demonstrating novelty effects, we then turned our attention to determining which regions predicted subsequent memory performance as indexed by the postscan recognition memory test. That is, we sought to determine whether the magnitude of activation during encoding differed for items later remembered relative to those later forgotten. The voxel-based subsequent memory comparison (high-confidence remembered > forgotten trials) for pictures revealed several brain regions that were more active during the encoding of subsequently remembered than during the encoding of subsequently forgotten trials, including the right hippocampus, right parahippocampal and fusiform gyri, left parahippocampal and fusiform gyri extending into the hippocampus, bilateral posterior cingulate gyrus, and right intraparietal sulcus. Although the voxel-based statistical threshold used in this study was p = 0.001, lowering our statistical threshold to p = 0.05 revealed activation in the posterior RIPC, which is consistent with previous findings (Brewer et al., 1998). Stereotaxic localizations of the picture MTL subsequent memory responses were similar to previously reported subsequent memory responses for pictures (Brewer et al., 1998) and for words (Wagner et al., 1998b). The voxels demonstrating a picture subsequent memory response in the right hippocampus, left parahippocampal/fusiform region extending into the hippocampus, and right parahippocampal/fusiform region overlapped with MTL voxels revealing picture and word novelty responses (see Figs. 3, 4). The voxel-based subsequent memory analysis for the words did not reveal any prefrontal or medial temporal regions that met our significance criteria. This null result likely reflects a lack of power in this comparison because due to the fact that there were relatively few forgotten word trials (participants forgot 20.3% of the words compared with 39.4% of the pictures).
Regions that demonstrated subsequent memory responses also revealed novelty responses
To explore further possible subsequent memory effects for words in view of Wagner et al.'s (1998b) findings of word subsequent memory effects (see also Alkire et al., 1998; Henson et al., 1999; Petersson et al., 1999; Wagner et al., 1999), as well as to investigate the relationship between subsequent memory effects and novelty effects, ROI analyses were conducted on MTL regions demonstrating a voxel-based picture subsequent memory effect. These analyses revealed that MTL regions that demonstrated picture subsequent memory effects also revealed picture and word novelty responses and trends for word subsequent memory effects (Fig. 4). The right hippocampus (Fig. 4, region a) demonstrated a main effect of subsequent memory [F(1,8) = 11.85], with the absence of a stimulus × subsequent memory interaction (F< 1). An ANOVA examining memory effects for words alone revealed a trend for a subsequent memory response [F(1,8) = 4.67; p < 0.07 uncorrected], which suggests that both picture and word subsequent memory effects were present in this region. Considering the effects of novelty in this region, we observed a main effect of novelty [F(1,8) = 14.59] and a trend toward a reliable stimulus × novelty interaction [F(1,8) = 4.89; p < 0.06 uncorrected], suggesting that there was a greater novelty response for pictures [F(1,8) = 53.50] than for words [F(1,8) = 17.59].
Medial temporal regions that were more active during the encoding of pictures that were subsequently remembered than during the encoding of pictures that were subsequently forgotten.a, 28, −30, −6; hippocampus. b, −28, −37, −9; BA35/36/37. c, 31, −40, −6; BA 35/36/37. Graphs a–c, Subsequent memory percent signal change time courses. Graphs a1 –c1, Novelty percent signal change time courses for the same ROIs depicted in a–c, respectively. Medial temporal regions that demonstrated picture subsequent memory effects also revealed picture and word novelty responses and trends for word subsequent memory effects. Graphs a–c, Remembered pictures, solid white line withwhite squares; forgotten pictures, dashed white line with white squares; remembered words,solid yellow line with yellow squares; forgotten words, dashed yellow line with yellow squares. Graphs a1 –c1, Novel pictures, solid white line with white squares; repeated pictures, dashed white linewith white squares; novel words, solid yellow line with yellow squares; repeated words,dashed yellow line with yellow squares.
The left parahippocampal/fusiform and right parahippocampal/ fusiform regions (Fig. 4, regions b, c) observed in the voxel-based picture subsequent memory analysis also revealed main effects of subsequent memory in ROI analyses [left,F(1,8) = 30.82; right,F(1,8) = 11.42]. The stimulus × subsequent memory interactions were not reliable for either region [left, F(1,8) = 1.40; NS; right,F(1,8) = 1.24; NS]. ANOVAs examining subsequent memory effects for words alone revealed a trend for a word subsequent memory effect in the left parahippocampus/fusiform [F(1,8) = 6.49; p < 0.04 uncorrected] but no word subsequent memory effect in the right parahippocampus/fusiform [F(1,8) = 2.59; NS]. The hemisphere × stimulus × memory interaction was not significant (F < 1). Considering the effects of novelty in these regions, analyses revealed main effects of novelty in both areas [left, F(1,8) = 34.69; right, F(1,8) = 50.80], a trend for a reliable stimulus × novelty interaction in the left parahippocampus/fusiform [F(1,8) = 9.80; p < 0.02 uncorrected], and a significant stimulus × novelty interaction in the right parahippocampus/fusiform [F(1,8) = 17.54]. These results suggest that novelty effects were greater for pictures [left, F(1,8) = 82.67; right, F(1,8) = 83.53] than for words [left, F(1,8) = 21.76; right,F(1,8) = 10.35] in these regions.
Regions that demonstrated novelty responses also revealed subsequent memory responses
Finally, to examine further the relationship between regions demonstrating novelty effects and those demonstrating subsequent memory effects, we examined the presence of subsequent memory effects in the prefrontal and temporal ROIs observed to demonstrate novelty responses. In general, those regions demonstrating novelty responses were also observed to demonstrate subsequent memory effects. Specifically, the posterior LIPC and posterior RIPC revealed trends for subsequent memory effects [LIPC, F(1,8) = 8.23;p < 0.03 uncorrected; RIPC,F(1,8) = 4.69; p < 0.07 uncorrected]. Both left and right posterior fusiform regions [left, F(1,8) = 11.65; right,F(1,8) = 12.78] and the left anterior fusiform [F(1,8) = 13.04] demonstrated reliable subsequent memory effects, as did bilateral parahippocampal/fusiform regions [left,F(1,8) = 35.59; right,F(1,8) = 27.21]. None of these regions revealed significant stimulus × subsequent memory interactions. It should be noted that the absence of stimulus × subsequent memory interactions should be interpreted cautiously because the ability to detect content-sensitive subsequent memory responses likely was limited by the modest number of forgotten word trials. The left anterior LIPC revealed a trend for a subsequent memory effect [F(1,8) = 5.69; p < 0.05 uncorrected] and a trend for a stimulus × subsequent memory interaction [F(1,8) = 4.67;p < 0.07 uncorrected]. This latter effect reflected the presence of a subsequent memory effect in this region for words [F(1,8) = 10.33] but not for pictures (F < 1). Finally, in contrast to all the other regions that had significant novelty effects in which we explored subsequent memory responses, the left lateral temporal lobe did not reveal a reliable main effect of subsequent memory [F(1,8) = 1.44; NS]. At present, it is unclear whether the absence of a subsequent memory effect in this region reflects a lack of sensitivity/power or meaningful functional differences between the processes mediated by this region and those subserved by other prefrontal and temporal regions.
DISCUSSION
The present results forward our understanding of prefrontal and temporal contributions to encoding by (1) clearly demonstrating the presence of at least three patterns of encoding-related activation in prefrontal and temporal regions that depend in part on event content and (2) revealing that the same prefrontal, medial temporal, and lateral temporal regions that are sensitive to stimulus novelty also predict subsequent explicit memory, suggesting that these regions actively contribute to encoding.
Posterior RIPC and bilateral posterior fusiform
Consistent with previous reports (Kelley et al., 1998; Wagner et al., 1998a), the posterior RIPC (BA 6/44) responded preferentially to picture encoding (Table 1). This observation builds on the proposal that the inferior prefrontal cortex maintains items in working memory (Petrides, 1994; Owen et al., 1996) and converges with the hypothesis that the RIPC mediates control processes supporting access to and maintenance of visuospatial and visuo-object representations (Awh and Jonides, 1998; Wagner, 1999). Importantly, the present study extends these results by demonstrating a qualitatively similar encoding pattern in posterior fusiform regions (BA 37) (Table 1). Other neuroimaging studies have revealed posterior fusiform activation during the processing of visual stimuli, such as scenes (Stern et al., 1996;Gabrieli et al., 1997), faces (Haxby et al., 1994; Kanwisher et al., 1997), objects (Martin et al., 1996; Chao et al., 1999; Ishai et al., 1999), and words (Bookheimer et al., 1995; Chao et al., 1999), with activation magnitude varying with stimulus novelty (Stern et al., 1996;Buckner et al., 1998). In humans, damage to lingual and fusiform gyri results in visual agnosia and prosapagnosia (Meadows, 1974; Damasio et al., 1990), underscoring the importance of this region for object recognition. Functionally and anatomically, the lingual and fusiform gyri in humans may be analogous to the inferotemporal cortex in monkeys, which plays an important role in object discrimination, recognition, and memory processes (Iwai and Mishkin, 1969; Mishkin et al., 1983; Desimone, 1996). The present observation of subsequent memory effects in posterior RIPC and posterior fusiform regions suggests that the operations mediated by these regions impact the efficacy of encoding visuo-object representations.
Posterior LIPC and left anterior fusiform
A different encoding pattern was observed in the posterior LIPC (BA 6/44) and left anterior fusiform (Table 1), with these regions preferentially responding to word encoding (Kelley et al., 1998; Wagner et al., 1998a). These results converge with neuroimaging evidence, suggesting that the posterior LIPC supports access to and maintenance of phonological codes (Paulesu et al., 1993; Awh et al., 1996; Fiez, 1997; Fiez and Petersen, 1998; Poldrack et al., 1999), and extend these previous findings by revealing a qualitatively similar pattern in the left anterior fusiform cortex. A similar fusiform/inferior temporal region has been hypothesized to play a role in lexical/phonological retrieval (Price and Friston, 1997; Brunswick et al., 1999; Mummery et al., 1999), having been observed to be engaged during (1) word, object, letter, and color naming (Bookheimer et al., 1995; Price and Friston, 1997), (2) reading tasks not requiring overt naming (Price et al., 1996; Buchel et al., 1998), and (3) naming of tactile and auditory stimuli (Buchel et al., 1998; Foundas et al., 1998). Moreover, lesions to the left BA 37 can yield anomia without semantic comprehension deficits (Raymer et al., 1997; Foundas et al., 1998). The present results suggest that the posterior LIPC and left anterior fusiform regions mediate phonological/lexical processes involved in word reading and picture naming. The present subsequent memory effects indicate that these processes may impact the efficacy of encoding of phonological representations.
Neuroanatomical studies in nonhuman primates have demonstrated strong connections between the ventrolateral prefrontal cortex and inferotemporal regions (Barbas, 1988; Felleman and Van Essen, 1991), and electrophysiological and cortical cooling studies suggest that the prefrontal cortex modulates inferotemporal processes (Fuster et al., 1985; Tomita et al., 1999). The present observations that fusiform activation is related to event content and subsequent explicit memory raise the possibility that fusiform activation partially derives from top-down modulation from the posterior RIPC and LIPC, and that it contributes to the encoding of object and phonological/lexical features into episodic memory. However, additional evidence regarding the temporal dynamics and interactions across frontal–temporal regions is necessary to further explore prefrontal–fusiform functional interactions.
Anterior LIPC and left lateral temporal lobe
A third qualitatively different encoding pattern was observed in the anterior LIPC and left lateral temporal cortex. The anterior LIPC has been observed to predict subsequent memory for words (Wagner et al., 1998b). This result was extended by demonstrating that both the anterior LIPC (BA 45/47) and left lateral temporal cortex (BA 21/22) were associated with word specific encoding responses, with the anterior LIPC also revealing a trend for word specific subsequent memory effects (Table 1). Primate anatomical studies have demonstrated that the ventrolateral prefrontal cortex is connected with the lateral temporal cortex (Petrides and Pandya, 1988; Felleman and Van Essen, 1991). Although the posterior LIPC appears to be more active during phonologically biased tasks, the anterior LIPC appears to be more active during semantically biased processing (Fiez, 1997; Poldrack et al., 1999) and thus is hypothesized to contribute to the accessing of and working with long-term semantic knowledge (Kapur et al., 1994; Demb et al., 1995; Fiez, 1997; Wagner, 1999). Within the lateral temporal lobe, BA 21/22 has been shown to be active during tasks that require the semantic processing of words (Fiez et al., 1996;Vandenberghe et al., 1996; Price et al., 1997) and inanimate object information (Martin et al., 1996; Mummery et al., 1998; Chao et al., 1999). Patients with damage in this region have semantic processing deficits including semantic dementia (Hart and Gordon, 1990; Hodges et al., 1992). Interactions between the anterior LIPC and lateral temporal cortex may support the encoding of abstract semantic attributes into memory, although at present it is unclear why the lateral temporal cortex did not predict subsequent memory.
Medial temporal lobe
Within the MTL, posterior hippocampal and parahippocampal regions were recruited during both picture and word encoding, with the magnitude of MTL activity during encoding being observed to predict subsequent memory (Brewer et al., 1998; Fernandez et al., 1998, 1999;Grunwald et al., 1998; Wagner et al., 1998b). The finding of posterior hippocampal/MTL activation complements previous neuroimaging evidence of posterior MTL involvement in encoding processes. Consistent with these previous studies, word encoding was associated with greater left MTL responses, whereas picture encoding activated both hemispheres of the MTL equivalently (Stern et al., 1996; Dolan and Fletcher, 1997;Gabrieli et al., 1997; Martin et al., 1997; Kelley et al., 1998). Thus, the two hemispheres of the MTL differentially contribute to the encoding of events as a function of their content. Moreover MTL voxels activated by word encoding overlapped with those activated by picture encoding, suggesting that the encoding of ostensibly verbal and nonverbal stimuli rely on similar MTL regions within a given hemisphere. This similarity may reflect the ability of participants to access visuo-object codes associated with words and phonological codes associated with pictures.
Novelty and encoding
Prefrontal and MTL regions respond differentially to novel relative to repeated stimuli (Tulving et al., 1994; Knight, 1996; Stern et al., 1996; Schacter and Buckner, 1998). Some have posited that this differential response indicates that these regions mediate the detection of situationally novel events (Knight, 1996; Tulving et al., 1996; Dolan and Fletcher, 1997), with one consequence of novelty detection being the facilitated encoding of these novel experiences (Knight, 1996; Tulving et al., 1996). However, a link between regions demonstrating a sensitivity to novelty and subsequent explicit memory has not been established previously. The present results provide the first evidence that the same ventrolateral prefrontal and temporal regions that are sensitive to novelty also predict subsequent explicit memory, supporting the hypothesis that these regions contribute to encoding. However, it should also be emphasized that these regions likely do not simply mediate novelty detection, because they predicted subsequent memory even when situational novelty was held constant. The colocalization of novelty and subsequent memory effects indicates that the operations mediated by ventrolateral prefrontal and temporal regions, operations that appear to impact the efficacy of encoding, are engaged to a greater extent during novel events, perhaps yielding richer and more effective episodic traces for initial relative to repeated experiences.
Summary
In this study colocalized novelty and subsequent memory responses were observed in prefrontal, lateral, and medial temporal cortices, suggesting that these regions are actively engaged in encoding processes, with novelty perhaps facilitating encoding. Moreover, qualitatively similar encoding patterns were observed in anatomically connected regions within the human ventrolateral prefrontal and lateral temporal cortices. Three distinct prefrontal–temporal circuits likely differentially contribute to the encoding of visuospatial, phonological/lexical, and semantic stimulus attributes. Prefrontal regions may contribute to encoding by modulating the processing of these stimulus representations in lateral temporal cortices, with medial temporal regions mediating associations between stimulus representations and other episodic features to form long-term, flexible memory traces. Future research that integrates high spatial and temporal resolution approaches, including fMRI and MEG, will provide additional insight into the temporal organization and top-down/bottom-up interactions that occur along the frontal–temporal axis during encoding.
Footnotes
This work was supported by the National Science Foundation (B.A.K.), the Alzheimer's Association (C.E.S.), and the National Institute on Aging (Grant AG 05778; A.D.W.). We would like to thank Terry Campbell, Seth Sherman, Eena Khalil, and Doug Greve for their assistance with this study.
Correspondence should be addressed to Brenda A. Kirchhoff, Department of Psychology, Boston University, 64 Cummington Street, Boston, MA 02215. E-mail address: brendak{at}bu.edu.
