Abstract
Intense debate surrounds the role of medial temporal lobe (MTL) structures in recognition memory. Using high-resolution fMRI and analyses of pattern similarity in humans, we examined the encoding computations subserved by MTL subregions. Specifically, we tested the theory that MTL cortex supports memory by encoding overlapping representations, whereas hippocampus supports memory by encoding pattern-separated representations. Consistent with this view, the relationship between encoding pattern similarity and subsequent memory dissociated MTL cortex and hippocampus: later memory was predicted by greater across-item pattern similarity in perirhinal cortex and in parahippocampal cortex, but greater pattern distinctiveness in hippocampus. Additionally, by comparing neural patterns elicited by individual stimuli regardless of subsequent memory, we found that perirhinal cortex and parahippocampal cortex exhibited differential content sensitivity for multiple stimulus categories, whereas hippocampus failed to demonstrate content sensitivity. These data provide novel evidence that complementary MTL encoding computations subserve declarative memory.
Introduction
While known to be critical for declarative memory, the nature of medial temporal lobe (MTL) computations subserving recognition memory remains controversial. One class of theories posits that MTL cortex [MTLc; perirhinal cortex (PRc) and parahippocampal cortex (PHc)] and hippocampus are computationally distinct, differentially supporting item versus relational memory (Eichenbaum and Cohen, 2001; Hannula et al., 2006), memory for items versus multi-item or item-context conjunctions (Davachi et al., 2003; Norman and O'Reilly, 2003; Diana et al., 2007), or familiarity versus recollection (Brown and Aggleton, 2001; Yonelinas et al., 2002). Neuropsychological and functional MRI (fMRI) reports of hippocampal/MTLc dissociations supporting this broad perspective (for review, see Eichenbaum et al., 2007; but see Stark et al., 2002; Manns et al., 2003) have relied on behavioral measures of complementary forms of memory expression; however, whether these measurements successfully identify qualitatively distinct forms of memory has remained controversial (Squire et al., 2007; Wixted and Squire, 2011). Here, guided by computational theories, and without relying on behavioral differentiation between forms of recognition (e.g., item vs relational or conjunctive; familiarity vs recollection), we used fMRI and pattern similarity analysis to test the hypothesis that qualitatively distinct encoding mechanisms in MTLc and hippocampus support later recognition memory.
Complementary learning systems (CLS) theory posits that MTLc and hippocampus support complementary forms of learning and memory through distinct encoding and retrieval computations (McClelland et al., 1995; Norman and O'Reilly, 2003; Norman, 2010). MTLc is posited to build overlapping representations that encode commonalities across stimuli, allowing memory computations that are sensitive to degree of global match (similarity; e.g., Hintzman, 1984) between a current stimulus and previously encountered stimuli. By contrast, hippocampus is posited to build distinct, pattern-separated representations (e.g., O'Reilly and McClelland, 1994), allowing memory to be computed via pattern completion of distinct details surrounding a stimulus's past encounter.
The present study examined these complementary encoding computations (overlapping vs pattern-separated representations) in the human MTL. Using high-resolution fMRI data collected during incidental encoding, we assessed the relationship between across-item encoding pattern similarity and subsequent memory in PRc, PHc, and hippocampus. We hypothesized that across-item neural pattern similarity in MTLc would predict subsequent memory: high representational overlap with other studied items would result in high global match of that item at retrieval, thereby increasing the strength of the global-match-sensitive MTLc memory signal. Conversely, we hypothesized that across-item neural pattern distinctiveness in hippocampus would predict subsequent memory: low representational overlap with other studied items would increase the likelihood of successful hippocampal pattern completion of the distinct event features associated with a given item during retrieval, thereby increasing the likelihood of a diagnostic hippocampal memory signal.
In a separate analysis, we also examined visual category representations in MTLc and hippocampus regardless of memory outcome. Anatomical projections to PRc and PHc (Suzuki and Amaral, 1994) suggest that the two regions differentially represent complex visual objects versus visuospatial information; by contrast, hippocampus is posited to integrate such inputs (Davachi, 2006; Manns and Eichenbaum, 2006). Despite revealing differential content sensitivity, extant fMRI data also suggest that PRc and PHc code for multiple content types (Diana et al., 2008; Preston et al., 2010; Dudukovic et al., 2011; Liang et al., 2013). Here, we assessed MTLc and hippocampal content sensitivity for faces, bodies, scenes, and objects using representational similarity analysis (Kriegeskorte et al., 2008).
Materials and Methods
Participants.
Eighteen participants were recruited from the Stanford University community. One participant was excluded due to excessive motion, leaving 17 participants aged 18–38 years (mean, 25.35; SD, 6.14; 12 male). Participants gave informed consent in line with procedures approved by the Stanford Institutional Review Board, and received monetary compensation at a rate of $20/h.
Materials.
The stimuli were 96 images previously used in nonhuman primate (Kiani et al., 2007) and human (Kriegeskorte et al., 2008) studies of categorical neural responses in inferotemporal cortex (we thank N. Kriegeskorte and R. Kiani for sharing their stimuli). All images were color photographs centered on a gray background. Stimuli were subdivided into faces (12 animal, 12 human), body parts (12 animal, 12 humans), inanimate objects (38), and scenes (10).
Behavioral procedure.
During incidental encoding, participants viewed a series of 96 images while undergoing fMRI (Fig. 1). Each fMRI run consisted of the same 96 stimuli presented in a random order, as well as 48 null trials (4 lead-in, 4 lead-out, 40 interleaved). Within a run, each stimulus was presented for 300 ms with a 2500 ms intertrial interval (ITI). Each participant completed between 13 and 16 runs (mean, 15.6; SD, 0.87).
While viewing the stimuli, participants performed a fixation cross color-change detection task. Specifically, during the ITI, a white fixation cross was centered on the gray background; during each stimulus presentation, the fixation cross changed from white to either blue or green, and participants reported the color via button press. The order of fixation cross colors was randomized independently of stimulus order.
Immediately following the scan session, participants were given an unexpected recognition memory test consisting of the 96 studied stimuli and 96 lures drawn from the same categories as the studied items. For each item, participants were given unlimited time to make a recognition decision indicating whether the item had been encountered in the scanner and their response confidence (i.e., “high-confidence old,” “low-confidence old,” “low-confidence new,” and “high-confidence new”). Due to a low frequency of “high-confidence new” responses to studied items (mean, 0.16; SD, 0.12; 5 participants used this response ≤5 times), data were collapsed across confidence levels to yield “old” and “new” responses.
fMRI data analysis.
Data were acquired on a 3T Signa MRI system (GE Medical) using an eight-channel receive-only head coil. Functional data were acquired using a T2*-weighted spiral in–out sequence consisting of 20 slices parallel to the long axis of the hippocampus (TR, 2.8 s; TE, 30.6 ms; FOV, 22 cm; voxel size, 2 × 2 × 2.5 mm). The first four volumes of each functional run were discarded to allow for T1 stabilization. Functional data were slice-time and motion corrected using SPM5 (http://www.fil.ion.ucl.ac.uk/spm/) and custom Matlab routines. To maintain high-spatial resolution and more precise anatomical localization, fMRI data were neither smoothed nor normalized (Zeineh et al., 2003; Carr et al., 2010). Data were high-pass filtered and voxel-based analyses were conducted according to a general linear model (GLM). The presentation of each stimulus was modeled as an impulse and convolved with a canonical hemodynamic response function. A single regressor was assigned to all presentations of each stimulus, resulting in 96 regressors of interest; motion, block, and linear drift were modeled as regressors of no interest. The resulting stimulus-specific β maps were normalized by the SD of the residual variance to transform them to t maps.
Analyses were performed using a region-of-interest (ROI) approach targeting MTL subregions. Anatomically defined ROIs (bilateral PRc, bilateral PHc, and bilateral hippocampus) were demarcated on the high-resolution structural images acquired perpendicular to the long axis of the hippocampus for each individual using established procedures and were coregistered to functional space (Fig. 2a). In addition, hippocampal subfields—bilateral subiculum, bilateral CA1, and bilateral CA2/CA3/dentate gyrus (CA2/3DG; CA2, CA3, and dentate gyrus cannot be segmented at the acquired resolution)—were demarcated for follow-up analyses (for details, see Olsen et al., 2009; Preston et al., 2010).
Analysis of neural pattern similarity.
Separately for each participant and for each ROI, we computed two neural similarity scores for each stimulus that reflect (1) the global similarity of a given stimulus's neural representation to the neural representations of other stimuli from the same stimulus category (within-category similarity), and (2) the global similarity of a given stimulus's neural representation to the neural representations of other stimuli from different stimulus categories (across-category similarity). The neural representation of each stimulus was operationalized as an ROI-specific t map corresponding to that stimulus [i.e., a vector of the voxelwise BOLD responses evoked by that stimulus (in units of t values) such that each observation in the vector is the stimulus-evoked response of a single voxel within the ROI]. The neural similarity scores for each stimulus were calculated as the mean of the Pearson correlations between the ROI-specific t map corresponding to that stimulus and the ROI-specific t maps corresponding to each of the other stimuli either (1) drawn from the same category (within-category similarity) or (2) drawn from the other stimulus categories (across-category similarity; Fig. 2b). This yielded a within-category similarity score and an across-category similarity score for each stimulus for each participant-specific ROI. Similarity scores were transformed to z-scores separately for each participant relative to that participant's within-category similarity score mean and SD (see Table 1 for raw scores; all reported results remain significant when raw scores are used for analyses). For all ANOVAs performed on these similarity scores, we report Huynh–Feldt-corrected p values when Huynh–Feldt epsilons are <1.
Results
Behavioral results
Concurrent with fMRI, participants repeatedly encoded images of faces, bodies, scenes, and objects while performing an incidental fixation cross color-change detection task. Task accuracy was significantly above chance (chance, 0.50; mean, 0.87; SD, 0.10; t(16) = 14.89; p < 10−10). During a subsequent unexpected memory test, participants made old/new recognition decisions about studied images and lures drawn from the same categories as the studied images (Fig. 1). D' scores indicated that memory performance was significantly above chance (mean, 1.09; SD, 0.25; t(16) = 17.76; p < 10−11; hit rate: mean, 0.61; SD, 0.10; false alarm rate: mean, 0.23; SD, 0.11).
fMRI results
Neural pattern similarity and MTL content representation
To assess whether MTL subregions exhibit differential category sensitivity, within-category and across-category similarity scores were sorted based on category (face, body, object, or scene) and similarity scores for each category were averaged separately for each ROI, yielding face, body, object, and scene within-category and across-category similarity scores for each ROI for each participant. To the extent that a given region is sensitive to a stimulus category, within-category similarity scores for that category should be higher than the corresponding across-category similarity scores (Kriegeskorte et al., 2008). To foreshadow our results, we also observed unexpected instances in which within-category similarity scores were lower than the corresponding across-category similarity scores; these instances are difficult to interpret but may arise when stimuli within a category are highly heterogeneous (e.g., face or body categories that consist of both animals and humans) and/or a region is sensitive to an additional stimulus dimension that cuts across categories.
ANOVA of these similarity scores revealed a significant region (PRc/PHc/hippocampus) × similarity type (within-category/across-category) × category (face/body/object/scene) interaction (Fig. 3, Table 1; F(6,96) = 52.14, Huynh–Feldt corrected pH-F < 10−16). Follow-up 2 (region) × 2 (similarity type) × 4 (category) ANOVAs revealed significant region × similarity type × category interactions when any pair of regions was included in the analysis (all pH-F < 10−3).
To more closely examine these differential relationships between similarity type and category, we performed 2 (similarity type) × 4 (category) ANOVAs separately for each region. Within PRc, a main effect of higher within-category than across-category similarity (F(1,16) = 7.77, p = 0.013) was qualified by a significant interaction with category (F(3,48) = 3.81, pH-F = 0.020). Follow-up paired t tests revealed that within-category similarity was greater than across-category similarity for faces and objects (t(16) = 2.13 and 3.16, p = 0.049 and 0.006), within-category similarity was lower than across-category similarity for bodies (t(16) = 2.78, p = 0.013), and the two similarity types did not differ for scenes (t(16) = 1.23, p = 0.24). Within PHc, a main effect of higher within-category than across-category similarity (F(1,16) = 193.66, p < 10−9) was qualified by a significant interaction with category (F(3,48) = 121.04, pH-F < 10−12). Follow-up paired t tests revealed that within-category similarity was greater than across-category similarity for objects and scenes (t(16) = 7.68 and 12.71, both p < 10−6; the magnitude of this effect was greater for scenes than for objects, F(1,16) = 68.85, p < 10−6). By contrast, within-category similarity was lower than across-category similarity for faces and bodies (t(16) = 2.16 and 5.91, p = 0.046 and < 10−4). Unlike PRc and PHc, analyses within hippocampus did not reveal a main effect of similarity type (F < 1). Although hippocampus exhibited a marginal similarity type × category interaction (F(3,48) = 2.93, pH-F = 0.058), within-category and across-category similarity scores did not significantly differ for any category within hippocampus (all p > 0.097). Follow-up analyses revealed that the finding of comparable within-category and across-category similarity scores in hippocampus was consistent across CA1, CA2/3DG, and subiculum (Table 1; subfield × similarity type, F < 1; subfield × similarity type × category, F(6,96) = 1.18, pH-F = 0.33).
Neural pattern similarity and MTL encoding computations
Given our primary objective of examining the computations subserved by MTL subregions, our main set of analyses assessed whether across-item neural pattern similarity was differentially associated with subsequent memory across MTL subregions. Due to (1) the theoretical expectation that global match and pattern separation computations may be most relevant for items drawn for the same stimulus category, and (2) the above-reported finding of category coding within MTL cortical subregions, all subsequent memory analyses used within-category similarity scores. To assess the relationship between across-item neural pattern similarity and later recognition memory, stimuli for each participant were back-sorted according to subsequent memory (i.e., whether they were subsequently identified as “old” (remembered) or “new” (forgotten) (Brewer et al., 1998; Wagner et al., 1998; Paller and Wagner, 2002). Within-category neural similarity scores for later-remembered and later-forgotten stimuli were averaged separately within each participant-specific ROI, yielding a single remembered and a single forgotten similarity score within each ROI for each participant.
Initial analysis of neural similarity revealed a significant region (PRc/PHc/hippocampus) × subsequent memory (remembered/forgotten) interaction (Fig. 4a, Table 1; F(2,32) = 6.49; pH-F = 0.0098). Follow-up 2 (region) × 2 (subsequent memory) ANOVAs revealed that this interaction was driven primarily by an interaction between PRc/hippocampus and subsequent memory (F(1,16) = 22.71, p < 10−3): within PRc, across-item similarity scores were greater for subsequently remembered than subsequently forgotten stimuli (t(16) = 3.02, p = 0.0082), whereas within hippocampus, similarity scores were numerically, but nonsignificantly lower for subsequently remembered than subsequently forgotten stimuli (t(16) = −1.39, p = 0.18). This hippocampal pattern was consistent across CA1, CA2/3/DG, and subiculum (Table 1): a 3 (subfield) × 2 (subsequent memory) ANOVA did not reveal an interaction between subfield and subsequent memory (F < 1), but revealed a marginally significant effect of subsequent memory (F(1,16) = 3.12, p = 0.096), again with subsequently remembered items having lower similarity scores than subsequently forgotten items. Neither of the 2 (region) × 2 (subsequent memory) interactions involving PHc was significant (both p > 0.08), and similarity scores within PHc did not differ between subsequently remembered and subsequently forgotten stimuli (t < 1).
Because the relationship between neural similarity and subsequent memory may be strongest for items with relatively low or high neural similarity, a second analysis examined this relationship after excluding stimuli with intermediate neural similarity scores. For each participant and for each ROI, stimuli were sorted into terciles according to their similarity scores, and subsequent memory performance (proportion remembered) was computed separately for the upper and lower terciles. Analysis of the proportion of items remembered from each tercile revealed a significant region (PRc/PHc/hippocampus) × tercile (upper/lower) interaction (Fig. 4b; F(2,32) = 4.99, pH-F = 0.020). Follow-up 2 (region) × 2 (tercile) ANOVAs revealed interactions between PRc/hippocampus and tercile and between PHc/hippocampus and tercile (F(1,16) = 13.20 and 5.32, p < 10−3 and p = 0.035): within PRc and PHc, subsequent memory performance was significantly higher for the upper tercile (high similarity scores) than for the lower tercile (low similarity scores; t(16) = 2.51 and 2.45, p = 0.023 and 0.026); the patterns in PRc and PHc did not significantly differ (F < 1). By contrast, within hippocampus, subsequent memory performance was numerically, but nonsignificantly lower for the upper tercile than for the lower tercile (t < 1).
Finally, and most critically, because memory-related differences in neural similarity could be obscured by nuisance variables—specifically, item effects, category effects, and univariate signal amplitudes—when averaging over all remembered and overall forgotten stimuli, we used logistic regression to model memory outcome for each stimulus using similarity scores in MTL subregions as predictor variables while controlling for these nuisance factors (Fig. 4c). Specifically, the set of all stimulus–participant combinations (1632: 17 participants × 96 stimuli) served as data points for a logistic regression model that predicted memory outcome (remembered or forgotten) based on within-category neural similarity scores in MTL subregions. Due to collinearity between similarity scores in PRc and PHc, separate logistic regressions were conducted using similarity scores from each of these regions in conjunction with hippocampus similarity scores. Univariate signal amplitude associated with each stimulus in each ROI [extracted according to a finite impulse response model with MarsBaR (http://marsbar.sourceforge.net), summed over the second and third TR, and z-scored within each participant] and the stimulus category were included in the model as control predictors. Both models assumed the maximal random effects structure: participant, stimulus, and corresponding slope terms for each predictor variable were modeled as random effects.
The inclusion of univariate signal amplitude and stimulus category as control variables in the models ensured that these nuisance factors did not either (1) spuriously drive or (2) obscure observed relationships between neural similarity and memory outcome. Specifically, increases in univariate amplitude for a subset of stimuli (e.g., subsequently remembered stimuli) can produce increases in neural similarity for these stimuli by (1) increasing signal-to-noise for these stimuli, or (2) selectively activating a subset of voxels, thereby increasing neural similarity via this shared set of voxels. The inclusion of univariate signal amplitude as a control predictor to address these concerns has been previously reported (Ritchey et al., 2012; Xue et al., 2012), and we conducted analyses of simulated data that confirmed that this procedure attenuated increases in neural pattern similarity resulting from increased univariate amplitude (all simulation code and outcomes are available upon request from the authors). Similarly, the inclusion of category as a control predictor was motivated by the finding that memory outcome varied as a function of stimulus category, and neural pattern similarity varied as a function of stimulus category differentially across regions. The only control variable that emerged as a significant predictor of subsequent memory was the finding that objects were more likely to be remembered than scenes (χ2(1) = 8.43 and 13.10, p = 0.0037 and < 10−3, in models including PRc and PHc, respectively).
Turning to the critical findings, likelihood ratio tests from a regression including PRc and hippocampus revealed that PRc similarity score was a significant positive predictor of subsequent memory (χ2(1) = 6.03, p = 0.014), whereas hippocampus similarity score was a significant negative predictor of subsequent memory (χ2(1) = 9.40, p = 0.0022). Moreover, likelihood ratio tests from a regression including PHc and hippocampus revealed that PHc similarity score also was a significant positive predictor of subsequent memory (χ2(1) = 6.72, p = 0.0096), whereas hippocampus similarity score once again emerged as a significant negative predictor of subsequent memory (χ2(1) = 9.34, p = 0.0022). Additional logistic regressions (run separately for each subfield due to collinearity between similarity scores in the subfields) revealed that similarity scores in all hippocampal subfields were significant or marginally significant negative predictors of subsequent memory (χ2(1) = 5.28, 5.21, and 3.32; p = 0.022, 0.022, and 0.068 for subiculum, CA1, and CA2/3/DG, respectively).
Impact of analysis parameters
A final set of analyses explored the dependence of our results on our analysis procedure. We reran the logistic regression models using alternate calculations of neural similarity scores to determine whether the results depended on the manner in which neural similarity scores were calculated. First, we found that using raw (rather than z-scored) similarity scores as predictor variables did not change any of the reported results: PRc and PHc similarity scores remained significant positive predictors of subsequent memory (both p < 0.05), whereas hippocampus similarity scores in both models remained significant negative predictors of subsequent memory (both p < 10−3). Second, we calculated similarity scores that reflected the mean of the Pearson correlations between the pattern of response to a given stimulus and the patterns of response to all other stimuli, regardless of stimulus category (all-category similarity score). The use of all-category similarity scores as predictor variables only modestly affected the results: PRc and PHc z-scored all-category similarity scores remained significant positive predictors of subsequent memory (both p < 0.05), whereas hippocampus z-scored all-category similarity scores in the PRc and PHc models were marginal (p = 0.054) and significant (p < 0.05) negative predictors of subsequent memory, respectively; PRc and PHc raw all-category similarity scores were marginal (p = 0.063) and significant (p < 0.05) positive predictors of subsequent memory, whereas hippocampus raw all-category similarity scores in both models were significant negative predictors of subsequent memory (both p < 0.05). Finally, we calculated similarity scores in which an exponential function, er, was applied to each pairwise correlation, r, before the mean of these scores was calculated; this exponential function has been used in summed-similarity recognition and categorization models (Nosofsky, 1988), and results in high pairwise correlations being weighted more heavily than low pairwise correlations. The use of exponential similarity scores as predictor variables did not change any of the above-reported results with the exception that the PRc raw all-category similarity score became a significant, rather than marginal, positive predictor of subsequent memory (p < 0.01). Thus, our logistic regression results were robust to a number of calculations of neural similarity, with PRc and PHc similarity scores consistently emerging as significant positive predictors of subsequent memory and hippocampus similarity score consistently emerging as a significant negative predictor of subsequent memory.
Second, we examined whether our results were robust across a different treatment of nuisance variables. Specifically, we asked whether the primary logistic regression results (using z-scored within-category similarity scores) remained significant when the regression models were run in a hierarchical fashion: we used logistic regression to predict memory outcome based on univariate signal amplitude and stimulus category, and then used linear regression to predict the residuals of these models based on neural similarity scores (in separate models that included either PRc and hippocampus or PHc and hippocampus). All models included corresponding random effects slope terms for each predictor variable, and the linear regression models included additional random effect intercept terms for participant and for stimulus (because accounting for these effects in both models would be redundant). Using this procedure, we found that PRc and PHc similarity scores were significant positive predictors of the memory outcome residuals (both p < 0.05), whereas hippocampus similarity scores from both models were significant negative predictors of these residuals (both p < 0.01). Thus, all results remain significant when the nuisance variance is completely removed before the predictors of interest are fit to the data.
Third, to further assess whether our results were robust to univariate signal amplitude, we examined whether our key findings remain significant when excluding all voxels showing significantly higher responses to remembered relative to forgotten stimuli. An additional GLM was run for each participant in which a single regressor was assigned to all subsequently remembered stimuli and a single regressor was assigned to all subsequently forgotten stimuli; motion, block, and linear drift were modeled as regressors of no interest. Voxels showing significantly higher responses to remembered relative to forgotten stimuli at a liberal uncorrected threshold of p < 0.05 were removed on a participant-by-participant basis and neural similarity scores were recalculated using the remaining voxels (Xue et al., 2010). We found that (1) PRc similarity scores remained significantly higher for remembered relative to forgotten stimuli, (2) all previously significant region × subsequent memory interactions remained significant, and (3) when we submitted scores to a logistic regression analysis, PRc and PHc similarity scores remained positive predictors of subsequent memory, whereas hippocampus similarity score remained a negative predictor of subsequent memory (all p < 0.05). Thus, the observed similarity–subsequent memory relationships do not depend on the presence of voxels that were more responsive to remembered relative to forgotten stimuli.
Discussion
Using high-resolution fMRI and analyses of neural pattern similarity, the present experiment revealed distinct computational and representational functional dissociations between human hippocampus, PRc, and PHc. First, we observed a dissociable relationship between across-item pattern similarity in hippocampus and in MTLc and subsequent recognition memory: later recognition was predicted by greater pattern distinctiveness within hippocampus, but greater pattern similarity in PRc and PHc. These data address a central theoretical debate regarding functional distinctions within the MTL, providing compelling new evidence that hippocampus and MTLc implement complementary learning mechanisms. Second, we observed differential sensitivity to visual categories within the MTL hierarchy: MTLc demonstrated greater content sensitivity than hippocampus, with dissociable categorical effects in PRc and PHc. These data extend the literature on categorical representation in ventral temporal cortex to the MTL, and complement recent multivariate observations of content-sensitive responses across the MTLc.
Extensive debate surrounds the role of MTL subregions in memory, with controversy particularly focused on the nature of PRc and hippocampal computations (Brown and Aggleton, 2001; Holdstock et al., 2002; Mayes et al., 2002; Stark et al., 2002; Yonelinas et al., 2002; Davachi et al., 2003; Manns et al., 2003; Norman and O'Reilly, 2003; Stark and Squire, 2003; Kirwan and Stark, 2004; Ranganath et al., 2004; Diana et al., 2007; Eichenbaum et al., 2007; Squire et al., 2007; Wixted and Squire, 2011). Initial fMRI data bearing on this debate revealed that PRc and hippocampal encoding activations differentially predict later item recognition versus source recollection (Davachi et al., 2003; Ranganath et al., 2004), item versus associative recognition (Kirwan and Stark, 2004), recognition accompanied by knowing versus remembering (Uncapher and Rugg, 2005; Montaldi et al., 2006), and graded versus high-threshold confidence judgments (Daselaar et al., 2006). While suggestive of computational differentiation within the MTL, these observations constitute single dissociations between PRc and hippocampus as a function of putatively distinct behavioral expressions of memory, and thus have been subject to interpretative challenges (Squire et al., 2007; Wixted and Squire, 2011; but see Bowles et al., 2007 for lesion data supporting a PRc/hippocampal double dissociation between familiarity and recollection).
Leveraging advances in distributed pattern analyses of fMRI data, the present study revealed a double dissociation between MTLc and hippocampus in the relationship between across-item encoding pattern similarity and subsequent recognition memory performance, providing critical new insights into the encoding computations subserved by PRc, PHc, and hippocampus. Consistent with the hypothesis that MTLc encodes overlapping representations that support later recognition memory, greater across-item representational similarity in PRc and PHc predicted later memory success. Conversely, consistent with the hypothesis that hippocampus encodes pattern-separated representations that enable later memory expression, greater representational distinctiveness in hippocampus predicted later memory success. We discuss each result in turn.
The positive relationship between across-item MTLc pattern similarity for a given item and subsequent memory for that item is consistent with global match models of memory, in which a recognition-memory decision depends on the sum similarity (global match) of a test probe to stored representations of previously encountered items (Gillund and Shiffrin, 1984; Hintzman, 1984). In particular, our findings are consistent with the dual-process CLS model of recognition memory (Norman and O'Reilly, 2003; Norman, 2010), in which the MTLc model (1) encodes overlapping representations that capture regularities across experience and (2) exhibits sensitivity to the degree of global match at test. From this perspective, items with encoding patterns more similar to those of other items (high global match) should be better recognized at test; this is precisely the pattern we observed for both PRc and PHc.
The positive relationship between global match and subsequent memory in PRc and PHc suggests a common cortical learning mechanism along the anterior/posterior extent of the parahippocampal gyrus. Previous studies have reported that BOLD encoding activity in PRc predicts item memory or subjective familiarity, whereas BOLD encoding activity in PHc predicts source memory or subjective recollection (Davachi et al., 2003; Ranganath et al., 2004; for review, see Diana et al., 2007). These prior observations do not necessarily reflect dissociable learning and retrieval mechanisms in PRc and PHc, however, as more recent work has demonstrated that when source-related stimulus content is manipulated, memory-related BOLD responses in PRc and PHc dissociate according to retrieved stimulus content rather than qualitative memory processes. Specifically, during encoding, BOLD responses in PRc and PHc differentially predict subsequent memory for object-related source information (Awipi and Davachi, 2008; Staresina and Davachi, 2008; Staresina et al., 2011) versus scene-related source information (Davachi et al., 2003; Staresina et al., 2011). Consistent with these recent findings, our multivariate approach revealed that PRc and PHc implement analogous, rather than distinct, encoding computations that support subsequent recognition memory.
In contrast, the negative relationship between across-item hippocampal pattern similarity for a given item and subsequent memory for that item is consistent with the hypothesis that hippocampal encoding mechanisms yield pattern-separated neural representations (O'Reilly and McClelland, 1994; McClelland et al., 1995; O'Reilly and Rudy, 2001). Previous work on pattern separation in rodent (Leutgeb et al., 2007) and human (Bakker et al., 2008; Lacy et al., 2011) hippocampus examined the relationship between changes in sensory input and changes in elicited neural response. Here, using across-item pattern similarity analysis, we provide critical new evidence that greater hippocampal pattern separation during encoding is predictive of later memory. Moreover, although the degree of pattern separation elicited by a change in sensory input has been found to be greater in CA3 and dentate gyrus relative to CA1 (Leutgeb et al., 2007; Bakker et al., 2008; Lacy et al., 2011), we did not observe differences between hippocampal subfields in the relationship between neural pattern similarity and subsequent memory. It is important to note, however, that “pattern separation”-like responses have been observed in all hippocampal subfields (Kirwan and Stark, 2007). Thus, while hippocampal subfields may vary in the degree of pattern separation triggered by changes in sensory input, successful subsequent memory expression may depend on engagement of pattern separation in multiple subfields during learning (Yassa and Stark, 2011).
The present findings contribute to an emerging literature on the relationship between neural pattern similarity at encoding and subsequent memory. Using standard-resolution whole-brain fMRI, Xue and colleagues (Xue et al., 2010) observed that, in multiple regions outside the MTL, consistency in the neural pattern of response elicited by an item across repetitions of that item was positively associated with subsequent memory. We instead used high-resolution fMRI of the MTL to focus on an item's pattern similarity to patterns elicited by other previously encountered items, examining relationships between global similarity, pattern separation, and subsequent memory. Strength of category representation, as well as across-item within-category neural pattern similarity, in temporal and prefrontal regions have also been linked to successful subsequent memory (Kuhl et al., 2012). The increased anatomical specificity of our high-resolution approach suggests that the relationship between neural pattern similarity and subsequent memory is not uniform, and instead dissociates across MTLc and hippocampus. Our finding of qualitatively distinct mechanisms in MTLc and hippocampus opens new avenues for research that targets how behaviorally distinct expressions of memory relate to the extent of engagement of specific MTLc and hippocampal encoding mechanisms, rather than simply to overall BOLD signal amplitudes in these regions.
Addressing a secondary aim, separate representational similarity analyses targeting content sensitivity revealed differential sensitivity to visual categories across the MTL: PRc showed sensitivity to faces and objects, PHc showed sensitivity to scenes and objects, and hippocampus did not exhibit category sensitivity. The MTLc findings are broadly consistent with previous fMRI studies, wherein multivariate pattern analyses assessed content representations in the MTL using blocks of stimuli drawn from the same category (Diana et al., 2008; Liang et al., 2013). Here, we observed differential category sensitivity in MTLc using an intermixed, event-related paradigm, wherein neural patterns were computed at the individual stimulus, rather than the categorical level, minimizing the likelihood that our results arose from attentional biases to contextual category sets. More broadly, our results extend recent representational similarity analyses of inferotemporal cortex (Kriegeskorte et al., 2008), demonstrating that visual category structure previously observed across inferotemporal cortex is preserved at subsequent levels of the visual-processing hierarchy, namely anatomically defined MTLc. By contrast, the absence of category sensitivity in hippocampus is consistent with (1) hippocampus's purported role in the integration of information from distinct processing streams (Davachi, 2006) and in pattern separation of overlapping inputs (O'Reilly and McClelland, 1994), and (2) the observation that hippocampus shows sensitivity to individual events from the same broad category, such as sensitivity to individual spatial locations (Hassabis et al., 2009), individual retrieved memories (Chadwick et al., 2010), or individual images (Bonnici et al., 2012). Finally, although the content sensitivity and subsequent memory analyses were designed to answer separate questions, the relationship between these two properties (e.g., the relative degree to which MTLc regions perform global match computations over stimuli to which they are categorically sensitive vs insensitive) remains an important avenue for future research.
Collectively, the present high-resolution fMRI data demonstrate dissociable relationships between MTL subregions in terms of computational properties and representational content. By examining the similarity across neural representations encoded during experience, our findings reveal the promise of pattern similarity analyses of high-resolution fMRI data and provide novel evidence that complementary PRc, PHc, and hippocampal computations support learning that enables future recognition of things past.
Notes
Supplemental material for this article is available at http://memorylab.stanford.edu/Publications/LaRocque_JNeurosci_simulations.html. This presents simulations exploring the relationship between univariate signal amplitude and multivariate pattern similarity. Code, results, and discussion are included. This material has not been peer reviewed.
Footnotes
This work was supported by grants from the National Institute of Mental Heath (R01-MH076932 to A.D.W.; F32-MH087012 to V.A.C.) and the National Science Foundation (Integrative Graduate Education and Research Traineeship Fellowship 0801700 and Graduate Research Fellowship to K.F.L.).
The authors declare no competing financial interests.
- Correspondence should be addressed to either Karen F. LaRocque or Anthony D. Wagner, Jordan Hall, Building 420, Stanford, CA 94305-2130, klarocqu{at}stanford.edu or awagner{at}stanford.edu