Abstract
Repeated exposure to a visual stimulus is associated with corresponding reductions in neural activity, particularly within visual cortical areas. It has been argued that this phenomenon of repetition suppression is related to increases in processing fluency or implicit memory. However, repetition of a visual stimulus can also be considered in terms of the similarity of the pattern of neural activity elicited at each exposure—a measure that has recently been linked to explicit memory. Despite the popularity of each of these measures, direct comparisons between the two have been limited, and the extent to which they differentially (or similarly) relate to behavioral measures of memory has not been clearly established. In the present study, we compared repetition suppression and pattern similarity as predictors of both implicit and explicit memory. Using functional magnetic resonance imaging, we scanned 20 participants while they viewed and categorized repeated presentations of scenes. Repetition priming (facilitated categorization across repetitions) was used as a measure of implicit memory, and subsequent scene recognition was used as a measure of explicit memory. We found that repetition priming was predicted by repetition suppression in prefrontal, parietal, and occipitotemporal regions; however, repetition priming was not predicted by pattern similarity. In contrast, subsequent explicit memory was predicted by pattern similarity (across repetitions) in some of the same occipitotemporal regions that exhibited a relationship between priming and repetition suppression; however, explicit memory was not related to repetition suppression. This striking double dissociation indicates that repetition suppression and pattern similarity differentially track implicit and explicit learning.
Introduction
A fundamental goal of cognitive neuroscience is to understand how the brain's responses to environmental stimuli relate to behavioral measures of learning and memory. Two tools that have gained prominence for investigating visual representation and memory using functional magnetic resonance imaging (fMRI) are repetition suppression and multi-voxel pattern analysis (MVPA). Repetition suppression (also fMRI adaptation or repetition attenuation) is a decrease in blood oxygen level-dependent (BOLD) response to repeated stimuli relative to new stimuli (Schacter and Buckner, 1998; Wiggs and Martin, 1998; Grill-Spector and Malach, 2001). MVPA examines spatially distributed response patterns elicited by different stimuli or classes of stimuli, using classification algorithms to decode stimulus category (for review, see Norman et al., 2006) or by quantifying the neural similarity (correlation) between stimuli or trials (Haxby et al., 2001; Kriegeskorte et al., 2008; Drucker and Aguirre, 2009).
Repetition suppression is frequently linked to implicit memory measures such as repetition priming (Maccotta and Buckner, 2004; Wig et al., 2005; Zago et al., 2005; Turk-Browne et al., 2006; Epstein et al., 2008), although the relationship is not always robust (Bunzeck et al., 2006; Ganel et al., 2006; Salimpoor et al., 2010). The relationship between repetition suppression and explicit memory has been studied less often, with some studies showing a positive relationship (Gonsalves et al., 2005; Turk-Browne et al., 2006) and others showing a negative one (Wagner et al., 2000; Xue et al., 2011).
Pattern similarity, as measured by fMRI across repeated exposures of a stimulus, is higher when stimuli are subsequently remembered versus subsequently forgotten (Xue et al., 2010; LaRocque et al., 2013). Likewise, fMRI pattern similarity between encoding and recognition of a stimulus relates to the subjective experience of remembering (Ritchey et al., 2012). Thus, consistency in the representation of a stimulus is positively related to explicit memory for that stimulus. However, to our knowledge, the relationship between pattern similarity and implicit measures of visual memory has not been investigated.
Here we directly compared repetition suppression and pattern similarity as predictors of implicit memory (repetition priming) and explicit memory (recognition memory). Motivated by prior findings, we predicted that priming would be associated with reductions in neural activity across repetitions (repetition suppression; Maccotta and Buckner, 2004; Wig et al., 2005; Zago et al., 2005; Turk-Browne et al., 2006; Epstein et al., 2008), while successful recognition memory would be associated with consistency of representation across repetitions (pattern similarity; Xue et al., 2010; Ritchey et al., 2012). Critically, we sought to determine whether each of these relationships would be present within common, high-level visual cortical areas.
Using fMRI, we scanned participants while they made speeded indoor/outdoor judgments for repeated scene pictures. Repetition priming (facilitated categorization across repetitions) was treated as a measure of implicit memory. Explicit memory was measured by a surprise recognition test administered after scanning. We examined how repetition suppression and pattern similarity measures related to both priming and subsequent recognition memory. Within occipitotemporal cortex, we observed a double dissociation: repetition suppression predicted priming, but not subsequent recognition, while pattern similarity predicted subsequent recognition, but not priming.
Materials and Methods
Participants.
Twenty participants (seven females) were recruited from the Yale University community and gave written informed consent in compliance with procedures approved by the Yale University Human Subjects Committee. Participants had normal or corrected-to-normal vision. Four additional participants were scanned but were excluded from the study before data analysis (two for poor performance, two for excessive head motion). Participants were paid for their participation.
Apparatus.
fMRI data were acquired at the Magnetic Resonance Research Center at Yale University, on a 3 T Siemens Trio equipped with a standard 12-channel head-coil. T1-weighted anatomical images were acquired using a 3D MPRAGE sequence (TR = 2530 ms, TE = 2.77 ms, time to inversion = 1100 ms, voxel size = 1 × 1 × 1 mm, matrix size = 256 × 256 × 256). T2*-weighted images sensitive to BOLD contrasts were acquired using a gradient-echo echo-planar pulse sequence (TR = 2000 ms, TE = 25 ms, voxel size = 3.5 × 3.5 × 4 mm, matrix size = 64 × 64 × 34). Stimuli were presented using PsychoPy (Peirce, 2007) and displayed through an LCD projector on a rear-projection screen. Responses were recorded using a two-button fiber-optic response pad system.
Stimuli.
For the main analysis, stimuli were 160 color images of scenes, of which 80 were indoor scenes and 80 were outdoor scenes. Images were 350 × 350 pixels. Outdoor scenes were generic, nonfamous landscapes and cityscapes (Epstein and Ward, 2010); indoor scenes were from a variety of categories (e.g., living room, hallway, etc.). None of the scenes included people. Images were selected from a stock image website (www.sxc.hu). Scenes were presented at the center of the screen and subtended ∼10° × 10° of visual angle. For the functional localizer, 140 additional scenes and 140 images of faces were used.
Design.
The experiment session consisted of five experimental scan runs (6 min 34 s each) and two functional localizer scan runs (7 min 18 s each), followed by a behavioral subsequent memory test. During the experimental scan runs, participants viewed 80 unique scenes (40 indoor, 40 outdoor); the remaining 80 scenes were used as foils in the subsequent recognition test (the assignment of scenes to these two conditions was counterbalanced across every two participants). The scenes were divided evenly such that in each run (scan) 16 different scenes were presented. Within each run, each scene was presented twice with 2–7 trials between presentations (i.e., the second presentation occurred at lag 3–8; range 24–108 s lag, average 57.9 s [4.8 trials]). Two predesigned sequences were alternated every two participants.
The functional localizer consisted of 24 s blocks during which participants viewed 12 images (250 ms each, 1.75 s interstimulus interval, ISI) of either all faces or all scenes. Before each block, participants were cued to the upcoming image category. Participants indicated the gender of the faces or whether the scenes were indoor or outdoor.
Procedure.
On experimental trials, each scene stimulus was presented for 3 s. During this time, participants indicated whether the scene was indoors or outdoors by pressing button box keys corresponding to their index and middle fingers, respectively. Participants were highly accurate at this task (indoor vs outdoor; M = 91.7 ± 21.0%). Scene stimuli were separated by a 9 s active baseline task in which a series of six arrows was presented (150 ms each, 1.25 s ISI), each pointing left or right (Fig. 1). Participants indicated the direction of each arrow (left = index finger, right = middle finger) by button press. The direction of each arrow was randomly determined. The arrow task helped to prevent further encoding of the scenes and was used as baseline for the fMRI analysis. Mean accuracy on the arrow task was 83.9%, with no significant difference in performance on trials in which the stimulus was later remembered or later forgotten, indicating no difference in overall alertness between memory conditions (F(1,19) = 0.233, p = 0.635).
Each scene was presented for 3000 ms. Participants were asked to identify whether the image was of an outdoor or indoor scene. Following the stimulus, a series of arrows was presented for 9 s, during which participants indicated the direction the arrows were pointing. This prevented further encoding of the scene and was used as the fMRI baseline.
After all scans (i.e., the main experiment scans followed by the localizer scans) were completed, participants exited the scanner and we tested their memory for all of the scenes shown during the in-scanner experimental trials. In the recognition test, 160 scenes were shown (80 old, 80 new). The test was self-paced, but participants were encouraged to respond quickly without losing accuracy. Each participant was asked to rate on a scale of 1 through 5 how confident he/she was that each scene was seen before in the scanner: Ratings were as follows: (1) did not see before, (2) probably did not see before, (3) unsure, (4) probably saw before, and (5) definitely saw before. Ratings of 1–3 were classified as “forgotten” and ratings of 4–5 were classified as “remembered.” To prevent intentional encoding during the scanned experimental trials (and to encourage sufficient forgetting of scenes), participants were not told beforehand that there would be a subsequent recognition test. For inclusion in the fMRI analyses, a participant needed a 10 scene minimum in both subsequent memory conditions. All participants met this requirement.
Data analysis.
Functional images were corrected for motion and for differences in slice timing. Data were registered to MNI152 standard space template (2 mm isotropic voxel size). Voxels were smoothed using a 5 mm full-width, half-maximum Gaussian filter (Xue et al., 2010), which may reduce noise and improve sensitivity after motion correction (Kamitani and Sawahata, 2010). Cortical reconstruction of each participant's data was performed with FreeSurfer software (http://surfer.nmr.mgh.harvard.edu; Fischl, 2012). For each run, a quadratic polynomial was fit and removed to eliminate drift. Data were analyzed using FSL (Smith et al., 2004) and in-house Python scripts.
Our analyses were run on 48 bilateral cortical regions of interest (ROIs) taken from the probabilistic Harvard–Oxford anatomical atlas (threshold at 25% probability). Because these ROIs are in a standard space, the ROIs were identical for each participant. We were also interested in probing functionally defined, scene-selective cortex. Thus, using data from the localizer scans, for each participant we identified bilateral parahippocampal place area (PPA) (Epstein and Kanwisher, 1998). The PPA responds to spatial layout (Epstein, 2008; Epstein and Ward, 2010) and has shown sensitivity to scene information both in repetition suppression and MVPA paradigms (Kravitz et al., 2011; Morgan et al., 2011; Epstein and Morgan, 2012). To identify PPA ROIs, we selected, for each participant, the peak voxel (from a scene>face contrast) in both the left and right hemispheres, located at or near the collateral sulcus, including posterior parahippocampal cortex and fusiform gyrus. Four millimeter radius spheres were generated, centered at these peak voxels. In all cases, the peak voxel responded more strongly to scenes than faces at a significance threshold of p < 0.001. The resulting participant-specific bilateral spherical PPA ROIs contained 93 voxels each (Fig. 2).
Independently localized bilateral PPA from a representative participant. PPA ROIs consisted of 4 mm radius spheres centered at the voxel exhibiting the peak response to a scene>face contrast (from the localizer task).
Both our repetition suppression and pattern similarity analyses were based on the results of an initial general linear model (GLM). In our GLM, each trial served as a separate regressor (80 scenes × 2 repetitions = 160 total regressors in the GLM). Hemodynamic response function was fit to the 3 s stimulus presentation using a second degree gamma function with 2.25 s hemodynamic delay and 1.25 s dispersion. This GLM resulted in β-values that were extracted for each trial and for each voxel within each of the anatomical ROIs and for each voxel within functionally defined PPA.
Repetition suppression was computed for each scene by subtracting the mean β-value—across all voxels within an ROI—at the second presentation of that scene from the mean response at the first presentation of that scene. For pattern similarity analyses, for each trial and each ROI, a vector array was created, which represented the β-value for each voxel within that ROI. Similarity was defined as the Pearson correlation between two vector arrays (e.g., between the two repetitions of a scene); the higher the correlation between the two patterns, the greater the similarity across repetitions. The r values from these correlations were z-transformed for all statistical analyses.
To determine the relationship between these neural measures and implicit and explicit memory, we used either ANOVA or mixed-effects analysis as implemented by lme4 (Bates et al., 2012) in R (R Development Core Team, 2012). Mixed-effects modeling is a powerful statistical tool that offers many advantages over conventional t test, regression, and AN(C)OVA in sophisticated fMRI designs (Mumford and Poldrack, 2007; Chen et al., 2013), especially in cases where there is an unequal number of trials in each condition for each participant (e.g., here, participants remembered different numbers of scenes). Our analysis approach with the mixed-effects models was to treat repetition suppression or pattern similarity (along with other potentially confounding factors) as predictors of behavior (either priming or subsequent recognition). To determine whether a factor (e.g., pattern similarity) was a significant predictor, we used likelihood ratio tests to compare a model with that factor (plus other factors of noninterest) to a model without that factor (but with the same factors of noninterest).
Results
We used subsequent recognition (i.e., whether a scene was remembered or forgotten during the surprise recognition test; see Materials and Methods) as a measure of explicit memory and repetition priming (the change in reaction time, across repetitions, for the indoor/outdoor decision) as a measure of implicit memory.
Behavioral performance
Subsequent memory
Scenes from the experimental scans were grouped according to subsequent memory performance: 35.8% of scenes were forgotten [mean frequencies (±SD): rating = 1, “did not see before” (15.0 ± 7.1%); rating = 2, “probably did not see before” (16.9 ± 7.4%); rating = 3, “unsure” (3.9 ± 5.4%)]. The remainder of scenes were remembered [rating = 4, “probably saw before” (15.2 ± 7.5%); rating = 5, “definitely saw before” (49.0 ± 15.7%)]. For new scenes (i.e., scenes that were not presented during the experimental scans), an average of 15.9 ± 3.8% was falsely remembered. To determine whether lag between scene repetition influenced subsequent memory performance, we performed a one-way ANOVA with three lag “bins” (i.e., lag 3–4, lag 5–6, and lag 7–8; binning increased statistical power). We found a significant main effect of lag, reflecting a higher probability of a scene being subsequently remembered as lag increased (F(1,19) = 22.66, p < 0.001; Mlag3−4 = 55.29 ± 17.9%, Mlag5–6 = 65.45 ± 16.1%, Mlag7–8 = 70.37 ± 15.3%). Note: because of this relationship between lag and subsequent recognition, we included lag as a predictor of no interest in all mixed-effects models (both those predicting subsequent recognition and those predicting priming).
Repetition priming
Response time (RT) was assessed via ANOVA with factors of presentation number (first vs second), subsequent explicit memory (remembered vs forgotten), and lag (bins 3–4, 5–6, and 7–8). The main effect of repetition was significant (Mpresentation1 = 1062 ± 21 ms, Mpresentation2 = 941 ± 21 ms, F(1,19) = 80.38, p < 0.001), with faster RTs at the second presentations of scenes than the first (repetition priming). There was no main effect of lag (F(1,19) = 0.001, p = 0.974), nor an interaction between repetition and lag (F(1,19) = 0.502, p = 0.487); thus, priming did not vary across lags. The main effect of subsequent memory approached significance (Mremembered = 1023 ± 23 ms, Mforgotten = 980 ± 21 ms, F(1,19) = 3.588, p = 0.07; Table 1), reflecting a trend toward longer RTs being associated with better memory. There was a significant interaction between subsequent memory and repetition, reflecting a greater reduction in RT for scenes that were later remembered compared with forgotten (F(1,19) = 5.933, p = 0.025).
RTs for the indoor/outdoor categorization as a function of presentation number and subsequent memory
fMRI-whole brain analyses
The following fMRI analyses were performed for each of the 48 anatomical ROI (see Materials and Methods). To correct for multiple comparisons, we used a Bonferroni-corrected threshold of p < 0.00104 [0.05/48], unless otherwise stated.
Repetition suppression
To test for repetition suppression, we applied a linear mixed-effects model for which presentation number (first vs second) was treated as a predictor of the BOLD response β-values on each trial (see Materials and Methods). We also included lag (unbinned) as a fixed effect (to control for any relationship between lag and repetition suppression) and subject number as a random effect. Since repetition and lag did not vary across participants, we did not include a random slope term. Here, we compared a model including repetition (first vs second) and lag against a model with lag only to determine whether repetition predicted the BOLD response (i.e., a repetition suppression effect) beyond what was predicted by lag alone. Nine ROIs, including medial frontal, lateral temporal, and occipital regions, exhibited a negative relationship between repetition and the BOLD response (i.e., repetition suppression) and two ROIs, angular gyrus and cuneal cortex, exhibited a positive relationship between repetition and the BOLD response (i.e., repetition enhancement) at p < 0.05 (corrected; Table 2, column 3).
Summary of analyses for each cortical ROI from the Harvard–Oxford atlas
We next investigated the relationship between repetition suppression and repetition priming (our measure of implicit memory). For each participant, we calculated the magnitude of repetition suppression and the magnitude of repetition priming separately for each of the 80 scenes; these stimulus-level values of repetition suppression were used to predict stimulus-level values of repetition priming using a linear mixed-effects model. Lag (unbinned) was treated as a fixed effect. Because the amount of repetition suppression could vary across participants, we included a random slope for repetition suppression and random intercept for subject number as random effects. We then compared models with versus without repetition suppression as a predictor of priming. Fifteen regions, including prefrontal and occipitotemporal regions, exhibited a significant relationship between repetition suppression and priming (Table 2, column 4; Fig. 3, left); in all cases this reflected greater repetition suppression predicting greater repetition priming. Thus, the magnitude of repetition suppression associated with an individual stimulus was strongly (and positively) related to the corresponding magnitude of repetition priming.
Harvard–Oxford atlas cortical ROIs in which repetition suppression predicted repetition priming (left column brain) and in which pattern similarity predicted subsequent memory (right column brain). All effects significant at p < 0.05 (corrected).
Finally, we investigated the relationship between repetition suppression and subsequent memory using a logistic mixed-effects model [subsequent memory was transformed into a binary (Remembered/Forgotten) measure]. Critically, no regions displayed a significant positive relationship between repetition suppression and subsequent memory at either p < 0.05 (corrected) or at a more liberal threshold of p < 0.01 (uncorrected; Table 2, column 5). However, three occipitotemporal regions did show a positive relationship between overall BOLD response (collapsing across repetition) and subsequent memory at p < 0.05 (corrected; Table 2, column 1). Of these regions, the posterior parahippocampal gyrus exhibited this effect at initial scene presentation (Table 2, column 2), consistent with previous reports of subsequent memory effects (Brewer et al., 1998; Wagner et al., 1998; Garoff et al., 2005). Thus, while greater BOLD activity was associated with better subsequent memory, reductions in BOLD activity across repetitions were not predictive of subsequent memory.
Pattern similarity
Pattern similarity was defined as the correlation coefficient (r) from a Pearson correlation between the pattern of β-values elicited on separate trials (either corresponding to repetitions of the same scene or to two different scenes). The higher the correlation between the two patterns, the greater the similarity across repetitions.
We first tested whether pattern similarity was greater across repetitions of the same scene (i.e., between first and second presentations of scene “A”—same condition) than across two different scenes (i.e., between the first presentation of scene A and second presentations of other lag-matched scenes that were not A—different condition). Thus, for each subject, there were 80 same correlations. For the different condition, we found the correlation between the first presentations of each of the 80 scenes and all second presentations of nonidentical scenes that occurred between 2 and 7 trials after the first scene (i.e., at lag 3–8). To control for potential differences in pattern similarity related to lag, we binned same and different trials into three lag groups, each with a bin size of 2: i.e., lag 3–4, lag 5–6, and lag 7–8). Correlation values were then entered into a second-level ANOVA with factors of “match” (same condition vs different condition) and repetition lag (bins 3–4, 5–6, and 7–8). At a threshold of p < 0.05, corrected, no regions displayed greater similarity for same versus different trials (Table 2, column 6). At a less stringent threshold of p < 0.01 (uncorrected), two regions showed higher pattern similarity for same versus different scenes: lingual gyrus and occipital pole. Thus, there was modest evidence that our pattern similarity measure differentiated between individual scene stimuli.
Critically, we next investigated how pattern similarity across repetitions of the same scene related to subsequent recognition memory for that scene. Pattern similarity was used to predict subsequent memory using a logistic mixed-effects model [subsequent memory was transformed into a binary (Remembered/Forgotten) measure]. Lag (unbinned) was treated as a fixed effect. Because the amount of pattern similarity could vary across participants, we included a random slope for pattern similarity and random intercept for subject number as random effects. We then compared models with versus without pattern similarity as a predictor of subsequent memory. Five regions exhibited a significant relationship between pattern similarity and subsequent recognition memory (Table 2, column 7; Fig. 3, right), with greater similarity for remembered scenes than forgotten scenes: superior occipital cortex, lingual gyrus, and temporal/occipital aspects of the fusiform gyrus. No regions showed greater similarity for forgotten scenes than remembered scenes. We also confirmed that for each of these five regions, pattern similarity was a stronger predictor of subsequent memory than BOLD activity collapsed across repetitions (all p's < 0.01 (uncorrected)) or BOLD activity at initial presentations (all p's < 0.01 (uncorrected)).
Because pattern similarity across presentations of different exemplars of a category has been also shown to predict subsequent explicit memory (Kuhl et al., 2012b; LaRocque et al., 2013), we sought to confirm that the relationship between similarity and subsequent memory observed here was selective to repetitions of the same exemplar (and not global or category-level similarity). Thus, we ran a mixed-effects model that directly tested whether similarity across repetitions of the same scene better predicted subsequent memory than similarity across repetitions of different scenes (where different scenes fell within the same lag range of 3–8). Within all of the ROIs that exhibited a relationship between pattern similarity and subsequent memory (at corrected and uncorrected threshold; see Table 2), we found that within-item similarity predicted subsequent memory above and beyond across-item similarity (p's<0.05).
We next tested for a relationship between pattern similarity and repetition priming using a linear mixed-effects model. No regions exhibited a significant relationship between pattern similarity and repetition priming at either p < 0.05 (corrected) or p < 0.01 (uncorrected; Table 2, column 8).
Finally, we assessed whether the regions that exhibited a relationship between repetition suppression and priming overlapped with those exhibiting a relationship between pattern similarity and subsequent recognition memory. At the level of p < 0.05 (corrected), the superior lateral occipital cortex exhibited both effects (Fig. 3, yellow), reflecting a strong double dissociation. At a more liberal threshold of p < 0.01 (uncorrected) for each analysis, the double dissociation was also observed in the intracalcarine cortex and occipital extent of the fusiform gyrus (Fig. 4A). Together, these results indicate that, in visual cortical regions, similarity of the distributed pattern of neural activity across repetitions of a scene was predictive of later memory for that scene, but was unrelated to repetition priming (Fig. 4B).
A, Harvard–Oxford atlas ROIs and a representative example of functionally defined PPA for which repetition suppression positively predicting priming and pattern similarity positively predicted subsequent memory [each effect significant at p < 0.01 (uncorrected)]. A, Anterior; S, superior; P, posterior; I, inferior. B, Repetition suppression (green) and pattern similarity (purple) as predictors of repetition priming (left) subsequent memory (right) in the ROIs. Prediction strength is calculated using by-subject coefficients from linear (priming) and logistic (subsequent memory) mixed-effects models. Error bars indicate ±1 SE of the mean difference between the two measures.
PPA ROI
Because we exclusively used scene stimuli in the present study, we were also interested in whether a similar dissociation between repetition suppression and pattern similarity would be observed in a functionally defined, category-selective region. In particular, we were interested in the PPA based on our earlier work (Turk-Browne et al., 2006; Xu et al., 2007). We therefore defined bilateral PPA on a subject-by-subject basis (see Materials and Methods), and repeated the critical analyses described above using this ROI. Because this was a region of a priori interest, we used a significance threshold of p < 0.05, uncorrected. As with the previous analyses, p values for the mixed-effects models were obtained by likelihood ratio tests of full models including the factor in question and lag against the model without the factor in question (i.e., lag only).
Within the PPA, the repetition suppression effect was highly significant (χ2(1) = 134.16, p < 0.001). Additionally, pattern similarity across repetitions was greater when considering repetitions of the same scene versus pairs of different scenes (F(1,19) = 5.489, p = 0.03), reflecting sensitivity to individual scenes. Of primary interest was whether repetition suppression and pattern similarity differentially predicted repetition priming and subsequent recognition memory. Indeed, we found that repetition suppression in the PPA significantly predicted repetition priming (χ2(1) = 10.991, p < 0.001), but not subsequent explicit memory (χ2(1) = 2.498, p = 0.114; Fig. 4B). Conversely, pattern similarity significantly predicted subsequent memory (χ2(1) = 9.981, p = 0.002), but not repetition priming (χ2(1) = 0.083, p = 0.773; Fig. 4B). Pattern similarity in the PPA also predicted subsequent memory beyond BOLD amplitude at the first presentation (χ2(1) = 5.409, p = 0.02) or amplitude across both presentations (χ2(1) = 5.309, p = 0.02). Furthermore, within-scene similarity was a better predictor of subsequent memory than across-scene similarity (χ2(1) = 15.433, p < 0.001). Thus, within a scene-selective cortical area (PPA), we again observed a clear double dissociation: repetition suppression, but not pattern similarity, predicted repetition priming, whereas pattern similarity across repetitions, but not repetition suppression, predicted subsequent explicit memory (Figs. 3, 4).
Relationship between repetition suppression and pattern similarity
Given the double dissociation between repetition suppression and pattern similarity for repetition priming and recognition memory, we also directly investigated the relationship between pattern similarity and repetition suppression. To this end, we applied a linear mixed-effects model for which repetition suppression was treated as a predictor of pattern similarity. We also included lag (unbinned) as a fixed effect. Because the amount of repetition suppression could vary across participants, we included a random slope for repetition suppression and random intercept for subject number as random effects. At a threshold of p < 0.05 (corrected), no cortical regions showed a relationship between repetition suppression and pattern similarity. At p < 0.01 (uncorrected), two regions along the fusiform gyrus showed a significant negative relationship between the two, indicating that greater repetition suppression across scene presentations was associated with less pattern similarity (Table 2, column 9). The PPA also demonstrated this negative relationship (χ2(1) = 6.820, p = 0.009).
Discussion
We compared neural responses across repetitions of visual scenes using two different fMRI measures—repetition suppression and multi-voxel pattern analysis—to determine how they correspond to implicit (priming) and explicit (recognition) memory. We observed a clear double dissociation. Across prefrontal, parietal, and occipitotemporal regions, repetition suppression predicted repetition priming, but not subsequent recognition memory. In contrast, pattern similarity in occipitotemporal regions predicted subsequent recognition memory, but not priming. This double dissociation was also evident in functionally defined PPA. This novel double dissociation suggests that repetition suppression and pattern similarity track different aspects of visual encoding and representation. Below we separately consider how repetition suppression and pattern similarity relate to (1) each other, (2) implicit memory, and (3) explicit memory. To preview, we propose that repetition suppression reflects processing fluency, while distributed patterns of activity reveal the consistency of stimulus-specific representation.
Repetition suppression and pattern similarity
Few studies have directly compared repetition suppression and MVPA. Although there is evidence that these measures show similar sensitivity for oriented gratings in early visual areas (Sapountzis et al., 2010), there are several examples of dissociations—particularly in higher level visual areas. For example, in the lateral portion of object-selective lateral occipital cortex (LOC; Grill-Spector, 2003), differences in object shape are better tracked by MVPA than repetition suppression, whereas in ventral LOC, differences in object shape are better tracked by repetition suppression than MVPA (Drucker and Aguirre, 2009). For visual scenes, both MVPA and repetition suppression reveal sensitivity to landmark identity within scene-selective posterior regions such as retrosplenial complex and PPA (Morgan et al., 2011). However, category information (e.g., jungle vs mountain) is better captured by MVPA (Epstein and Morgan, 2012). Our study showed a modest, but significant, negative relationship between stimulus-specific repetition suppression and pattern similarity: scenes that produced more repetition suppression showed less pattern similarity. Thus, while a primary advantage of MVPA—as opposed to univariate analyses—is thought to be the increased sensitivity it affords (Norman et al., 2006), there are contexts in which repetition suppression may be more sensitive than MVPA to visual properties of a repeated stimulus (Drucker and Aguirre, 2009) or behavioral facilitation that occurs across repetitions (shown here). Regarding the tuning and spatial organization of the underlying neural population (Drucker and Aguirre, 2009), our view is consistent with a possibility raised by Epstein and Morgan (2012): that MVPA capitalizes on “coarse-grain” clustering of information, while repetition suppression may reflect dynamic processes that “operate on top of the underlying neural code.”
Implicit memory
We found that repetition suppression across a wide range of cortical areas was related to repetition priming (our measure of implicit memory). However, one complication in understanding the relationship between repetition suppression and implicit memory is that the mechanisms of repetition suppression remain debated (for review, see Grill-Spector et al., 2006). Adaptive filtering—one prominent account of repetition suppression—holds that additional exposure to a stimulus causes pruning of cells that poorly represent stimulus features, resulting in a smaller, more selective, population response. With a more sparse population representing the stimulus, activation of the population is presumably more efficient, transmitting perceptual decisions downstream more quickly, resulting in behavioral priming. At the level of individual cells, this type of sharpening should lead to changes in the distributed patterns that represent visual stimuli; however, to the extent that pruning of cells is widely and evenly distributed, it is unlikely that fMRI can provide evidence for pruning at voxel level. Thus, while an extreme account in which pruning occurs at voxel level would predict a negative relationship between pattern similarity and priming (i.e., greater dissimilarity associated with greater priming), a more likely scenario is that fMRI pattern similarity across repetitions will not be sensitive to pruning and, therefore, will be unrelated to priming. Here, we did not observe any regions for which pattern similarity predicted priming (either positively or negatively).
Another leading account of repetition suppression is that the entire population of relevant neurons exhibits dampened or shortened activity to repeated stimuli (Henson and Rugg, 2003), reflective of processing fluency and, therefore, behavioral priming. By this account, largely the same population of neurons is activated at each repetition. As such, pattern similarity would be relatively independent of the magnitude of dampening (repetition suppression) and, therefore, independent of priming, consistent with what we found here. Thus, while the present results do not differentiate between these leading models of repetition suppression, they are consistent with them.
An additional consideration is that our measure of implicit memory likely reflects multiple levels of priming. Because our task involved repetition of a stimulus (e.g., scene A), decision (“indoor or outdoor?”), and response (e.g., “no”), we cannot isolate the level at which priming occurred. The wide range of cortical sites that linked repetition suppression and priming (prefrontal, temporal, and occipital areas) is consistent with the idea that multiple forms of priming may have occurred. Response learning (i.e., simple stimulus-response mappings), which can allow the brain to bypass elaborate perceptual processing, may be a particularly robust form of learning (Dobbins et al., 2004). However, repetition suppression within lateral temporal and ventral occipitotemporal cortex has been shown to dissociate from responses in prefrontal cortex, with prefrontal cortex relatively more sensitive to task demands and occipitotemporal regions exhibiting repetition suppression independent of task (Xu et al., 2007; Race et al., 2009). Thus, we believe repetition suppression observed in occipitotemporal regions here is more likely related to perceptual facilitation than response learning. We also predict that within occipitotemporal regions, pattern similarity across repetitions would be largely unaffected by stimulus-response reversals.
Explicit memory
A long history of research demonstrates that implicit and explicit measures of memory are dissociable (Squire et al., 1993). As reviewed in the introduction, attempts to link explicit memory to repetition suppression have yielded mixed results. We did not observe a positive relationship between repetition suppression and subsequent recognition in any of our 48 cortical ROIs—even at a relaxed statistical threshold. We observed a negative relationship at a more liberal (uncorrected) threshold in two ROIs (frontal pole, angular gyrus); however, neither of these ROIs exhibited overall repetition suppression. Interestingly, we observed a modest relationship between priming and subsequent recognition, consistent with findings from Turk-Browne et al. (2006). However, this effect was largely driven by relatively long RTs during the first presentation of subsequently remembered scenes. Thus, the lack of relationship between repetition suppression and subsequent recognition in the present study indicates that increases in processing fluency do not correspond to better long-term, explicit memory.
Accumulating evidence indicates that explicit memory is related to neural pattern similarity. Our finding that similarity across repetitions predicted subsequent recognition replicates the findings of Xue et al. (2010), who observed relationships between pattern similarity and subsequent recognition of faces in similar neural regions (e.g., superior LOC). Explicit memory is also positively related to item-level pattern similarity in occipitotemporal regions when comparing encoding and recognition trials (i.e., when an encoded item is “repeated” in a recognition test) (Ritchey et al., 2012). Moreover, within ventral temporal cortex, similarity across different items within a category (e.g., similarity between two different scenes) is positively related to subsequent explicit memory (Kuhl et al., 2012a; LaRocque et al., 2013). However, our study confirmed that explicit memory was better predicted by similarity across repetitions of the same scene than repetitions of different scenes. Thus, consistent item-level representation of a stimulus is associated with better explicit memory.
How might consistent representations support explicit memory? One possibility is that when a stimulus repeats, the original episode is reactivated (Xue et al., 2010). Thus, while the second presentation of a stimulus is, nominally, a second encoding event, pattern completion mechanisms supported by the hippocampus may trigger reactivation of the initial presentation (Kuhl et al., 2010). Such reactivation at the second presentation may simply be a reflection of how well the stimulus was initially learned (i.e., better initial encoding = higher probability of reactivation = greater pattern similarity). However, we confirmed that pattern similarity was a better predictor of subsequent recognition than the BOLD response at initial encoding. Thus, a reactivation account of the present results and Xue et al. (2010) would more likely require that reactivation plays a causal role in strengthening. One test of this account would be to assess whether similarity across repetitions is related to hippocampal activity (a potential marker of pattern completion) at the second presentation of an item. Alternatively, similarity may reflect consistent engagement of item-level processing (e.g., processing the same stimulus features across repetitions) or consistent attention across repetitions. However, an attention account appears unlikely in that consistent attention would also be expected to increase repetition suppression effects (Yi and Chun, 2005) whereas we found that pattern similarity and repetition suppression were not positively related.
In summary, our findings reveal that repetition suppression and pattern similarity differentially predict implicit and explicit memory. Further characterizing how these neural measures relate to learning and memory will advance our understanding of the underlying neural codes and their relevance to behavior.
Footnotes
Acknowledgements: This work was supported by research grants from the National Institutes of Health (R01 EY014193 to M.M.C.) and the National Science Foundation (to E.J.W.). We thank the members of the Yale Visual Cognitive Neuroscience Laboratory for helpful feedback and G. Lupyan for discussion and comments on previous versions of this manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Emily J. Ward, Department of Psychology, Yale University, 2 Hillhouse Avenue, New Haven, CT 06511. emily.ward{at}yale.edu
