Episodic memory retrieval is assumed to rely on the rapid reactivation of sensory information that was present during encoding, a process termed “ecphory.” We investigated the functional relevance of this scarcely understood process in two experiments in human participants. We presented stimuli to the left or right of fixation at encoding, followed by an episodic memory test with centrally presented retrieval cues. This allowed us to track the reactivation of lateralized sensory memory traces during retrieval. Successful episodic retrieval led to a very early (∼100–200 ms) reactivation of lateralized alpha/beta (10–25 Hz) electroencephalographic (EEG) power decreases in the visual cortex contralateral to the visual field at encoding. Applying rhythmic transcranial magnetic stimulation to interfere with early retrieval processing in the visual cortex led to decreased episodic memory performance specifically for items encoded in the visual field contralateral to the site of stimulation. These results demonstrate, for the first time, that episodic memory functionally relies on very rapid reactivation of sensory information.
SIGNIFICANCE STATEMENT Remembering personal experiences requires a “mental time travel” to revisit sensory information perceived in the past. This process is typically described as a controlled, relatively slow process. However, by using electroencephalography to measure neural activity with a high time resolution, we show that such episodic retrieval entails a very rapid reactivation of sensory brain areas. Using transcranial magnetic stimulation to alter brain function during retrieval revealed that this early sensory reactivation is causally relevant for conscious remembering. These results give first neural evidence for a functional, preconscious component of episodic remembering. This provides new insight into the nature of human memory and may help in the understanding of psychiatric conditions that involve the automatic intrusion of unwanted memories.
Perceived information can reverberate with stored memory traces (Tulving et al., 1983). This fast and involuntary process is considered a decisive ingredient for the “mental time travel” implied in episodic memory retrieval, reinstating sensory features of study episodes and biasing subsequent retrieval processes leading to the experience of recollection (Tulving, 1983; Tulving et al., 1983). This fundamental memory mechanism, termed “ecphory,” has long been described phenomenologically and studied behaviorally (Semon, 1911; Tulving, 1976; 1982). Neurophysiological evidence for such fast reinstatement processes has not emerged until recently (Waldhauser et al., 2012; Wimber et al., 2012; Jafarpour et al., 2014). Using electrophysiological methods, these recent studies show that a reactivation of sensory memory traces can occur very rapidly, within 500 ms after onset of a retrieval cue. However, whether this early reactivation process functionally drives episodic memory retrieval, as assumed in the theoretical idea of ecphory, remains unclear. We used a visual-half field paradigm to identify the oscillatory signature of early reactivation (Experiment 1) and, moreover, to show that ecphoric reactivation is functionally relevant for episodic memory retrieval (Experiment 2).
Retrieval from episodic memory leads to a reactivation of sensory memory traces in the hemisphere contralateral to the visual field of initial presentation (Slotnick and Schacter, 2006; Waldhauser et al., 2012). We made use of this principle to track the rapid reactivation of individual memory representations, measuring and modulating brain oscillatory activity by means of electroencephalography (EEG, Experiment 1) and rhythmic transcranial magnetic stimulation (rTMS, Experiment 2). Participants were engaged in instructed or noninstructed encoding of everyday objects presented either in the left or right visual field (LVF/RVF; see Fig. 1). In a subsequent retrieval task, memory cues were presented at the center of the screen and participants engaged in item recognition followed by a source memory task. Source memory was examined to test for the ability to retrieve contextual details of the study episode, which is considered a hallmark of episodic memory (Tulving, 1983).
Previous studies were able to localize memory reactivation effects during retrieval to lateralized visual cortical areas active during encoding (Gratton, 1998; Slotnick, 2004; Slotnick and Schacter, 2006) and investigated the timing of lateral reactivation in terms of event-related potential effects (Gratton, 1998; Slotnick and Schacter, 2010). However, it still remains unclear whether and how rapid visual cortical reactivation is causally relevant to explicit episodic memory (Slotnick and Schacter, 2010; Thakral et al., 2013), leaving a fundamental question of memory theory unanswered. To shed light on these questions, we investigated brain oscillations as a highly sensitive, physiologically relevant measure of cortical activity. Following recent ideas (Hanslmayr et al., 2012), a desynchronization of oscillatory power in the α/β frequency bands indicates an increase in the complexity of firing patterns, thereby allowing for higher information coding capacity during memory encoding and retrieval. Thus, we expected a specific decrease of α/β oscillations in the brain hemisphere contralateral to the visual field of encoding as a marker of memory reactivation. In line with the theoretical notion of ecphory, this α/β power decrease was hypothesized to occur very rapidly (i.e., within 500 ms) (Tulving et al., 1983; Waldhauser et al., 2012; Jafarpour et al., 2014), before the occurrence of signatures of recollection and controlled retrieval processes (Rugg and Curran, 2007). Applying rTMS at the neural sources, in the time range, and with a frequency as identified in the first experiment was expected to counteract the rapid reactivation of visual cortical areas and lead to disrupt source memory performance. This would give strong evidence for the functional relevance of ecphoric processes for episodic remembering.
Materials and Methods
Materials and procedure
Materials, behavioral task, trial timing, and basic procedure were identical for Experiments 1 and 2 (see Fig. 1). The study was approved by the Ethical Review Board at the University of Konstanz.
A total of 240 everyday objects (Rossion and Pourtois, 2004) were used in each experiment. Half of these items were presented during encoding, serving as old items during retrieval. Objects subtended a visual angle of 5.6° × 4° and were presented to the left or right visual field (LVF/RVF), 4° below and 6° lateral from central fixation. Items were preselected into 8 sets, so that semantic categories of the depicted objects, frequency of occurrence, vividness, visibility, and orientation (portrait vs landscape) were balanced across conditions.
The experiment was performed in two blocks, pertaining to the two encoding conditions (instructed and noninstructed encoding). Each block comprised the encoding phase, a distracter task, and a final retrieval phase. In the noninstructed encoding condition, participants were supposed to engage in judging the size of each object. In the intentional encoding condition, subjects were simply instructed to intentionally encode each object the best they could for later retrieval. During encoding, participants responded with the index and middle finger of the right hand whether an item was larger or smaller than a shoebox or whether the item was easy or difficult to encode. Allocation of response buttons to index or middle finger was counterbalanced across subjects. The encoding phase was followed by a 3 min distracter task to prevent selective rehearsal, consisting of counting backwards aloud from a three digit number in steps of three. During the subsequent retrieval task, 60 encoded items were presented at the center of the screen as old items together with the same amount of new items. Participants were instructed to first engage in an old/new recognition test and subsequently tested for source memory, indicating whether an item was presented to the left or right visual field during encoding. Participants responded with the index and middle finger of the right hand whether an item was old or new and whether the item was endorsed as old, whether the item was initially presented to right or left of fixation. Allocation of response buttons to index or middle finger was counterbalanced across subjects in the old/new task, but the middle finger was always used to indicate source endorsement to the RVF and the index finger was used to indicate LVF presentation. Succession of encoding conditions, old/new status of items, and presentation of the old items to the left or right visual field during encoding were counterbalanced across subjects.
Presentation of items within encoding phases was randomized so that each VF condition was followed by an item from the same or opposite VF condition with equal probability and under the constraint that the same VF could occur twice in a row. During retrieval, the sequence of items encoded in the left and right VF was also randomized under the same constraint (no more than two items from the same VF condition in a row). Additionally, old items were randomly intermixed with new items so that no more than four old or new items could occur in a row.
Analysis of behavioral data.
Behavioral data during the final memory test were analyzed based on (1) hits (i.e., the number of correctly identified old items) and (2) source hits (i.e., the number of hits for which VF of presentation was correctly identified in addition to correct item recognition). Number of hits and source hits were each divided by the number of all old items for each subject. Arguably, dividing the number of source hits by the number of hits would give a cleaner measure of pure source memory performance in addition to simple item recognition. However, we assumed that source memory performance is already constrained by item memory performance in our case, so that a “new” response to an old item (i.e., miss) prevents a correct source memory judgment, as only an “old” response prompts a subsequent source memory task. In addition, item recognition performance may already be biased by source memory demands so that only items are endorsed as hits if a correct source memory judgment can be made. As a consequence of our calculation method, source hit rates may appear rather low. Chance level for source hits is <50% because the probability to endorse an item as being presented in the LVF or RVF already depends on the probability to endorse an item as old or new (for detailed discussion, see, e.g., Cycowicz et al., 2001; Batchelder and Riefer, 1990). In addition to the different types of hits, we report false alarms (new items that were incorrectly classified as old). For Experiment 2, we also investigated source false alarms (new items that were incorrectly endorsed as old and for which a source memory judgment was given). This analysis was conducted to determine whether rTMS affects the tendency to misattribute a new item to the VF contralateral or ipsilateral of stimulation. Behavioral data were analyzed in uncorrected repeated-measures ANOVAs or uncorrected two-sided paired t tests.
Sixteen right-handed (Oldfield, 1971) subjects (11 female) with a mean age of 24 years (range 21–37 years), normal or corrected-to-normal vision, and no history of neurological of psychiatric disease participated in this study. Participants received course credit or a monetary reward of 20 € for participating in the experiment. Informed consent was acquired from each participant before the experiment.
EEG was recorded using equidistant 128 Ag/AgCl scalp electrodes (DC-100 Hz, 512 Hz sampling rate) referenced to common average. A time-frequency representation of the EEG signal (1–30 Hz) was derived by means of Morlet wavelets (width 5). Because we had no hypotheses concerning the different encoding tasks and because no statistical differences between encoding tasks were obtained (see Results; Table 1), EEG power was collapsed across encoding tasks. To quantify event-related signal changes on sensor level, poststimulus power change was calculated in relation to a prestimulus baseline period (−500 to 0 ms) for both encoding and retrieval phases. No significant electrode clusters emerged between conditions in the baseline interval in a fieldtrip cluster statistic calculated at all electrodes for the frequency ranges of interest during encoding and retrieval.
To identify time windows and the frequency range of interest, EEG power during encoding was first compared between LVF and RVF conditions in a sliding-time window fieldtrip cluster statistic (Maris and Oostenveld, 2007). Here, we continuously applied cluster statistics for time and frequency windows of 200 ms and 2 Hz in steps of 100 ms and 1 Hz to the data from 1 to 30 Hz and from 0 to 1500 ms (Staudigl and Hanslmayr, 2013). Following our hypotheses, we directly compared posterior electrodes over the respective hemisphere contralateral to the VF of presentation between LVF and RVF conditions. We did so by inverting EEG power values in the RVF condition for each time and frequency point at left-hemispheric parieto-occipital sensors through multiplication with −1. Then, we compared the result in each time-frequency window with the corresponding (noninverted) power values at right-hemispheric sensors for the LVF condition. Each cluster statistic then reveals whether there is a contralateral power decrease, with the highest likelihood of significant effects if there is a strong contralateral decrease in both LVF and RVF conditions. To correct for multiple comparisons, the resulting 630 p values were adjusted following the false discovery rate procedure (Benjamini and Hochberg, 1995; Benjamini and Yekutieli, 2001). The result of this analysis indicated electrode clusters yielding significant EEG power differences between LVF and RVF conditions over the respective contralateral hemispheres in the alpha/ lower beta range (8–20 Hz), most prominent between 200 and 700 ms after stimulus presentation (see Fig. 3A, top).
However, the sliding cluster analyses can only suggest the presence of a statistical difference between VF conditions at the contralateral hemispheres, but it is not able to reliably show the actual topographical clusters that differ between VF conditions and to take into account ipsilateral and contralateral sensors. To this end, we subtracted noninverted data in the RVF from the LVF condition and calculated a fieldtrip cluster statistic in the time (200–700 ms) and frequency (8–20 Hz) window suggested by the sliding analysis, allowing us to identify significant electrode clusters in each hemisphere (see Fig. 3A, middle). The mean EEG power at these clusters interacted significantly between VF (LVF/RVF) and left and right brain hemispheres (LH/RH; see Fig. 3A, bottom), as indicated by a two-way ANOVA.
To investigate lateralized activity during retrieval, EEG power differences between contralateral and ipsilateral electrode clusters identified during encoding were averaged over LVF and RVF conditions and compared over time in a frequency range between 1 and 30 Hz. Results were masked by the results of running Wilcoxon signed-rank test applied at each time (∼50 ms) and frequency (∼0.5 Hz) bin resulting from the Morlet wavelet time-frequency transformation. Because our hypotheses were concerned with rapid contralateral decreases in α/β power, we focused on early (<500 ms) interaction effects as indicators for ecphoric processes. Negative values in Figure 3B (top) indicate uncorrected significant (p < 0.05) differences between contralateral − ipsilateral electrodes collapsed across both VF conditions. Based on the results of this exploratory analysis, the interaction between VF (LVF/RVF) and left and right brain hemispheres was tested in an uncorrected two-way repeated-measures ANOVA taking into account mean EEG power (10–25 Hz, 100–200 ms) at the LH/RH clusters identified during encoding (see Fig. 3B, bottom, for mean power depending on condition and hemisphere and Fig. 3B, middle, for topographical distribution of mean EEG power). To shed light on hemisphere specificity of these effects (see Fig. 4), additional two-sided t tests were performed in the 100–200 ms time window, comparing LVF-RVF differences in the hemisphere-specific peak frequencies at the LH and RH electrode clusters.
To test whether reinstatement strength reflecting in a contralateral α/β power decrease is linked to behavioral performance, we compared the amount of lateralization between source hits and source misses in a two-sided t test. Lateralization was defined as the mean difference between the contralateral − ipsilateral electrode clusters in the 10–25 Hz frequency band from 100 to 200 ms. We expected lateralization to be more negative for source hits than source misses. In a second step, we also investigated whether single trials could be identified as source hits and source misses (i.e., hits that were incorrectly attributed to the VF opposite to the VF of actual presentation) based on the amount of contralateral − ipsilateral lateralization at the electrode clusters in the 10–25 Hz frequency band by means of a contingency table (Table 2). In this approach, EEG signals were not baseline corrected to avoid misclassification of trials due to potentially higher baseline noise in nonaveraged single trials. Instead, a lateralization index was estimated for each trial by correcting for the total power at the contralateral and ipsilateral electrodes (lateralization index = contralateral − ipsilateral/contralateral + ipsilateral) (compare Händel et al., 2011). A lateralization index <0 signifies a trial as having a relative power decrease at contralateral electrodes. We hypothesized that such trials would be classified as source hits with a frequency above chance, whereas trials with a lateralization index >0 would more often be classified as source misses. We tested this assumption by means of a χ2 test for all trials from all participants (i.e., fixed-effects analysis). To make the data more accessible, we also report classification performance in percentage ((correctly classified source hits + correctly classified source misses)/all items) × 100). In addition, we also tested whether the mean differences between observed cell frequencies and expected cell frequencies in χ2 tests across all participants were significantly different from zero on the group level by means of a one-sample t test (i.e., random-effects analysis).
To identify sources of oscillatory activity, we used the Dynamic Imaging of Coherent Sources beam-forming approach after calculating the cross-spectral density matrix. We used individual electrode positions as acquired with a sensor digitization tool (Xensor; www.ant-neuro.com) and the FieldTrip standard MRI. Dynamic Imaging of Coherent Sources constructs adaptive spatial filters to localize power for each grid point in the entire brain (Gross et al., 2001; Gross et al., 2003). For each individual and the time periods of interest during encoding and retrieval, filters were calculated using activity following the encoding stimulus and the recognition cue, respectively, and baseline periods corresponding to the length of the poststimulus period of interest (Dalal et al., 2008), including the trials from both LVF and RVF conditions. For encoding data, a baseline period from −500 to 0 ms and poststimulus period from 200 to 700 ms were chosen. For retrieval data, a baseline period of −100 to 0 ms and a poststimulus period of 100 to 200 ms were chosen. Cross-spectral density and source power were estimated using frequency analysis with Slepian multitapers as implemented in FieldTrip (Oostenveld et al., 2011) for the frequency range observed at sensor level (i.e., 8–20 Hz for encoding and 10–25 Hz for retrieval). The resulting average source estimate for each condition in the time intervals of interest was corrected for source activity in the baseline interval and subsequently statistically compared with the other condition and interpolated to the standard MNI brain. In a final step, we aimed at identifying the voxels that differentiated with most sensitivity between conditions across both hemispheres. To this end, we statistically compared the RVF-LVF power difference in the right hemisphere with the corresponding effect in the left hemisphere in a one-sided FieldTrip source cluster statistic. This was done again for both encoding and retrieval effects. The results were again interpolated to the standard MNI brain (see Fig. 5). The obtained maximum interhemispheric difference between conditions during retrieval was determined at MNI coordinates ±40, −78, 0. Source localization results were visualized using the Caret software (http://brainvis.wustl.edu).
Twenty-four right-handed subjects (14 female) with a mean age of 23 years (range 18–27 years) and normal or corrected-to-normal vision completed the experiment. In addition to a general assessment of history of neurological and psychiatric disease and medication, an additional TMS screening was conducted before the experiment (Rossi et al., 2011). Furthermore, a detailed explanation of the TMS method and its risks was provided for the subjects (Rossi et al., 2009). Informed consent was acquired from each participant before the experiment. One participant was excluded before the start of the study due to unclear status of inner ear damage, and one further person aborted the experiment because of neck pain during phosphene stimulation. Participants received course credit or a monetary reward of 20 € for participating in the experiment and an extra 10 € for taking part in structural MRI assessment.
Behavioral task and procedure were almost identical to Experiment 1. Stimulation conditions were applied in a within-subject fashion during the retrieval phase of both blocks. This required pausing the procedure every 40 trials to relocate the coil between the hemispheres and change the tilt of the coil according to TMS and sham conditions. Old, new, LVF, and RVF items were equally distributed between these segments of 40 trials. Four subjects received only 36 trials per stimulation block due to experimental error, but counterbalancing and matching of condition and stimulus material were preserved. rTMS was applied to the right and left hemisphere during each of the two retrieval phases. Sham was applied to only one hemisphere during a particular retrieval block to keep the blocks at reasonable size and length. Succession of rTMS and Sham conditions was counterbalanced across subjects so that stimulation conditions and hemispheres were equally often followed and preceded by the other conditions.
TMS was applied with a Magstim Rapid2 Transcranial Magnetic Stimulator via a Magstim figure-of-eight coil. At the beginning of an experimental session, phosphene threshold (PT) was identified in each subject, using the modified binary search procedure (Tyrrell and Owens, 1988; Anderson and Johnson, 2006; Romei et al., 2010). To this end, subjects were blindfolded and single TMS pulses were applied with the coil centered midline 2 cm above the inion. Participants had to indicate the presence or absence of a phosphene by answering “yes” or “no.” PT is defined as the percentage of maximum stimulator output above which subjects consistently report seeing phosphenes. PT served as an individual marker of cortical excitability and was used to determine stimulator output during the experiment. rTMS was applied with 90% PT output, which corresponded to a mean stimulator output of 61.3% (SD 7.44%). If no reliable PT could be determined (n = 3), a fixed stimulator output of 60% was used (Romei et al., 2010). No participant reported perception of phosphenes during the experiment.
To deliver magnetic pulses with high anatomical precision, rTMS was guided by a neuronavigation system, which coregisters the individual MRI with the position of the TMS coil using a 3D tracking device (ANTVisor; www.ant-neuro.com). Individual high-resolution T1-weighted MRIs were acquired from a Siemens Skyra 3 T scanner (flip angle = 7°; TR = 2500 ms; TE = 4.06 ms) or based on other structural MRI scans from various sources if already existing for the participant. Maximum magnetic field strength was applied to the neural source of α/β oscillatory decreases in the lateral occipital cortex as identified during retrieval in Experiment 1 (MNI coordinates: ±40, −78, 0). Four TMS pulses were applied with a duration of 57 ms between pulses, with application starting at 33.5 ms and lasting until 204.5 ms after cue presentation, corresponding to a frequency of 17.5 Hz (see Fig. 6A).
Sham was applied by tilting the TMS coil in an angle of ∼60° away from the scalp surface but still touching the head. Thereby, participants still experienced clicking noise and slight vibrating sensation at the scalp surface (Sauseng et al., 2009).
Episodic memory retrieval is accompanied by a rapid decrease of α/β oscillatory power
In Experiment 1 (Fig. 1), high-density EEG was measured during encoding and retrieval to reveal the oscillatory signature of memory reactivation. In a first step, we aimed at identifying encoding-related lateralized brain activity. On a behavioral level, no difference between encoding tasks (instructed vs noninstructed) or visual field conditions (LVF vs RVF) were obtained, on either item recognition or on source memory performance (all t(15) < 1.361, p > 0.05; Table 1). In addition, there was no difference between blocks determined by encoding conditions in terms of false alarm rates (t(15) = 1.156, p = 0.266; instructed: mean ± SD, 11.6 ± 8.91%; noninstructed: 14.1 ± 8.65%). This allowed us to collapse the EEG data across encoding instructions to enhance signal-to-noise ratio and to directly compare VF conditions on a neural level. We compared EEG activity between RVF and LVF conditions at the respective left- or right-hemispheric posterior contralateral electrodes in a sliding cluster statistic (see Materials and Methods; Fig. 2) to isolate the time and frequency windows most sensitive to the expected contralateral power decreases at encoding. This was achieved by inverting power values for the RVF condition at the left-hemispheric electrodes and comparing them with power at right-hemispheric electrodes for the LVF condition. This analysis indicated maximum power differences at contralateral electrodes between the VF conditions from 200–700 ms in the α/β range (8–20 Hz, padj < 0.05, FDR corrected; Fig. 3A, top) (Genovese et al., 2002). Although this result pinpointed the time and frequency range of lateral differences between VF conditions, the analysis could neither indicate whether the effect is stronger on contralateral than ipsilateral electrodes nor answer exactly which electrodes are most sensitive to hemisphere and VF-specific effects. Thus, we subtracted power in the LVF condition from power in the RVF condition in the identified time-frequency range, such that positive values reflect a stronger α/β power decrease in the RVF condition, whereas negative values reflect a power decrease in the LVF condition (Fig. 3A, middle). We calculated a cluster statistic on all electrodes to identify the clusters most sensitive to VF specific decreases (Maris and Oostenveld, 2007). In line with our hypothesis, this analysis revealed one left-hemispheric and one right-hemispheric electrode cluster (left: p = 0.038; right: p = 0.006), each most sensitive to a power decrease in the contralateral VF (Fig. 3A, middle). This VF × Hemisphere interaction was supplemented by a two-way repeated-measures ANOVA taking into account mean power at the identified electrode clusters (F(1,15) = 62.039, p < 0.001), suggesting a significant power decrease at contralateral electrodes differing from ipsilateral activity for each VF condition (see Fig. 3A, bottom). Finally, we conducted a beamformer source analysis of the 8–20 Hz α/β oscillatory activity between 200 and 700 ms (Gross et al., 2001; Gross et al., 2003). To obtain the maximum interhemispheric difference between conditions, we subtracted the interpolated RVF-LVF power difference at source level in the right hemisphere from the effect in the left hemisphere and vice versa. The source analysis, localizing the maximum difference between contralateral and ipsilateral hemispheres, revealed neural generators of the EEG effect in the lateral (middle and inferior) occipital gyrus (LOC, BA 18/19; see Fig. 5).
To reveal the neural signature of memory reactivation, we performed statistical analyses on those electrode clusters showing significant lateralization differences between LVF and RVF at encoding. Following previous EEG studies of memory reactivation (Wimber et al., 2012; Jafarpour et al., 2014; Johnson et al., 2015) and in line with the theoretical notion of ecphory (Tulving et al., 1983), we focused our analysis at retrieval on a time-range preceding the typical timing of recollection effects, before 500 ms after presentation of the retrieval cue. As indicated by a continuous Wilcoxon sign-rank test, a significant difference between ipsilateral and contralateral electrode clusters emerged very early, 100–200 ms after cue presentation (p < 0.05), and in a similar frequency band as during encoding, between 10 and 25 Hz (Fig. 3B, top). This difference, again, was due to a power decrease at the electrode cluster contralateral to the VF of presentation during encoding, as indicated by a significant VF × hemisphere interaction (F(1,15) = 8.773, p = 0.01, see Fig. 3B). This effect appeared to be different between hemispheres in terms of peak frequencies. As shown in Figure 4A, differences between VF conditions were clustered ∼20 Hz in the RH cluster, whereas the LH cluster displayed a more specific effect at ∼10 Hz. Frequency specificity of the two hemispheres was confirmed in a statistical analysis, showing that only the LH cluster showed a significant VF difference at 10 Hz (t(15) = 2.332, p = 0.034) and only the RH cluster showed a significant VF difference at 20 Hz (t(15) = 2.616, p = 0.02; Fig. 4B). However, based on our previous studies and following the idea that we tap into similar oscillatory processes in the two hemispheres, we continued our analyses with data integrated over the whole frequency range showing a mean contralateral versus ipsilateral difference (Waldhauser et al., 2012).
To check whether contralateral versus ipsilateral differences in the 10–25 Hz frequency range are indeed due to a poststimulus decrease, and not due to a prestimulus shift of attention, we repeated the same analysis taking into account the raw data from the baseline (−500 to 0 ms) interval. This analysis yielded no significant results (F(1,15) < 3.554, p > 0.05).
According to theories on episodic memory, an ecphoric reactivation of trace information is a prerequisite for the recollection of details of an episode. Thus, we expected it to be most pronounced for source hits, for which the source of encoding could be correctly remembered. In line with this idea, the contralateral versus ipsilateral α/β power decrease was significantly greater for source hits (−7.13 ± 9.631%), compared with source misses (t(15) = 4.061, p = 0.001), the latter actually showing a relative power increase at contralateral electrodes (7.27 ± 12.534%). The lateralization effects for source hits and source misses were both significantly different from zero (source hits: t(15) = −2.962, p = 0.01; source misses: t(15) = 2.320, p = 0.035). These results suggest that the laterality of EEG power predicts to which VF an item is attributed to, in a way that subjects tend to attribute an item to the VF that is contralateral to the hemisphere displaying a power decrease. As shown in Table 2, this was also the case on a single-trial level. Source hit trials more often showed a negative lateralization index (i.e., a power decrease at electrodes contralateral to the VF in which the item was actually presented during encoding). Source miss trials more often showed a positive lateralization index (i.e., a power decrease over the ipsilateral hemisphere) (Table 2). A dependency of lateralization and response was confirmed statistically in a fixed-effects analysis (χ12 = 4.731, p = 0.03), showing that 53% of all items were classified correctly as source hits and source misses based on EEG lateralization. This statistical dependency was also confirmed in a random-effects analysis on group level, with the mean difference between observed and expected cell values being significantly greater than zero (0.951 ± 1.6374; t(15) = 2.25, p = 0.04). Notably, 11 (of 16) participants numerically showed this classification pattern (i.e., mean differences >0) on the single-trial level. Lateralization and memory performance were independent when taking into account the baseline interval from −500 to 0 ms on a single-trial level (χ12 = 1.8467, p = 0.174). Classification of source hits and source misses across all subjects based on EEG lateralization was close to chance (49%). This accorded with a random-effects analysis when comparing individual differences between expected and observed frequencies against zero (0.647 ± 1.317; t(15) = 1.885, p = 0.079).
Finally, we aimed at localizing the neural generators of α/β power decreases for source hits. Neural generators of this EEG effect corresponded closely to the sources observed during encoding, localized to the middle and inferior LOC (BA 18/19; Fig. 5).
Together, these data indicate a very rapid reactivation of neural signatures established during encoding, which are visible in a power decrease in the α/β frequency band. Source analysis suggests that α/β decreases can be localized to the LOC, a region that is known to be constitutive for object recognition during perception and encoding (Konen and Kastner, 2008) and that is sensitive to hemisphere-specific retrieval of lateralized visual memory traces (Slotnick, 2004; Slotnick and Schacter, 2006).
The very early (∼100 ms) re-emergence of α/β oscillatory decreases during retrieval is in line with previous EEG studies of early reactivation (Wimber et al., 2012; Jafarpour et al., 2014; Johnson et al., 2015) and suggests that retrieval correlates with a rapidly occurring ecphoric process. However, it is unclear whether such early reactivations are functionally relevant to successful episodic retrieval. Theoretically, these early reactivation signatures could accompany memory retrieval, but they might not be causally relevant to the retrieval process. To investigate this question, we tested whether counteracting early sensory cortical activity with rTMS in the hemisphere contralateral to the site of encoding reduces retrieval of the episodic memory trace.
Counteracting early sensory cortical activity through rTMS reduces episodic memory
Behavioral task and procedures in Experiment 2 were identical to Experiment 1 (Fig. 1), except that, instead of measuring EEG, rTMS and Sham stimulation were applied during the retrieval phase of the experiment. Small breaks during the retrieval phases were used to relocate the TMS coil for the stimulation conditions (Sham, TMS) on the different stimulation sites, left hemisphere (LH) or right hemisphere (RH) in each subject. rTMS was applied at the average center EEG frequency observed during retrieval (17.5 Hz) at the neural sources of the EEG retrieval effect obtained in Experiment 1 via a neuronavigation system (ANT-Visor; www.ant-neuro.com). rTMS was centered at the time interval of maximum EEG differences between 100 and 200 ms. Driving neural assemblies in the LOC at α/β frequencies with 17.5 Hz rTMS (Thut et al., 2011; Hanslmayr et al., 2014) should counteract the decrease of α/β power observed in Experiment 1 and therefore impair episodic memory retrieval (Waldhauser et al., 2012).
As in Experiment 1, encoding condition (instructed vs noninstructed) had no effect on later item recognition or source memory performance, nor did visual field at encoding (all t(23) < 1.686, p > 0.05; Table 1). Thus, behavioral data were again collapsed across these encoding conditions. In addition, we collapsed memory performance for items for which TMS was applied to the hemisphere contralateral versus ipsilateral to the VF of presentation during encoding. We assumed that rTMS should specifically decrease episodic memory performance for items that were presented contralateral to the hemisphere of stimulation. A two-way ANOVA on the percentage of correct source hits on all old items revealed a significant interaction between Stimulation (Sham vs rTMS) and Hemisphere (contralateral vs ipsilateral to VF of presentation; F(1,23) = 4.617, p = 0.042; Table 3; Fig. 6B). No main effect for Stimulation or Hemisphere occurred in this analysis (F(1,23) < 3.449, p > 0.05). Source memory performance was significantly lower when rTMS was applied at the hemisphere contralateral to the VF of encoding compared with contralateral source memory performance in the Sham condition (t(23) = 2.410, p = 0.024; Table 3; Fig. 6B). No difference between rTMS and Sham was found for ipsilateral stimulation (t(23) = 0.160, p = 0.874, not significant; Table 3; Fig. 6B). Source hits did not differ between contralateral and ipsilateral hemispheres when analyzing rTMS and Sham stimulation data separately (t(23) < 1.973, p > 0.05; compare Table 3).
No main or interaction effects were obtained for item recognition as measured in hit rates, regardless of source memory performance (F(1,23) < 2.474, p > 0.05; Table 3), suggesting that rTMS only had an influence on source memory but not item memory. However, there was no significant evidence (F(1,23) = 0.013, p = 0.911) that rTMS affected source hits to a larger extent than hits as tested in a 2 × 2 × 2 ANOVA with factors Type (source hits vs hits), Stimulation (Sham vs rTMS), and Hemisphere (contralateral vs ipsilateral), possibly because both measures are not fully independent from each other. To further explore the nature of the rTMS effect on memory, several control analyses were performed. For instance, we investigated in how far rTMS introduced a response bias. To this end, we tested whether rTMS increased false alarm rates and whether a potential effect would be dependent on the hemisphere of stimulation in a two-way repeated-measures ANOVA with factors Stimulation (rTMS, Sham) and Hemisphere (LH, RH). There was no significant main or interaction effect differentiating false alarm rates during LH (mean ± SD, 9.5 ± 9.00%) or RH (9.1 ± 4.28%), rTMS and LH (8.4 ± 8.29%) or RH (9.0 ± 7.15%) Sham stimulation (all F(1,23) < 0.449, p > 0.05). Second, we tested whether rTMS increased or decreased the tendency to attribute false alarms to the visual field contralateral to the site of stimulation (Table 3, source false alarms). Again, no significant main or interaction effect emerged in the two-way repeated-measures ANOVA with factors Stimulation (rTMS vs Sham) and Hemisphere (contralateral vs ipsilateral; all F(1,23) < 3.206, p > 0.05).
Together, these analyses show that rTMS specifically affects source memory performance for items previously presented in the VF contralateral to the stimulated hemisphere compared with Sham stimulation. Such an effect could not be observed for hits regardless of source memory performance, but the reduction in source hit performance was not significantly different from performance for hits in general. Finally, we could rule out any unspecific effect of rTMS on response execution or memory bias by showing that Stimulation had no effect on the endorsement of new items in terms of false alarm rates. This suggests that α/β power decreases are especially relevant for episodic memory through providing rapid sensory reactivation as a basis for later source memory performance.
We here show that very rapid reactivation of sensory information is functionally relevant for episodic memory retrieval. This conclusion is supported by the results of two independent experiments described above. First, lateralized encoding patterns of α/β power decreases reemerge rapidly in visual cortical areas during retrieval. Second, interfering with these early reactivation patterns reduces episodic memory retrieval. These findings add to the literature in demonstrating very rapid, context-specific memory reactivation, termed ecphory (Waldhauser et al., 2012; Wimber et al., 2012; Jafarpour et al., 2014; Johnson et al., 2015). Our results go one critical step beyond the previous findings in showing that these rapid reactivations of sensory memory traces are functionally relevant for episodic memory retrieval and directly affect the ability to retrieve contextual details of the study episode. These findings are a major conceptual advance for episodic memory research, providing first evidence that ecphoric processes are causally related to episodic memory retrieval.
Ecphory has long been suggested as a prerequisite for the “mental time travel” implied in retrieval from episodic memory (Tulving et al., 1983). It is assumed that incoming sensory information from retrieval cues reverberates with stored memory traces, leading to their immediate and involuntary reactivation (Tulving, 1982). While the theoretical idea has a long tradition in cognitive psychology (Semon, 1911) and is corroborated by behavioral data (Sheldon and Moscovitch, 2010), neural evidence for this process has been sparse. Typically, neural correlates of episodic memory retrieval were identified in a time range not before ∼300 ms after onset of a retrieval cue (Rugg and Curran, 2007). However, recent studies showed early reactivation patterns during episodic retrieval that are in line with our results (Waldhauser et al., 2012; Wimber et al., 2012; Jafarpour et al., 2014; Johnson et al., 2015). Albeit using different analysis strategies and stimuli, these studies show that a replay of context information can occur rapidly, well within 500 ms after presentation of a retrieval cue. The exact neural mechanisms of this sensory reactivation and its interaction with controlled retrieval processes are still unclear and deserve further investigation. According to cognitive theories, ecphory is a necessary but not sufficient prerequisite for episodic retrieval (Tulving, 1982, 1983; Moscovitch, 2008). The sensory information needs to be further processed by higher retrieval mechanisms, possibly through hippocampal-neocortical loops (Horner et al., 2012; Staresina et al., 2012) and potentially guided by prefrontal and parietal control (Ranganath and Paller, 1999; Cabeza et al., 2008). As our data suggest, disrupting early sensory reactivation via external stimulation negatively affects the recollection (i.e., retrieval of contextual details) of episodic memories. Interestingly, our results match closely with one of our earlier studies that showed that inhibiting retrieval of interfering information leads to an increase in α/β oscillatory power in the hemisphere housing unwanted memory traces that interfere with the retrieval of a target memory trace (Waldhauser et al., 2012). This suggests that early modulations of α/β power are a decisive ingredient of successful remembering and can already act in concert with retrieval goals and be biased by higher-order cognitive control mechanisms.
Discussing the possible involvement of prefrontal control processes calls into question whether “ecphory” is the right label for the early sensory reactivation effects observed in the present data. However, as already mentioned in early papers on this topic, higher-order control mechanisms are likely to interact with rapid sensory reactivation during ecphory (Tulving et al., 1983; Johnson, 1992; Lepage et al., 2000; Rugg and Wilding, 2000). Recent neuroscientific studies also suggest that it is difficult to view these processes in separation, even at early stages of memory processing and during involuntary retrieval (Kompus, 2011; Kompus et al., 2011; Waldhauser et al., 2012).
In line with our hypothesis, a decrease of α/β oscillatory power at the hemisphere contralateral to the VF of encoding was most pronounced for source hits, to the extent that single trials could be classified as source hits or source misses on the basis of EEG lateralization. In the same vein, rTMS had a selective influence on source memory performance (albeit not to the extent of yielding significant differences between source hits, and hits; i.e., item memory). This pattern of results suggests that α/β power decreases are particularly relevant for source memory performance, which is in accordance with the theoretical notion of ecphory because correct source memory judgments require the highest amount of ecphoric information (Tulving et al., 1983). Unexpectedly, in the EEG analysis, source misses showed α/β lateralization in the opposite direction compared with source hits. Interpretation of this result has to remain speculative at this point. It appears that visual cortical activity predicted later memory decisions, such that a retrieved memory representation is attributed to the VF contralateral to the hemisphere where a decrease of α/β power occurred. It could be the case that memory representations attributed to the wrong VF were initially stored in the ipsilateral hemisphere during encoding. Alternatively, this misattribution may be due to nonsystematic fluctuations in α/β activity that led to illusory recollection of the wrong hemifield (e.g., Lange et al., 2014). Finally, it might be the case that EEG lateralization in general reflects a process of source reconstruction, and not sensory reactivation. However, this possibility seems rather unlikely because reconstructive processes during memory retrieval are more effortful and typically observed later (>600 ms) during retrieval processing (Johansson and Mecklinger, 2003; Herron, 2007). Future studies, possibly combining EEG and high-resolution fMRI or using electrophysiological methods that allow for assessing gamma oscillations (see below), might allow to distinguish between true versus false source memories in this paradigm (Slotnick and Schacter, 2004; Sederberg et al., 2007).
Our findings are in line with the idea that α/β power decreases reflect sensory information of episodic memory traces (Hanslmayr et al., 2012). However, we cannot conclude that these are the only frequency bands that are involved in ecphoric processing because we restrained analyses to low frequencies up to 30 Hz because EEG is not ideally suited to pick up the presumably very local high-frequency activities in the gamma range (Lopes da Silva, 2013). Furthermore, we restrained rTMS to the center frequency of the α/β range observed in Experiment 1 but did not apply stimulation with another frequency. Thus, although we have strong reason to assume that α/β power in visual cortex plays a decisive role for ecphoric processes, we cannot conclude that ecphory is specific to the α/β range. It might be that gamma, together with theta and alpha oscillations, also plays a decisive role in ecphoric processes (Osipova et al., 2006; Osipova et al., 2008). These are important questions that are beyond the scope of our study and should be addressed by future studies using techniques that allow for investigating these high-frequency activities (MEG or intracranial EEG) and applying different control frequencies in stimulation protocols (using TMS or transcranial alternating current stimulation). Another interesting question that is beyond the scope of our study is the role of hippocampal-neocortical interactions during early retrieval processes. For instance, does the rapid reactivation of sensory information depend on the hippocampus or is it a purely cortical or thalamocortical phenomenon (Staudigl et al., 2012; Headley and Weinberger, 2015; Ketz et al., 2015)? Our results add an important angle to this research topic by suggesting that the earliest interactions between sensory information and stored memory traces may occur in the α/β oscillatory band, beyond the long-discussed role of theta and gamma oscillations for memory processing. Finally, another open question is whether similar results would be observed in different sensory modalities or with different visual stimuli. Future studies, together with past research using different stimulus material and different experimental manipulations, are required to generalize our findings (Gratton et al., 1997; Wheeler and Buckner, 2003; Slotnick and Schacter, 2006; Waldhauser et al., 2012).
Together, our results show that retrieval from episodic memory leads to a very rapid reactivation of encoding activity, which is visible in α/β power decreases in visual brain regions. Affecting the cortical generators of the α/β power decrease with rTMS in the same frequency range hampered episodic memory retrieval. These observations suggest that episodic memory retrieval relies on ecphoric processes. A deeper understanding of ecphory and its manipulation with cortical stimulation techniques potentially offers new perspectives for the treatment of neuropsychiatric disorders (McNamara et al., 2001). For example, patients suffering from post-traumatic stress disorder suffer from the rapid intrusive reactivation of sensory memories pertaining to their traumatic experiences (Reynolds and Brewin, 1999). Treating memory intrusion through the external induction of oscillatory activity could be an important future therapeutic mean to assist patients in controlling unwanted memories.
This work was supported by the Swedish Research Council (Grant VR 435-2011-7163) and the University of Konstanz Young Scholar Fund (Grant 83946931) to G.T.W., and the German Research Council (Grant DFG HA 5622/1-1) to S.H. We thank Martin Dahl for valuable help during data collection and Tzvetan Popov for assistance with EEG source analysis.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Gerd T. Waldhauser, Department of Neuropsychology, Institute of Cognitive Neuroscience, Ruhr-University Bochum, GAFO 05/602, Universitätsstrasse 150, 44801 Bochum, Germany.