Abstract
The medial temporal lobe (MTL) is widely implicated in supporting episodic memory and navigation, but its precise functional role in organizing memory across time and space remains elusive. Here we examine the specific cognitive processes implemented by MTL structures (hippocampus and entorhinal cortex) to organize memory by using electrical brain stimulation, leveraging its ability to establish causal links between brain regions and features of behavior. We studied neurosurgical patients of both sexes who performed spatial-navigation and verbal-episodic memory tasks while brain stimulation was applied in various regions during learning. During the verbal memory task, stimulation in the MTL disrupted the temporal organization of encoded memories such that items learned with stimulation tended to be recalled in a more randomized order. During the spatial task, MTL stimulation impaired subjects' abilities to remember items located far away from boundaries. These stimulation effects were specific to the MTL. Our findings thus provide the first causal demonstration in humans of the specific memory processes that are performed by the MTL to encode when and where events occurred.
SIGNIFICANCE STATEMENT Numerous studies have implicated the medial temporal lobe (MTL) in encoding spatial and temporal memories, but they have not been able to causally demonstrate the nature of the cognitive processes by which this occurs in real-time. Electrical brain stimulation is able to demonstrate causal links between a brain region and a given function with high temporal precision. By examining behavior in a memory task as subjects received MTL stimulation, we provide the first causal evidence demonstrating the role of the MTL in organizing the spatial and temporal aspects of episodic memory.
- brain stimulation
- entorhinal cortex
- episodic memory
- hippocampus
- medial temporal lobe
- spatiotemporal memory
Introduction
The medial temporal lobe (MTL) plays a key role in encoding episodic memories for various types of spatial and temporal information (Cohen and Eichenbaum, 1993; Eichenbaum, 2004; Ekstrom et al., 2011). The importance of the MTL for memory is now well accepted, as researchers have reported concordant evidence from multiple methods, including observational studies of lesion patients (Scoville and Milner, 1957), experiments in rodents (O'Keefe and Dostrovsky, 1971), and, more recently, with data from noninvasive neuroimaging (Henson, 2005). However, although we know that the MTL is vital for episodic memory in general, we do not precisely understand the computational nature of the processes that the MTL uses to encode individual episodic memories in various contexts (Guderian et al., 2015; Howard and Eichenbaum, 2015; Maguire et al., 2016; Douglas et al., 2017).
Traditional methods for investigating the anatomical basis of human cognitive processes, such as lesion and neuroimaging approaches, have provided a plethora of information regarding the role of the MTL in memory (Bohbot et al., 2004; Suthana et al., 2009; Copara et al., 2014; Kolarik et al., 2018), but are unable to demonstrate causal links between a given brain region and a set of functions with high temporal precision. Localization of lesions is often uncontrolled, and the permanent nature of brain injury results in poor temporal resolution. Thus, one cannot always use lesion studies to perfectly identify the specific circumstances under which a given brain region is necessary for a given behavior (Rorden and Karnath, 2004). Analogously, neuroimaging studies are correlational, and are therefore unable to provide conclusive evidence about the necessity of a given brain region for a specific task in human subjects (Friston et al., 2002; Rorden and Karnath, 2004; Ramsey et al., 2010).
A different approach for probing brain function in humans is neuromodulation. Neuromodulation is promising because it can establish causal relationships between a brain region and particular behavioral functions (S. H. Lee and Dan, 2012; Knotkova and Rasche, 2014). Neuromodulation techniques such as optogenetics, transcranial magnetic stimulation (TMS), and electrical brain stimulation (EBS) allow researchers to transiently alter processing in a region and to determine the effects of this modulation on task performance (Knotkova and Rasche, 2014; Suthana and Fried, 2014). Showing that a cognitive process is transiently altered when a particular region is specifically stimulated demonstrates a causal link between the two (Suthana and Fried, 2014). Optogenetics is currently only implemented in animals, and TMS is only able to target neocortical structures (Klomjai et al., 2015). However, EBS is a viable research approach for certain neurosurgical patients, because it can target subcortical structures, such as the hippocampus, making it a useful approach to directly map cognitive function in humans.
Previous work has shown that 50 Hz MTL stimulation impairs spatial and episodic memory overall (Jacobs et al., 2016). We move beyond the results of that work here by using an improved analytic approach on this dataset to identify the specific features of memory that were perturbed by stimulation. To foreshadow our results, we find that in the verbal domain, MTL stimulation disrupted the temporal ordering of episodic memory and impaired the recall of items from early list positions. Analogously, we found that stimulation specifically disrupted spatial memories for objects located far from boundaries, which we hypothesized were encoded with MTL-dependent representations, potentially based on grid or place cells. By performing quantitative, item-level analyses of behavioral data collected during brain stimulation, our findings provide the first causal and temporally precise demonstration of the specific cognitive processes that the MTL uses to organize episodic memories across time and space.
Materials and Methods
We analyzed data from 49 epilepsy patients (32 female, 17 male) who had surgically implanted electrodes for localization of seizure foci as part of their evaluation for epilepsy surgery. Patients performed verbal-episodic and spatial memory tasks that we adapted such that direct brain stimulation was applied during some learning trials.
In each session, a selected electrode pair was connected to an electrical stimulator (Grass Technologies or Blackrock Microsystems). Stimulators were configured to deliver bipolar stimulation current between a pair of electrodes, each with a surface area of 0.059 cm2. MTL structures were stimulated using depth electrodes separated by either 5 or 10 mm, whereas strip and grid electrodes targeted other neocortical structures. Stimulation was applied at 50 Hz with a balanced biphasic stimulation pulse of 300 μs per phase for 1 s. Under clinical supervision, we began each session by manually testing a range of currents for each site, beginning at 0.5 mA and slowly increasing to a maximum of 1.5 mA (depth electrodes) or 3.0 mA (surface electrodes). The final stimulation current that was used for the cognitive experiments was the maximum current for each site that could be applied without inducing patient symptoms, epileptiform afterdischarges, or seizures.
We determined the anatomical location of each implanted electrode by examining an MRI scan, which provided a high-resolution image of the hippocampus and MTL (0.5 × 0.5 × 2 mm resolution). Depth electrodes in the MTL were localized via a semiautomatic process. First, MTL subregions were labeled within the MRI using a multi-atlas-based segmentation technique (H. Wang et al., 2013; Yushkevich et al., 2015). A neuroradiologist then identified each electrode contact on a post-implant CT scan, which was then coregistered with the MRI (Avants et al., 2008), and an anatomic label for each contact was automatically generated. Finally, the neuroradiologist visually confirmed the accuracy of the labeled location of the implant (Suthana et al., 2012; Jacobs et al., 2016). We designated a stimulation site as targeting a particular region if at least one electrode of the bipolar pair was in the region. Note that unlike our previous study, which analyzed stimulation effects at the level of individual sessions (Jacobs et al., 2016), here we analyzed stimulation-related changes to memory performance at the subject level, which is a slightly more conservative approach but nonetheless provided convergent results.
Each subject provided informed, written consent before participation. Our multisite study was approved by local institutional review boards (IRBs), the IRB of the University of Pennsylvania (data coordination site), and the Human Research Protection Official at the Space and Naval Warfare Systems Center Pacific. The raw data are publically available (http://memory.psych.upenn.edu/) and were analyzed previously (Jacobs et al., 2016) but the results presented here are novel.
Verbal task.
Thirty-nine patients (23 with MTL stimulation) performed the Free Recall task (Sederberg et al., 2003) while stimulation was applied during the learning of some items. Figure 1A presents a timeline of this task. During each trial, subjects were asked to memorize a list of 12 words sequentially presented as text on the computer screen. Each word was presented for 1600 ms, followed by a blank screen for 750–1000 ms. Lists consisted of high-frequency nouns (http://memory.psych.upenn.edu/Word_Pools). After a 20 s math distractor task, subjects were given 30 s to verbally recall the words in any order. We recorded the verbal word responses for later manual scoring.
Overview of verbal and spatial memory tasks. A, Timeline of the verbal memory task. Half of the words on stimulated lists are encoded with the stimulator active. B, Overview of the spatial task. Half of the trials occurred with the stimulator active during both encoding periods. Taken from Jacobs et al. (2016) with permission.
Lists consisted of two types: stimulated lists, in which half of the words on the list were delivered simultaneously with electrical brain stimulation, and control lists, in which all 12 words on a list were presented without stimulation. Each session included 25 lists, consisting of 20 stimulated lists and 5 control lists in a random order. For each stimulated list, stimulation occurred in a blocked pattern: the stimulator was active during the presentation of a pair of consecutive words and then inactive for the following pair. Thus, in total, on each stimulated list the stimulator was active for half the total words. When the stimulator activated, it was timed to occur 200 ms before the presentation of the first word in each block, continuing for 4.6 s, until the disappearance of the second word. The onset of stimulation was balanced, such that a random half of the stimulation lists began with a nonstimulated block and the others began with a stimulated block.
Data analysis: Free Recall task.
Our analyses of data from the Free Recall task followed methods used in other studies (Kahana, 2012), adapted for examining the behavioral effects of stimulation. To examine how the effects of stimulation on memory varied over the time course of each stimulation cycle, we separately measured recall probability according to the position (“phase”) of an item within a stimulated or nonstimulated block. For this analysis, we averaged recall probability for each phase within stimulated and nonstimulated blocks and then normalized relative to matched positions on control lists.
For some analyses we separately measured memory performance as a function of list position. We defined the primacy period as the first two items on each list, consistent with previous Free Recall studies using similar list lengths (Craik, 1970; Fischler et al., 1970). This classification fit the data because, after drops in performance between the recall rates for the first two list positions of >0.06, memory performance was roughly comparable across positions 3–5 (changes in recall rates of <0.025).
To assess the effect of stimulation on erroneous recalls, we measured the rates of prior list intrusions (PLIs; Darley and Murdock, 1971), which occurred when subjects incorrectly recalled an item from the previous list during the current list's recall period. Many PLI probabilities were close to zero; therefore, we assessed significance using nonparametric statistics.
In free recall, subjects exhibit a strong tendency to cluster recall sequences based on the temporal proximity of the items during study. We examined the effect of stimulation on this clustering effect by computing two measures of temporal clustering: temporal clustering factors (TCFs; Polyn et al., 2009b) and conditional response probabilities (CRPs; Kahana, 1996). The TCF is a number that measures the mean tendency for recall transitions to occur between items presented at nearby list positions. A TCF of 1 indicates perfect temporal contiguity, with the subject only making transitions to temporally adjacent items, whereas a TCF of 0.5 indicates that the subject is making transitions randomly (Polyn et al., 2009a). The CRP is a curve that indicates the conditional probability of a particular item being recalled as a function of the difference in its position in the learned list relative to that of the previously recalled item. We performed all TCF and CRP comparisons with nonparametric tests. To rule out the possibility that differences in mean recall rates between stimulated and control lists led to apparent changes in TCFs, we used a downsampling procedure to artificially match recall rates between stimulated and nonstimulated lists.
We also examined the effects of stimulation on non-MTL regions. Five subjects received stimulation in the frontal lobe, two in the cingulate, and two in the lateral temporal lobe. We termed these “neocortex” regions. To identify potential results caused by sample size differences between the MTL and neocortex datasets, we reperformed our analyses using a bootstrap procedure. In this procedure we chose a random subset of nine MTL electrodes and then computed the effect of interest and confidence intervals across 1000 iterations.
Spatial task.
In the spatial memory task (Fig. 1B), 26 subjects received MTL stimulation while participating in a virtual navigation paradigm that is reminiscent of the Morris (1984) water maze. Eight of these subjects also participated in the verbal task with MTL stimulation. The environment was rectangular (1.8:1 aspect ratio), and was surrounded by a continuous boundary. There were four distal visual cues (landmarks), one centered on each side of the rectangle, to aid with orienting. Each trial started with two 5 s encoding periods, during which subjects were driven to an object from a random starting location. At the beginning of an encoding period, the object appeared and, over the course of 5 s, the subject was automatically driven directly toward it. The 5 s period consisted of three intervals: first, the subject was rotated toward the object (1 s), second, the subject was driven toward the object (3 s), and, finally, the subject paused while at the object location (1 s). After a 5 s delay with a blank screen, the same process was repeated from a different starting location. Alternating trials (24 of the 48) were designated as stimulation trials, during which stimulation was applied throughout the 5 s of time while the object was visible to the subject for both encoding periods.
After both encoding periods for each item, there was a 5 s pause followed by the test period. The subject was placed in the environment at a random starting location with the object hidden and then asked to freely navigate using a joystick to the location where they thought the object was located. When they reached their chosen location, they pressed a button to record their response. They then received feedback on their performance via an overhead view of the environment showing the actual and reported object locations. Between stimulation and nonstimulation trials, starting location, starting orientation, and object location were all counterbalanced. This was achieved by creating a set of location triads for the stimulated conditions and transposing them across the environment's diagonal for use in nonstimulation trials, ensuring that the geometric relationship between the start and object locations was matched in stimulation and nonstimulation trials.
Data analysis: spatial memory task.
Performance in the spatial task was measured by computing the distance between the reported and actual locations for each object. In the same manner as Jacobs et al. (2016), we normalized this euclidean distance metric into a memory score (MS) between the range of 0 and 1, where a value of 1 indicates a perfect response and a value of 0 indicates the worst possible response given the object's location. This normalization took the form of ranking the subject's actual response compared with all other possible responses. This normalization procedure removes potential bias in the results by accounting for the fact that the distribution of possible response error distances varies according to the object's distance from the boundaries; it also corrects for the rectangular shape of the environment. Namely, objects near boundaries had a larger maximum possible error distance than objects in the interior. Subjects with an average MS of less than chance (0.5) were excluded from all analyses. We used Tukey's honest significant difference (HSD) correction for multiple comparisons when performing post hoc analyses of ANOVA tests.
Previous studies have shown that boundaries play a crucial role in guiding navigational behavior (Hartley et al., 2004; Chan et al., 2012; S. A. Lee, 2017, S.A. Lee et al., 2018), so we chose to analyze the effect of object location on subject performance. To this end, we divided the environment into “boundary” and “interior” regions of equal area by creating an inner rectangle with an identical aspect ratio to the environment itself.
We hypothesized that subjects sometimes used view-based spatial memory strategies, which rely on facing the same direction during encoding and recall. Such strategies would be most effective where salient visual scenes were most prominent, which occurred when the subject was close to the environment's boundaries. To test whether some subjects might have used such a strategy, we labeled the directions that subjects faced at the end of the learning trials and the test trials as the “headings” for that trial. The circular mean of the two learning headings was then subtracted from the test heading to compute the trial's learning–test heading difference. All heading statistics were calculated with circular statistics (Berens, 2009).
We conducted a simulation to assess the possibility that our task's design led to artificially decreased heading differences between learning and test for certain areas of the environment. In a single iteration of this procedure, we simulated 1000 learning and test trials, with randomly generated start and end locations. For each simulated test trial, the simulated “subject” drove in a straight line from the start to the end location. Then, across these 1000 trials, we computed the mean vector length (R̄) of the trialwise learning–test heading differences. This entire procedure was then repeated 100 times to establish confidence intervals for R̄.
Results
We sought to determine how electrical stimulation of the hippocampus and entorhinal cortex influences the precise temporal and spatial organization of memory. To answer this question, we conducted new analyses of a previously published dataset (Jacobs et al., 2016) in which subjects performed spatial and episodic memory tasks with concurrent brain stimulation in the MTL. Going beyond the previous study, which reported that MTL stimulation reduced verbal and spatial memory performance overall, our new analyses show that MTL stimulation selectively disrupted the temporal organization of verbal memory and the ability to encode spatial locations in an environment without visual cues from boundaries.
Verbal episodic memory
In the verbal memory task, subjects learned two types of word lists: stimulated lists and control lists. On stimulated lists, electrical stimulation was present for alternating blocks of two items at a time. Therefore, items on stimulated lists consisted of two categories: those delivered with stimulation (“stimulated items”) and those delivered without (“nonstimulated items on stimulated lists”). Control lists consisted entirely of items delivered without stimulation. Our data analyses separately examined recall rates across items from different categories. As reported in Jacobs et al. (2016), recall rates were lower for stimulated items relative to nonstimulated items (t(22) = −2.29, p = 0.04, paired t test), indicating that MTL stimulation impaired memory encoding. Going beyond this earlier work, we examined the time course of the effects of stimulation and whether stimulation affected memory for the order of learned items.
Effect of stimulation on verbal memory encoding
First, we examined whether the memory impairment from stimulation lingered after stimulation ended. To do this, we measured recall performance for nonstimulated items on stimulated lists, as well as for items on control lists (Fig. 2A). We found significant differences across all three conditions (one-way ANOVA; F(1,22) = 3.92, p = 0.02). Recall rates were lower for stimulated items compared with nonstimulated items (HSD-corrected post hoc t(22) = −2.29, p = 0.04, paired t test) and for nonstimulated items on stimulated lists relative to items on control lists (HSD-corrected t(22) = −1.73, p = 0.07, paired t test). These results indicate that the memory impairment from stimulation persists after the stimulation interval moderately impairing recall rates for items learned right after the stimulator was turned off.
Effect of MTL stimulation on verbal memory encoding in free recall. A, Probability of item recall averaged across all serial positions, separately computed for stimulated items (red), nonstimulated items on stimulated lists (blue), and control lists (green), then averaged across subjects. B, Recall probabilities separately computed for primacy (items 1–2) and non-primacy items (items 3–12) in the same way as A. C, Change in mean recall probability for stimulated lists versus control lists (position matched), computed relative to stimulation onset. D, Probabilities of incorrectly recalling items from different sources (labeled “item type”) while recalling items on a stimulated list. *p ≤ 0.05, **p < 0.01, †p < 0.1. Error bars are SEM.
Electrophysiological studies have suggested that different neural patterns underlie the encoding of items from early and late positions within an individual list (Sederberg et al., 2006; Manning et al., 2011; Serruya et al., 2014). To compare the role of the MTL in encoding early versus late items, we measured the impact of stimulation across the course of each list. We computed recall probabilities for each stimulation condition separately for primacy (items 1 and 2; see Materials and Methods) and for non-primacy items (items 3–12; Fig. 2B). As expected (Murdock, 1962), overall recall probabilities for primacy items were higher than for non-primacy items. However, the effects of MTL stimulation varied over the course of the list. To assess this effect, we performed a two-way repeated-measures ANOVA with factors list position (primacy/non-primacy) and stimulation condition (stim, nonstim item on stim lists, and control lists). We found that MTL stimulation impaired the recall of items more for primacy than non-primacy positions (ANOVA interaction F(1,44) = 2.78, p = 0.047). We confirmed that stimulation significantly impaired recall of primacy items (HSD-corrected post hoc t(22) = −1.95, p = 0.04, paired t test) and that this impairment was not present for non-primacy items (p > 0.8, t test).
As mentioned above, on stimulated lists the stimulator was enabled in a two-on-two-off cycle across items. To examine how memory performance varied according to the phase of the stimulation cycle, we compared the effect of stimulation on memory performance in these intervals relative to position-matched controls (see Materials and Methods). Memory performance was more strongly impaired for the second stimulated item compared with the first such item (Fig. 2C; t(22) = −2.10, p = 0.042, paired t test), thus indicating that the impairment of memory from stimulation accumulates gradually or has a delayed onset.
In addition to comparing mean accuracy rates, an additional way to assess the effects of stimulation on memory is to investigate the types of errors that are made during recall (Darley and Murdock, 1971). To test whether stimulation influenced the types of recall errors that subjects made, we examined PLIs, defined as recalls of items from the previous list rather than the current one (Fig. 2D). We found that stimulated items on a previous list had a lower probability of being the source of a PLI compared with nonstimulated items on a previous list (z = −2.12, p = 0.034, n = 23, signed rank test). This finding suggests that when an item is learned in the presence of MTL stimulation, it is less strongly maintained in memory.
Effect of stimulation on the temporal organization of memory
We next examined whether MTL stimulation altered the temporal structure of memory. In a standard delayed Free Recall task without stimulation, subjects tend to recall items in the order that they were encoded (Howard and Kahana, 1999). We hypothesized that stimulation might disrupt this phenomenon. To examine this issue, we present the results of two separate analyses of the effect of stimulation on the temporal structure of episodic memory recalls. We begin by examining recall order at the beginning of each list, followed by a broader analysis of the overall temporal order of responses across the complete list (Polyn et al., 2009a).
To examine the effects of MTL stimulation on recall order, we computed the mean probability of recalling an item from each list position in each of the first four output positions (Fig. 3). On control lists, as expected, there was a tendency for items to be recalled in the order they were viewed. However, MTL stimulation disrupted this pattern, as we show by comparing recall rates at each position between control and stimulation lists (Figs. 3A–D). Subjects exhibited decreased probabilities for recalling each of the first four items in their proper orders (items 1, 3, and 4: p values <0.01; item 2: p = 0.078). Notably, when subjects recalled three or more items on a stimulated list, they most often recalled the third item first. These results indicate that stimulation hindered subjects from encoding temporal structure during learning.
Effect of MTL stimulation on recall order for the beginning of the list. Plots show the probability of recall at different recall-output positions. Probabilities were averaged within each subject and then averaged across subjects. Plots from left to right show results separately for items that were presented at positions 1–4 (A–D respectively). *p < 0.05, **p < 0.01, ***p < 10−3, ****p < 10−4, †p < 0.1.
In the Free Recall task, item recalls tend to be temporally clustered, such that items consecutively recalled are more often learned from nearby list positions (Howard and Kahana, 2002). We examined the effect of stimulation on temporal clustering by computing two measures of list-wide temporal contiguity, the TCF (Polyn et al., 2009a) and the lag-CRP function (Kahana, 1996), and testing whether they changed with MTL stimulation (Fig. 4A). TCFs, which measure the correlation between item ordering during encoding and recall, were higher for control lists compared with conditions with stimulation (p values <0.05, rank sum tests). Despite the theoretical insensitivity of the TCF to recall counts (Polyn et al., 2009a), to rule out the possibility that the temporal factor was lowered by the diminished recall rates on stimulated lists, we reperformed this analysis after matching recall counts between control and stimulated lists with random subsampling. However, this analysis replicated the same pattern of results (p values <0.02), confirming our original interpretation that stimulation specifically impaired temporal clustering in addition to diminishing the mean recall rate.
Analysis of the effect of MTL stimulation on temporal clustering of item recalls. A, TCFs for items from each stimulation condition, averaged across subjects. B, CRP plot for control and stimulated lists, averaged across subjects. C, Difference in recall probability from stimulation (stim list − control). *p < 0.05, **p < 0.01, ***p < 10−3
To visualize the effect of MTL stimulation on the dynamics of memory, we computed the lag-CRP for each list condition. Overall, both stimulated and control lists show higher recall probabilities at short lags, as expected (Kahana, 1996). However, CRPs for stimulated lists were flatter than for control lists (Fig. 4B). In particular, with stimulation there was a significant decrease in recall probability for item transitions at lag = −1 (p = 0.032, rank sum test; Fig. 4C). These results support the notion that MTL stimulation disrupts the temporal organization of memory, by decreasing subjects' tendencies both to recall items in their viewed order and to make temporally contiguous recalls.
An unexpected feature of our data was that the CRPs for control lists were rather symmetric, as opposed to showing a moderate asymmetry (Kahana, 1996). To explain this pattern, we separately examined CRPs for items from different list positions. The CRP for the first half of each list showed a normal forward asymmetry for both control and stimulated lists, and a significantly lowered recall probability at lag = −1 for stimulated compared with control lists (p < 0.05, rank sum test). In contrast, the CRP for the second half of the list was symmetric and showed no significant differences between stimulation and control conditions (p > 0.1). Thus, the symmetry of the aggregate CRP was caused by recalls in the second half of lists deviating from expected patterns.
Stimulation outside the MTL
To compare whether the memory modulation patterns we observed were specific to stimulation in the MTL, we compared recall performance between sessions where stimulation was applied in MTL regions (hippocampus and entorhinal cortex) versus stimulation outside the MTL (neocortex). Recapitulating past results (Jacobs et al., 2016), neocortical stimulation did not significantly impair recall rates compared with control lists (Fig. 5A; p > 0.1, one-sample t test). This effect was significantly different compared with the effect of MTL stimulation (t(31) = −1.76, p = 0.044, unpaired t test). Within the neocortex group, stimulation electrodes were placed in different subregions across subjects (see Materials and Methods). However, we did not observe significant differences in the effects of stimulation between these subregions (one-way ANOVA, F(2) = 0.73, p = 0.52). There was also no significant effect of neocortical stimulation on temporal clustering (Fig. 5B). Thus, at least with the type of stimulation we used, stimulation's ability to disrupt the temporal organization of memory is specific to MTL sites and is not a brain-wide phenomenon. Finally, we note that these regional differences were not a result of differing sample sizes between MTL and neocortex, because we found the same pattern of results after reperforming this analysis with sample sizes matched using random subsampling (see Materials and Methods).
Comparison of effects of stimulation in different regions during free recall. A, Change in recall probability due to stimulation (stimulated lists − control lists). This measure was separately computed for stimulation that was applied in hippocampus/entorhinal cortex (labeled MTL) or in neocortex. B, Change in temporal clustering factor (stimulated lists − control lists) due to stimulation applied to MTL and neocortex regions. *p < 0.05, **p < 0.01.
Spatial memory
Our spatial memory task tested subjects' ability to remember the locations where items had been observed in a virtual reality environment. We began our analyses by examining overall task performance, as measured by our MS measure, for subjects with stimulation electrodes in the MTL. Patients showed a range of mean memory scores, ranging from 0.51 to 0.95. Visually, the distribution of memory scores appeared to comprise more than one performance group (Fig. 6A). We determined quantitatively that splitting our subject population into two groups provided the best fit for this performance distribution using the k-means gap statistic (Tibshirani et al., 2001). This split our subjects into two performance groups, “good-performers” and “bad-performers”, using a threshold of MS = 0.75.
Behavioral analysis of memory performance in the spatial memory task without MTL stimulation. A, Histogram of memory scores across subjects with electrodes in the MTL. B, Average memory score for good- and bad-performing subjects, separately computed for remembered locations near and far from boundaries, then averaged across subjects. *p < 0.05.
We were interested in understanding the source of this performance difference. Prior work suggests that boundaries, in particular, are an important influence on spatial navigation and memory (Hartley et al., 2004; Chan et al., 2012; S. A. Lee, 2017). Furthermore, data from subjects performing this same task without stimulation showed differences in both memory performance and neural signals near boundaries (S. A. Lee et al., 2018). This body of earlier work motivated us to consider that one way the two performance groups could be distinguished is by their behavior in relation to boundaries. Thus, our subsequent analyses separately considered boundary trials and interior trials for each of the two subject performance groups.
Assessing spatial memory strategies
We hypothesized that part of the reason that the good-performers in our task show improved performance is because they were more effective at using visual information from nearby boundaries to assist with encoding object locations. We tested this by using an ANOVA to examine the effects of object location (boundary/interior, a repeated measure), subject condition group, and their interaction on memory score for trials without stimulation (Fig. 6B). Although this analysis showed that MS did not vary significantly with object location as a main factor (F(1,22) = 2.16, p = 0.15), there was a significant interaction between subject group (i.e., good or bad) and object location (F(1,22) = 4.21, p = 0.018). This indicated that good-performers showed significantly better memory performance near boundaries compared with bad-performers. Post hoc tests confirmed that good- but not bad-performing subjects showed significantly greater MS for items located near boundaries (HSD-corrected post hoc tests: good-performers, p = 0.047; bad-performers, p = 0.93).
We confirmed that this pattern was robust by analyzing a separate dataset of 69 subjects who performed the same task without stimulation (S. A. Lee et al., 2018). Here we again found that good-performers showed a significantly larger improvement in MS near boundaries than bad-performers (interaction F(1,67) = 4.94, p = 0.028, two-way ANOVA), and that only good-performers demonstrated boundary-related performance improvements (post hoc tests: good-performers, p = 0.032; bad-performers, p = 0.99). This replication of the findings from our main dataset supports the view that good-performing subjects exhibit improved memory performance when remembering locations near boundaries in this task.
The finding that one group of subjects showed increased memory performance for remembering locations near boundaries indicated to us that it was possible that these subjects varied their memory strategy for objects in different locations. We hypothesized that the subjects who showed increased memory performance for objects near boundaries used a strategy in which they attended to visual-boundary cues during encoding and made use of those same environmental features to guide retrieval. Figure 7A presents a visualization of how this type of “visual-boundary-based” cuing could occur in this task. This technique would be more effective when subjects are near boundaries because the increased size of visual cues make them more salient. By contrast, Figure 7B depicts a “visual-boundary-agnostic” representation, in which subjects remember each location based on a global sense of their position relative to the environment. This type of representation would likely be equally useful for remembering locations both near and far from boundaries.
Analysis of navigational representations used by subjects independent of MTL stimulation. A, First-person visualization of a subject using visual-boundary-based cuing on a trial where they remembered a location near a boundary. B, Same but for a trial where the subject does not use visual-boundary-based cuing on a trial where they remembered a location near the interior. C, Probability density functions of differences in headings between learning and test trials. Length of resultant vector (R̄) for the differences between learning and test headings for each category are also indicated. Large values imply significant clustering at around 0. Heading differences were averaged across subjects in each category. ***p < 0.001.
Based on our finding that only good-performers showed better memory performance when remembering objects near boundaries (Fig. 6B), we hypothesized that good spatial encoding of boundary locations is likely to involve, at least in part, the use of visual-boundary-based scene representations. To test this hypothesis, we estimated the spatial representation used on each trial by measuring the difference between each subject's heading at the end of learning and test (see Materials and Methods). We computed the distribution of learning–test heading differences separately for the interior and boundary regions of the environment, as well as for good- and bad-performers. If a subject had a similar heading between learning and test on a given trial, it indicates that they matched visual scene information between encoding and recall. Consistent with our predictions, good-performing subjects near boundaries showed more similar headings between learning and test compared with other memory conditions (Fig. 7C; pairwise k tests, all p values <0.001). This result supports our hypothesis that good-performing subjects were more likely to use visual-boundary-based representations to assist with remembering objects near boundaries.
Objects located in the interior of the environment can easily be approached from any direction, whereas objects near boundaries are most often approached by driving from the center of the environment. To ensure that this aspect of the task design did not influence our results, we conducted a simulation of the mean learning–test heading differences that would be expected by chance (see Materials and Methods). The heading differences for good-performers near boundaries in our data were strongly clustered near zero with a mean resultant vector length R̄ of 0.82 (Fig. 7C). This mean resultant vector length was well outside the range of R̄ values that we observed in our randomized simulations (p < 0.01; simulated R̄ range 0.23–0.62), indicating that the learning–test heading similarities for boundary items were not artifacts of the task design.
Effect of stimulation
Overall, performance in the spatial memory task decreased with MTL stimulation (Jacobs et al., 2016). Given our results indicating that good-performing subjects use different memory strategies near boundaries, we assessed the effects of MTL stimulation on memory score for each subject performance group and environment area using a two-way ANOVA (Fig. 8A). This analysis showed that stimulation's effect on MS was modulated significantly by performance group (F(1,24) = 8.33, p < 0.01) and by object location (F(1,24) = 4.55, p = 0.038). Thus, MTL stimulation impaired performance more in the interior of the environment and more for good-performing subjects. A post hoc test confirmed that the memory impairment from stimulation was statistically significant for good-performing subjects in the interior of the environment (p = 0.01, one-sample t test, Bonferroni-corrected αcrit = 0.016).
Analysis of effect of MTL stimulation on memory for objects from different locations. A, Difference in memory score (×100) due to MTL stimulation for different object locations and subject types. Negative values indicate impairment from stimulation. Impairments were first averaged across trials and then across subjects. *p < 0.05, †p < 0.1. B, Scatter plot for individual subjects of the differences in memory score between stimulated and nonstimulated and between boundary and interior trials. Each point represents by how much stimulation impairs boundary trials more than interior trials. A negative value indicates increased impairment for interior trials. Line represents best-fit trend line from linear regression.
Figure 8B illustrates our pattern of results more fully by plotting the relation between each subject's mean memory score and the differential effect of stimulation on their mean memory score for boundary versus interior trials. There is a significant negative relation between these two variables (t(24) = −2.21, p = 0.038, one-sample t test), which indicates that, with increasing subject performance, stimulation caused greater memory impairment for items near the interior of the environment. These results indicate that visual-boundary-agnostic representations, which subjects likely use to remember items in the environment's interior, were specifically impaired by MTL stimulation.
Comparing the effects of MTL stimulation on memory performance between the spatial and verbal tasks, we found a weakly positive but nonsignificant correlation across subjects (ρ = 0.19, p = 0.65). However, this analysis may be underpowered because only eight subjects contributed to this comparison.
Discussion
By identifying the nature of the changes in memory performance that related to direct electrical brain stimulation, we have provided causal evidence to suggest that the human MTL supports the encoding of memories according to their spatial and temporal features. This bolsters our understanding of the specific cognitive processes implemented by the MTL to allow humans to maintain a framework of when and where past events occurred. By using temporally restricted methods to causally identify the means by which the MTL maintains episodic memory representations, our results provide strong evidence that is consistent with findings from previous lesion (Bohbot et al., 1998, 2004; Spiers et al., 2001a,c; Kolarik et al., 2018), imaging (Staresina and Davachi, 2009; Suthana et al., 2009; Copara et al., 2014), and intracranial EEG studies (Ekstrom and Bookheimer, 2007; Howard et al., 2012; Foster et al., 2013; Miller et al., 2013; Watrous et al., 2013).
Theoretical models have suggested that the MTL plays a role in supporting episodic memory coding with place cells and time cells (Howard and Eichenbaum, 2015). Place cells activate to encode an animal's presence at a specific location in a given environment, representing that location in an allocentric manner relative to the overall constellation of the environment's spatial geometry (O'Keefe and Nadel, 1978). Analogously, time cells activate at particular moments in temporally structured intervals, relative to the overall scaling of the event sequence (Eichenbaum, 2014). Based on the properties of place and time cells, it suggested that the role of the MTL, and specifically the hippocampus, in memory is to represent the overall spatiotemporal context for memory coding (Squire et al., 2004; Howard and Eichenbaum, 2015; Ekstrom and Ranganath, 2017). Our findings indicate that MTL place and time cells could be closely involved in the encoding of spatial and temporal memories (Miller et al., 2013), as the nature of the memory impairments we observed from stimulation is what would be expected if place and time cell representations were disrupted.
Notably, specific features of time cells correspond to aspects of our verbal memory findings. In rodents, the time cells that represent early moments in an interval each have shorter activations compared with those that represent later moments (MacDonald et al., 2011). As a result, the hippocampal population representation of temporal context evolves more rapidly at the beginning of a given interval (Howard et al., 2015). This phenomenon could relate to our finding that the disruptive effect of stimulation was stronger for primacy items. MTL stimulation may be more disruptive at the beginning of each list if the temporal context representations in this interval were more transient, or fragile, perhaps because of the shorter-lived responses of the underlying time cells.
It is important theoretically to note that stimulation caused decreased temporal clustering (Fig. 4), as evidenced by a flattened lag-CRP curve for items learned while the stimulator was active. The clustering of item recalls based on the times when they were encoded is a key element of retrieved-context models of episodic memory (Howard and Kahana, 1999); our results therefore support these models generally and indicate that the MTL has a key functional role in the neural instantiation and encoding of episodic contexts (Howard et al., 2005). Notably, stimulation in our task occurred during encoding, whereas the item clustering changes we measured were during recall. Thus our findings causally show that clustering of recalls is, at least in part, a result of neural signals during encoding, perhaps due to the construction of associations from each viewed item to the current temporal context represented by time cells.
Analogous to how stimulation disrupts encoding of memory representations that may rely on MTL time cells, our data from the spatial task indicate that MTL stimulation could also disrupt the encoding of items that may rely on place and grid cells. Because the encoding of items in the interior of the environment was more strongly disrupted by stimulation, it suggests that encoding these locations was more directly supported by the MTL. Our finding that MTL stimulation does not significantly interfere with memories of objects located near boundaries does not necessarily indicate that the MTL has no role in the encoding of items near boundaries; indeed, MTL boundary cells provide input to place cells (Barry et al., 2006). Together these findings suggest that non-MTL structures can support view-based memory strategies, facilitating the encoding of certain types of memory items independently of the MTL. Consistent with this interpretation, there is evidence from fMRI studies that spatial scene recognition is mediated by both MTL and non-MTL structures, such as the striatum, parahippocampal place area, retrosplenial cortex, and occipital place area (Epstein et al., 2007; Doeller et al., 2008; Park and Chun, 2009; Park et al., 2011; Julian et al., 2016). Furthermore, it has also been directly shown that subjects with MTL damage have intact scene recognition abilities (Spiers et al., 2001b). These findings support our view that even when the MTL was impaired by stimulation, performance was relatively unaffected on boundary trials because extra-MTL areas were able to compensate for the deficit. Further support for this claim would arise from using a paradigm with manipulations to directly identify allocentric spatial representations, rather than our approach of inferring strategy based on mean behavior.
A potentially surprising feature of our results is that, in the spatial task, good- but not bad-performing subjects exhibited differential memory performance according to the location of an object. This emphasizes that strategy selection may be an important element of memory, for settings where multiple memory systems could potentially accomplish a task (Squire, 1992; Iaria et al., 2003; Doll et al., 2015). Because we found a correlation between subject mean performance and the tendency for stimulation to impair memory encoding far from boundaries, it suggests that good-performers were more likely to alter their spatial memory strategy depending on the object's location relative to the boundaries. This indicates that these subjects preferentially used MTL-based strategies in the interior of the environment while recruiting extra-MTL brain areas for objects near boundaries to support visual-boundary-based encoding.
Our primary result is showing that the MTL is responsible for encoding the spatial and temporal structure of particular types of episodic memories. By using causal and temporally reversible methods, we provide perhaps the strongest evidence yet for this claim. In addition to being important for our fundamental understanding of brain function and memory systems, our results have implications for guiding the future use of brain stimulation for cognitive enhancement.
Memory includes a diverse range of cognitive and neural processes. Our findings suggest that to develop brain stimulators for memory enhancement, it might be useful to tune these devices to a specific memory strategy or behavioral process. Neuromodulation in the MTL may be particularly useful for modulating memories, based on the activity of hippocampal place and time cells. Nonetheless it may still be possible that other types of neocortical stimulation could be used to modulate memory. Several types of studies have demonstrated a role for the neocortex in episodic memory, including studies with lesions (Duarte et al., 2010), direct lateral-temporal stimulation (Ezzyat et al., 2018; Kucewicz et al., 2018), and transcranial magnetic stimulation (J.X. Wang et al., 2014). Thus, probing the effects of neocortical stimulation is likely to be a focus of much future work, as it is possible that our low sample size in these regions led us to underestimate its therapeutic potential. One approach that could be useful for such enhancement is to use a closed-loop approach to stimulation by measuring ongoing neuronal activity in real-time and dynamically varying the nature of the stimulation that will be applied (Ezzyat et al., 2017, 2018). Given the complexity of human memory and cognition, this type of dynamic approach would be useful by allowing stimulation to vary according to instantaneous internal neural states as well as external environmental demands.
Footnotes
This work was supported by the DARPA Restoring Active Memory program (Cooperative Agreement N66001-14-2-4032) and the National Institutes of Health (MH061975). The views, opinions, and/or findings expressed are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. We thank Blackrock Microsystems for providing neural recording and stimulation equipment, and Michael Kahana and Daniel Rizzuto for assisting with data collection and for scientific discussions.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Joshua Jacobs, Columbia University, 351 Engineering Terrace, Mail Code 8904, 1210 Amsterdam Avenue, New York, NY 10027. joshua.jacobs{at}columbia.edu
