Abstract
New memories are believed to be consolidated over several hours of post-task sleep. The reactivation or “replay” of hippocampal cell assemblies has been proposed to provide a key mechanism for this process. However, previous studies have indicated that such replay is restricted to the first 10–30 min of post-task sleep, suggesting that it has a limited role in memory consolidation. We performed long-duration recordings in sleeping and behaving male rats and applied methods for evaluating the reactivation of neurons in pairs as well as in larger ensembles while controlling for the continued activation of ensembles already present during pre-task sleep (“preplay”). We found that cell assemblies reactivate for up to 10 h, with a half-maximum timescale of ∼6 h, in sleep following novel experience, even when corrected for preplay. We further confirmed similarly prolonged reactivation in post-task sleep of rats in other datasets that used behavior in novel environments. In contrast, we saw limited reactivation in sleep following behavior in familiar environments. Overall, our findings reconcile the duration of replay with the timescale attributed to cellular memory consolidation and provide strong support for an integral role of replay in memory.
SIGNIFICANCE STATEMENT Neurons that are active during an experience reactivate again afterward during rest and sleep. This replay of ensembles of neurons has been proposed to help strengthen memories, but it has also been reported that replay occurs only in the first 10–30 min of sleep, suggesting a circumscribed role. We performed long-duration recordings in the hippocampus of rats and found that replay persists for several hours in sleep following novel experience, far beyond the limits found in previous reports based on shorter recordings. These findings reconcile the duration of replay with the hours-long timescales attributed to memory consolidation.
Introduction
Sleep is necessary for memory consolidation (Rasch and Born, 2013); disruption of sleep for 3–6 h following hippocampus-dependent tasks impairs the formation of long-term memories (Havekes and Abel, 2017). One possible mechanism for memory consolidation in sleep is replay, the process by which neuronal assemblies active during tasks reactivate during subsequent sleep (Marr, 1971; Buzsáki, 2015). However, previous accounts have indicated that replay is limited to the first 10–30 min of sleep following a task (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Tatsuno et al., 2006; Ji and Wilson, 2007). This disparity between the short duration of replay and the long time course of sleep's effects on memory has presented a major challenge for the field, leading many to argue that replay cannot effectively serve as a mechanism for memory consolidation (Sutherland et al., 2010; Tononi and Cirelli, 2014). However, a careful inspection of previous studies reveals that replay was largely tested in well trained animals following memory tasks or exploration in familiar environments and that most studies did not maintain unit recordings beyond the first hour of post-task sleep (Havekes and Abel, 2017). Because novel experience is a powerful trigger for plasticity and learning in the hippocampus (van de Ven et al., 2016; Bittner et al., 2017), we re-examined the extent and duration of reactivation of rat hippocampal CA1 neurons in long-duration recordings following experiments in which both the task and the environment were designed to be novel experiences for the animal.
Materials and Methods
Surgery
Data from a total of 10 Long–Evans male rats weighing 250–400 g were analyzed in this study and were previously used in other studies (Mizuseki et al., 2009; Grosmark and Buzsáki, 2016; Miyawaki and Diba, 2016). We will hereby refer to these individually as the Miyawaki, Mizuseki, and Grosmark datasets. Protocols originally used to obtain these data were approved by the Animal Care and Use Committees at the University of Wisconsin-Milwaukee, Rutgers University-Newark, and New York University, respectively.
All surgeries were performed in a stereotaxic frame under isoflurane anesthesia, with recording started only after full recovery (>5 d) from surgery. In all three datasets, recordings were performed using implanted 32-channel and 64-channel “Buzsaki” and “Buzsaki-sp” silicon probes (NeuroNexus). The Miyawaki dataset was obtained with a NeuraLynx data-acquisition system at 30 and 32 kHz sampling rate; the Mizuseki recordings used a DataMAX system (RC Electronics) at 20 kHz sampling rate; Grosmark recordings were performed on Amplipex recording systems sampling at 20 kHz. All datasets were recorded from the CA1 layer of the hippocampus with two stainless steel screws implanted above the cerebellum used for referencing and grounding. Mizuseki et al. (2009) also recorded simultaneously from the medial entorhinal cortex, which was not used in the present study. In addition, Mizuseki et al. (2009) and Grosmark and Buzsáki (2016) obtained the electromyogram (EMG) using wires placed in nuchal muscles. In all datasets, behavior was monitored and position of head-mounted LEDs was tracked using an overhead digital camera. In all datasets, spikes were detected by filtering and thresholding the wide-band signal using custom software in NDManager and its plugins (Hazan et al., 2006; http://ndmanager.sourceforge.net) and automatically sorted with KlustaKwik (Harris et al., 2000; http://klusta-team.github.io/klustakwik), followed by manual inspection and reclustering using the Klusters package (Hazan et al., 2006). All data underwent additional manual inspection and reclustering to ensure consistent standards across datasets (Csicsvari et al., 1998; Barthó et al., 2004).
All three datasets featured recordings sessions with sleep and rest before and after (“PRE” and “POST”) a maze (“MAZE”) epoch. Recordings were paused for between 3 and 45 min in between these epochs (to untangle recording cables, prepare setups, transfer animals, etc.). Additional criteria were implemented for all recordings to ensure unit stability across PRE and POST periods. Any units whose isolation distance changed by >50% across PRE-POST (Schmitzer-Torbert et al., 2005) were excluded from the analyses. Additionally, any units whose firing rates dropped <30% of the mean firing rate for combined PRE and POST in any 1 h time window were also excluded. Units were categorized into pyramidal or interneurons based on their wave shape, refractory period, firing rates, and burstiness (Csicsvari et al., 1998; Barthó et al., 2004).
Behavior
Three of the animals (rats R, T, and K) from the Miyawaki dataset, which provided initial data used to generate Figures 1, 2, and 4 (individual sessions shown in Fig. 1-1), were trained to drink from water wells while water-restricted (30 min ad libitum water per 24 h) but had not been placed on any track environments. Following recovery from surgery for electrode implantation, animals were again water-restricted to motivate running on the track. Recording in the home cage commenced ∼6 h into the dark cycle (PRE). Three hours before the start of light cycle, animals were transferred to the track (MAZE). Water rewards were supplied at either ends of the track to motivate running. For rats R and K, each extended track running session was composed of two consecutive blocks. In the first block, during the first 20 trials, the track was obstructed to confine the animal to the reward platform for 2 min after each trial. After 20 trials, the second block began, during which animals could freely explore the track and platforms for water rewards. For rat R, recordings were performed on this same track for 2 additional days. These additional track sessions were not considered novel experience and were not pooled with the other data. In the third animal, rat T, no obstruction block was performed and the rat ran freely for the entire session. Three recording sessions were performed with this animal with the shape of the track modified from I to L to U on consecutive days; all of these sessions were considered novel experience. The Miyawaki track sessions lasted ∼3 h. Following the track sessions, animals were returned to the home cage and recordings (POST) resumed during their sleep and waking rest. Animals had ad libitum access to food in the home cage. To compare correlations across entire light and dark cycles, we also identified and isolated among these data a total of 20 long-duration sessions from five Long–Evans male rats spent entirely in the home cage; these sessions are individually depicted in Fig. 7-1.
As described by Grosmark and Buzsáki (2016), the four animals from the Grosmark dataset “were pretrained to search for water on a geometrically unrelated open-field 'cheese-board' maze for several days before novelty maze sessions. Once electrodes reached the CA1 pyramidal layer and the animals were well acclimatized to running for water reward as well as to the 'familiar' room as determined by the observation that the animals engaged in uninterrupted sleep in this room, a 'novelty' session was recorded. A novelty session consisted of a 'PRE' epoch in the familiar room, a novelty run in one of the three novel rooms and 'POST' epoch back in the familiar room. Only one novelty room was used per novelty session. Note also that […] in the present study the animals had never been inside of the novel rooms, ensuring that the animals had no experience of the maze context, even fleeting ones during the plugging of the electrophysiological headstages, before novelty exposure.” These animals received water rewards for completing runs across linear or circular mazes. “The RUN sessions were terminated once the animals were satiated and no longer ran for reward.” (Grosmark and Buzsáki, 2016). The recordings were all performed during the light cycle.
In the Mizuseki dataset, animals were water-restricted for ∼24 h before experiments and had been previously trained to alternate between two arms of a figure eight maze, with additional running on a running wheel in a delay area (Pastalkova et al., 2008; Mizuseki et al., 2009). One of these sessions was recorded on the first day of exposure (for 65 min) to the linear track (“novel”), though the animal had previously performed the alternation task and had multiple days of experience in the recording room. The other sessions were performed after at least 10 d of exposure to the linear track (“familiar”) placed at varying orientations in the room. Sleep and rest before (PRE) and after (POST) the maze epochs were recorded. These recordings were all performed during the light cycle.
Sleep scoring
In the Miyawaki dataset, sleep scoring was performed using EMG, movement and theta-delta power. EMG was smoothed with a 1 s Gaussian filter and z-scored and a Schmitt-trigger threshold set at 0 and 0.5 was used to detect state transitions between low and high EMG power. Periods with low and high EMG power were labeled as sleep and wake, respectively. The theta (5–10 Hz) over delta (1–4 Hz) plus (10–14 Hz) band ratio of the power spectral density was used to detect transitions between high theta and low theta, using custom MATLAB software, followed by visual inspection. Sleep states with high theta were classified as rapid eye movement (REM) and the remainder were classified as non-REM. Wake periods with high theta were labeled as “active” and the remaining were labeled “quiet”.
In the Grosmark et al. (2016) dataset, “sleep scoring was performed using hippocampal LFP (theta/delta ratio), accelerometer (movement), and EMG data” and “all state scoring was performed using TheStateEditor developed by Andres Grosmark in the Buzsáki lab (https://github.com/buzsakilab/buzcoderough/tree/master/BehavioralStateDetection)”. Active wake was “characterized by high theta-delta ratio and active movement/EMG-activity”; quiet wake (labeled “drowsy” by Grosmark) was “characterized by low overall spectral power, low movement/EMG-activity”; NREM was “characterized by high delta/theta ratio, and very low movement/EMG activity”; periods labeled as “intermediate” by Grosmark “which are short (∼30 s) states which occur at the NREM to REM transition and are characterized by highly-elevated pyramidal layer spindle (12–20 Hz) power and low movement/EMG-activity” were considered as part of NREM here; REM was “characterized by high theta/delta ratio and very low movement/EMG activity occurring after NREM episodes” (Grosmark et al., 2016).
In the Mizuseki dataset, sleep versus wake were scored by visual inspection of the animal and the local field potentials from CA1 and the entorhinal cortex while the animal was in its home cage. Quiet and active wake were not further separated. Theta (5–11 Hz) over delta (1–4 Hz) plus (12–14 Hz) band ratio of the power spectral density was used to detect transitions between high and low theta using custom MATLAB software. Sleep states with high theta were classified as REM and the remainder were classified as non-REM. These periods were further “cross-validated with experimenter notes taken while observing theta activity on-line in sleep session and verifying that the rat was sleeping” (Mizuseki et al., 2011).
Experimental design and statistical analysis
A total of 16 sessions composed of PRE/MAZE/POST recordings from 10 male Long–Evans rats were used. Statistical comparisons between POST and PRE were performed in each animal separately for the EV and cell assembly analyses, with the sample size depending on the number of isolated putative pyramidal neurons for each session. All individual sessions are shown in Fig. 1-1 (; including numbers of putative pyramidal neurons recorded in each session) to demonstrate consistency of findings across animals and datasets. Pooled results are shown in Figures 1b, 2c, and 3b, with error bars indicating SEM pooled over sessions. Statistical analyses were performed using MATLAB. All correlations shown in figures were calculated using Pearson's r and significance level was set at α = 0.001. Exact p values are provided in the Results.
Explained variance measure.
Spike times were binned into 250 ms time bins, creating an N × T matrix, where N is the number of neurons and T is the number of time bins. The time bin size was chosen based on a previous study (Tatsuno et al., 2006) and because it matches the timescales for ∼2 theta-cycle oscillations and sharp-wave ripples. Pearson's correlations, R, were determined for spike counts from neuronal pairs recorded on different shanks in 15 min sliding windows (window length 15 min, sliding 5 min steps) in PRE and POST and in the entire MAZE session to produce P, an M-dimensional vector, where M is the number of cell pairs. To assess similarity between P vectors from different windows, the Pearson correlation R of these vectors (i.e., the correlation between cell pair correlations) was determined (e.g., R[PRE,POST], R[PRE,MAZE] and R[MAZE,POST]). The main element of interest here is R[MAZE,POST], but it may be contaminated by preexisting correlations before the animal has been exposed to a maze (Kudrimoti et al., 1999; Ribeiro et al., 2004; Tatsuno et al., 2006). To address this issue, we calculated the explained variance for a window WIN, based on the square of partial correlation (Kudrimoti et al., 1999):
averaged over all PRE(k) windows for which WIN ≠ PRE(k). To further assess how much of EV can be generated by chance, we also calculated the time-reversed explained variance (REV) for each window WIN, as proposed in a previous study (Tatsuno et al., 2006):
averaged over all PRE(k) windows for which WIN ≠ PRE(k). Error bars for EV and REV in Figures 1a, 3a, 4a, 5, 6, 7a, and Fig. 1-1 indicate SD over PRE(k). For results pooled across animals and sessions, as shown in Figures 1b and 3b, error bars indicate SEM pooled over sessions. For Figure 1c–e analyses involving pooled pairwise relationships, NREM periods within PRE and POST were concatenated in each session and pairwise correlation vectors were generated. These vectors were pooled across sessions for PRENREM, MAZE, and POSTNREM. Residuals for POSTNREM and MAZE (Fig. 1e) were obtained from the respective regression lines with PRENREM.
For Figure 4 analyses on the amount of exposure required to produce significant reactivation, P of MAZE was calculated from spike taken from the first 1 min, 5 min, 15 min, 30 min, or the entire period of exploration in the MAZE session. To extract a timescale for reactivation from sessions that showed some evidence of decay in EV, we first measured the maximum EV across all 15 min (sliding) windows. We then determined the best-fit line through each window which was composed of at least 50% NREM sleep, starting up to 1 h before the maximum EV until the end of the session. The time point at which the EV dropped to half of the maximum EV was then obtained from the regression line (Fig. 1-1, ).
Temporal bias measure.
The temporal bias was calculated from the cross correlogram for each cell pair (calculated with 1 ms bins) by taking the sum of the counts from +1 to +125 ms minus the counts from −1 to −125 ms (Skaggs and McNaughton, 1996). To compare the temporal order across all pairs in POST and PRE versus MAZE, we took the difference between the number of pairs in the upper-right and lower-left quadrants of the scatter plots (i.e. “positively related”; Fig. 1f, scatter plots) and the number of pairs in the upper-left and lower-right quadrants (i.e. “negatively related”). This population temporal bias was greater in POST versus PRE. To determine the significance of this difference between POST and PRE (Fig. 1g), we compared the observed difference value against a null distribution obtained from 10,000 independent shuffles of the cell-pair identities in PRE and POST.
Cell assembly reactivation.
To investigate cell assembly reactivation, we used a recently developed technique (Lopes-dos-Santos et al., 2013; Trouche et al., 2016) based on independent component analysis (ICA) of spike trains projected onto bases obtained from principal component analysis. Briefly, a correlation matrix was generated using the N × T matrix of neuronal spike counts in 250 ms windows after it was z-scored. Principal components were calculated following eigenvalue decomposition. To eliminate spurious activity patterns, only principal components having eigenvalues above the Marcenko–Pastur distribution were retained. The z-scored activity matrix was then projected on the subspace spanned by these principal components. The resulting reduced activity matrix was then used to generate the ICA weight matrix. Each neuron received a weight for the corresponding independent component (assembly pattern). Given the random values of weights and their sign, each component was normalized to have sum 1. The dot product of each z-scored bin with the weight vector was then used to calculate the activation strength of the assembly pattern. An average reactivation strength was obtained by subtracting the mean activation strength during PRE and averaging over all of the independent components in a given time window.
Results
To examine reactivation following novel experience, we pre-trained rats to alternate drinking from two water wells in their home cages. On the first day of recording, in generating the Miyawaki dataset, the rats were placed on a linear track with water wells on platforms at each end of the track. These animals had not previously experienced any linear tracks or other large mazes, which ensured that running for water across a track was a very novel experience on the first day. On subsequent days, the length and shape of the track were modified with new segments, but the task itself was not novel. The track sessions on the first day and on subsequent days showed similar results and were combined in these analyses. For all analyses, we included only putative pyramidal units that displayed stable and isolated clusters (see Materials and Methods). Each recording session was divided into a MAZE period (for the task itself) as well as PRE and POST periods (for sleep or rest in the home cage before and after the task).
Because co-firing is considered to bind neurons into ensembles and drives synaptic plasticity, we first examined coactivity in pairs of neurons (Fig. 1a,b); we used a commonly used “explained variance” (EV) measure (Kudrimoti et al., 1999; Tatsuno et al., 2006; Johnson et al., 2010) to test whether coactivity in 250 ms time windows during MAZE persists in POST beyond the levels present in PRE. This control was necessary because cell pairs showed significant correlations (Pearson's r = 0.176, p = 5.386 × 10−28, no. of pairs = 3809) between PRE and MAZE (Tatsuno et al., 2006; Dragoi and Tonegawa, 2011, 2013; Fig. 1c) and fairly strong correlations (Pearson's r = 0.508, p = 7.482 × 10−249, no. of pairs = 3809) between PRE and POST sleep (Tatsuno et al., 2006; Dragoi and Tonegawa, 2014; Fig. 1d). However, after controlling for existing PRE correlations, regression analysis revealed a significant correlation (Pearson's r = 0.448, p = 3.219 × 10−187, no. of pairs = 3809) between MAZE activation and POST NREM sleep patterns (Fig. 1e). The time window used in this measure was chosen to correspond to the duration of typical hippocampal sharp-wave ripples and ∼2 theta oscillation cycles; therefore, EV captures neuronal activation and reactivation compressed within theta sequences and sharp-wave ripple sequences. The mean reactivation was higher in NREM compared with quiet waking rest [EV(NREM) = 0.201 ± 0.1464, EV(quiet wake) = 0.083 ± 0.06, p = 0.043]. In all sessions involving exploration of a novel track segment we observed elevated reactivation for the entire ∼3 h duration of POST (see individual sessions in Fig. 1-1), with surprisingly little evidence of decay over this time (Fig. 1b).
Neuronal reactivation persists for hours following novel experience. a, Reactivation as assessed by EV (black; mean ± SD. in 15 min bins, sliding in 5 min steps; Kudrimoti et al., 1999) was significantly enhanced in POST sleep following novel MAZE in a sample recording (37 cells, 497 pairs), relative to the chance level (green; mean + SD) assessed by reversed EV (REV;Tatsuno et al., 2006; see Materials and Methods). Detected brain states are indicated in the background. b, Results pooled across five POST sessions from three animals (Miyawaki dataset; mean ± SEM for n = 5 sessions, 205 cells and 3809 pairs; for individual sessions, see Figure 1-1). The total ratio of time in each sleep/wake state is shown in the background for each bin as a proportion of the y-axis limit. c, Neuronal pairwise correlations persist from PRE to MAZE. d, Much stronger correlations are observed across PRE and POST NREM sleep. These relationships are controlled for in the EV measure (Kudrimoti et al., 1999; Tatsuno et al., 2006). e, After regressing out for the correlations with PRE, strong partial correlations are evident between POST NREM and MAZE periods. f, Scatter plots of the temporal bias (total counts in the cross-correlogram from +1 to +125 ms minus counts from −1 to −125 ms; Skaggs and McNaughton, 1996) of cell pairs (3809 pairs from 5 sessions) on MAZE versus PRE and POST (same axes scales for both plots). g, The population temporal bias (number of scatter plot points in upper-right and lower-left quadrants minus those in upper-left and lower-right quadrants) was significantly greater in POST compared with PRE (p = 0.001, permutation test of observed value of POST − PRE compared against 10000 shuffles of cell pair identity). **p < 0.01, ***p < 0.001.
Figure 1-1
We and others previously reported that firing rates of hippocampal neurons increase after novel waking experience (Karlsson and Frank, 2008; Larkin et al., 2014; Miyawaki and Diba, 2016), with higher firing rates in POST compared with PRE (Miyawaki and Diba, 2016). To rule out that these firing-rate differences account for the increased EV in POST compared with PRE, we repeated this analysis after subsampling to equalize the firing rate for each neuron in PRE and POST but found nearly identical results. In particular, the correlation between MAZE and POST NREM, partialed out for PRE NREM, remained significant (median Pearson's r = 0.454, median p = 6.417 × 10−197 for 100 different random subsamples equalizing PRE and POST spike counts of each cell; no. of pairs = 3809). We also redid the analyses with different size time bins, ranging from 50 to 500 ms, for calculating correlation. These also produced highly similar and consistent results (Pearson's r50ms = 0.507, p = 5.916 × 10−248, r100ms = 0.505, p = 3.968 × 10−245, r500ms = 0.3633, p = 3.420 × 10−119; no. of pairs = 3809), confirming the robustness of the observation.
To examine whether the reactivation in POST is consistent with temporal sequences in the firing patterns of neurons (Skaggs and McNaughton, 1996; Lee and Wilson, 2002; Ji and Wilson, 2007), we used methods introduced by Skaggs and McNaughton (1996) to evaluate temporally ordered replay. We measured the temporal bias in the spike times of neuronal pairs [the difference in the number of positive lag (+1 to +125 ms) and negative lag (−1 to −125 ms) spike counts in the cross correlogram] in PRE, POST, and MAZE epochs (see Materials and Methods). The temporal bias in POST was significantly correlated to the temporal bias in MAZE (Fig. 1f; Pearson's r = 0.274, p = 1.1 × 10−66, n = 3809 cell pairs). The temporal bias correlation between PRE and MAZE was weaker, yet also significant (Pearson's r = 0.053, p = 0.0011), consistent with previous reports (Dragoi and Tonegawa, 2011, 2013). Importantly, in POST compared with PRE, there was a greater concentration of cell pairs that showed a bias consistent with the MAZE patterns (Fig. 1f, number of points in upper-right and lower-left quadrants minus points in the upper-left and lower-right quadrants of the scatter plots; p = 0.0010 for POST − PRE compared with 10,000 shuffles), indicating that the temporal spiking patterns in POST more strongly resemble the patterns observed during MAZE (Skaggs and McNaughton, 1996; Wikenheiser and Redish, 2013). Our results therefore demonstrate that the neuronal firing patterns in post-task sleep are consistent with a temporal replay of the sequences experienced during behavior.
To extend our analyses to ensembles of neurons and track their reactivation at a finer temporal resolution, we used a recent technique that identifies coactivate cell assemblies using ICA of neural population vectors (Lopes-dos-Santos et al., 2013; Trouche et al., 2016). The resulting ICA weight vectors were extracted from the MAZE period and their reactivation (or pre-activation) was examined in both POST and PRE (Fig. 2a). In both the sample session and pooled sessions, cell assemblies showed greater activation in POST compared with PRE (Fig. 2b,c). As expected, these reactivations occurred in conjunction with sharp-wave ripple oscillations in the CA1 pyramidal layer (Buzsáki, 2015; Fig. 2d). Importantly, the average of the POST sleep cell assembly reactivations lasted well beyond previously reported limits and well into the third hour of post-task sleep, in agreement with results from the EV analysis.
Assembly reactivation persists for hours following novel experience. a, Replay was assessed using ICA of cell assemblies (Lopes-dos-Santos et al., 2013). In a sample recording showing rasters for 37 neurons (right axis) spanning PRE, MAZE, and POST periods, ICA was used to define cell assemblies from the MAZE period (3 of 10 identified assemblies shown on far right). (Re)Activation strength of these three assemblies (left) is shown in corresponding colors. Ripple band local-field potential is shown below each panel. b, (Re)Activation of the third assembly from a is shown in PRE (left) and POST (right) panels for the same recording as Figure 1a, with sleep/wake states shown in the background. The average (re)activation strength (POST - PRE) of all 10 cell assemblies for this recording is overlaid in red (mean in 15 min windows sliding in 5 min steps; right axis). c, Mean (±SEM) cell assembly reactivation strength pooled across five sessions from three animals (no. of assemblies = 54; Miyawaki dataset), with the total ratio of time in each sleep/wake state in the background for each bin. Assembly reactivation strength remained significantly >0 throughout 3 h of POST. d, Assembly reactivation was coincident with hippocampal sharp-wave ripple events.
Because these sessions involved long ∼3 h durations of waking experience, we asked whether briefer exploration in a novel maze would lead to similarly prolonged replay. To answer this question, we extended these same analyses to the Grosmark dataset, generously provided by Grosmark et al. (2016), with sessions composed of PRE and POST sleep before and after ∼45 min (range: 34–51 min) of MAZE experience where both the track and the recording room were designed to be novel. Figure 3a demonstrates a sample recording where POST replay is seen for >4 h following MAZE. In the pooled Grosmark et al. (2016) sessions as well, replay was prolonged for several hours in POST sleep in both EV and ICA measures (Fig. 3b). Based on the half-maxima of the EV in these individual sessions, reactivation persisted for a mean of 2.3 h (range = 1.9–2.85 h) following MAZE. Thus, ∼45 min of experience on a novel track was sufficient to promote long-lasting hippocampal replay, well beyond previously reported limits.
Replay following brief novel experience. a, Reactivation (EV; mean ± SD. from 108 cells, 4984 pairs) in POST following brief (45 min) novel track experience in a sample session from the Grosmark dataset (Grosmark et al., 2016). Sleep/wake states are indicated in the background. b, Reactivation analysis results pooled across five POST sessions from four animals (total 278 cells, 8810 pairs; for individual Grosmark sessions, see Figure 1-1) shows extended replay for ∼3 h following novel experience (mean ± SEM for EV, black, left axis; Assembly reactivation strength, red, right axis). Note that these animals demonstrated less NREM sleep after this point, making it difficult to assess the continuation of replay. The total ratio of time in each sleep/wake state is shown in the background for each bin as a proportion of the y-axis limit. Because individual sessions varied in duration, N varies from 4–5 in different bins.
During early exploration of a novel environment, neuronal ensembles composed of place-fields are still in the process of forming and stabilizing (Frank et al., 2004; Cheng and Frank, 2008; Feng et al., 2015). We therefore asked whether even early MAZE experience could leave a lasting trace in POST sleep. To examine this question, we used only the first 1, 5, 15, or 30 min initial segments of exploration to define the pairwise correlation structure of MAZE and examined whether we could detect significant reactivation of those early ensembles in POST. Although POST reactivation more strongly resembles MAZE ensembles defined over longer periods (Fig. 4a,b), we found that the activation of cell pairs in just the first 1 min of MAZE was sufficient to produce significant reactivation in POST. This observation therefore indicates that ensembles that already emerge within the first few minutes of novel experience continue to reactivate during post-task sleep.
Reactivation of the onset of MAZE experience. a, We iteratively used the first 1 min, 5 min, 15 min, 30 min, or entirety of the behavior session to define the MAZE pairwise correlation template and calculated reactivation (EV; mean ± SD) in POST accordingly (sleep/wake states indicated in background). The strength of reactivation increased with the length of the initial period used to define the template in this sample session, but some reactivation could be seen of even the first 1 min of MAZE. b, Pooled results from EV in NREM in POST (n = 5 sessions from the Miyawaki dataset, with 205 cells and 3809 pairs) for these incremental durations of MAZE experiences. All durations produced significant Pearson partial correlation coefficients (1 min, p = 1.82 × 10−13; 5 min, p = 4.835 × 10−32; 15 min, p = 3.647 × 10−57; 30 min, p = 3.917 × 10−84; all MAZE, p = 3.219 × 10−187). ***p > 0.001.
In previous reports, reactivation following experience in familiar environments was found only in the first 10–30 min of sleep (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Tatsuno et al., 2006; Ji and Wilson, 2007). To examine the role of novelty versus familiarity, in one animal we recorded PRE, MAZE, and POST over 3 consecutive days of experience on the same linear track (Fig. 5). Replay was lower but still enhanced during POST after the second day of track running (Fig. 5, middle). However, by the third day on the track, the observed replay was essentially at the chance level (Fig. 5, right). To examine this question in additional animals, we analyzed another four sessions in three rats from the Mizuseki dataset (Mizuseki et al., 2009). In one of these sessions, the animal was newly introduced to the linear track, but had previously performed delayed alternation in a figure eight maze with a running wheel (Pastalkova et al., 2008; Mizuseki et al., 2009) in this recording room. Remarkably consistent with the Miyawaki sessions, replay persisted for >3 h in POST sleep following this novel experience (Fig. 6a; time of half-maximum = 2.22 h). In contrast, in three other sessions recorded from two animals after extensive exposure to the track (Fig. 6b), replay was much more limited and was significant for <30 min (range = 15–30 min, mean = 23.33 min, Student's t test one-tailed paired p < 0.001) of POST NREM sleep. Similar results were seen when we assessed reactivation in the ICA-defined cell assemblies (Fig. 6c,d). In sum, our observations indicate that the novelty of the environment experienced during spatial behavior is a key determinant of the duration of reactivation in subsequent sleep.
Reactivation decreases following repeated exposure to an environment. Reactivation (EV; mean ± SD) in POST following repeated exposure of an animal to the same environment (rat R, Env 1) on Day 1 (58 cells, 1332; left), Day 2 (86 cells, 2966 pairs; middle), and Day 3 (83 cells, 2717 pairs; right). The strength of reactivation shows a progressive decrease over days of exposure. Sleep/wake states are indicated in the background.
Limited reactivation in sleep following behavior in a familiar environment. a, Extended reactivation (EV; mean ± SD. from 42 cells, 567 pairs) is observed in POST following a 65 min session on a novel track from the Mizuseki dataset. Sleep/wake states are indicated in the background. b, In contrast, very limited reactivation (EV; mean ± SD from 3 sessions from the Mizuseki dataset; 44 cells, 248 pairs) is observed in POST following behavior on a highly familiar track (>10 exposures). Horizontal gray lines along the x-axis indicate intervals during which recording was paused by the experimenter. c, (Re)Activation of a sample ICA assembly (gray) is shown in PRE (left) and POST (right) panels for the Roy Day 3 Env 1 session, with sleep/wake states shown in the background. The average (re)activation strength of all 20 cell assemblies for this recording session is overlaid in red (mean in 15 min windows sliding in 5 min steps; right axis). d, Reactivation strength pooled across four POST sessions following MAZE in a familiar environment shows evidence for little or no reactivation (p = 0.6 for all POST; p = 0.74 for NREM only; one-tailed one-sample t test against mean 0). Error bars denote +SEM.
To better determine the temporal extent of reactivation following novel experience in MAZE, in two sessions from the Miyawaki dataset we were able to extend our recordings for 10 h of POST sleep (Fig. 7a). In both these sessions, reactivation peaked at ∼3–5 h following MAZE, then showed progressive decrease, dropping past half-maximum at ∼ 6.28 h into POST sleep. These sessions appeared to feature longer reactivation than the Grosmark sessions. Thus, it may be that the longer MAZE periods in our sessions led to a longer reactivation period. However, the longer replay we observed could also be attributed to the more continuous sleep we obtained in our light-cycle recordings compared with Grosmark et al. (2016) (e.g., compare fractions of sleep in Fig. 1b, and Fig. 3b), whereas fragmentation of sleep may limit the persistence of reactivation. Nevertheless, in all three datasets of recordings examined, we found that reactivation extended for several hours following novel experience.
Reactivation in long-duration sleep recordings. In long-duration recordings from two sessions from two different animals, (a) replay (EV, black, mean ± SD vs REV, green, mean ± SD) persisted for the duration of the recording (timestamp corresponding to half-maximum = 6.55 h for rat R, 58 cells, 1332 pairs and = 6.01 h for rat T, 17 cells, 102 pairs). b, The vector of pairwise correlations was compared across the recordings (from PRE to MAZE to POST in 15 min bins). As noted, strong correlations exist within and across PRE and POST sleep, though correlations with MAZE are stronger in POST for several hours. As sleep progresses, different pairs dominate, with weaker correlations across, rather than within, late sleep and early sleep. For additional sleep sessions across entire light/dark cycles, see Figure 7-1.
Figure 7-1
We next examined the activation patterns of neurons over the course of sleep in the correlations between neuronal pairs across our long-duration recordings (15 min bins). Several interesting features can be gleaned from these figures (Fig. 7b). As noted previously, there are large correlations between all recorded sleep periods, including those that extend from PRE to POST sleep. The MAZE patterns are largely divergent from those during sleep, though some correlations were still noted between PRE and MAZE (Fig. 1c). Thus, the MAZE experience lead to a correlation pattern that is largely unique. Although these patterns persist in POST sleep, compared with PRE, they occur alongside the stronger patterns that are in turn unique to sleep. As can also be seen in Figure 7b, the correlation structure during POST shifts over time so that different neuronal groups are dominant at different times. For example, relatively weaker apparent correlations exist across than within the end and beginning of the Day 3 session recorded from Rat T, with the dominant patterns shifting over hours-long periods. Additional long-duration sessions (n = 20 long 6–12 h light or dark cycle sessions across 5 animals; Figure 7-1) further corroborate this viewpoint.
Discussion
These results demonstrate that cell assemblies associated during learning reactivate during subsequent sharp-wave ripples for a much longer timespan than has been generally appreciated. In particular, we found that cell assemblies in sleep reprise MAZE activity for up to 7 h following novel experience. Remarkably, similar timescales have been reported both for the post-task consolidation window during which sleep deprivation impairs memory formation in different hippocampus-dependent learning tasks (Havekes and Abel, 2017). For example, deprivation of sleep in the 4–5 h after object-location exploration disrupted subsequent memory (Florian et al., 2011; Havekes et al., 2014; Prince et al., 2014). Importantly, this memory deficit was observed even when the first hour of post-task sleep, and presumably concurrent replay, was left intact (Prince et al., 2014), indicating that processes beyond the first hour of sleep remain important for memory consolidation. Other studies investigating contextual fear conditioning (Graves et al., 2003; Vecsey et al., 2009) and object-recognition memory (Palchykova et al., 2006) reported a similar 5 h post-task consolidation window, consistent with our report. Our results are also consistent with a recent report that enduring stability of neural ensemble patterns are supported by parvalbumin-positive interneurons (Ognjanovski et al., 2017) following contextual fear conditioning.
The duration of replay we report here also closely matches the hours-long timescales reported for protein signaling following memory (often referred to as “cellular” or “synaptic” consolidation; McGaugh, 2000; Dudai, 2004). In particular, cAMP and protein kinase A signaling pathways, which regulate the cAMP response element binding protein, are critically involved in multiple forms of synaptic plasticity linked to memory, as well as in gene expression in the hours immediately following learning (Abel et al., 2013). Importantly, these pathways are impaired by 5 h of sleep-deprivation (Bourtchouladze et al., 1998; Vecsey et al., 2009). Protein synthesis through the mammalian target of rapamycin kinase complex, which is likewise implicated in both plasticity and memory formation (Hoeffer and Klann, 2010), is also reduced upon sleep-deprivation over a 5 h post-task window (Vecsey et al., 2012; Tudor et al., 2016). Although more research is needed to understand the inter-relationship between reactivation and protein-signaling, these results help to reconcile previous discrepancies between the time windows considered for cellular/synaptic and systems consolidation of memory.
We used two different methods for evaluating and tracking reactivation in POST relative to PRE: the EV method based on pairwise correlations (Kudrimoti et al., 1999; Tatsuno et al., 2006) and an ICA method based on ensemble coactivity (Peyrache et al., 2009; Lopes-dos-Santos et al., 2011, 2013). Although these methods are not designed to evaluate temporal sequences, we used a 250 ms time window to capture coactivation on the timescale of sharp-wave ripples during which replay sequences are observed (Nádasdy et al., 1999; Buzsáki, 2015). Furthermore, we examined the temporal structure of spike times within this window using a third method, by comparing the temporal bias of spike times in pairs of neurons, and found that POST firing sequences showed significant fidelity to MAZE sequences, and greater temporal fidelity compared with PRE sequences, consistent with previous accounts of temporal replay (Skaggs and McNaughton, 1996; Lee and Wilson, 2002; Wikenheiser and Redish, 2013). Preliminary analysis of our data with additional methods that are sensitive to sequential structure at the population level (Chen et al., 2016; Maboudi et al., 2018b) have also produced consistent results to what we report here (Maboudi et al., 2018a), but further work is needed to evaluate and differentiate temporal relationships across PRE, MAZE, and POST.
Importantly, studies by different investigators, some using methods identical to ours, had previously observed much shorter durations for replay than we report here (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Tatsuno et al., 2006; Ji and Wilson, 2007). This discrepancy may arise from two reasons. First, with the exception of Tatsuno et al. (2006), these studies did not examine reactivation beyond the first 1 h of post-task sleep as we have, potentially missing re-emergent reactivation in subsequent sleep. Second, the MAZE experience in these studies involved exploration of environments that were highly familiar to the animal, whereas we observed that the time course of reactivation depended on the novelty of the task environment. Interestingly, Tatsuno et al. (2006) introduced novel objects to animals in a familiar environment but failed to find evidence of protracted replay. This was in contrast to Ribeiro et al. (2004) who performed similar experiments that appeared to show reactivation lasting for days but did not control for extant correlation in PRE. Our analysis (Fig. 1c,d) demonstrates the importance of these controls, in agreement with Tatsuno et al. (2006), which may otherwise give an illusion of replay. Overall, our findings indicate that prolonged reactivation in post-task sleep is contingent upon exposure to a novel environment, which induces a new hippocampal map (or “global remapping”), rather than the experience of novel objects or events in a familiar environment, for which the hippocampus makes use of the same spatial map (or “rate remapping”; Muller and Kubie, 1987; Leutgeb et al., 2005). An intriguing possibility is that other experiences, such as a shock (Moita et al., 2004) or exposure to a predator's odor (Wang et al., 2012), which produce global remapping can generate a similarly prolonged reactivation in subsequent sleep.
The long duration of replay demonstrated in our study counters the notion that the bulk of neuronal patterns in sleep following learning is random or noisy, and strengthens the argument that memory can be consolidated during sleep through a systems level process involving hippocampal replay (Buzsáki, 1989; Diekelmann and Born, 2010). We propose that novelty increases the firing rates of those hippocampal neurons activated in the environment (Hirase et al., 2001) through increased membrane excitability alongside potentiation of synapses that bind the activated cell assemblies (Takeuchi et al., 2014). Some of these processes are likely enhanced by activation of membrane dopaminergic receptors (Takeuchi et al., 2016), which promotes the reactivation of these assemblies in post-task sleep. Over the course of sleep, reactivation during sharp-wave ripples promotes the consolidation of memories (Maingret et al., 2016). But as memories are consolidated, hippocampal firing rates then decrease (Miyawaki and Diba, 2016), likely as a consequence of synaptic downscaling also triggered by sharp-wave ripples (Tononi and Cirelli, 2014; Miyawaki and Diba, 2016; Norimoto et al., 2018). It is interesting and worthwhile to note that while extended replay is clearly a significant phenomenon, even in POST sleep the MAZE patterns are not the most dominant activation patterns. Additionally, a small but significant correlation exists between PRE and MAZE periods, consistent with the notion of preplay (Dragoi and Tonegawa, 2011, 2013). Thus, replay must contend with activity patterns that are unique to sleep, including those that carry onto subsequent experience (Dragoi and Tonegawa, 2014; Grosmark and Buzsáki, 2016). However, the function performed by these other (non-replay) activities within sleep still remains elusive. Because network, cellular, and synaptic processes are intimately intertwined, additional research is needed to better understand whether and how reactivation and non-reactivation activities during sleep contribute to consolidation and other sleep functions by engaging proteins and plasticity processes to strengthen and weaken specific synaptic connections.
Footnotes
This work was supported by the National Institute of Mental Health R01MH109170 and R01MH117964 to K.D., the German BMBF 01GQ1506 to S.C., and the Toray Science Foundation to K.M. We thank Asohan Amarasingham, Laura Ball, Caleb Kemere, Andrew P. Maurer, and Brendon Watson for helpful comments, and to Andres Grosmark, John Long, and Gyorgy Buzsaki for publicly sharing their valuable data. Data and analyses code will be made available to readers upon reasonable request.
The authors declare no competing financial interests.
- Correspondence should be addressed to Kamran Diba at kdiba{at}umich.edu
References
In this issue
