Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Reverse replay of behavioural sequences in hippocampal place cells during the awake state

Abstract

The hippocampus has long been known to be involved in spatial navigational learning in rodents1,2, and in memory for events in rodents3,4, primates5 and humans6. A unifying property of both navigation and event memory is a requirement for dealing with temporally sequenced information. Reactivation of temporally sequenced memories for previous behavioural experiences has been reported in sleep in rats7,8. Here we report that sequential replay occurs in the rat hippocampus during awake periods immediately after spatial experience. This replay has a unique form, in which recent episodes of spatial experience are replayed in a temporally reversed order. This replay is suggestive of a role in the evaluation of event sequences in the manner of reinforcement learning models. We propose that such replay might constitute a general mechanism of learning and memory.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: A sequence of place fields.
Figure 2: Reverse replay events during a single lap.
Figure 3: Reverse replay events, by lap, for an entire recording session.
Figure 4: Analysis of reverse replay across all recording sessions.

Similar content being viewed by others

References

  1. O'Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon, London, 1978)

    Google Scholar 

  2. Morris, R. G. M., Garrud, P., Rawlins, J. N. P. & O'Keefe, J. Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982)

    Article  ADS  CAS  Google Scholar 

  3. Olton, D. S. & Samuelson, R. J. Remembrance of places past: spatial memory in rats. J. Exp. Psychol. Anim. Behav. Process. 2, 97–116 (1976)

    Article  Google Scholar 

  4. Steele, R. J. & Morris, R. G. M. Delay-dependent impairment of a matching-to-place task with chronic and intrahippocampal infusion of the NMDA-antagonist d-AP5. Hippocampus 9, 118–136 (1999)

    Article  CAS  Google Scholar 

  5. Gaffan, D. Scene-specific memory for objects: a model of episodic memory impairment in monkeys with fornix transection. J. Cogn. Neurosci. 6, 305–320 (1994)

    Article  CAS  Google Scholar 

  6. Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957)

    Article  CAS  Google Scholar 

  7. Louie, K. & Wilson, M. A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29, 145–156 (2001)

    Article  CAS  Google Scholar 

  8. Lee, A. K. & Wilson, M. A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194 (2002)

    Article  CAS  Google Scholar 

  9. Wilson, M. A. & McNaughton, B. L. Dynamics of the hippocampal ensemble code for space. Science 261, 1055–1058 (1993)

    Article  ADS  CAS  Google Scholar 

  10. O'Keefe, J. & Dostrovsky, J. The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Res. 34, 171–175 (1971)

    Article  CAS  Google Scholar 

  11. Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. Numerical Recipes in C: The Art of Scientific Computing 2nd edn (Cambridge Univ. Press, Cambridge, 1992)

    MATH  Google Scholar 

  12. Ranck, J. B. Jr. Studies on single neurons in dorsal hippocampal formation and in septum in unrestrained rats. I. Behavioral correlates and firing repertoires. Exp. Neurol. 42, 461–531 (1973)

    Google Scholar 

  13. Buzsaki, G. Hippocampal sharp waves: their origin and significance. Brain Res. 398, 242–252 (1986)

    Article  CAS  Google Scholar 

  14. Buzsaki, G. A two-stage model of memory trace formation: a role for ‘noisy’ brain states. Neuroscience 31, 551–570 (1989)

    Article  CAS  Google Scholar 

  15. Kaelbling, L. P., Littman, M. L. & Moore, A. W. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)

    Article  Google Scholar 

  16. Sutton, R. S. Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9–44 (1988)

    Google Scholar 

  17. Schultz, W., Dayan, P. & Montague, P. R. A neural subtstrate of prediction and reward. Science 275, 1593–1599 (1997)

    Article  CAS  Google Scholar 

  18. Foster, D. J., Morris, R. G. M. & Dayan, P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10, 1–16 (2000)

    Article  CAS  Google Scholar 

  19. Lin, L. J. Self-improving reactive agents based on reinforcement learning, planning and teaching. Mach. Learn. 8, 293–321 (1992)

    Google Scholar 

  20. Sutton, R. S. & Singh, S. P. in Proceedings of the Eighth Yale Workshop on Adaptive and Learning Systems 31–36 (Yale Univ., New Haven, 1994)

    Google Scholar 

  21. Cichosz, P. An analysis of experience replay in temporal difference learning. Cybernet. Syst. 30, 341–363 (1999)

    Article  Google Scholar 

  22. Reynolds, S. Experience Stack Reinforcement Learning for Off-policy Control. Cognitive Science Technical Report CSRP-02–1 (Birmingham Univ., Birmingham, 2002)

    Google Scholar 

  23. Stickgold, R., Whidbee, D., Schirmer, B., Patel, V. & Hobson, J. A. Visual discrimination task improvement: a multi-step process occurring during sleep. J. Cogn. Neurosci. 12, 246–254 (2000)

    Article  CAS  Google Scholar 

  24. Siapas, A. G. & Wilson, M. A. Coordinated interactions between hippocampal wripples and cortical spindles during slow-wave sleep. Neuron 21, 1123–1128 (1998)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank V. Ego-Stengel, P. Dayan and M. Fee for comments on the manuscript. This work was supported by an NIH grant to M.A.W.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David J. Foster.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Figures

This file contains Supplementary Figures 1–7 (PDF 365 kb)

Supplementary Figure Legends

This file contains text to accompany the above Supplementary Figures. (DOC 23 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Foster, D., Wilson, M. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006). https://doi.org/10.1038/nature04587

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature04587

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing