Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A reservoir of time constants for memory traces in cortical neurons

Nature Neuroscience volume 14, pages 366–372 (2011)Cite this article

Subjects

Abstract

According to reinforcement learning theory of decision making, reward expectation is computed by integrating past rewards with a fixed timescale. In contrast, we found that a wide range of time constants is available across cortical neurons recorded from monkeys performing a competitive game task. By recognizing that reward modulates neural activity multiplicatively, we found that one or two time constants of reward memory can be extracted for each neuron in prefrontal, cingulate and parietal cortex. These timescales ranged from hundreds of milliseconds to tens of seconds, according to a power law distribution, which is consistent across areas and reproduced by a 'reservoir' neural network model. These neuronal memory timescales were weakly, but significantly, correlated with those of monkey's decisions. Our findings suggest a flexible memory system in which neural subpopulations with distinct sets of long or short memory timescales may be selectively deployed according to the task demands.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Behavioral task and schematic illustration of memory traces.
Figure 2: An example neuron in ACCd showing multiplicative modulation of memory traces by the epoch code.
Figure 3: Firing rates and memory traces for six neurons, two for each of the three recorded areas.
Figure 4: Distribution of the timescales characterizing the reward memory traces across neurons.
Figure 5: Distribution of behavioral timescales and their relationship with the neural memory timescales.
Figure 6: Stability of behavioral and neural memory timescales in an experimental session.
Figure 7: Neural responses (memory traces) in the model and distribution of timescales of the memory traces in model neurons.
Figure 8: Distribution of amplitudes of the memory traces in the neural data and model.

Similar content being viewed by others

References

  1. Kable, J.W. & Glimcher, P.W. The neurobiology of decision: consensus and controversy. Neuron 63, 733–745 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Rushworth, M.F. & Behrens, T.E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008).

    Article  CAS  PubMed  Google Scholar 

  3. Wang, X.-J. Decision making in recurrent neural circuits. Neuron 60, 215–234 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Soltani, A., Lee, D. & Wang, X.-J. Neural mechanism for stochastic behavior during a competitive game. Neural Netw. 19, 1075–1090 (2006).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Sutton, R.S. & Barto,, A.G. Reinforcement Learning, an Introduction (MIT Press, Cambridge, Massachusetts, 1998).

  6. Behrens, T.E., Woolrich, M.W., Walton, M.E. & Rushworth, M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214–1221 (2007).

    Article  CAS  PubMed  Google Scholar 

  7. Daw, N.D., O′Doherty, J.P., Dayan, P., Seymour, B. & Dolan, R.J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Lau, B. & Glimcher, P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555–579 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  9. Corrado, G.S., Sugrue, L.P., Seung, H.S. & Newsome, W.T. Linear-nonlinear-Poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581–617 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Kennerley, S.W., Walton, M.E., Behrens, T.E., Buckley, M.J. & Rushworth, M.F. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9, 940–947 (2006).

    Article  CAS  PubMed  Google Scholar 

  11. Lee, D., Conroy, M.L., McGreevy, B.P. & Barraclough, D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45–58 (2004).

    Article  PubMed  Google Scholar 

  12. Kim, S., Hwang, J., Seo, H. & Lee, D. Valuation of uncertain and delayed rewards in primate prefrontal cortex. Neural Netw. 22, 294–304 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  13. Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002).

    Article  PubMed  Google Scholar 

  14. Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20, 335–352 (2007).

    Article  PubMed  Google Scholar 

  15. Verstraeten, D., Schrauwen, B., D′Haene, M. & Stroobandt, D. An experimental unification of reservoir computing methods. Neural Netw. 20, 391–403 (2007).

    Article  CAS  PubMed  Google Scholar 

  16. Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Bertschinger, N. & Natschlager, T. Real-time computation at the edge of chaos in recurrent neural networks. Neural Comput. 16, 1413–1436 (2004).

    Article  PubMed  Google Scholar 

  18. Langton, C.G. Computation at the edge of chaos: phase transitions and emergent computations. Physica D 42, 12–37 (1990).

    Article  Google Scholar 

  19. Seo, H. & Lee, D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27, 8366–8377 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Seo, H., Barraclough, D.J. & Lee, D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 17, i110–i117 (2007).

    Article  PubMed  Google Scholar 

  21. Seo, H., Barraclough, D.J. & Lee, D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 29, 7278–7289 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Barraclough, D.J., Conroy, M.L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404–410 (2004).

    Article  CAS  PubMed  Google Scholar 

  23. Lapish, C.C., Durstewitz, D., Chandler, L.J. & Seamans, J.K. Successful choice behavior is associated with distinct and coherent network states in anterior cingulate cortex. Proc. Natl. Acad. Sci. USA 105, 11963–11968 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Sigala, N., Kusonoki, M., Nimmo-Smith, I., Gaffan, D. & Duncan, J. Hierarchical coding for sequential task events in the monkey prefrontal cortex. Proc. Natl. Acad. Sci. USA 105, 11969–11974 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Jin, D.Z., Fujii, N. & Graybiel, A.N. Neural representation of time in cortico-basal ganglia circuits. Proc. Natl. Acad. Sci. USA 106, 19156–19161 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Trefethen, L.N. & Embree, M. Spectra and Pseudospectra: The Behavior of Nonnormal Matrices and Operators (Princeton University Press, Princeton, New Jersey, 2005).

  27. Murphy, B.K. & Miller, K.D. Balanced amplification: a new mechanism of selective amplification of neural activity patterns. Neuron 61, 635–648 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Ganguli, S., Huh, D. & Sompolinsky, H. Memory traces in dynamical systems. Proc. Natl. Acad. Sci. USA 105, 18970–18975 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Goldman, M.S. Memory without feedback in a neural network. Neuron 61, 621–634 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Schneidman, E., Berry, M.J., Segev, R. & Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 1007–1012 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Brunel, N., Hakim, V., Isope, P., Nadal, J.-P. & Barbour, B. Optimal information storage and the distribution of synaptic weights: perceptron versus purkinje cell. Neuron 43, 745–757 (2004).

    CAS  PubMed  Google Scholar 

  32. Ganguli, S. et al. One-dimensional dynamics of attention and decision making in LIP. Neuron 58, 15–25 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Duncan, J. An adaptive coding model of neural function in prefrontal cortex. Nat. Rev. Neurosci. 2, 820–829 (2001).

    Article  CAS  PubMed  Google Scholar 

  34. Rigotti, M., Rubin, D.B.D., Wang, X.-J. & Fusi, S. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Front. Comput. Neurosci. 4, 24 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Botvinick, M.M., Braver, T.S., Barch, D.M., Carter, C.S. & Cohen, J.D. Conflict monitoring and cognitive control. Psychol. Rev. 108, 624–652 (2001).

    Article  CAS  PubMed  Google Scholar 

  36. Holroyd, C.B. & Coles, M.G.H. The neural basis of human error processing: reinforcement learning, dopamine and error-related negativity. Psychol. Rev. 109, 679–709 (2002).

    Article  PubMed  Google Scholar 

  37. Wallis, J.D. & Kennerley, S.W. Heterogeneous reward signals in prefrontal cortex. Curr. Opin. Neurobiol. 20, 191–198 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).

    Article  CAS  PubMed  Google Scholar 

  39. Roitman, J.D. & Shadlen, M.N. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J. Neurosci. 22, 9475–9489 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Bayer, H.M. & Glimcher, P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129–141 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Rainer, G. & Miller, E.K. Time course of object-related neural activity in the primate prefrontal cortex during a short-term memory task. Eur. J. Neurosci. 15, 1244–1254 (2002).

    Article  PubMed  Google Scholar 

  42. Machens, C.K., Romo, R. & Brody, C.D. Functional, but not anatomical, separation of “what” and “when” in prefrontal cortex. J. Neurosci. 30, 350–360 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Shafi, M. et al. Variability in neuronal activity in primate cortex during working memory tasks. Neuroscience 146, 1082–1108 (2007).

    Article  CAS  PubMed  Google Scholar 

  44. Curtis, C.E. & Lee, D. Beyond working memory: the role of persistent activity in decision making. Trends Cogn. Sci. 14, 216–222 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  45. Passingham, D. & Sakai, K. The prefrontal cortex and working memory: physiology and brain imaging. Curr. Opin. Neurobiol. 14, 163–168 (2004).

    Article  PubMed  Google Scholar 

  46. Lebedev, M.A., Messinger, A., Kralik, J.D. & Wise, S.P. Representation of attended versus remembered locations in prefrontal cortex. PLoS Biol. 2, e365 (2004).

    Article  PubMed  PubMed Central  Google Scholar 

  47. Funahashi, S., Chafee, M.V. & Goldman-Rakic, P.S. Prefrontal neuronal activity in rhesus monkeys performing a delayed anti-saccade task. Nature 365, 753–756 (1993).

    Article  CAS  PubMed  Google Scholar 

  48. Rainer, G., Rao, S.G. & Miller, E.K. Prospective coding for objects in primate prefrontal cortex. J. Neurosci. 19, 5493–5505 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Brody, C.D., Hernandez, A., Zainos, A. & Romo, R. Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cereb. Cortex 13, 1196–1207 (2003).

    Article  PubMed  Google Scholar 

  50. Bromberg-Martin, E.S., Matsumoto, M., Nakahara, H. & Hikosaka, O. Multiple timescales of memory in lateral habenula and dopamine neurons. Neuron 67, 499–510 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank J. Mazer and M.W. Jung for comments on an earlier version of the manuscript, and R. Chaudhuri, M. Harre and J. Murray for discussions. This work was supported by the US National Institutes of Health grant R01 MH062349 and the Swartz Foundation (A.B. and X.-J.W.), and by US National Institutes of Health grants R01 MH073246 (X.-J.W. and D.L.) and DA029330 (D.L.).

Author information

Authors and Affiliations

  1. Department of Neurobiology and Kavli Institute of Neuroscience, Yale University School of Medicine, New Haven, Connecticut, USA

    Alberto Bernacchia, Hyojung Seo, Daeyeol Lee & Xiao-Jing Wang

Authors
  1. Alberto Bernacchia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyojung Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daeyeol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao-Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All of the authors participated in the research design and the preparation of the manuscript. H.S. collected the data, A.B. and H.S. analyzed data, and A.B. and X.-J.W. performed modeling.

Corresponding author

Correspondence to Xiao-Jing Wang.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–4 and Supplementary Text (PDF 1784 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bernacchia, A., Seo, H., Lee, D. et al. A reservoir of time constants for memory traces in cortical neurons. Nat Neurosci 14, 366–372 (2011). https://doi.org/10.1038/nn.2752

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2752

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing