Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Robust timing and motor patterns by taming chaos in recurrent neural networks

Nature Neuroscience volume 16, pages 925–933 (2013)Cite this article

Subjects

Abstract

The brain's ability to tell time and produce complex spatiotemporal motor patterns is critical for anticipating the next ring of a telephone or playing a musical instrument. One class of models proposes that these abilities emerge from dynamically changing patterns of neural activity generated in recurrent neural networks. However, the relevant dynamic regimes of recurrent networks are highly sensitive to noise; that is, chaotic. We developed a firing rate model that tells time on the order of seconds and generates complex spatiotemporal patterns in the presence of high levels of noise. This is achieved through the tuning of the recurrent connections. The network operates in a dynamic regime that exhibits coexisting chaotic and locally stable trajectories. These stable patterns function as 'dynamic attractors' and provide a feature that is characteristic of biological systems: the ability to 'return' to the pattern being generated in the face of perturbations.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Complexity without chaos.
Figure 2: Generation and stability of complex spatiotemporal motor patterns.
Figure 3: Improved timing capacity.
Figure 4: Innate training decreases the neural variance and results in Weber-like timing.
Figure 5: Robustness against noise.
Figure 6: Suppression of chaos.
Figure 7: Effects of training on network structure.

Similar content being viewed by others

References

  1. Mauk, M.D. & Buonomano, D.V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).

    Article  CAS  PubMed  Google Scholar 

  2. Buhusi, C.V. & Meck, W.H. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005).

    Article  CAS  PubMed  Google Scholar 

  3. Ivry, R.B. & Schlerf, J.E. Dedicated and intrinsic models of time perception. Trends Cogn. Sci. 12, 273–280 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Church, R.M., Meck, W.H. & Gibbon, J. Application of scalar timing theory to individual trials. J. Exp. Psychol. Anim. Behav. Process. 20, 135–155 (1994).

    Article  CAS  PubMed  Google Scholar 

  5. Durstewitz, D. Self-organizing neural integrator predicts interval times through climbing activity. J. Neurosci. 23, 5342–5353 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Simen, P., Balci, F., de Souza, L., Cohen, J.D. & Holmes, P. A model of interval timing by neural integration. J. Neurosci. 31, 9238–9253 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Miall, C. The storage of time intervals using oscillating neurons. Neural Comput. 1, 359–371 (1989).

    Article  Google Scholar 

  8. Matell, M.S., Meck, W.H. & Nicolelis, M.A. Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. Behav. Neurosci. 117, 760–773 (2003).

    Article  PubMed  Google Scholar 

  9. Ahrens, M.B. & Sahani, M. Observers exploit stochastic models of sensory change to help judge the passage of time. Curr. Biol. 21, 200–206 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Buonomano, D.V. & Laje, R. Population clocks: motor timing with neural dynamics. Trends Cogn. Sci. 14, 520–527 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Medina, J.F. & Mauk, M.D. Computer simulation of cerebellar information processing. Nat. Neurosci. 3 (suppl.), 1205–1211 (2000).

    Article  CAS  PubMed  Google Scholar 

  12. Buonomano, D.V. & Mauk, M.D. Neural network model of the cerebellum: temporal discrimination and the timing of motor responses. Neural Comput. 6, 38–55 (1994).

    Article  Google Scholar 

  13. Buonomano, D.V. & Merzenich, M.M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).

    Article  CAS  PubMed  Google Scholar 

  14. Durstewitz, D. & Deco, G. Computational significance of transient dynamics in cortical networks. Eur. J. Neurosci. 27, 217–227 (2008).

    Article  PubMed  Google Scholar 

  15. Rabinovich, M., Huerta, R. & Laurent, G. Transient dynamics for neural processing. Science 321, 48–50 (2008).

    Article  CAS  PubMed  Google Scholar 

  16. Buonomano, D.V. & Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).

    Article  CAS  PubMed  Google Scholar 

  17. Hahnloser, R.H.R., Kozhevnikov, A.A. & Fee, M.S. An ultra-sparse code underlies the generation of neural sequence in a songbird. Nature 419, 65–70 (2002).

    Article  CAS  PubMed  Google Scholar 

  18. Long, M.A., Jin, D.Z. & Fee, M.S. Support for a synaptic chain model of neuronal sequence generation. Nature 468, 394–399 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Crowe, D.A., Averbeck, B.B. & Chafee, M.V. Rapid sequences of population activity patterns dynamically encode task-critical spatial information in parietal cortex. J. Neurosci. 30, 11640–11653 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Li, J.X. & Lisberger, S.G. Learned timing of motor behavior in the smooth eye movement region of the frontal eye fields. Neuron 69, 159–169 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. London, M., Roth, A., Beeren, L., Hausser, M. & Latham, P.E. Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature 466, 123–127 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Izhikevich, E.M. & Edelman, G.M. Large-scale model of mammalian thalamocortical systems. Proc. Natl. Acad. Sci. USA 105, 3593–3598 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  23. van Vreeswijk, C. & Sompolinsky, H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).

    Article  CAS  PubMed  Google Scholar 

  24. Brunel, N. Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J. Physiol. Paris 94, 445–463 (2000).

    Article  CAS  PubMed  Google Scholar 

  25. Banerjee, A., Series, P. & Pouget, A. Dynamical constraints on using precise spike timing to compute in recurrent cortical networks. Neural Comput. 20, 974–993 (2008).

    Article  PubMed  Google Scholar 

  26. Sompolinsky, H., Crisanti, A. & Sommers, H.J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259–262 (1988).

    Article  CAS  PubMed  Google Scholar 

  27. Monteforte, M. & Wolf, F. Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).

    CAS  Google Scholar 

  28. Jaeger, H. & Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004).

    Article  CAS  PubMed  Google Scholar 

  29. Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Rajan, K., Abbott, L.F. & Sompolinsky, H. Stimulus-dependent suppression of chaos in recurrent neural networks. Physical Rev. E Stat. Nonlin. Soft Matter Phys. 82, 011903 (2010).

    Article  CAS  Google Scholar 

  31. Doya, K. in Proc. IEEE Int. Symp. Circuits and Syst. 2777–2780 (1992).

  32. Jaeger, H., Maass, W. & Principe, J. Special issue on echo state networks and liquid state machines. Neural Netw. 20, 287–289 (2007).

    Article  Google Scholar 

  33. Ganguli, S., Huh, D. & Sompolinsky, H. Memory traces in dynamical systems. Proc. Natl. Acad. Sci. USA 105, 18970–18975 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Song, S., Sjostrom, P.J., Reigl, M., Nelson, S. & Chklovskii, D.B. Highly nonrandom feature of synaptic connectivity in local cortical circuits. PLoS Biol. 3, e66 (2005).

    Article  CAS  Google Scholar 

  36. Watts, D.J. & Strogatz, S.H. Collective dynamics of 'small-world' networks. Nature 393, 440–442 (1998).

    Article  CAS  PubMed  Google Scholar 

  37. Janssen, P. & Shadlen, M.N. A representation of the hazard rate of elapsed time in the macaque area LIP. Nat. Neurosci. 8, 234–241 (2005).

    Article  CAS  PubMed  Google Scholar 

  38. Bueti, D., Lasaponara, S., Cercignani, M. & Macaluso, E. Learning about time: plastic changes and interindividual brain differences. Neuron 75, 725–737 (2012).

    Article  CAS  PubMed  Google Scholar 

  39. Coull, J. & Nobre, A. Dissociating explicit timing from temporal expectation with fMRI. Curr. Opin. Neurobiol. 18, 137–144 (2008).

    Article  CAS  PubMed  Google Scholar 

  40. Merchant, H., Zarco, W., Pérez, O., Prado, L. & Bartolo, R. Measuring time with different neural chronometers during a synchronization-continuation task. Proc. Natl. Acad. Sci. USA 108, 19784–19789 (2011).

    Article  PubMed  PubMed Central  Google Scholar 

  41. Pastalkova, E., Itskov, V., Amarasingham, A. & Buzsaki, G. Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Ivry, R.B., Keele, S.W. & Diener, H.C. Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988).

    Article  CAS  PubMed  Google Scholar 

  43. Medina, J.F., Garcia, K.S., Nores, W.L., Taylor, N.M. & Mauk, M.D. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Buonomano, D.V. Decoding temporal information: a model based on short-term synaptic plasticity. J. Neurosci. 20, 1129–1141 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Litwin-Kumar, A. & Doiron, B. Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 1498–1505 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Liu, J.K. & Buonomano, D.V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Goldman, M.S. Memory without feedback in a neural network. Neuron 61, 621–634 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Wang, X.J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001).

    Article  CAS  PubMed  Google Scholar 

  50. Skarda, C.A. & Freeman, W.J. How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161–173 (1987).

    Article  Google Scholar 

  51. Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks. GMD Report No. 148 (German National Research Center for Computer Science) (2001).

  52. Haykin, S. Adaptive Filter Theory (Prentice Hall, 2002).

  53. Kantz, H. A robust method to estimate the maximal Lyapunov exponent of a time series. Phys. Lett. A 185, 77–87 (1994).

    Article  Google Scholar 

  54. Boffetta, G., Lacorata, G., Radaelli, G. & Vulpiani, A. Detecting barriers to transport: a review of different techniques. Physica D 159, 58–70 (2001).

    Article  Google Scholar 

  55. Fagiolo, G. Clustering in complex directed networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 76, 026107 (2007).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank A. Garfinkel and R. Huerta for helpful discussions and comments on the manuscript. This work was supported by the US National Institutes of Health (NS077340), the National Science Foundation (II-1114833), the Pew Charitable Trusts, and Consejo Nacional de Investigaciones Científicas y Técnicas (Argentina).

Author information

Author notes

  1. Rodrigo Laje

    Present address: Present addresses: Departamento de Ciencia y Tecnología, Universidad Nacional de Quilmes, Bernal, Argentina, and Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina.,

Authors and Affiliations

  1. Department of Neurobiology, University of California, Los Angeles, California, USA

    Rodrigo Laje & Dean V Buonomano

  2. Department of Psychology, University of California, Los Angeles, California, USA

    Dean V Buonomano

  3. Brain Research Institute, University of California, Los Angeles, California, USA

    Dean V Buonomano

  4. Integrative Center for Learning and Memory, University of California, Los Angeles, California, USA

    Dean V Buonomano

Authors
  1. Rodrigo Laje
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dean V Buonomano
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

R.L. and D.V.B. designed the experiments and wrote the code, and R.L. performed most of the simulations and data analysis. R.L. designed and performed the stability and structural experiments. D.V.B. conceived of the approach, and R.L. and D.V.B. wrote the paper.

Corresponding author

Correspondence to Dean V Buonomano.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8 and Supplementary Modeling (PDF 725 kb)

Supplementary Matlab Routines

Supplementary Matlab Routines (ZIP 3501 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Laje, R., Buonomano, D. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat Neurosci 16, 925–933 (2013). https://doi.org/10.1038/nn.3405

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3405

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing