Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Decoding reveals the contents of visual working memory in early visual areas

Abstract

Visual working memory provides an essential link between perception and higher cognitive functions, allowing for the active maintenance of information about stimuli no longer in view1,2. Research suggests that sustained activity in higher-order prefrontal, parietal, inferotemporal and lateral occipital areas supports visual maintenance3,4,5,6,7,8,9,10,11, and may account for the limited capacity of working memory to hold up to 3–4 items9,10,11. Because higher-order areas lack the visual selectivity of early sensory areas, it has remained unclear how observers can remember specific visual features, such as the precise orientation of a grating, with minimal decay in performance over delays of many seconds12. One proposal is that sensory areas serve to maintain fine-tuned feature information13, but early visual areas show little to no sustained activity over prolonged delays14,15,16. Here we show that orientations held in working memory can be decoded from activity patterns in the human visual cortex, even when overall levels of activity are low. Using functional magnetic resonance imaging and pattern classification methods, we found that activity patterns in visual areas V1–V4 could predict which of two oriented gratings was held in memory with mean accuracy levels upwards of 80%, even in participants whose activity fell to baseline levels after a prolonged delay. These orientation-selective activity patterns were sustained throughout the delay period, evident in individual visual areas, and similar to the responses evoked by unattended, task-irrelevant gratings. Our results demonstrate that early visual areas can retain specific information about visual features held in working memory, over periods of many seconds when no physical stimulus is present.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Design of working memory experiment and resulting time course of fMRI activity.
Figure 2: Orientation decoding results for areas V1–V4.
Figure 3: Time-resolved decoding of individual fMRI time points.

References

  1. Baddeley, A. Working memory: looking back and looking forward. Nature Rev. Neurosci. 4, 829–839 (2003)

    Article  CAS  Google Scholar 

  2. Luck, S. J. & Vogel, E. K. The capacity of visual working memory for features and conjunctions. Nature 390, 279–281 (1997)

    Article  ADS  CAS  Google Scholar 

  3. Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971)

    Article  ADS  CAS  Google Scholar 

  4. Miyashita, Y. & Chang, H. S. Neuronal correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68–70 (1988)

    Article  ADS  CAS  Google Scholar 

  5. Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996)

    Article  CAS  Google Scholar 

  6. Courtney, S. M., Ungerleider, L. G., Keil, K. & Haxby, J. V. Transient and sustained activity in a distributed neural system for human working memory. Nature 386, 608–611 (1997)

    Article  ADS  CAS  Google Scholar 

  7. Pessoa, L., Gutierrez, E., Bandettini, P. & Ungerleider, L. Neural correlates of visual working memory: fMRI amplitude predicts task performance. Neuron 35, 975–987 (2002)

    Article  CAS  Google Scholar 

  8. Curtis, C. E. & D'Esposito, M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn. Sci. 7, 415–423 (2003)

    Article  Google Scholar 

  9. Todd, J. J. & Marois, R. Capacity limit of visual short-term memory in human posterior parietal cortex. Nature 428, 751–754 (2004)

    Article  ADS  CAS  Google Scholar 

  10. Vogel, E. K. & Machizawa, M. G. Neural activity predicts individual differences in visual working memory capacity. Nature 428, 748–751 (2004)

    Article  ADS  CAS  Google Scholar 

  11. Xu, Y. & Chun, M. M. Dissociable neural mechanisms supporting visual short-term memory for objects. Nature 440, 91–95 (2006)

    Article  ADS  CAS  Google Scholar 

  12. Magnussen, S. & Greenlee, M. W. The psychophysics of perceptual memory. Psychol. Res. 62, 81–92 (1999)

    Article  CAS  Google Scholar 

  13. Pasternak, T. & Greenlee, M. W. Working memory in primate sensory systems. Nature Rev. Neurosci. 6, 97–107 (2005)

    Article  CAS  Google Scholar 

  14. Offen, S., Schluppeck, D. & Heeger, D. J. The role of early visual cortex in visual short-term memory and visual attention. Vision Res. 10.1016/j.visres.2007.12.022 (in the press)

  15. Bisley, J. W., Zaksas, D., Droll, J. A. & Pasternak, T. Activity of neurons in cortical area MT during a memory for motion task. J. Neurophysiol. 91, 286–300 (2004)

    Article  Google Scholar 

  16. Zaksas, D. & Pasternak, T. Directional signals in the prefrontal cortex and in area MT during a working memory for visual motion task. J. Neurosci. 26, 11726–11742 (2006)

    Article  CAS  Google Scholar 

  17. Kamitani, Y. & Tong, F. Decoding the visual and subjective contents of the human brain. Nature Neurosci. 8, 679–685 (2005)

    Article  CAS  Google Scholar 

  18. Kastner, S. & Ungerleider, L. G. Mechanisms of visual attention in the human cortex. Annu. Rev. Neurosci. 23, 315–341 (2000)

    Article  CAS  Google Scholar 

  19. Sasaki, Y. et al. The radial bias: a different slant on visual orientation sensitivity in human and nonhuman primates. Neuron 51, 661–670 (2006)

    Article  CAS  Google Scholar 

  20. Tong, F. Primary visual cortex and visual awareness. Nature Rev. Neurosci. 4, 219–229 (2003)

    Article  CAS  Google Scholar 

  21. Kosslyn, S. M., Ganis, G. & Thompson, W. L. Neural foundations of imagery. Nature Rev. Neurosci. 2, 635–642 (2001)

    Article  CAS  Google Scholar 

  22. Roelfsema, P. R. Elemental operations in vision. Trends Cogn. Sci. 9, 226–233 (2005)

    Article  Google Scholar 

  23. Super, H., Spekreijse, H. & Lamme, V. A. A neural correlate of working memory in the monkey primary visual cortex. Science 293, 120–124 (2001)

    Article  CAS  Google Scholar 

  24. Ress, D., Backus, B. T. & Heeger, D. J. Activity in primary visual cortex predicts performance in a visual detection task. Nature Neurosci. 3, 940–945 (2000)

    Article  CAS  Google Scholar 

  25. Geng, J. J., Ruff, C. C. & Driver, J. Saccades to a remembered location elicit spatially specific activation in the human retinotopic visual cortex. J. Cogn. Neurosci. 21, 230–245 (2009)

    Article  Google Scholar 

  26. Miller, E. K., Li, L. & Desimone, R. Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J. Neurosci. 13, 1460–1478 (1993)

    Article  CAS  Google Scholar 

  27. Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nature Rev. Neurosci. 7, 358–366 (2006)

    Article  CAS  Google Scholar 

  28. Logothetis, N. K. et al. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001)

    Article  ADS  CAS  Google Scholar 

  29. Lee, H., Simpson, G. V., Logothetis, N. K. & Rainer, G. Phase locking of single neuron activity to theta oscillations during working memory in monkey extrastriate visual cortex. Neuron 45, 147–156 (2005)

    Article  CAS  Google Scholar 

  30. Wang, X. J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank D. Brady and B. Wolfe for technical support, and J. Gore and the Vanderbilt University Institute of Imaging Science for MRI support. This work was supported by a grant from the National Eye Institute, National Institutes of Health to F.T. and a postgraduate fellowship from the Natural Sciences and Engineering Research Council of Canada to S.A.H.

Author Contributions F.T. devised and designed the experiments, S.A.H. and F.T. programmed the experiments, S.A.H. conducted the experiments and carried out the analyses with assistance from F.T., F.T. and S.A.H. wrote the paper together.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Frank Tong.

Supplementary information

Supplementary Information

This file contains Supplementary Methods, Supplementary Data, Supplementary References and Supplementary Figures 1-7 with Legends (PDF 2982 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Cite this article

Harrison, S., Tong, F. Decoding reveals the contents of visual working memory in early visual areas. Nature 458, 632–635 (2009). https://doi.org/10.1038/nature07832

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature07832

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing