Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Cortical activity patterns predict speech discrimination ability

Abstract

Neural activity in the cerebral cortex can explain many aspects of sensory perception. Extensive psychophysical and neurophysiological studies of visual motion and vibrotactile processing show that the firing rate of cortical neurons averaged across 50–500 ms is well correlated with discrimination ability. In this study, we tested the hypothesis that primary auditory cortex (A1) neurons use temporal precision on the order of 1–10 ms to represent speech sounds shifted into the rat hearing range. Neural discrimination was highly correlated with behavioral performance on 11 consonant-discrimination tasks when spike timing was preserved and was not correlated when spike timing was eliminated. This result suggests that spike timing contributes to the auditory cortex representation of consonant sounds.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Spectrograms of each speech sound grouped by manner and place of articulation.
Figure 2: Neurograms depicting the onset response of rat A1 neurons to 20 English consonants.
Figure 3: Predictions of consonant discrimination ability based on onset response similarity.
Figure 4: Behavioral discrimination of consonant sounds.
Figure 5: Both average A1 responses and trial-by-trial neural discrimination predicted consonant discrimination ability when temporal information was maintained.
Figure 6: Predictions of consonant discrimination ability based on nearest-neighbor classifier.
Figure 7: Neural discrimination using the onset activity pattern from individual multiunit sites was best correlated with behavior.

Similar content being viewed by others

References

  1. DeWeese, M.R., Hromadka, T. & Zador, A.M. Reliability and representational bandwidth in the auditory cortex. Neuron 48, 479–488 (2005).

    Article  CAS  Google Scholar 

  2. Parker, A.J. & Newsome, W.T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

    Article  CAS  Google Scholar 

  3. Liu, J. & Newsome, W.T. Correlation between speed perception and neural activity in the middle temporal visual area. J. Neurosci. 25, 711–722 (2005).

    Article  CAS  Google Scholar 

  4. Pruett, J.R. Jr., Sinclair, R.J. & Burton, H. Neural correlates for roughness choice in monkey second somatosensory cortex (SII). J. Neurophysiol. 86, 2069–2080 (2001).

    Article  Google Scholar 

  5. Romo, R. & Salinas, E. Flutter discrimination: neural codes, perception, memory and decision making. Nat. Rev. Neurosci. 4, 203–218 (2003).

    Article  CAS  Google Scholar 

  6. Britten, K.H., Shadlen, M.N., Newsome, W.T. & Movshon, J.A. The analysis of visual motion: a comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745–4765 (1992).

    Article  CAS  Google Scholar 

  7. Narayan, R., Grana, G. & Sen, K. Distinct time scales in cortical discrimination of natural sounds in songbirds. J. Neurophysiol. 96, 252–258 (2006).

    Article  Google Scholar 

  8. Schnupp, J.W., Hall, T.M., Kokelaar, R.F. & Ahmed, B. Plasticity of temporal pattern codes for vocalization stimuli in primary auditory cortex. J. Neurosci. 26, 4785–4795 (2006).

    Article  CAS  Google Scholar 

  9. Walker, K.M., Ahmed, B. & Schnupp, J.W. Linking cortical spike pattern codes to auditory perception. J. Cogn. Neurosci. 20, 135–152 (2008).

    Article  Google Scholar 

  10. Ahissar, E. et al. Speech comprehension is correlated with temporal response patterns recorded from auditory cortex. Proc. Natl. Acad. Sci. USA 98, 13367–13372 (2001).

    Article  CAS  Google Scholar 

  11. Orduna, I., Mercado, E. III, Gluck, M.A. & Merzenich, M.M. Cortical responses in rats predict perceptual sensitivities to complex sounds. Behav. Neurosci. 119, 256–264 (2005).

    Article  Google Scholar 

  12. Wang, L., Narayan, R., Grana, G., Shamir, M. & Sen, K. Cortical discrimination of complex natural stimuli: can single neurons match behavior? J. Neurosci. 27, 582–589 (2007).

    Article  CAS  Google Scholar 

  13. Bertoncini, J., Bijeljac-Babic, R., Blumstein, S.E. & Mehler, J. Discrimination in neonates of very short CVs. J. Acoust. Soc. Am. 82, 31–37 (1987).

    Article  CAS  Google Scholar 

  14. Blumstein, S.E. & Stevens, K.N. Acoustic invariance in speech production: evidence from measurements of the spectral characteristics of stop consonants. J. Acoust. Soc. Am. 66, 1001–1017 (1979).

    Article  CAS  Google Scholar 

  15. Fowler, C.A., Brown, J.M., Sabadini, L. & Weihing, J. Rapid access to speech gestures in perception: evidence from choice and simple response time tasks. J. Mem. Lang. 49, 396–413 (2003).

    Article  Google Scholar 

  16. Jongman, A. Duration of frication noise required for identification of English fricatives. J. Acoust. Soc. Am. 85, 1718–1725 (1989).

    Article  CAS  Google Scholar 

  17. Kuhl, P.K. & Miller, J.D. Speech perception by the chinchilla: voiced-voiceless distinction in alveolar plosive consonants. Science 190, 69–72 (1975).

    Article  CAS  Google Scholar 

  18. Reed, P., Howell, P., Sackin, S., Pizzimenti, L. & Rosen, S. Speech perception in rats: use of duration and rise time cues in labeling of affricate/fricative sounds. J. Exp. Anal. Behav. 80, 205–215 (2003).

    Article  Google Scholar 

  19. Miller, G.A. & Nicely, P.E. An analysis of perceptual confusions among some English consonants. J. Acoust. Soc. Am. 27, 338–352 (1955).

    Article  Google Scholar 

  20. Steinschneider, M., Fishman, Y.I. & Arezzo, J.C. Representation of the voice onset time (VOT) speech parameter in population responses within primary auditory cortex of the awake monkey. J. Acoust. Soc. Am. 114, 307–321 (2003).

    Article  Google Scholar 

  21. Steinschneider, M., Reser, D., Schroeder, C.E. & Arezzo, J.C. Tonotopic organization of responses reflecting stop consonant place of articulation in primary auditory cortex (A1) of the monkey. Brain Res. 674, 147–152 (1995).

    Article  CAS  Google Scholar 

  22. Tavabi, K., Obleser, J., Dobel, C. & Pantev, C. Auditory evoked fields differentially encode speech features: an MEG investigation of the P50m and N100m time courses during syllable processing. Eur. J. Neurosci. 25, 3155–3162 (2007).

    Article  Google Scholar 

  23. Young, E.D. Neural representation of spectral and temporal information in speech. Phil. Trans. R. Soc. Lond. B Biol. Sci. 363, 923–945 (2008).

    Article  Google Scholar 

  24. Steinschneider, M. et al. Intracortical responses in human and monkey primary auditory cortex support a temporal processing mechanism for encoding of the voice onset time phonetic parameter. Cereb. Cortex 15, 170–186 (2005).

    Article  Google Scholar 

  25. Cooke, J.E., Zhang, H. & Kelly, J.B. Detection of sinusoidal amplitude modulated sounds: deficits after bilateral lesions of auditory cortex in the rat. Hear. Res. 231, 90–99 (2007).

    Article  Google Scholar 

  26. Dewson, J.H. III, Pribram, K.H. & Lynch, J.C. Effects of ablations of temporal cortex upon speech sound discrimination in the monkey. Exp. Neurol. 24, 579–591 (1969).

    Article  Google Scholar 

  27. Heffner, H.E. & Heffner, R.S. Effect of restricted cortical lesions on absolute thresholds and aphasia-like deficits in Japanese macaques. Behav. Neurosci. 103, 158–169 (1989).

    Article  CAS  Google Scholar 

  28. Rybalko, N., Suta, D., Nwabueze-Ogbo, F. & Syka, J. Effect of auditory cortex lesions on the discrimination of frequency-modulated tones in rats. Eur. J. Neurosci. 23, 1614–1622 (2006).

    Article  Google Scholar 

  29. Wetzel, W., Ohl, F.W., Wagner, T. & Scheich, H. Right auditory cortex lesion in Mongolian gerbils impairs discrimination of rising and falling frequency-modulated tones. Neurosci. Lett. 252, 115–118 (1998).

    Article  CAS  Google Scholar 

  30. Steinschneider, M., Volkov, I.O., Noh, M.D., Garell, P.C. & Howard, M.A. III. Temporal encoding of the voice onset time phonetic parameter by field potentials recorded directly from human auditory cortex. J. Neurophysiol. 82, 2346–2357 (1999).

    Article  CAS  Google Scholar 

  31. Wong, S.W. & Schreiner, C.E. Representation of CV-sounds in cat primary auditory cortex: intensity dependence. Speech Commun. 41, 93–106 (2003).

    Article  Google Scholar 

  32. Kluender, K.R., Diehl, R.L. & Killeen, P.R. Japanese quail can learn phonetic categories. Science 237, 1195–1197 (1987).

    Article  CAS  Google Scholar 

  33. Ramus, F., Hauser, M.D., Miller, C., Morris, D. & Mehler, J. Language discrimination by human newborns and by cotton-top tamarin monkeys. Science 288, 349–351 (2000).

    Article  CAS  Google Scholar 

  34. Sinnott, J.M. & Brown, C.H. Perception of the American English liquid /ra-la/ contrast by humans and monkeys. J. Acoust. Soc. Am. 102, 588–602 (1997).

    Article  CAS  Google Scholar 

  35. Toro, J.M., Trobalon, J.B. & Sebastian-Galles, N. Effects of backward speech and speaker variability in language discrimination by rats. J. Exp. Psychol. Anim. Behav. Process. 31, 95–100 (2005).

    Article  Google Scholar 

  36. Dooling, R.J., Okanoya, K. & Brown, S.D. Speech perception by budgerigars (Melopsittacus undulatus): the voiced-voiceless distinction. Percept. Psychophys. 46, 65–71 (1989).

    Article  CAS  Google Scholar 

  37. Foffani, G. & Moxon, K.A. PSTH-based classification of sensory stimuli using ensembles of single neurons. J. Neurosci. Methods 135, 107–120 (2004).

    Article  Google Scholar 

  38. Weliky, M., Fiser, J., Hunt, R.H. & Wagner, D.N. Coding of natural scenes in primary visual cortex. Neuron 37, 703–718 (2003).

    Article  CAS  Google Scholar 

  39. Grace, J.A., Amin, N., Singh, N.C. & Theunissen, F.E. Selectivity for conspecific song in the zebra finch auditory forebrain. J. Neurophysiol. 89, 472–487 (2003).

    Article  Google Scholar 

  40. Buonomano, D.V. & Merzenich, M. A neural network model of temporal code generation and position-invariant pattern recognition. Neural Comput. 11, 103–116 (1999).

    Article  CAS  Google Scholar 

  41. VanRullen, R., Guyonneau, R. & Thorpe, S.J. Spike times make sense. Trends Neurosci. 28, 1–4 (2005).

    Article  CAS  Google Scholar 

  42. Richmond, B.J., Optican, L.M. & Spitzer, H. Temporal encoding of two-dimensional patterns by single units in primate primary visual cortex. I. Stimulus-response relations. J. Neurophysiol. 64, 351–369 (1990).

    Article  CAS  Google Scholar 

  43. Ohl, F.W. & Scheich, H. Orderly cortical representation of vowels based on formant interaction. Proc. Natl. Acad. Sci. USA 94, 9440–9444 (1997).

    Article  CAS  Google Scholar 

  44. Qin, L., Chimoto, S., Sakai, M. & Sato, Y. Spectral-shape preference of primary auditory cortex neurons in awake cats. Brain Res. 1024, 167–175 (2004).

    Article  CAS  Google Scholar 

  45. Versnel, H. & Shamma, S.A. Spectral-ripple representation of steady-state vowels in primary auditory cortex. J. Acoust. Soc. Am. 103, 2502–2514 (1998).

    Article  CAS  Google Scholar 

  46. Buonomano, D.V. & Merzenich, M.M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).

    Article  CAS  Google Scholar 

  47. Kawahara, H. Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited. Proc. 1997 IEEE Int. Conf. Acoust., Speech, Signal Process 2, 1303–1306 (1997).

    Article  Google Scholar 

  48. Engineer, N.D. et al. Environmental enrichment improves response strength, threshold, selectivity, and latency of auditory cortex neurons. J. Neurophysiol. 92, 73–82 (2004).

    Article  Google Scholar 

  49. Rennaker, R.L., Ruyle, A.M., Street, S.E. & Sloan, A.M. An economical multi-channel cortical electrode array for extended periods of recording during behavior. J. Neurosci. Methods 142, 97–105 (2005).

    Article  CAS  Google Scholar 

  50. Rennaker, R.L., Street, S., Ruyle, A.M. & Sloan, A.M. A comparison of chronic multi-channel cortical implantation techniques: manual versus mechanical insertion. J. Neurosci. Methods 142, 169–176 (2005).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors would like to thank J. Roland, R. Jain and D. Listhrop for assistance with microelectrode mappings. We would like to thank R. Rennaker for technical assistance and training and for providing microelectrode arrays and inserter. We would also like to thank M. Perry, C. Heydrick, A. McMenamy, A. Meepe, C. Dablain, J. Choi, V. Badhiwala, J. Riley, N. Hatate, P. Kan, M. Lazo de la Vega and A. Hudson for help with behavioral training. We would also like to thank S. Blumstein, Y. Cohen, H. Read, S. Denham, L. Miller, S. Edelman, V. Dragoi, H. Abdi, P. Assmann, X. Wang and R. Romo for their suggestions about earlier versions of the manuscript. This work was supported by grants from the US National Institute for Deafness and Other Communicative Disorders and the James S. McDonnell Foundation.

Author information

Authors and Affiliations

Authors

Contributions

C.T.E., C.A.P., R.S.C. and A.C.R. collected behavioral training data. C.T.E., C.A.P., Y.H.C., R.S.C., V.J. and K.Q.C. recorded anesthetized cortical responses. J.A.S. recorded awake cortical responses. M.P.K. and C.T.E. wrote the manuscript and performed data analysis. All authors discussed the paper and commented on the manuscript.

Corresponding authors

Correspondence to Crystal T Engineer or Michael P Kilgard.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8, Supplementary Table 1, Supplementary Data (PDF 1110 kb)

Supplementary Video 1

A1 spatiotemporal activity evoked by consonant onset. The video illustrates the spatiotemporal activity patterns evoked in primary auditory cortex by the onset of twenty different consonant sounds. The color of each polygon indicates the multi-unit activity at each of 63 recording sites in a single rat. The map of characteristic frequency (CF) illustrates the topographic organization of tone frequency tuning. The blue lines under each activity map indicate the average firing rate for all 63 A1 sites. The red lines mark the time at which each spatial activity pattern occurs relative to response onset. (MPG 2459 kb)

Supplementary Video 2

A1 spatiotemporal activity evoked by consonant-vowel syllables. The video illustrates the spatiotemporal activity patterns evoked in primary auditory cortex by words beginning with twenty different consonant sounds followed by /a/ as in 'sad'. The color of each polygon indicates the multi-unit activity at each of 63 recording sites in a single rat. The map of characteristic frequency (CF) illustrates the topographic organization of tone frequency tuning. The blue lines under each activity map indicate the average firing rate for all 63 A1 sites. The red lines mark the time at which each spatial activity pattern occurs relative to response onset. (MPG 4245 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Engineer, C., Perez, C., Chen, Y. et al. Cortical activity patterns predict speech discrimination ability. Nat Neurosci 11, 603–608 (2008). https://doi.org/10.1038/nn.2109

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2109

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing