Skip to main content
SpringerLink
Log in

Predictions of Speech Chimaera Intelligibility Using Auditory Nerve Mean-Rate and Spike-Timing Neural Cues

  • Research Article
  • Published:
Journal of the Association for Research in Otolaryngology Aims and scope Submit manuscript

Abstract

Perceptual studies of speech intelligibility have shown that slow variations of acoustic envelope (ENV) in a small set of frequency bands provides adequate information for good perceptual performance in quiet, whereas acoustic temporal fine-structure (TFS) cues play a supporting role in background noise. However, the implications for neural coding are prone to misinterpretation because the mean-rate neural representation can contain recovered ENV cues from cochlear filtering of TFS. We investigated ENV recovery and spike-time TFS coding using objective measures of simulated mean-rate and spike-timing neural representations of chimaeric speech, in which either the ENV or the TFS is replaced by another signal. We (a) evaluated the levels of mean-rate and spike-timing neural information for two categories of chimaeric speech, one retaining ENV cues and the other TFS; (b) examined the level of recovered ENV from cochlear filtering of TFS speech; (c) examined and quantified the contribution to recovered ENV from spike-timing cues using a lateral inhibition network (LIN); and (d) constructed linear regression models with objective measures of mean-rate and spike-timing neural cues and subjective phoneme perception scores from normal-hearing listeners. The mean-rate neural cues from the original ENV and recovered ENV partially accounted for perceptual score variability, with additional variability explained by the recovered ENV from the LIN-processed TFS speech. The best model predictions of chimaeric speech intelligibility were found when both the mean-rate and spike-timing neural cues were included, providing further evidence that spike-time coding of TFS cues is important for intelligibility when the speech envelope is degraded.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

FIG. 1
FIG. 2
FIG. 3
FIG. 4
FIG. 5
FIG. 6
FIG. 7
FIG. 8
FIG. 9

Similar content being viewed by others

References

  • Apoux F, Yoho SE, Youngdahl CL, Healy E (2013) Can envelope recovery account for speech recognition based on temporal fine structure? Proceedings of Meetings on Acoustics 19(1):050072

    Article  Google Scholar 

  • Baer T, Moore BCJ, Gatehouse S (1993) Spectral contrast enhancement of speech in noise for listeners with sensorineural hearing impairment: effects on intelligibility, quality, and response times. J Rehabil Res Dev 30(1):49–72

    CAS  PubMed  Google Scholar 

  • Bentsen T, Harte JM, Dau T (2011) Human cochlear tuning estimates from stimulus-frequency otoacoustic emissions. J Acoust Soc Am 129(6):3797–3807

    Article  PubMed  Google Scholar 

  • Bondy J, Bruce IC, Becker S, Haykin S (2004) Predicting speech intelligibility from a population of neurons. In: Thrun S, Saul L, Schölkopf B (eds) Advances in neural information processing systems 16. MIT Press, Cambridge, MA, pp 1409–1416

    Google Scholar 

  • Bruce IC (2004) Physiological assessment of contrast-enhancing frequency shaping and multiband compression in hearing aids. Physiol Meas 25(4):945–956

    Article  PubMed  Google Scholar 

  • Bruce IC, Dinath F, Zeyl T (2007) Insights into optimal phonemic compression from a computational model of the auditory periphery. In: Auditory Signal Processing in Hearing-Impaired Listeners, Internationl Symposium on Audiological and Auditory Research (ISAAR), p 73–81

  • Bruce IC, Léger AC, Moore BC, Lorenzi C (2013) Physiological prediction of masking release for normal-hearing and hearing-impaired listeners. Proceedings of Meetings on Acoustics: ICA 2013 Montreal, Acoustical Society of America 133(5):1–8

  • Bruce IC, Léger AC, Wirtzfeld MR, Moore BC, Lorenzi C (2015) Spike-time coding and auditory-nerve degeneration best explain speech intelligibility in noise for normal and near-normal low-frequency hearing. In: Abstracts of the 38th ARO Midwinter Research Meeting

  • Burnham KP, Anderson DR (2002) Model selection and multimodel inference, a practical information-theoretic approach, 2nd edn. Springer, New York

    Google Scholar 

  • Chi T, Gao Y, Guyton MC, Ru P, Shamma S (1999) Spectro-temporal modulation transfer functions and speech intelligibility. J Acoust Soc Am 106(5):2719–2732

    Article  CAS  PubMed  Google Scholar 

  • Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) Lexcial information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences. J Exp Psychol 134(2):222–241

    Article  Google Scholar 

  • Delgutte B (1997) Auditory neural processing of speech. The handbook of phonetic sciences pp:507–538

  • Dinath F, Bruce IC (2008) Hearing aid gain prescriptions balance restoration of auditory nerve mean-rate and spike-timing representations of speech. In: Proceedings of 30th International IEEE Engineering in Medicine and Biology Conference, IEEE, Piscataway, NJ, p 1793–1796

  • Drullman R (1995) Temporal envelope and fine structure cues for speech intelligibility. J Acoust Soc Am 97(1):585–592

    Article  CAS  PubMed  Google Scholar 

  • Dudley H (1939) The vocoder. Bell Labs Record 17:122–126

    Google Scholar 

  • Elhilali M, Chi T, Shamma SA (2003)A spectro-temporal modulation index (STMI) for assessment of speech intelligibility. Speech Comm 41(2, 3):331–348

  • Flanagan JL (1980) Parametric coding of speech spectra. J Acoust Soc Am 68(2):412–419

    Article  Google Scholar 

  • Fogerty D, Humes LE (2012)The role of vowel and consonant fundamental frequency, envelope, and temporal fine structure cues to the intelligibility of words and sentences. J Acoust Soc Am 131(2):1490–1501

  • Franck BAM, Sidonne C, van Kreveld-Bos GM, Dreschler WA, Verschuure H (1999) Evaluation of spectral enhancement in hearing aids, combined with phonemic compression. J Acoust Soc Am 106(3):1452–1464

    Article  CAS  PubMed  Google Scholar 

  • French NR, Steinberg JC (1947) Factors governing the intelligibility of speech sounds. J Acoust Soc Am 19:90–119

    Article  Google Scholar 

  • Ghitza O (2001) On the upper cutoff frequency of the auditory critical-band envelope detectors in the context of speech perception. J Acoust Soc Am 110(3):1628–1640

    Article  CAS  PubMed  Google Scholar 

  • Gilbert G, Lorenzi C (2006) The ability of listeners to use recovered envelope cues from speech fine structure. J Acoust Soc Am 119(4):2438–2444

    Article  PubMed  Google Scholar 

  • Gilbert G, Bergeras I, Voillery D, Lorenzi C (2007) Effects of periodic interruptions on the intelligibility of speech based on temporal fine-structure or envelope cues. J Acoust Soc Am 122(3):1336–1339

    Article  PubMed  Google Scholar 

  • Greenwood DD (1990) A cochlear frequency-position function for several species–29 years later. J Acoust Soc Am 87(6):2592–2605

    Article  CAS  PubMed  Google Scholar 

  • Hartline HK (1974) Studies on the excitation and inhibition in the retina, Edited by Floyd Ratliff. The Rockefeller University Press, New York

  • Heinz MG, Swaminathan J (2009) Quantifying envelope and fine-structure coding in auditory nerve responses to chimaeric speech. J Assoc Res Otolaryngol 10(3):407–423

    Article  PubMed  PubMed Central  Google Scholar 

  • Hines A, Harte N (2010) Speech intelligibility from image processing. Speech Comm 52(9):736–752

    Article  Google Scholar 

  • Hines A, Harte N (2012) Speech intelligibility prediction using a neurogram similarity index measure. Speech Comm 54(2):306–320

    Article  Google Scholar 

  • Hopkins K, Moore BCJ, Stone MA (2010) The effects of the addition of low-level, low-noise noise on the intelligibility of sentences processed to remove temporal envelope information. J Acoust Soc Am 128(4):2150–2161

    Article  PubMed  Google Scholar 

  • Hossain ME, Jassim WA, Zilany MSA (2016) Reference-free assessment of speech intelligibility using bispectrum of an auditory neurogram. PLoS One 11(3):e0150,415

    Article  Google Scholar 

  • Ibrahim RA, Bruce IC (2010) Effects of peripheral tuning on the auditory nerve’s representation of speech envelope and temporal fine structure cues. In: Lopez-Poveda EA, Palmer AR, Meddis R (eds) The neurophysiological basis of auditory perception. Springer, New York, pp 429–438

    Chapter  Google Scholar 

  • Jackson BS, Carney LH (2005) The spontaneous-rate histogram of the auditory nerve can be explained by only two or three spontaneous rates and long-range dependence. J Assoc Res Otolaryngol 6(2):148–159

    Article  PubMed  PubMed Central  Google Scholar 

  • Jassim WA, Zilany MS (2016) Speech quality assessment using 2d neurogram orthogonal moments. Speech Comm 80:34–48

    Article  Google Scholar 

  • Johnson DH (1980) The relationship between spike rate and synchrony in responses of auditory-nerve fibers to single tones. J Acoust Soc Am 68(4):1115–1122

    Article  CAS  PubMed  Google Scholar 

  • Jørgensen S, Ewert SD, Dau T (2013) A multi-resolution envelope-power based model for speech intelligibility. J Acoust Soc Am 134(1):436–446

    Article  PubMed  Google Scholar 

  • Joris PX, Yin TCT (1992) Responses to amplitude-modulated tones in the auditory nerve of the cat. J Acoust Soc Am 91(1):215–232

    Article  CAS  PubMed  Google Scholar 

  • Joris PX, Schreiner CE, Rees A (2004) Neural processing of amplitude-modulated sounds. Physiol Rev 84(2):541–577

    Article  CAS  PubMed  Google Scholar 

  • Joris PX, Bergevin C, Kalluri R, McLaughlin M, Michelet P, van der Heijden M, Shera CA (2011) Frequency selectivity in old-world monkeys corroborates sharp cochlear tuning in humans. Proc Natl Acad Sci 108(42):17,516–17,520

    Article  CAS  Google Scholar 

  • Kates JM, Arehart KH (2014) The hearing-aid speech perception index (HASPI). Speech Comm 65:75–93

    Article  Google Scholar 

  • Kiang NYS, Watanabe T, Thomas EC, Clark LF (1965) Discharge patterns of single fibers in the cat’s auditory nerve. Res. Monogr. No. 35, M.I.T. Press, Cambridge

  • Léger AC, Desloge JG, Braida LD, Swaminathan J (2015a) The role of recovered envelope cues in the identification of temporal fine-structure speech for hearing-impaired listeners. J Acoust Soc Am 137(1):505–508

    Article  PubMed  PubMed Central  Google Scholar 

  • Léger AC, Reed CM, Desloge JG, Swaminathan J, Braida LD (2015b)Consonant identification in noise using Hilbert-transform temporal fine-structure speech and recovered-envelope speech for listeners with normal and impaired hearing. J Acoust Soc Am 138(1):389–403

  • Liberman MC (1978) Auditory-nerve response from cats raised in a low-noise chamber. J Acoust Soc Am 63(2):442–455

    Article  CAS  PubMed  Google Scholar 

  • Logan BF Jr (1977) Information in the zero crossings of bandpass signals. Bell Syst Tech J 56(4):487–510

    Article  Google Scholar 

  • Lopez-Poveda EA, Eustaquio-Martin A (2013) On the controversy about the sharpness of human cochlear tuning. J Assoc Res Otolaryngol 14(5):673–686

    Article  PubMed  PubMed Central  Google Scholar 

  • Lorenzi C, Gilbert G, Carn H, Garnier S, Moore BCJ (2006) Speech perception problems of the hearing impaired reflect inability to use temporal fine structure. Proc Natl Acad Sci U S A 103(49):18,866–18,869

    Article  CAS  Google Scholar 

  • Lyzenga J, Festen JM, Houtgast T (2002) A speech enhancement scheme incorporating spectral expansion evaluated with simulated loss of frequency selectivity. J Acoust Soc Am 112(3):1145–1157

    Article  CAS  PubMed  Google Scholar 

  • Mesgarani N, David SV, Fritz JB, Shamma SA (2008) Phoneme representation and classification in primary auditory cortex. J Acoust Soc Am 123(2):899–909

    Article  PubMed  Google Scholar 

  • Miller RL, Schilling JR, Franck KR, Young ED (1997)Effects of acoustic trauma on the representation of the vowel /ε/ in cat auditory nerve fibers. J Acoust Soc Am 101(6):3602–3616

  • Moore BCJ (2008) The role of temporal fine structure processing in pitch perception, masking, and speech perception for normal-hearing and hearing-impaired people. J Assoc Res Otolaryngol 9(4):399–406

    Article  PubMed  PubMed Central  Google Scholar 

  • Nie K, Stickney G, Zeng FG (2005) Encoding frequency modulation to improve cochlear implant performance in noise. IEEE Trans Biomed Eng 52(1):64–73

    Article  PubMed  Google Scholar 

  • Nie K, Atlas L, Rubinstein J (2008) Single sideband encoder for music coding in cochlear implants. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008), p 4209–4212

  • Paliwal K, Wójcicki K (2008) Effect of analysis window duration on speech intelligibilty. IEEE Signal Processing Letters 15:785–788

    Article  Google Scholar 

  • Pascal J, Bourgeade A, Lagier M, Legros C (1998) Linear and nonlinear model of the human middle ear. J Acoust Soc Am 104(3):1509–1516

    Article  CAS  PubMed  Google Scholar 

  • Rice SO (1973) Distortion produced by band limitation of an FM wave. Bell Syst Tech J 52(5):605–626

    Article  Google Scholar 

  • Rose JE, Brugge JF, Anderson DJ, Hind JE (1967) Phase-locked response to low-frequency tones in single auditory nerve fibers of the squirrel monkey. J Neurophsiology 30(4):769–793

    CAS  Google Scholar 

  • Rosen S (1992) Temporal information in speech: acoustic, auditory and linguistic aspects. Philos Trans: Biol Sci 336(1278):367–373

    Article  CAS  Google Scholar 

  • Ruggero MA, Temchin AN (2005) Unexceptional sharpness of frequency tuning in the human cochlea. Proc Natl Acad Sci U S A 102(51):18,614–18,619

    Article  CAS  Google Scholar 

  • Sachs MB, Young ED (1979) Encoding of steady-state vowels in the auditory nerve: representation in terms of discharge rate. J Acoust Soc Am 66(2):470–479

    Article  CAS  PubMed  Google Scholar 

  • Sachs MB, Young ED (1980) Effects of nonlinearities on speech encoding in the auditory nerve. J Acoust Soc Am 68(3):858–875

    Article  CAS  PubMed  Google Scholar 

  • Sachs MB, Voigt HF, Young ED (1983) Auditory nerve representation of vowels in background noise. J Neurophysiol 50(1):27–45

    CAS  PubMed  Google Scholar 

  • Shamma SA (1985) Speech processing in the auditory system II: Lateral inhibition and the central processing of speech evoked activity in the auditory nerve. J Acoust Soc Am 78(5):1622–1632

  • Shamma SA (1998) Spatial and temporal processing in the auditory system. In: Koch C, Segev I (eds) Methods of neuronal modeling: from ions to networks, 2nd edn. MIT Press, Cambridge, MA, pp 411–460 

  • Shamma S, Lorenzi C (2013) On the balance of envelope and temporal fine structure in the encoding of speech in the early auditory system. J Acoust Soc Am 133(5):2818–2833

    Article  PubMed  PubMed Central  Google Scholar 

  • Shannon RV, Zeng FG, Kamath V, Wygonski J, Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270(5234):303–304

    Article  CAS  PubMed  Google Scholar 

  • Sheft S, Ardoint M, Lorenzi C (2008) Speech identification based on temporal fine structure cues. J Acoust Soc Am 124(1):562–575

    Article  PubMed  PubMed Central  Google Scholar 

  • Shera CA, Guinan JJ Jr, Oxenham AJ (2002) Revised estimates of human cochlear tuning from otoacoustic and behavioral measurements. Proc Natl Acad Sci 99(5):3318–3323

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Shera CA, Guinan JJ Jr, Oxenham AJ (2010) Otoacoustic estimation of cochlear tuning: validation in the chinchilla. J Assoc Res Otolaryngol 11(3):343–365

    Article  PubMed  PubMed Central  Google Scholar 

  • Simpson AM, Moore BCJ, Glasberg BR (1990) Spectral enhancement to improve the intelligibility of speech in noise for hearing-impaired listeners. Acta Otolaryngol Suppl 469:101–107

    CAS  PubMed  Google Scholar 

  • Sit JJ, Simonson AM, Oxenham AJ, Faltys MA, Sarpeshkar R (2007) A low-power asynchronous interleaved sampling algorithm for cochlear implants that encodes envelope and phase information. IEEE Trans Biomed Eng 54(1):138–149

    Article  PubMed  Google Scholar 

  • Smith ZM, Delgutte B, Oxenham AJ (2002) Chimaeric sounds reveal dichotomies in auditory perception. Nature 416(6876):87–90

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Stone MA, Moore BCJ (1992) Spectral feature enhancement for people with sensorineural hearing impairment: effects on speech intelligibility and quality. J Rehabil Res Dev 29(2):39–56

    Article  CAS  PubMed  Google Scholar 

  • Studebaker GA (1985) A “rationalized” arcsine transform. J Speech Hear Res 28(3):455–462

    Article  CAS  PubMed  Google Scholar 

  • Swaminathan J, Heinz MG (2012) Psychophysiological analyses demonstrate the importance of neural envelope coding for speech perception in noise. J Neurosci 32(5):1747–1756

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Swaminathan J, Reed CM, Desloge JG, Braida LD, Delhorne LA (2014) Consonant idenfication using temporal fine structure and recovered envelope cues. J Acoust Soc Am 135(4):2078–2090

    Article  PubMed  PubMed Central  Google Scholar 

  • Tillman TW, Carhart R (1966)An expanded test for speech discrimination utilizing CNC monosyllabic words. Brooks Air Force Base, TX Northwestern University Auditory Test No. 6, USAF School of Aerospace Medicine Technical Report, p 1–12

  • Voelcker HB (1966) Toward a unified theory of modulation, part I: phase-envelope relationships. Proc IEEE 54(3):340–353

    Article  Google Scholar 

  • Voigt HF, Sachs MB, Young ED (1982) Representation of whispered vowels in discharge patterns of auditory-nerve fibers. Hear Res 8(1):49–58

    Article  CAS  PubMed  Google Scholar 

  • Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  PubMed  Google Scholar 

  • Wiener FM, Ross DA (1946) The pressure distribution in the auditory canal in a progressive sound field. J Acoust Soc Am 18(2):401–408

    Article  Google Scholar 

  • Wirtzfeld MW (2017) Predicting speech intelligibility and quality from model auditory nerve fiber mean-rate and spike-timing activity. PhD thesis, McMaster University, Hamilton, ON, Canada

  • Young ED, Oertel D (2003) The cochlear nucleus. In: Shepherd GM (ed) Synaptic organization of the brain. Oxford University Press, NY, chap 4, p 125–163

  • Young ED, Sachs MB (1979) Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers. J Acoust Soc Am 66(5):1381–1403

    Article  CAS  PubMed  Google Scholar 

  • Zeng FG, Nie K, Liu S, Stickney G, Rio ED, Kong YY, Chen H (2004) On the dichotomy in auditory perception between temporal envelope and fine structure cues. J Acoust Soc Am 116(3):1351–1354

    Article  PubMed  Google Scholar 

  • Zilany MSA, Bruce IC (2006) Modeling auditory-nerve responses for high sound pressure levels in the normal and impaired auditory periphery. J Acoust Soc Am 120(3):1446–1466

    Article  PubMed  Google Scholar 

  • Zilany MSA, Bruce IC (2007a) Predictions of speech intelligibility with a model of the normal and impaired auditory-periphery. In: Proceedings of 3rd International IEEE EMBS Conference on Neural Engineering, IEEE, Piscataway, NJ

  • Zilany MSA, Bruce IC (2007b) Representation of the vowel /ε/ in normal and impaired auditory nerve fibers: model predictions of responses in cats. J Acoust Soc Am 122(1):402–417

  • Zilany MSA, Bruce IC, Nelson PC, Carney LH (2009) A phenomenological model of the synapse between the inner hair cell and auditory nerve: long-term adaptation with power-law dynamics. J Acoust Soc Am 126(5):2390–2412

    Article  PubMed  PubMed Central  Google Scholar 

  • Zilany MSA, Bruce IC, Carney LH (2014) Updated parameters and expanded simulation options for a model of the auditory periphery. J Acoust Soc Am 135(1):283–286

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank Laurel Carney and Hubert de Bruin for advice on the experiment design; Sue Becker for the use of her amplifier, headphones, and testing room; Malcolm Pilgrim and Timothy Zeyl for assistance with running the experiment; Dan Bosnyak and Dave Thompson for assistance with the acoustic calibration; Jason Boulet and the anonymous reviewers for very helpful comments on earlier versions of the manuscript; and the subjects for their participation. This research was supported by the Natural Sciences and Engineering Research Council of Canada (Discovery Grant No. 261736), and the human experiments were approved by the McMaster Research Ethics Board (#2010 051).

Author information

Author notes

    Authors and Affiliations

    1. Department of Electrical and Computer Engineering, McMaster University, 1280 Main Street West, Hamilton, L8S 4K1, ON, Canada

      Michael R. Wirtzfeld, Rasha A. Ibrahim & Ian C. Bruce

    Authors
    1. Michael R. Wirtzfeld
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Rasha A. Ibrahim
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Ian C. Bruce
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Ian C. Bruce.

    Ethics declarations

    Conflict of Interest

    The authors declare that they have no conflict of interest.

    Additional information

    Michael R. Wirtzfeld and Rasha A. Ibrahim contributed equally to this study.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Wirtzfeld, M.R., Ibrahim, R.A. & Bruce, I.C. Predictions of Speech Chimaera Intelligibility Using Auditory Nerve Mean-Rate and Spike-Timing Neural Cues. JARO 18, 687–710 (2017). https://doi.org/10.1007/s10162-017-0627-7

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s10162-017-0627-7

    Keywords

    Search

    Navigation