Research papersNeuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex
Section snippets
Introduction: no place (for unisensory auditory neurons) to hide
While peripheral transduction in the cochlea appears specifically designed to capture the physical characteristics of sound, the auditory cortex is not purely a sound processor. The advent of new techniques, coupled with new conceptual models, has made it clear that even at the level of A1, auditory processing is influenced by multimodal stimuli Following on initial indications from experiments in humans (Sams et al., 1991, Calvert et al., 1997, Giard and Peronnet, 1999) and a parallel line of
Anatomical convergence of visual and somatosensory inputs in auditory cortex
Anatomical evidence forms a framework for thinking about potential non-auditory influences on auditory cortex, though these data cannot tell us whether and to what extent these input routes function in multisensory integration. As was already apparent in our initial review of this topic (Schroeder et al., 2003), there is strong evidence for potential cortical and subcortical sources of non-auditory input to auditory cortex and more recent investigations continue to support and elaborate on this
Response timing of non-auditory signals in auditory cortex
Analysis of the response timing to stimulus onset in auditory cortex can be used to establish a model of auditory and non-auditory input timing into auditory cortex. In this section, we first consider the important experimental variables affecting response latency. Because differences across experimental paradigms and analytical methods can greatly influence reported latency values, we have summarized response timing data from our group, which employed the same paradigm (and largely concur with
Multisensory mechanisms in auditory cortex and relationships to perception and behavior
Low level multisensory integration in auditory cortex has the potential to greatly benefit perception and behavior because input from multiple modalities can enhance the representation of sound that the rest of the auditory cortices (and higher-order regions) receives. Audiovisual integration in low level auditory areas has been linked to specific perceptual benefits such as improved speech understanding (Grant et al., 1998, Sekiyama et al., 2003, van Wassenhove et al., 2005, Schroeder et al.,
Conclusions and future directions
The overwhelming evidence of multisensory integration at even the lowest levels of the auditory system suggests that multisensory interaction may be the rule rather than the exception for brain processing in humans and other primates. This view presents a challenging paradox for the classic hierarchical view that increasingly complex information is incorporated into the neural code in a series of subsequent downstream areas. On the one hand, incorporation of multisensory information at low
Acknowledgements
We would like to thank our colleagues Tammy McGinnis, M. Noelle O’Connel and Aimee Mills for their assistance. We also thank Peter Lakatos, Arnaud Falchier and Troy A. Hackett for their helpful comments and suggestions. This work was supported by the National Institutes of Health DC 09918 and MH 61989.
References (93)
- et al.
Reaching beyond the classical receptive field of V1 neurons: horizontal or feedback axons?
J. Physiol. Paris
(2003) - et al.
Auditory cortical neuron response differences under isoflurane versus pentobarbital anesthesia
Hear. Res.
(2001) - et al.
Multisensory interplay reveals crossmodal influences on ‘sensory-specific’ brain regions, neural responses, and judgments
Neuron
(2008) - et al.
Is neocortex essentially multisensory?
Trends Cogn. Sci.
(2006) - et al.
Activation of the human posterior parietal and temporoparietal cortices during audiotactile interaction
Neuroimage
(2003) - et al.
Integration of heard and seen speech: a factor in learning disabilities in children
Neurosci. Lett.
(2003) The thalamic matrix and thalamocortical synchrony
Trends Neurosci.
(2001)- et al.
Parchment-skin illusion: sound-biased touch
Curr. Biol.
(1998) - et al.
Neuronal oscillations and multisensory interaction in primary auditory cortex
Neuron
(2007) - et al.
Relationships between behavior, brainstem and cortical encoding of seen and heard speech in musicians and non-musicians
Hear. Res.
(2008)
Multisensory convergence in calcarine visual areas in macaque monkey
Int. J. Psychophysiol.
Laminar origins and terminations of cortical connections of the occipital lobe in the rhesus monkey
Brain Res.
Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions
Schizophr. Res.
Seeing speech: visual information from lip movements modifies activity in the human auditory cortex
Neurosci. Lett.
Multisensory contributions to low-level, ‘unisensory’ processing
Curr. Opin. Neurobiol.
The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex
Brain Res. Cogn. Brain Res.
Neuronal oscillations and visual amplification of speech
Trends Cogn. Sci.
Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing
Int. J. Psychophysiol.
Auditory–visual speech perception examined by fMRI and PET
Neurosci. Res.
Cellular generators of the cortical auditory evoked potential initial component
Electroencephalogr. Clin. Neurophysiol.
Cross-modal interactions in the perception of musical performance
Cognition
Changes of single unit activity in the cat’s auditory thalamus and cortex associated to different anesthetic conditions
Neurosci. Res.
Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys
J. Neurosci.
Activation of auditory cortex during silent lipreading
Science
Heteromodal connections supporting multisensory integration at low levels of cortical processing in the monkey
Eur. J. Neurosci.
Auditory evoked responses. Intracranial and extracranial average evoked responses
Arch. Neurol.
The natural statistics of audiovisual speech
PLoS Comput. Biol.
Functional anatomy and interaction of fast and slow visual pathways in macaque monkeys
Cereb. Cortex
Single unit activity in the auditory cortex of the cat
Bull. Johns Hopkins Hosp.
Cortical connections of the auditory cortex in marmoset monkeys: core and medial belt regions
J. Comp. Neurol.
Thalamic connections of the auditory cortex in marmoset monkeys: core and medial belt regions
J. Comp. Neurol.
Increased cortical representation of the fingers of the left hand in string players
Science
Anatomical evidence of multimodal integration in primate striate cortex
J. Neurosci.
The case for feedforward multisensory convergence during early cortical processing
Neuroreport
Auditory cortical neurons respond to somatosensory stimulation
J. Neurosci.
Anesthesia changes frequency tuning of neurons in the rat primary auditory cortex
J. Neurophysiol.
Auditory–visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study
J. Cogn. Neurosci.
Sources of attention-sensitive visual event-related potentials
Brain Topogr.
Auditory–visual speech recognition by hearing-impaired subjects: consonant recognition, sentence recognition, and auditory–visual integration
J. Acoust. Soc. Am.
Multisensory convergence in auditory cortex. II. Thalamocortical connections of the caudal superior temporal plane
J. Comp. Neurol.
Sources of somatosensory input to the caudal belt areas of auditory cortex
Perception
Thalamocortical connections of the parabelt auditory cortex in macaque monkeys
J. Comp. Neurol.
Auditory enhancement of visual temporal order judgment
Neuroreport
Aspects of multisensory perception: the integration of visual and auditory information in musical experiences
Ann. NY Acad. Sci.
A new view of specific and nonspecific thalamocortical connections
Adv. Neurol.
Cited by (83)
Cognitive function mediates the relationship between visual contrast sensitivity and functional outcome in schizophrenia
2021, Journal of Psychiatric ResearchTopographical and laminar distribution of audiovisual processing within human planum temporale
2021, Progress in NeurobiologyCross-sensory inhibition or unisensory facilitation: A potential neural architecture of modality switch effects
2020, Journal of Mathematical PsychologyCitation Excerpt :Moreover, previous studies demonstrated that these synapses can be trained by experience, to implement the prior probability of the co-occurrence of audio–visual stimuli in close temporal and spatial proximity (Ursino et al., 2017). Various data in the literature confirm the existence of cross-modal links between the primary visual and auditory regions ((Alais et al., 2010; Driver & Noesselt, 2008; Foxe & Schroeder, 2005; Ghazanfar & Schroeder, 2006; Musacchia & Schroeder, 2009; Recanzone, 2009; Shams & Kim, 2010; Stein & Stanford, 2008; Ursino et al., 2014) for reviews). Finally, we wish to remark that these cross-modal synapses in the model are not strong enough to evoke a phantom response in the other area in the case of unisensory inputs, i.e., only a single area is active in the unisensory condition.