Trends in Neurosciences
ReviewCrossmodal binding through neural coherence: implications for multisensory processing
Introduction
The inputs delivered by different sensory organs provide us with both complementary and redundant information about the environment. Constantly, multisensory interactions occur in the brain to evaluate whether there is a matching of the information arriving through different channels, or whether the signals give rise to conflict and need to be processed separately. The outcome of these interactions is of critical importance for perception, cognitive processing and the control of action 1, 2, 3, 4. Recent studies have revealed that a vast number of cortical operations, including those carried out by primary regions, are shaped by inputs from multiple sensory modalities 4, 5, 6.
In recent years, an increasing number of studies have aimed at characterizing multisensory cortical regions, revealing multisensory processing in the superior temporal sulcus, the intraparietal sulcus and frontal regions, as well as insula and claustrum 4, 6, 7. Interestingly, there is increasing evidence that neurons in areas formerly considered as unisensory like, such as auditory belt areas 3, 4, 6, 8, 9, can also exhibit multisensory characteristics. Furthermore, numerous subcortical structures are involved in multisensory processing. In addition to the superior colliculus [1], these include the striatum [10], the cerebellum [11] and the amygdala [12], as well as nuclei of the thalamus [13].
Whereas the ubiquity and fundamental relevance of multisensory processing are becoming increasingly clear, the neural mechanisms underlying crossmodal interactions are much less well understood. In this article, we review recent studies that cast new light on this issue. Whereas classical studies have postulated a feedforward convergence of unimodal signals as the primary mechanism for multisensory integration 1, 2, there is now evidence that both feedback and lateral interactions are also relevant 4, 6, 14, 15. Beyond this changing view on the anatomical substrate of multisensory interactions, there is increasing awareness that complex dynamic interactions of cell populations, leading to coherent oscillatory firing patterns, might play a key role in mediating cross-systems integration in the brain 16, 17, 18, 19, 20, 21. Here we consider the hypothesis that synchronized oscillations (see Glossary) might provide a potential mechanism for crossmodal integration and for the selection of information that matches across different sensory channels.
Section snippets
Views on crossmodal integration
The classical view posits that multisensory integration occurs in a hierarchical manner by progressive convergence of pathways. In this view, sensory signals are merged only in higher association areas and specialized subcortical regions 1, 2. A core assumption of this approach is that perceptual information is primarily encoded in the firing rate of the cells involved. Multisensory integration, accordingly, is expressed by firing rate changes in neural populations receiving convergent inputs
Oscillatory activity in crossmodal processing
A variety of different paradigms have been used to study the role of oscillatory responses and neural coherence during multisensory processing (Table 1). Most studies have been performed in humans using EEG or MEG (see Glossary), with relatively few animal studies currently available. The approaches used address different aspects of multisensory processing, including (i) bottom-up processing of multisensory stimuli, (ii) crossmodally induced perceptual changes, (iii) modulation by top-down
Functional role of neural synchrony for crossmodal interactions
The data available so far support the hypothesis that coherence of oscillatory responses might play a crucial role in multisensory processing. They show that multisensory interactions can be accompanied by condition-specific changes in oscillatory responses which often, albeit not always, occur in the gamma band 28, 30, 32, 33, 34, 35, 37, 40, 41, 45, 46, 47, 48. The effects observed in EEG or MEG signals are likely to result not only from changes in oscillatory power but also in the phase
Outlook
Clearly, the hypothesis that neural synchrony might play a role in multisensory processing requires further experimental testing (Box 2). Thus far, only a relatively small number of multisensory studies have used coherence measures to explicitly address interactions across different neural systems. Substantial progress will require studies in humans with approaches suitable to capture dynamic cross-systems interactions in source space 53, 54. Furthermore, to characterize the role of correlated
Acknowledgements
We thank Markus Siegel, Peter König, Jörg Hipp and Gernot Supp for helpful comments on the manuscript. D.S. received support from a NARSAD 2006 young investigator award and the German Research Foundation (SE 1859/1–1). J.J.F. received support from a U.S. National Institute of Mental Health grant (NIMH–RO1 MH65350). A.K.E. acknowledges support by the European Union (IST-027268, NEST-043457), the German Research Foundation (GRK 1247/1), the German Federal Ministry of Education and Research
Glossary
- Electroencephalography (EEG)
- The standard noninvasive method to measure electrical potential changes arising from the brain. The EEG signal is recorded on the scalp surface and reflects the summed postsynaptic activity in the underlying cortical regions. A key advantage is the high temporal resolution of the method.
- Evoked oscillatory activity
- Oscillatory neuronal activity that is phase locked to the onset of eliciting events (e.g. sensory inputs). Evoked oscillatory activity can be quantified
References (78)
On the neuronal basis for multisensory convergence: a brief overview
Cogn. Brain Res.
(2002)- et al.
Multisensory spatial interactions: a window onto functional integration in the human brain
Trends Neurosci.
(2005) - et al.
Is neocortex essentially multisensory?
Trends Cogn. Sci.
(2006) Integration of touch and sound in auditory cortex
Neuron
(2005)- et al.
Multisensory perception: beyond modularity and convergence
Curr. Biol.
(2000) Temporal coding in the visual cortex: new vistas on integration in the nervous system
Trends Neurosci.
(1992)Cognitive functions of gamma-band activity: memory match and utilization
Trends Cogn. Sci.
(2004)A mechanism for cognitive dynamics: neuronal communication through neuronal coherence
Trends Cogn. Sci.
(2005)Human gamma-frequency oscillations associated with attention and memory
Trends Neurosci.
(2007)- et al.
Gamma and beta neural activity evoked during a sensory gating paradigm: effects of auditory, somatosensory and cross-modal stimulation
Clin. Neurophysiol.
(2006)
Bisensory stimulation increases gamma-responses over multiple cortical regions
Brain Res. Cogn. Brain Res.
Spatio-temporal frequency characteristics of intersensory components in audiovisual evoked potentials
Brain Res. Cogn. Brain Res.
Neuronal oscillations and multisensory interaction in primary auditory cortex
Neuron
Gamma-band activity over early sensory areas predicts detection of changes in audiovisual speech stimuli
Neuroimage
Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations
Neuropsychologia
Attention-dependent suppression of distracter visual input can be cross-modally cued as indexed by anticipatory parieto-occipital alpha-band oscillations
Brain Res. Cogn. Brain Res.
Neural synchrony in brain disorders: relevance for cognitive dysfunctions and pathophysiology
Neuron
Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions
Schizophr. Res.
Brain oscillations in perception and memory
Int. J. Psychophysiol.
Upper alpha ERD and absolute power: their meaning for memory performance
Prog. Brain Res.
New vistas for α-frequency band oscillations
Trends Neurosci.
Motivation, emotion, and their inhibitory control mirrored in brain oscillations
Neurosci. Biobehav. Rev.
Hippocampal sequence-encoding driven by a cortical multi-item working memory buffer
Trends Neurosci.
Modulation of induced gamma band activity in the human EEG by attention and visual information processing
Int. J. Psychophysiol.
Neuronal oscillations and visual amplification of speech
Trends Cogn. Sci.
The Merging of the Senses
Do early sensory cortices integrate crossmodal information?
Brain Struct. Funct.
Functional imaging of human crossmodal identification and object recognition
Exp. Brain Res.
Crossmodal processing in the human brain: insights from functional neuroimaging studies
Cereb. Cortex
Auditory-somatosensory multisensory processing in auditory association cortex: an fMRI study
J. Neurophysiol.
Multisensory integration in the basal ganglia
Eur. J. Neurosci.
Neural correlates of coherent audiovisual motion perception
Cereb. Cortex
Topographic distribution of modality-specific amygdalar neurons in alert monkey
J. Neurosci.
Auditory thalamus integrates visual inputs into behavioral gains
Nat. Neurosci.
The case for feedforward multisensory convergence during early cortical processing
Neuroreport
Visual feature integration and the temporal correlation hypothesis
Annu. Rev. Neurosci.
Dynamic predictions: oscillations and synchrony in top-down processing
Nat. Rev. Neurosci.
Correlated neuronal activity and the flow of neural information
Nat. Rev. Neurosci.
Modulation of neuronal interactions through neuronal synchronization
Science
Cited by (301)
The role of neural oscillations in visuo-motor communication at the time of saccades
2023, NeuropsychologiaEmerging computational motifs: Lessons from the retina
2023, Neuroscience ResearchThe impact of cardiac phases on multisensory integration
2023, Biological PsychologyResponse of the somatosensory cortex following thermal stimuli to dental implants
2022, Neuroscience Letters