Review
Sensory modalities are not separate modalities: plasticity and interactions

https://doi.org/10.1016/S0959-4388(00)00241-5Get rights and content

Abstract

Historically, perception has been viewed as a modular function, with the different sensory modalities operating independently of each other. Recent behavioral and brain imaging studies challenge this view, by suggesting that cross-modal interactions are the rule and not the exception in perception, and that the cortical pathways previously thought to be sensory-specific are modulated by signals from other modalities.

Introduction

Cross-modal integration is performed on a vast level in the brain and contributes significantly to adaptive behavior in our daily life. Very little is known about how integration is achieved or its underlying neural mechanisms, however, because the overwhelming majority of studies on perception have focused on one sensory modality. Studying perception in an isolated single modality would be justifiable if different modalities processed sensory inputs independently of each other, as separate ‘modules’. But are sensory modalities really separate modules? A variety of evidence seems to counter this notion of modularity. In this review, we summarize the evidence for vigorous interaction among sensory modalities.

Section snippets

Plasticity across sensory modalities

Both animal and human studies suggest that sensory modalities in early stages of development are not as inherently distinct and independent as was previously once thought. For example, in a study of cross-modal plasticity Sur et al. [1] removed the superior colliculus of both the ferret and the hamster on the day of birth by direct ablation. They also deprived the medial geniculate nucleus or the ventrobasal nucleus from their normal sensory input by sectioning the major input pathways. The

Interaction across modalities

In the phenomenological and psychophysical literature, a wealth of literature on mature perceptual systems indicates that there is vigorous interaction and integration among the sensory modalities—a far greater interaction than what is expected from a ‘naive modularity’ point of view. One of the most intriguing examples of cross-modal interaction is synesthesia, in which an involuntary conscious sensation (such as color) is induced by a stimulus in another modality (such as sound). Synesthesia

Neural mechanisms

At what point along the perceptual processing pathway do these cross-modal interactions take place? Recent data from brain imaging studies suggest that they occur at brain sites that used to be considered as modality-specific. For example, Calvert et al. [35] carried out a functional magnetic resonance imaging (fMRI) study that showed that the primary auditory cortex is activated when a talking face is viewed in the absence of sound. The activation was observed specifically in speech or

Conclusions

We have discussed a wide variety of evidence against the notion of strict modularity of sensory modalities. Both animal studies and human-deprivation cases provide evidence for a surprising degree of cross-modal plasticity in cortical processing. Psychophysical data indicate that interaction between modalities is the rule as opposed to the exception in brain function, and brain imaging and recording studies provide evidence against modularity and for interaction in areas traditionally thought

Acknowledgements

We apologize to the many researchers whose work was omitted owing to space constraints. This work was supported in part by NIH grant HD08506. We thank the section editors for their insightful comments.

References and recommended reading

Papers of particular interest, published within the annual period of review, ave been highlighted as:

  • • of special interest

  • •• of outstanding interest

References (38)

  • M.B Casey et al.

    Prenatal visual experience influences the development of turning bias in bobwhite quail chicks (Colinus virginianus)

    Dev Psychobiol

    (1998)
  • T Kujala et al.

    Electrophysiological evidence for cross-modal plasticity in humans with early- and late-onset blindness

    Psychophysiology

    (1997)
  • N Sadato et al.

    Activation of the primary visual cortex by Braille reading in blind subjects

    Nature

    (1996)
  • L.G Cohen et al.

    Functional relevance of cross-modal plasticity in blind humans

    Nature

    (1997)
  • H McGurk et al.

    Hearing lips and seeing voices

    Nature

    (1976)
  • I.P Howard et al.

    Human Spatial Orientation

    (1966)
  • I Rock et al.

    Vision and touch: an experimentally created conflict between the two senses

    Science

    (1964)
  • J.T Walker et al.

    Auditory-visual conflicts in the perceived duration of lights, tones, and gaps

    J Exp Psychol Hum Percept Perform

    (1981)
  • J.W Gebhard et al.

    On discriminating the rate of visual flicker and auditory flutter

    Am J Psychol

    (1959)
  • Cited by (0)

    View full text