Abstract
By binding multisensory signals, we get robust percepts and respond to our surroundings more correctly and quickly. How and where does the brain link cross-modal sensory information to produce such behavioral advantages? The classical role of sensory thalamus is to relay modality-specific information to the cortex. Here we find that, in the rat thalamus, visual cues influence auditory responses, which have two distinct components: an early phasic one followed by a late gradual buildup that peaks before reward. Although both bimodal presentation and reward value had similar effects on behavioral performance, the cross-modal effect on neural activity showed unique temporal dynamics: it affected the amplitude of the early component and starting level of the late component, whereas reward value affected only the slope of the late component. These results demonstrate that cross-modal cueing modulates gain in the sensory thalamus, potentially providing a priming influence on the choice of an optimal behavior.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Bushara, K.O. et al. Neural correlates of cross-modal binding. Nat. Neurosci. 6, 190–195 (2003).
Shams, L., Kamitani, Y., Thompson, S. & Shimojo, S. Sound alters visual evoked potentials in humans. Neuroreport 12, 3849–3852 (2001).
Wallace, M.T., Ramachandran, R. & Stein, B.E. A revised view of sensory cortical parcellation. Proc. Natl. Acad. Sci. USA 101, 2167–2172 (2004).
Sekiyama, K., Kanno, I., Miura, S. & Sugita, Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47, 277–287 (2003).
Barth, D.S., Goldberg, N., Brett, B. & Di, S. The spatiotemporal organization of auditory, visual, and auditory-visual evoked potentials in rat cortex. Brain Res. 678, 177–190 (1995).
Calvert, G.A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997).
Macaluso, E., Frith, C.D. & Driver, J. Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208 (2000).
Schroeder, C.E. et al. Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50, 5–17 (2003).
McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746–748 (1976).
Driver, J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature 381, 66–68 (1996).
Stein, B.E., Meredith, M.A., Huneycutt, W.S. & McDade, L. Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. J. Cogn. Neurosci. 1, 12–24 (1989).
Wilkinson, L.K., Meredith, M.A. & Stein, B.E. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp. Brain Res. 112, 1–10 (1996).
Jiang, W., Jiang, H. & Stein, B.E. Two corticotectal areas facilitate multisensory orientation behavior. J. Cogn. Neurosci. 14, 1240–1255 (2002).
Glimcher, P.W. The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26, 133–179 (2003).
Schultz, W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 1, 199–207 (2000).
Komura, Y. et al. Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature 412, 546–549 (2001).
Grier, J.B. Nonparametric indexes for sensitivity and bias: computing formulas. Psychol. Bull. 75, 424–429 (1971).
Lovelace, C.T., Stein, B.E. & Wallace, M.T. An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Brain Res. Cogn. Brain Res. 17, 447–453 (2003).
Winer, J.A., Kelly, J.B. & Larue, D.T. Neural architecture of the rat medial geniculate body. Hear. Res. 130, 19–41 (1999).
Rauschecker, J.P., Tian, B., Pons, T. & Mishkin, M. Serial and parallel processing in rhesus monkey auditory cortex. J. Comp. Neurol. 382, 89–103 (1997).
Kaas, J.H., Hackett, T.A. & Tramo, M.J. Auditory processing in primate cerebral cortex. Curr. Opin. Neurobiol. 9, 164–170 (1999).
Hu, B., Senatorov, V. & Mooney, D. Lemniscal and non-lemniscal synaptic transmission in rat auditory thalamus. J. Physiol. (Lond.) 479, 217–231 (1994).
Rauschecker, J.P. & Tian, B. Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proc. Natl. Acad. Sci. USA 97, 11800–11806 (2000).
von Melchner, L., Pallas, S.L. & Sur, M. Visual behaviour mediated by retinal projections directed to the auditory pathway. Nature 404, 871–876 (2000).
Bushara, K.O., Grafman, J. & Hallett, M. Neural correlates of auditory-visual stimulus onset asynchrony detection. J. Neurosci. 21, 300–304 (2001).
Brett-Green, B., Fifkova, E., Larue, D.T., Winer, J.A. & Barth, D.S. A multisensory zone in rat parietotemporal cortex: intra- and extracellular physiology and thalamocortical connections. J. Comp. Neurol. 460, 223–237 (2003).
Barth, D.S. & MacDonald, K.D. Thalamic modulation of high-frequency oscillating potentials in auditory cortex. Nature 383, 78–81 (1996).
Bhattacharya, J., Shams, L. & Shimojo, S. Sound-induced illusory flash perception: role of gamma band responses. Neuroreport 13, 1727–1730 (2002).
Linke, R., De Lima, A.D., Schwegler, H. & Pape, H.C. Direct synaptic connections of axons from superior colliculus with identified thalamo-amygdaloid projection neurons in the rat: possible substrates of a subcortical visual pathway to the amygdala. J. Comp. Neurol. 403, 158–170 (1999).
Stein, B.E. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp. Brain Res. 123, 124–135 (1998).
Stein, B.E., Jiang, W., Wallace, M.T. & Stanford, T.R. Nonvisual influences on visual-information processing in the superior colliculus. Prog. Brain Res. 134, 143–156 (2001).
Weeks, R. et al. A positron emission tomographic study of auditory localization in the congenitally blind. J. Neurosci. 20, 2664–2672 (2000).
Itaya, S.K. & Van Hoesen, G.W. Retinal innervation of the inferior colliculus in rat and monkey. Brain Res. 233, 45–52 (1982).
Wallace, M.T., Meredith, M.A. & Stein, B.E. Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80, 1006–1010 (1998).
Hikosaka, K., Iwai, E., Saito, H. & Tanaka, K. Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J. Neurophysiol. 60, 1615–1637 (1988).
Bell, A.H., Corneil, B.D., Meredith, M.A. & Munoz, D.P. The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can. J. Exp. Psychol. 55, 123–132 (2001).
Frens, M.A. & Van Opstal, A.J. Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res. Bull. 46, 211–224 (1998).
Ernst, M.O. & Bulthoff, H.H. Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169 (2004).
Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998).
Romanski, L.M. et al. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 2, 1131–1136 (1999).
Kobler, J.B., Isbey, S.F. & Casseday, J.H. Auditory pathways to the frontal cortex of the mustache bat, Pteronotus parnellii. Science 236, 824–826 (1987).
Takada, M., Itoh, K., Yasui, Y., Sugimoto, T. & Mizuno, N. Topographical projections from the posterior thalamic regions to the striatum in the cat, with reference to possible tecto-thalamo-striatal connections. Exp. Brain Res. 60, 385–396 (1985).
LeDoux, J.E., Farb, C.R. & Romanski, L.M. Overlapping projections to the amygdala and striatum from auditory processing areas of the thalamus and cortex. Neurosci. Lett. 134, 139–144 (1991).
Shammah-Lagnado, S.J., Alheid, G.F. & Heimer, L. Efferent connections of the caudal part of the globus pallidus in the rat. J. Comp. Neurol. 376, 489–507 (1996).
Dalley, J.W., Cardinal, R.N. & Robbins, T.W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771–784 (2004).
Brunia, C.H. & van Boxtel, G.J. Wait and see. Int. J. Psychophysiol. 43, 59–75 (2001).
Rektor, I., Kanovsky, P., Bares, M., Louvel, J. & Lamarche, M. Event-related potentials, CNV, readiness potential, and movement accompanying potential recorded from posterior thalamus in human subjects. A SEEG study. Neurophysiol. Clin. 31, 253–261 (2001).
Rafal, R.D. & Posner, M.I. Deficits in human visual spatial attention following thalamic lesions. Proc. Natl. Acad. Sci. USA 84, 7349–7353 (1987).
Harvey, M., Olk, B., Muir, K. & Gilchrist, I.D. Manual responses and saccades in chronic and recovered hemispatial neglect: a study using visual search. Neuropsychologia 40, 705–717 (2002).
Meredith, M.A. & Stein, B.E. Interactions among converging sensory inputs in the superior colliculus. Science 221, 389–391 (1983).
Acknowledgements
We thank F. Miles and M. Shidara for comments on the manuscript and S. Kitazawa and T. Kitamura for discussions. This work was partly supported by Special Coordination Funds for Promoting Science & Technology (Y.K.) and by Grants-in-Aid for Scientific Research nos. 17021015 and 17500273 (R.T.) from the Japanese Ministry of Education, Culture, Sports, Science and Technology.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Fig. 1
Neural responses to left and right tones in the initial assessment for spatial selectivity. (PDF 1538 kb)
Supplementary Fig. 2
The functional anatomy in the auditory thalamus. (PDF 3427 kb)
Rights and permissions
About this article
Cite this article
Komura, Y., Tamura, R., Uwano, T. et al. Auditory thalamus integrates visual inputs into behavioral gains. Nat Neurosci 8, 1203–1209 (2005). https://doi.org/10.1038/nn1528
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn1528
This article is cited by
-
Stronger responses in the visual cortex of sighted compared to blind individuals during auditory space representation
Scientific Reports (2019)
-
Multisensory integration in rodent tactile but not visual thalamus
Scientific Reports (2018)
-
Spatial localization of sound elicits early responses from occipital visual cortex in humans
Scientific Reports (2017)
-
Impact of language on functional connectivity for audiovisual speech integration
Scientific Reports (2016)
-
Prior physical exertion modulates allocentric distance perception: a demonstration of task-irrelevant cross-modal transfer
Experimental Brain Research (2016)