Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Auditory thalamus integrates visual inputs into behavioral gains

Nature Neuroscience volume 8, pages 1203–1209 (2005)Cite this article

Abstract

By binding multisensory signals, we get robust percepts and respond to our surroundings more correctly and quickly. How and where does the brain link cross-modal sensory information to produce such behavioral advantages? The classical role of sensory thalamus is to relay modality-specific information to the cortex. Here we find that, in the rat thalamus, visual cues influence auditory responses, which have two distinct components: an early phasic one followed by a late gradual buildup that peaks before reward. Although both bimodal presentation and reward value had similar effects on behavioral performance, the cross-modal effect on neural activity showed unique temporal dynamics: it affected the amplitude of the early component and starting level of the late component, whereas reward value affected only the slope of the late component. These results demonstrate that cross-modal cueing modulates gain in the sensory thalamus, potentially providing a priming influence on the choice of an optimal behavior.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Task design.
Figure 2: The cross-modal effect on neural activity in the auditory thalamus.
Figure 3: Bimodal modulation in extinction trials.
Figure 4: The effects of bimodal cues and reward values on the activity of neurons in the auditory thalamus.
Figure 5: Signal detection analysis on behavioral performance.
Figure 6: Correlation between neural responses and accuracy.
Figure 7: Correlation between neural responses and reaction time.

Similar content being viewed by others

References

  1. Bushara, K.O. et al. Neural correlates of cross-modal binding. Nat. Neurosci. 6, 190–195 (2003).

    Article  CAS  Google Scholar 

  2. Shams, L., Kamitani, Y., Thompson, S. & Shimojo, S. Sound alters visual evoked potentials in humans. Neuroreport 12, 3849–3852 (2001).

    Article  CAS  Google Scholar 

  3. Wallace, M.T., Ramachandran, R. & Stein, B.E. A revised view of sensory cortical parcellation. Proc. Natl. Acad. Sci. USA 101, 2167–2172 (2004).

    Article  CAS  Google Scholar 

  4. Sekiyama, K., Kanno, I., Miura, S. & Sugita, Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47, 277–287 (2003).

    Article  Google Scholar 

  5. Barth, D.S., Goldberg, N., Brett, B. & Di, S. The spatiotemporal organization of auditory, visual, and auditory-visual evoked potentials in rat cortex. Brain Res. 678, 177–190 (1995).

    Article  Google Scholar 

  6. Calvert, G.A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997).

    Article  CAS  Google Scholar 

  7. Macaluso, E., Frith, C.D. & Driver, J. Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208 (2000).

    Article  CAS  Google Scholar 

  8. Schroeder, C.E. et al. Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 50, 5–17 (2003).

    Article  Google Scholar 

  9. McGurk, H. & MacDonald, J. Hearing lips and seeing voices. Nature 264, 746–748 (1976).

    Article  CAS  Google Scholar 

  10. Driver, J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature 381, 66–68 (1996).

    Article  CAS  Google Scholar 

  11. Stein, B.E., Meredith, M.A., Huneycutt, W.S. & McDade, L. Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. J. Cogn. Neurosci. 1, 12–24 (1989).

    Article  CAS  Google Scholar 

  12. Wilkinson, L.K., Meredith, M.A. & Stein, B.E. The role of anterior ectosylvian cortex in cross-modality orientation and approach behavior. Exp. Brain Res. 112, 1–10 (1996).

    Article  CAS  Google Scholar 

  13. Jiang, W., Jiang, H. & Stein, B.E. Two corticotectal areas facilitate multisensory orientation behavior. J. Cogn. Neurosci. 14, 1240–1255 (2002).

    Article  Google Scholar 

  14. Glimcher, P.W. The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26, 133–179 (2003).

    Article  CAS  Google Scholar 

  15. Schultz, W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 1, 199–207 (2000).

    Article  CAS  Google Scholar 

  16. Komura, Y. et al. Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature 412, 546–549 (2001).

    Article  CAS  Google Scholar 

  17. Grier, J.B. Nonparametric indexes for sensitivity and bias: computing formulas. Psychol. Bull. 75, 424–429 (1971).

    Article  CAS  Google Scholar 

  18. Lovelace, C.T., Stein, B.E. & Wallace, M.T. An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection. Brain Res. Cogn. Brain Res. 17, 447–453 (2003).

    Article  Google Scholar 

  19. Winer, J.A., Kelly, J.B. & Larue, D.T. Neural architecture of the rat medial geniculate body. Hear. Res. 130, 19–41 (1999).

    Article  CAS  Google Scholar 

  20. Rauschecker, J.P., Tian, B., Pons, T. & Mishkin, M. Serial and parallel processing in rhesus monkey auditory cortex. J. Comp. Neurol. 382, 89–103 (1997).

    Article  CAS  Google Scholar 

  21. Kaas, J.H., Hackett, T.A. & Tramo, M.J. Auditory processing in primate cerebral cortex. Curr. Opin. Neurobiol. 9, 164–170 (1999).

    Article  CAS  Google Scholar 

  22. Hu, B., Senatorov, V. & Mooney, D. Lemniscal and non-lemniscal synaptic transmission in rat auditory thalamus. J. Physiol. (Lond.) 479, 217–231 (1994).

    Article  Google Scholar 

  23. Rauschecker, J.P. & Tian, B. Mechanisms and streams for processing of “what” and “where” in auditory cortex. Proc. Natl. Acad. Sci. USA 97, 11800–11806 (2000).

    Article  CAS  Google Scholar 

  24. von Melchner, L., Pallas, S.L. & Sur, M. Visual behaviour mediated by retinal projections directed to the auditory pathway. Nature 404, 871–876 (2000).

    Article  CAS  Google Scholar 

  25. Bushara, K.O., Grafman, J. & Hallett, M. Neural correlates of auditory-visual stimulus onset asynchrony detection. J. Neurosci. 21, 300–304 (2001).

    Article  CAS  Google Scholar 

  26. Brett-Green, B., Fifkova, E., Larue, D.T., Winer, J.A. & Barth, D.S. A multisensory zone in rat parietotemporal cortex: intra- and extracellular physiology and thalamocortical connections. J. Comp. Neurol. 460, 223–237 (2003).

    Article  Google Scholar 

  27. Barth, D.S. & MacDonald, K.D. Thalamic modulation of high-frequency oscillating potentials in auditory cortex. Nature 383, 78–81 (1996).

    Article  CAS  Google Scholar 

  28. Bhattacharya, J., Shams, L. & Shimojo, S. Sound-induced illusory flash perception: role of gamma band responses. Neuroreport 13, 1727–1730 (2002).

    Article  Google Scholar 

  29. Linke, R., De Lima, A.D., Schwegler, H. & Pape, H.C. Direct synaptic connections of axons from superior colliculus with identified thalamo-amygdaloid projection neurons in the rat: possible substrates of a subcortical visual pathway to the amygdala. J. Comp. Neurol. 403, 158–170 (1999).

    Article  CAS  Google Scholar 

  30. Stein, B.E. Neural mechanisms for synthesizing sensory information and producing adaptive behaviors. Exp. Brain Res. 123, 124–135 (1998).

    Article  CAS  Google Scholar 

  31. Stein, B.E., Jiang, W., Wallace, M.T. & Stanford, T.R. Nonvisual influences on visual-information processing in the superior colliculus. Prog. Brain Res. 134, 143–156 (2001).

    Article  CAS  Google Scholar 

  32. Weeks, R. et al. A positron emission tomographic study of auditory localization in the congenitally blind. J. Neurosci. 20, 2664–2672 (2000).

    Article  CAS  Google Scholar 

  33. Itaya, S.K. & Van Hoesen, G.W. Retinal innervation of the inferior colliculus in rat and monkey. Brain Res. 233, 45–52 (1982).

    Article  CAS  Google Scholar 

  34. Wallace, M.T., Meredith, M.A. & Stein, B.E. Multisensory integration in the superior colliculus of the alert cat. J. Neurophysiol. 80, 1006–1010 (1998).

    Article  CAS  Google Scholar 

  35. Hikosaka, K., Iwai, E., Saito, H. & Tanaka, K. Polysensory properties of neurons in the anterior bank of the caudal superior temporal sulcus of the macaque monkey. J. Neurophysiol. 60, 1615–1637 (1988).

    Article  CAS  Google Scholar 

  36. Bell, A.H., Corneil, B.D., Meredith, M.A. & Munoz, D.P. The influence of stimulus properties on multisensory processing in the awake primate superior colliculus. Can. J. Exp. Psychol. 55, 123–132 (2001).

    Article  CAS  Google Scholar 

  37. Frens, M.A. & Van Opstal, A.J. Visual-auditory interactions modulate saccade-related activity in monkey superior colliculus. Brain Res. Bull. 46, 211–224 (1998).

    Article  CAS  Google Scholar 

  38. Ernst, M.O. & Bulthoff, H.H. Merging the senses into a robust percept. Trends Cogn. Sci. 8, 162–169 (2004).

    Article  Google Scholar 

  39. Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1, 411–416 (1998).

    Article  CAS  Google Scholar 

  40. Romanski, L.M. et al. Dual streams of auditory afferents target multiple domains in the primate prefrontal cortex. Nat. Neurosci. 2, 1131–1136 (1999).

    Article  CAS  Google Scholar 

  41. Kobler, J.B., Isbey, S.F. & Casseday, J.H. Auditory pathways to the frontal cortex of the mustache bat, Pteronotus parnellii. Science 236, 824–826 (1987).

    Article  CAS  Google Scholar 

  42. Takada, M., Itoh, K., Yasui, Y., Sugimoto, T. & Mizuno, N. Topographical projections from the posterior thalamic regions to the striatum in the cat, with reference to possible tecto-thalamo-striatal connections. Exp. Brain Res. 60, 385–396 (1985).

    Article  CAS  Google Scholar 

  43. LeDoux, J.E., Farb, C.R. & Romanski, L.M. Overlapping projections to the amygdala and striatum from auditory processing areas of the thalamus and cortex. Neurosci. Lett. 134, 139–144 (1991).

    Article  CAS  Google Scholar 

  44. Shammah-Lagnado, S.J., Alheid, G.F. & Heimer, L. Efferent connections of the caudal part of the globus pallidus in the rat. J. Comp. Neurol. 376, 489–507 (1996).

    Article  CAS  Google Scholar 

  45. Dalley, J.W., Cardinal, R.N. & Robbins, T.W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neurosci. Biobehav. Rev. 28, 771–784 (2004).

    Article  CAS  Google Scholar 

  46. Brunia, C.H. & van Boxtel, G.J. Wait and see. Int. J. Psychophysiol. 43, 59–75 (2001).

    Article  CAS  Google Scholar 

  47. Rektor, I., Kanovsky, P., Bares, M., Louvel, J. & Lamarche, M. Event-related potentials, CNV, readiness potential, and movement accompanying potential recorded from posterior thalamus in human subjects. A SEEG study. Neurophysiol. Clin. 31, 253–261 (2001).

    Article  CAS  Google Scholar 

  48. Rafal, R.D. & Posner, M.I. Deficits in human visual spatial attention following thalamic lesions. Proc. Natl. Acad. Sci. USA 84, 7349–7353 (1987).

    Article  CAS  Google Scholar 

  49. Harvey, M., Olk, B., Muir, K. & Gilchrist, I.D. Manual responses and saccades in chronic and recovered hemispatial neglect: a study using visual search. Neuropsychologia 40, 705–717 (2002).

    Article  Google Scholar 

  50. Meredith, M.A. & Stein, B.E. Interactions among converging sensory inputs in the superior colliculus. Science 221, 389–391 (1983).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank F. Miles and M. Shidara for comments on the manuscript and S. Kitazawa and T. Kitamura for discussions. This work was partly supported by Special Coordination Funds for Promoting Science & Technology (Y.K.) and by Grants-in-Aid for Scientific Research nos. 17021015 and 17500273 (R.T.) from the Japanese Ministry of Education, Culture, Sports, Science and Technology.

Author information

Authors and Affiliations

  1. Neuroscience Research Institute, National Institute of Advanced Industrial Science and Technology, 1-1-1 Umezono, Tsukuba, 305-8568, Japan

    Yutaka Komura

  2. Department of Physiology, Faculty of Medicine, Toyama Medical and Pharmaceutical University, 2630 Sugitani, Toyama, 930-0194, Japan

    Ryoi Tamura & Teruko Uwano

  3. Core Research for Evolutionary Science and Technology, Japan Science and Technology Agency, 4-1-8 Honcho, Kawaguchi, 332-0012, Japan

    Ryoi Tamura, Teruko Uwano, Hisao Nishijo & Taketoshi Ono

  4. System Emotional Science, Graduate School of Medicine, Toyama Medical and Pharmaceutical University, 2630 Sugitani, Toyama, 930-0194, Japan

    Hisao Nishijo

  5. Molecular and Integrative Emotional Neuroscience, Graduate School of Medicine, Toyama Medical and Pharmaceutical University, 2630 Sugitani, Toyama, 930-0194, Japan

    Taketoshi Ono

Authors
  1. Yutaka Komura
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ryoi Tamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Teruko Uwano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hisao Nishijo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Taketoshi Ono
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taketoshi Ono.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Fig. 1

Neural responses to left and right tones in the initial assessment for spatial selectivity. (PDF 1538 kb)

Supplementary Fig. 2

The functional anatomy in the auditory thalamus. (PDF 3427 kb)

Supplementary Methods (PDF 92 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Komura, Y., Tamura, R., Uwano, T. et al. Auditory thalamus integrates visual inputs into behavioral gains. Nat Neurosci 8, 1203–1209 (2005). https://doi.org/10.1038/nn1528

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1528

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing