 |
|||||
|
|||||
|
|||||
|
|||||
|
|||||
The Journal of Neuroscience, May 18, 2005, 25(20):5004-5012; doi:10.1523/JNEUROSCI.0799-05.2005
Previous Article | Next Article
Behavioral/Systems/Cognitive
Multisensory Integration of Dynamic Faces and Voices in Rhesus Monkey Auditory Cortex
Asif A. Ghazanfar, Joost X. Maier, Kari L. Hoffman, andNikos K. Logothetis Max Planck Institute for Biological Cybernetics, 72076 Tuebingen, Germany
In the social world, multiple sensory channels are used concurrently to facilitate communication. Among human and nonhuman primates, faces and voices are the primary means of transmitting social signals (Adolphs, 2003; Ghazanfar and Santos, 2004). Primates recognize the correspondence between species-specific facial and vocal expressions (Massaro, 1998; Ghazanfar and Logothetis, 2003; Izumi and Kojima, 2004), and these visual and auditory channels can be integrated into unified percepts to enhance detection and discrimination. Where and how such communication signals are integrated at the neural level are poorly understood. In particular, it is unclear what role "unimodal" sensory areas, such as the auditory cortex, may play. We recorded local field potential activity, the signal that best correlates with human imaging and event-related potential signals, in both the core and lateral belt regions of the auditory cortex in awake behaving rhesus monkeys while they viewed vocalizing conspecifics. We demonstrate unequivocally that the primate auditory cortex integrates facial and vocal signals through enhancement and suppression of field potentials in both the core and lateral belt regions. The majority of these multisensory responses were specific to face/voice integration, and the lateral belt region shows a greater frequency of multisensory integration than the core region. These multisensory processes in the auditory cortex likely occur via reciprocal interactions with the superior temporal sulcus.
Key words: crossmodal; speech; vocalization; bimodal; superior temporal sulcus; temporal lobe
Received March 1, 2005; revised March 29, 2005; accepted April 12, 2005.
This article has been cited by other articles:
N. Kopco, I-F. Lin, B. G. Shinn-Cunningham, and J. M. Groh
Reference Frame of the Ventriloquism Aftereffect
J. Neurosci., November 4, 2009; 29(44): 13809 - 13814.
[Abstract] [Full Text] [PDF]
A. Falchier, C. E. Schroeder, T. A. Hackett, P. Lakatos, S. Nascimento-Silva, I. Ulbert, G. Karmos, and J. F. Smiley
Projection from Visual Areas V2 and Prostriata to Caudal Auditory Cortex in the Monkey
Cereb Cortex, October 29, 2009; (2009) bhp213v1.
[Abstract] [Full Text] [PDF]
L. H. Arnal, B. Morillon, C. A. Kell, and A.-L. Giraud
Dual Neural Routing of Visual Facilitation in Speech Processing
J. Neurosci., October 28, 2009; 29(43): 13445 - 13453.
[Abstract] [Full Text] [PDF]
S. A. Steckenfinger and A. A. Ghazanfar
Monkey visual behavior falls into the uncanny valley
PNAS, October 27, 2009; 106(43): 18362 - 18366.
[Abstract] [Full Text] [PDF]
L. Lemus, A. Hernandez, and R. Romo
Neural codes for perceptual discrimination of acoustic flutter in the primate auditory cortex
PNAS, June 9, 2009; 106(23): 9471 - 9476.
[Abstract] [Full Text] [PDF]
J. F. Bergan and E. I. Knudsen
Visual Modulation of Auditory Responses in the Owl Inferior Colliculus
J Neurophysiol, June 1, 2009; 101(6): 2924 - 2933.
[Abstract] [Full Text] [PDF]
S. Sadaghiani, J. X. Maier, and U. Noppeney
Natural, Metaphoric, and Linguistic Auditory Direction Signals Have Distinct Influences on Visual Motion Processing
J. Neurosci., May 20, 2009; 29(20): 6490 - 6499.
[Abstract] [Full Text] [PDF]
Y. Zahar, A. Reches, and Y. Gutfreund
Multisensory Enhancement in the Optic Tectum of the Barn Owl: Spike Count and Spike Timing
J Neurophysiol, May 1, 2009; 101(5): 2380 - 2394.
[Abstract] [Full Text] [PDF]
K. L. Campi, K. L. Bales, R. Grunewald, and L. Krubitzer
Connections of Auditory and Visual Cortex in the Prairie Vole (Microtus ochrogaster): Evidence for Multisensory Processing in Primary Sensory Areas
Cereb Cortex, April 24, 2009; (2009) bhp082v1.
[Abstract] [Full Text] [PDF]
A. J King
Visual influences on auditory spatial learning
Phil Trans R Soc B, February 12, 2009; 364(1515): 331 - 339.
[Abstract] [Full Text] [PDF]
C. Chandrasekaran and A. A. Ghazanfar
Different Neural Frequency Bands Integrate Faces and Voices Differently in the Superior Temporal Sulcus
J Neurophysiol, February 1, 2009; 101(2): 773 - 788.
[Abstract] [Full Text] [PDF]
R. M. Seyfarth and D. L. Cheney
Seeing who we hear and hearing who we see
PNAS, January 20, 2009; 106(3): 669 - 670.
[Full Text] [PDF]
J. Besle, C. Fischer, A. Bidet-Caulet, F. Lecaignard, O. Bertrand, and M.-H. Giard
Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans
J. Neurosci., December 24, 2008; 28(52): 14301 - 14310.
[Abstract] [Full Text] [PDF]
P. H. Rudebeck, D. M. Bannerman, and M. F. S. Rushworth
The contribution of distinct subregions of the ventromedial frontal cortex to emotion, social behavior, and decision making
Cogn Affect Behav Neurosci, December 1, 2008; 8(4): 485 - 497.
[Abstract] [PDF]
P. Yin, M. Mishkin, M. Sutter, and J. B. Fritz
Early Stages of Melody Processing: Stimulus-Sequence and Task-Dependent Neuronal Activity in Monkey Auditory Cortical Fields A1 and R
J Neurophysiol, December 1, 2008; 100(6): 3009 - 3029.
[Abstract] [Full Text] [PDF]
A. Fontanini and D. B. Katz
Behavioral States, Network States, and Sensory Response Variability
J Neurophysiol, September 1, 2008; 100(3): 1160 - 1168.
[Abstract] [Full Text] [PDF]
R.R Thompson, P.S Dickinson, J.D Rose, K.A Dakin, G.M Civiello, A Segerdahl, and R Bartlett
Pheromones enhance somatosensory processing in newt brains through a vasotocin-dependent mechanism
Proc R Soc B, July 22, 2008; 275(1643): 1685 - 1693.
[Abstract] [Full Text] [PDF]
C. Kayser, C. I. Petkov, and N. K. Logothetis
Visual Modulation of Neurons in Auditory Cortex
Cereb Cortex, July 1, 2008; 18(7): 1560 - 1574.
[Abstract] [Full Text] [PDF]
K. von Kriegstein, O. Dogan, M. Gruter, A.-L. Giraud, C. A. Kell, T. Gruter, A. Kleinschmidt, and S. J. Kiebel
From the Cover: Simulation of talking faces in the human brain improves auditory speech recognition
PNAS, May 6, 2008; 105(18): 6747 - 6752.
[Abstract] [Full Text] [PDF]
B. N. Carriere, D. W. Royal, and M. T. Wallace
Spatial Heterogeneity of Cortical Receptive Fields and Its Impact on Multisensory Interactions
J Neurophysiol, May 1, 2008; 99(5): 2357 - 2368.
[Abstract] [Full Text] [PDF]
A. A. Ghazanfar, C. Chandrasekaran, and N. K. Logothetis
Interactions between the Superior Temporal Sulcus and Auditory Cortex Mediate Dynamic Face/Voice Integration in Rhesus Monkeys
J. Neurosci., April 23, 2008; 28(17): 4457 - 4469.
[Abstract] [Full Text] [PDF]
U. Werner-Reiss and J. M. Groh
A Rate Code for Sound Azimuth in Monkey Auditory Cortex: Implications for Human Neuroimaging Studies
J. Neurosci., April 2, 2008; 28(14): 3747 - 3758.
[Abstract] [Full Text] [PDF]
R. Campbell
The processing of audio-visual speech: empirical and neural bases
Phil Trans R Soc B, March 12, 2008; 363(1493): 1001 - 1010.
[Abstract] [Full Text] [PDF]
U. Noppeney, O. Josephs, J. Hocking, C. J. Price, and K. J. Friston
The Effect of Prior Visual Information on Recognition of Speech and Sounds
Cereb Cortex, March 1, 2008; 18(3): 598 - 609.
[Abstract] [Full Text] [PDF]
K. Omata and K. Mogi
Fusion and combination in audio-visual integration
Proc R Soc A, February 8, 2008; 464(2090): 319 - 340.
[Abstract] [Full Text] [PDF]
B. E. Russ, A. L. Ackelson, A. E. Baker, and Y. E. Cohen
Coding of Auditory-Stimulus Identity in the Auditory Non-Spatial Processing Stream
J Neurophysiol, January 1, 2008; 99(1): 87 - 95.
[Abstract] [Full Text] [PDF]
K. K. Porter, R. R. Metzger, and J. M. Groh
Visual- and saccade-related signals in the primate inferior colliculus
PNAS, November 6, 2007; 104(45): 17855 - 17860.
[Abstract] [Full Text] [PDF]
H.-J. Lee, E. Truy, G. Mamou, D. Sappey-Marinier, and A.-L. Giraud
Visual speech circuits in profound acquired deafness: a possible role for latent multimodal connectivity
Brain, November 1, 2007; 130(11): 2929 - 2941.
[Abstract] [Full Text] [PDF]
V. Romei, M. M. Murray, L. B. Merabet, and G. Thut
Occipital Transcranial Magnetic Stimulation Has Opposing Effects on Visual and Auditory Stimulus Detection: Implications for Multisensory Interactions
J. Neurosci., October 24, 2007; 27(43): 11465 - 11472.
[Abstract] [Full Text] [PDF]
T. Noesselt, J. W. Rieger, M. A. Schoenfeld, M. Kanowski, H. Hinrichs, H.-J. Heinze, and J. Driver
Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices
J. Neurosci., October 17, 2007; 27(42): 11431 - 11441.
[Abstract] [Full Text] [PDF]
J. K. Bizley, F. R. Nodal, V. M. Bajo, I. Nelken, and A. J. King
Physiological and Anatomical Evidence for Multisensory Interactions in Auditory Cortex
Cereb Cortex, September 1, 2007; 17(9): 2172 - 2189.
[Abstract] [Full Text] [PDF]
L. M. Romanski
Representation and Integration of Auditory and Visual Stimuli in the Primate Ventral Lateral Prefrontal Cortex
Cereb Cortex, September 1, 2007; 17(suppl_1): i61 - i69.
[Abstract] [Full Text] [PDF]
G. Hein, O. Doehrmann, N. G. Muller, J. Kaiser, L. Muckli, and M. J. Naumer
Object Familiarity and Semantic Congruency Modulate Responses in Cortical Audiovisual Integration Areas
J. Neurosci., July 25, 2007; 27(30): 7881 - 7887.
[Abstract] [Full Text] [PDF]
R. Martuzzi, M. M. Murray, C. M. Michel, J.-P. Thiran, P. P. Maeder, S. Clarke, and R. A. Meuli
Multisensory Interactions within Human Primary Cortices Revealed by BOLD Dynamics
Cereb Cortex, July 1, 2007; 17(7): 1672 - 1679.
[Abstract] [Full Text] [PDF]
J. X. Maier and A. A. Ghazanfar
Looming Biases in Monkey Auditory Cortex
J. Neurosci., April 11, 2007; 27(15): 4093 - 4100.
[Abstract] [Full Text] [PDF]
C. Kayser, C. I. Petkov, M. Augath, and N. K. Logothetis
Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex
J. Neurosci., February 21, 2007; 27(8): 1824 - 1835.
[Abstract] [Full Text] [PDF]
M. Avillac, S. Ben Hamed, and J.-R. Duhamel
Multisensory Integration in the Ventral Intraparietal Area of the Macaque Monkey
J. Neurosci., February 21, 2007; 27(8): 1922 - 1932.
[Abstract] [Full Text] [PDF]
K. Kuraoka and K. Nakamura
Responses of Single Neurons in Monkey Amygdala to Facial and Vocal Emotions
J Neurophysiol, February 1, 2007; 97(2): 1379 - 1387.
[Abstract] [Full Text] [PDF]
T. Sugihara, M. D. Diltz, B. B. Averbeck, and L. M. Romanski
Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex
J. Neurosci., October 25, 2006; 26(43): 11138 - 11147.
[Abstract] [Full Text] [PDF]
M. M. Murray, C. Camen, S. L. Gonzalez Andino, P. Bovet, and S. Clarke
Rapid Brain Discrimination of Sounds of Objects
J. Neurosci., January 25, 2006; 26(4): 1293 - 1302.
[Abstract] [Full Text] [PDF]
|
|
|
|
 |
|