How are neurons with different modality preferences spatially distributed in multisensory cortical regions? Although spatial patterns of neurons have been well documented in the primary cortices (e.g., tonotopic maps in the auditory cortex, columnar structures in the visual cortex), data are lacking as to whether neurons in multimodal regions are organized in some way.
One functional magnetic resonance imaging (fMRI) study (Beauchamp et al., 2004) has suggested an orderly layout of neuron clusters sensitive to auditory and visual stimuli in the human superior temporal sulcus (STS), in which a putative multisensory area [the homolog of monkey superior temporal polysensory cortex—also known as TPO in the study by Seltzer and Pandya (1978)] (Fig. 1) is located. Consistent with this, an anatomical tracer injection study showed that afferent projections from cortical unisensory auditory and visual regions target nonoverlapping vertical modules within the upper bank of the STS (uSTS) (Seltzer et al., 1996). However, given the limited spatial resolution of fMRI and the lack of explicit functional information yielded by tracer injection techniques, the existence of functionally organized clusters in this multisensory region remains speculative without neurophysiological studies directly assessing spatial organizations of neuronal responses to sensory stimuli.
A simplified rendering of the anatomical parcellation of the STS based on the study by Seltzer and Pandya (1978). Unimodal regions are color-coded (green, auditory; blue, visual; yellow, somatosensory), and regions containing neurons that respond to more than one modality (multimodal regions) are colored in orange. The two transparent planes in the figure represent the approximate rostral and caudal boundaries of the recording area in the study by Dahl et al. (2009), which encompasses midportions of TPO. Left, A bottom-up view of the upper bank of the STS (regions shown: TAa, TPO-1, TPO-2, TPO-3, TPO-4, PGa) in which the multisensory nature of this region is clearly observed. Right, A top-down view of the lower bank of the STS (regions shown: PGa, IPa, TEa, OAa, TEm, Oa). STG, Superior temporal gyrus; MTG, middle temporal gyrus.
The recent study by Dahl et al. (2009) in The Journal of Neuroscience is the first successful attempt to use a stimulus-driven electrophysiological approach to address this question. Recordings of single-unit and multiunit activity within the midportion of the uSTS in rhesus monkeys in the presence of auditory, visual, or audiovisual stimuli, revealed that neurons responding to a single modality often were close to neurons having the same modality preference and to bimodal neurons. Stimulus exemplars from movies and/or recordings of conspecifics vocalizing, other animals making noises, or artificial dot motion patterns with corresponding pulsed broadband noise [Dahl et al. (2009), their Fig. 4A] were presented either in a single modality (visual/auditory alone) or in a synchronous bimodal pair. Units with a peak response amplitude significantly greater during stimulation than during baseline were selected for statistical analysis.
Among 532 sensory responsive units from two monkeys, some units responded preferentially to auditory (18.4%) or visual stimuli (28.4%), but most responded bimodally, either responding similarly to both modalities or most strongly to audiovisual stimuli (53.2%). Across neurons, responses to audiovisual stimuli were stronger than responses to unisensory auditory or visual stimuli [Dahl et al. (2009), their Fig. 1C,D], reflecting the multisensory nature of the uSTS.
The authors examined the spatial organization of neurons by determining the modality preferences of units recorded along the same or adjacent electrode penetrations systematically spaced (750 μm) on a recording grid [Dahl et al. (2009), their Fig. 2A]. Recording both auditory and visual unimodal units along a single penetration was significantly less likely in the actual data than in randomized data, in which all units were randomly assigned to individual penetrations; in contrast, penetrations with only bimodal units were significantly more likely in actual data than in randomized data [Dahl et al. (2009), their Fig. 2D].
To investigate whether this systematic organization held at a larger spatial scale (750 μm), the authors obtained an aggregate response for each penetration and observed a spatial pattern across penetrations [Dahl et al. (2009), their Fig. 2B]. Specifically, neighboring penetrations were significantly more likely to exhibit the same modality preference in actual data than in randomized data, suggesting that neuronal preferences were spatially organized.
The authors then pooled data across all units recorded on 2 × 2 neighboring penetrations (1.5 mm2), to compare their results with those from a previous fMRI report in which the data were obtained at a resolution of millimeters (Beauchamp et al., 2004). They found that these 1.5 mm2 blocks were more likely to have neighbors with the same modality preference than chance would predict. At this scale, units that were engaged in multisensory interactions (i.e., either supralinear or sublinear combinatory effects compared with the summation of unimodal units) were spatially clustered.
To more fully understand the spatial organization of neurons in the uSTS, it would be worthwhile to investigate a mapping that includes the third modality to which this area is sensitive (i.e., somatosensory stimuli). The uSTS in nonhuman primates can be parcellated into several distinct cytoarchitectonic zones (Fig. 1) based on architectonics (e.g., laminar differentiation, cell packing density) and connections (Seltzer and Pandya, 1978). The recordings in the midportion of the uSTS (C. Kayser, personal communication) were within the cytoarchitectonically defined TPO (more specifically, within TPO-2 and TPO-3) (Fig. 1). TPO receives input from auditory, visual, and somatosensory association regions (Seltzer and Pandya, 1989, 1994) and is potentially capable of bimodal or trimodal integration (Fig. 1) (Bruce et al. 1981). Therefore, any of the units recorded by Dahl et al. (2009), including those omitted from the analysis because of subthreshold responses, might have been responsive to somatosensory, somatosensory bimodal, and/or trimodal stimuli.
Examination of somatosensory as well as auditory and visual stimuli might also provide insight into why unisensory units tend to cluster with bimodal units [Dahl et al. (2009), their Fig. 2A]. The simplest explanation is that bimodal units incorporate the modality preference of nearby unisensory units. However, this interpretation is trivial given only two sensory modalities; with the addition of a third modality, one might observe a refuting situation in which bimodal units (e.g., audiovisual) cluster with noncorresponding unisensory units (e.g., somatosensory). Another possibility is that bimodal units, and the unimodal units they cluster with, are sensitive to a shared abstract stimulus dimension (i.e., emotionality or motion direction). A third possibility is that functional differences exist between bimodal units that cluster with one sensory modality over the other. For example, bimodal units with supralinear or sublinear interactions might cluster with visual units, whereas no-preference bimodal units might cluster with auditory units. However, these explanations are not mutually exclusive.
It would also be useful to know how spatial patterns of responsiveness in the uSTS generalize across subjects. In the study by Dahl et al. (2009), the spatial patterns presented in their Figure 2, B and C, from their two subjects are quite different. The differences in response profiles might be attributable to differences in the exact locations recorded in the two animals or might represent individual variability within this region. Studies similar to that by Dahl et al. (2009) but with a larger sample size would be useful to examine intersubject variability in the spatial organization of neural response profiles.
Another question that follows from the results of Dahl et al. (2009) is whether the neuronal spatial organization in the uSTS would also be observed in the thalamic/subcortical regions that send projections to the uSTS. As the authors briefly point out, the patchy organization of neurons in the uSTS may simply reflect a pattern of anatomical afferents. The neuronal organization of subcortical regions deserves consideration, given that the midportion of the uSTS receives strong projections from medial and central portions of the thalamic pulvinar nucleus (Yeterian and Pandya, 1989). The central portion of the pulvinar nucleus appears to be predominantly polymodal (Gattas et al. 1978) and is surrounded by regions that appear to be unimodal (Yeterian and Pandya, 1989). Furthermore, the pulvinar nucleus has response properties similar to multimodal regions of the uSTS and is connected with other cortical multisensory areas that themselves project to the uSTS (Yeterian and Pandya, 1989). Given the similarity in organization, modality preference, and response properties between the pulvinar nucleus and uSTS, it is plausible that the connections between these regions influence the topographical organization observed in the current study. Even if the spatial clustering of neurons in the uSTS appears to be a manifestation of afferent projections, there may still be functional significance to the organization within the uSTS (e.g., functionally distinct influences on regions to which neurons of the uSTS project).
The authors noted that spatial averaging biases responses toward a single modality preference and underestimates the contributions of multisensory units. This highlights one potential problem of conventional fMRI analyses in which the acquired spatial resolution (typically 2–3 mm3) combined with the application of spatial smoothing would result in an underestimate of brain regions involved in multimodal interactions, and may even lead to missing small clusters of unimodally responding neurons. Therefore, care should be taken when interpreting conventional fMRI findings in terms of the underlying functional properties of neurons in those voxels (Kayser et al., 2009). One analysis method that may reduce this problem is multivoxel pattern analysis (Norman et al., 2006), which takes advantage of pattern classification algorithms to establish mappings between spatially distributed brain activity and stimulus categories. To gain an in-depth understanding of multisensory interactions in the human brain, future studies should emphasize developing additional imaging methodologies capable of resolving responses at smaller spatial scales.
Together, Dahl et al. (2009) provide the first clear evidence that neurons in the midportion of the upper bank of the STS in rhesus monkeys are spatially organized according to their sensory modality preferences and that the organizational patterns are apparent at different spatial scales. Their results, in conjunction with what has been discussed here, might lead to a more complete spatial, and perhaps functional, profile of neuronal organization in high-level cortical fields.
Footnotes
-
We thank Dr. Christoph Kayser (corresponding author of the paper reviewed here) and Dr. Ingrid Johnsrude for their helpful comments on our manuscript.
-
Editor's Note: These short, critical reviews of recent papers in the Journal, written exclusively by graduate students or postdoctoral fellows, are intended to summarize the important findings of the paper and provide additional insight and commentary. For more information on the format and purpose of the Journal Club, please see http://www.jneurosci.org/misc/ifa_features.shtml.
- Correspondence should be addressed to Zane Z. Zheng, Centre for Neuroscience Studies, 62 Arch Street, Humphrey Hall, Kingston, ON K7L 3N6, Canada. 5zz2{at}queensu.ca