Elsevier

NeuroImage

Volume 57, Issue 2, 15 July 2011, Pages 624-633
NeuroImage

Syntactic and auditory spatial processing in the human temporal cortex: An MEG study

https://doi.org/10.1016/j.neuroimage.2011.04.034Get rights and content

Abstract

Processing syntax is believed to be a higher cognitive function involving cortical regions outside sensory cortices. In particular, previous studies revealed that early syntactic processes at around 100–200 ms affect brain activations in anterior regions of the superior temporal gyrus (STG), while independent studies showed that pure auditory perceptual processing is related to sensory cortex activations. However, syntax-related modulations of sensory cortices were reported recently, thereby adding diverging findings to the previous studies. The goal of the present magnetoencephalography study was to localize the cortical regions underlying early syntactic processes and those underlying perceptual processes using a within-subject design. Sentences varying the factors syntax (correct vs. incorrect) and auditory space (standard vs. change of interaural time difference (ITD)) were auditorily presented. Both syntactic and auditory spatial anomalies led to very early activations (40–90 ms) in the STG. Around 135 ms after violation onset, differential effects were observed for syntax and auditory space, with syntactically incorrect sentences leading to activations in the anterior STG, whereas ITD changes elicited activations more posterior in the STG. Furthermore, our observations strongly indicate that the anterior and the posterior STG are activated simultaneously when a double violation is encountered. Thus, the present findings provide evidence of a dissociation of speech-related processes in the anterior STG and the processing of auditory spatial information in the posterior STG, compatible with the view of different processing streams in the temporal cortex.

Research highlights

► Early syntactic processing activates the anterior superior temporal gyrus (aSTG) ► Auditory spatial processing activates the posterior superior temporal gyrus (pSTG) ► Simultaneous syntactic and perceptual processing by activating the aSTG and pSTG ► Syntactic and auditory spatial effects within the first 200 ms after the word onset

Introduction

Previous studies investigating early syntactic processes in auditory sentence comprehension reported an enhanced negativity around 100–200 ms, labeled early left anterior negativity (ELAN), that was elicited by sentences containing a syntactic word category violation (Friederici et al., 1993, Hahne and Friederici, 1999, Hahne and Friederici, 2002). It has been a point of discussion whether prosodic effects contribute to such early syntax-related responses. However, a recent finding provides contradicting evidence by showing that a change in the prosodic contour cannot account for the ELAN effect, thereby highlighting the interpretation of the ELAN as a marker for initial syntactic processes (B. Herrmann et al., 2011).

Another question that has been raised in the context of the interpretation of the ELAN is whether this component belongs to the family of the mismatch negativity (MMN), an early negativity associated with rule violations in auditory perception (Näätänen et al., 1978). The MMN is elicited by an infrequently presented auditory event among a series of frequently repeated auditory events reflecting a memory comparison process (Schröger, 2005). The MMN has its maximum at around 100–200 ms following the onset of the infrequent event and has been reported for frequency (Näätänen et al., 1978, Shalgi and Deouell, 2007), duration (Jemel et al., 2002) or spatial deviations (Schröger, 1996, Nager et al., 2003), for example.

The relation between processing syntactic and auditory perceptual information was first investigated by Hahne et al. (2002) who studied early syntactic and auditory spatial processing in combination using electroencephalography (EEG). In this study, infrequent spatial deviations within a spoken sentence elicited a MMN, whereas sentences containing a syntactic word category violation elicited an ELAN. Sentences including an infrequent spatial deviation as well as a syntactic violation led to a larger negativity around 125–175 ms than the single violations, although the amplitude was less than a complete addition of the two single violations. The results were taken as an indicator that early syntactic and physical acoustic information can be processed in parallel within the first 200 ms (Hahne et al., 2002).

However, it remains an open question what parallel processing in this context means as only slight differences in the EEG scalp distribution of the syntactic and auditory spatial violation effects were reported in the study of Hahne et al. (2002). The present magnetoencephalography (MEG) study aims to shed more light on this issue. It might be the case that both types of single deviations recruit the same brain regions and that the double deviation thus leads to stronger activation in these very same regions. On the other hand, different brain regions might be involved in processing syntactic versus spatial deviations and that these regions are activated simultaneously when a combined violation is encountered. According to dual stream auditory processing models (Rauschecker and Scott, 2009) spatial information is processed in the dorsal stream involving the posterior portion of the superior temporal cortex, while speech (intelligibility of speech) processes recruit regions anterior to Heschl's gyrus in the ventral stream. It could, therefore, be hypothesized that partly different regions are activated when speech-related and auditory spatial features are processed in parallel.

Previous studies on the localizations of the neural mechanisms that underlie early syntactic processes, have localized the sources of the magnetic ELAN (ELANm) to the superior temporal cortex (Groß et al., 1998, Knösche et al., 1999), and more specifically to the anterior parts of the superior temporal gyrus (aSTG) and the inferior frontal cortex (IFC; Friederici et al., 2000). More recently, a so called “sensory hypothesis” for early syntactic effects has been introduced in the visual domain (Dikker et al., 2009). This hypothesis is based on the observation that early sensory cortex activations were affected at around 100–200 ms when participants encountered a syntactic word category violation (Dikker et al., 2009, Dikker et al., 2010; B. Herrmann et al., 2009). It has been suggested that these early sensory effects rely on form properties associated with the syntactic category, e.g. overt category marking by an affix (Dikker et al., 2010). For the processing of syntactic violations in an auditory oddball paradigm, B. Herrmann et al. (2009) observed modulations of the primary auditory cortex (AC) and the superior temporal sulcus, thus, suggesting activations in regions different from previous localizations showing aSTG and IFC activations in processing word category violations (Friederici et al., 2000). One explanation for these diverging findings that has been proposed relates to the methodological approaches applied (Dikker et al., 2009). In the study conducted by Friederici et al. (2000), for instance, dipole modeling was constrained by functional magnetic resonance imaging (fMRI) results from a previous sentence processing study (Meyer et al., 2000). Dikker et al. (2009) argued that this constraint might not be valid, as fMRI lacks the temporal resolution to derive solid assumptions about an early stage in sentence processing. In the study by B. Herrmann et al. (2009), on the other hand, two-word utterances were presented in an auditory oddball paradigm in which syntactic processing is accompanied by an acoustic change (Shtyrov and Pulvermüller, 2007), thus, possibly biasing the source localization towards primary regions.

The underlying neural sources of the MMN have mainly been localized to auditory sensory cortex regions (Giard et al., 1990, Alho et al., 1998, Maess et al., 2007). Particularly relevant for the current study, the MMN and its magnetic counterpart (MMF, mismatch field) has been found sensitive to infrequent changes of interaural time and interaural level differences (ITD, ILD; Schröger, 1996, Schröger and Wolff, 1996, Kaiser et al., 2000, Nager et al., 2003). The ITD and ILD are two important auditory cues which allow spatial sound localization (Middlebrooks and Green, 1991). The MMN to changes in ITD/ILD has been shown to modulate brain activations in the posterior STG/AC (Kaiser et al., 2000, Tata and Ward, 2005, Sonnadara et al., 2006, Deouell et al., 2006). Sometimes, an additional neural generator localized to the right IFC has been reported to underlie the MMN mechanism (Giard et al., 1990, Jemel et al., 2002, Shalgi and Deouell, 2007).

In addition to the ELAN effect in the 100–200 ms time window, previous studies were able to disentangle the “early syntax effect” into different sub-stages, observing an additional very early syntax-related effect that modulated the M50 component (C. S. Herrmann et al., 2000; B. Herrmann et al., 2009, Herrmann et al., 2011). Furthermore, the detection of simple rule violations in an auditory oddball paradigm has not only been shown to elicit the MMN, but also to modulate the brain's activity very early, starting at around 30 ms (Boutros and Belger, 1999, Ermutlu et al., 2005, Slabu et al., 2010, Grimm et al., 2011).

In the present study, anatomically constrained magnetoencephalography (MEG) was used to further investigate the early syntactic and auditory perceptual parallel processing effect in the brain. On this account, auditory sentence materials were presented that varied in syntax (syntactically correct vs. syntactically incorrect) and auditory space (standard vs. infrequent ITD change). For the source analysis, a distributed source model was used without any priors regarding the location of the underlying cortical regions modulated by syntax and auditory space. Our main goals were to examine the neural mechanisms that underlie the processing of (1) syntactic violations, (2) auditory spatial violations (ITD change), and (3) double violations, i.e. in syntax and auditory space.

  • (1)

    The localization of the neural responses elicited by syntactic violations allowed us to test previous localizations of the ELANm (Friederici et al., 2000), and to examine whether auditory sensory cortices are sensitive to syntactic manipulations (Dikker et al., 2009). The former study predicts activations in the anterior STG with additional weaker frontal activations, whereas the latter view predicts the ELANm to be localized in auditory sensory cortices.

  • (2)

    We sought to localize the neural sources of the MMF elicited by infrequent ITD changes within naturally spoken sentences in order to have a condition which reflects auditory perceptual rule processing (Schröger, 1996, Schröger, 2005). We expected the posterior STG/AC to be sensitive to auditory spatial deviations (Kaiser et al., 2000, Deouell et al., 2006). Based on the dual pathways in the auditory system (Rauschecker and Scott, 2009), the neural sources of the ELANm and MMF were expected to differ in location.

  • (3)

    By localizing the neural responses to sentences including a double violation, we aimed to investigate how processing a syntactic violation and an auditory spatial violation in parallel is accomplished by different regions in the temporal cortex (Rauschecker and Scott, 2009). We expected the brain regions involved in processing the single syntactic and those involved in processing the auditory spatial violations to be activated simultaneously for sentences containing both violations.

Apart from the effects in the 100–200 ms time window (i.e., the ELANm and MMF), we intended to further elucidate on the very early syntax and simple rule violation effects that precede the ELANm and MMF, and ask whether parallel processing can be observed already at this processing stage.

Section snippets

Participants

Twenty-four healthy, native German-speaking adults (11 female, mean age = 25.3 years, standard error of the mean (SEM) = 1) participated in the MEG study. They were all right-handed as measured by the Edinburgh Handedness Inventory (Oldfield, 1971). The 20th percentile of the laterality quotient was 100 (range: 83–100). All participants gave written consent prior to testing and were paid seven Euros per hour. They had no known hearing deficit or neurological diseases in their history.

Stimulus material

The material

Behavioral task

The overall mean error rate was 2.9% (SEM = 0.5). Statistical comparisons did not reveal any differences between conditions.

Source localizationactivation strength

For each time window, the grand average activations of the violation conditions are depicted in Fig. 2. The corresponding STG activation time courses as well as a summary of the statistical comparisons for the two time windows are displayed in Fig. 3.

In the very early time window from 40 to 90 ms, the rmANOVA revealed a main effect of Syntax (F(1,23) = 33.82, P < 0.001, η²G = 

Discussion

The goal of the present MEG study was to investigate the underlying neural mechanisms of the parallel processing effect observed for early syntax-related and auditory perceptual-related responses, and to test previous localizations of early responses to syntactically incorrect sentences. The present data replicate earlier findings (Friederici et al., 1993, Friederici et al., 2000, Schröger, 1996, Hahne and Friederici, 1999, Hahne and Friederici, 2002, Hahne et al., 2002, Nager et al., 2003), by

Conclusions

The present MEG study revealed a number of similarities and differences in processing syntactic and auditory spatial information. Approximately 60 ms after stimulus onset, processing of syntactic and auditory spatial deviations led to modulations in the STG, showing that both deviations are processed in parallel when encountered in combination. Following these very early effects, clear evidence of a dissociation of speech-related processes and auditory spatial processes was observed in the

Acknowledgments

We thank Yvonne Wolff for carefully acquiring the data. BH was supported by the Deutsche Forschungsgemeinschaft, graduate program “Function of Attention in Cognition” at the University of Leipzig. We thank three anonymous reviewers for their very helpful comments.

References (74)

  • B. Herrmann et al.

    Localization of the syntactic mismatch negativity in the temporal cortex: an MEG study

    NeuroImage

    (2009)
  • B. Herrmann et al.

    Violation of syntax and prosody — disentangling their contributions to the early left anterior negativity (ELAN)

    Neuroscience Letters

    (2011)
  • M. Huotilainen et al.

    Combined mapping of human auditory EEG and MEG responses

    Electroencephalography and Clinical Neurophysiology

    (1998)
  • S.A. Kotz et al.

    Differentiation of syntactic processes in the left and right anterior temporal lobe: event-related brain potential evidence from lesion patients

    Brain and Language

    (2003)
  • M. Kubota et al.

    Magnetoencephalography detection of early syntactic processing in humans: comparison between L1 speakers and L2 learners of English

    Neuroscience Letters

    (2003)
  • B. Maess et al.

    Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing

    Brain Research

    (2006)
  • B. Maess et al.

    Localizing pre-attentive auditory memory-based comparison: magnetic mismatch negativity to pitch change

    NeuroImage

    (2007)
  • J.P. Mäkelä et al.

    Whole-head mapping of middle-latency auditory evoked magnetic fields

    Electroencephalography and Clinical Neurophysiology

    (1994)
  • M. Meyer et al.

    Neurocognition of auditory sentence comprehension: event related fMRI reveals sensitivity to syntactic violations and task demands

    Cognitive Brain Research

    (2000)
  • R. Näätänen et al.

    Early selective-attention effect on evoked potential reinterpreted

    Acta Psychologica

    (1978)
  • W. Nager et al.

    Tracking of multiple sound sources defined by interaural time differences: brain potential evidence in humans

    Neuroscience Letters

    (2003)
  • R.C. Oldfield

    The assessment and analysis of handedness: the Edinburgh Inventory

    Neuropsychologia

    (1971)
  • F. Pulvermüller et al.

    Automatic processing of grammar in the human brain as revealed by the mismatch negativity

    NeuroImage

    (2003)
  • N. Richter et al.

    Hemispheric specialization during discrimination of sound sources reflected by MMN

    Neuropsychologia

    (2009)
  • E. Schröger

    Interaural time and level differences: integrated or separated processing?

    Hearing Research

    (1996)
  • S. Shalgi et al.

    Direct evidence for differential roles of temporal and frontal components of auditory change detection

    Neuropsychologia

    (2007)
  • R.R. Sonnadara et al.

    Effects of spatial separation and stimulus probability on the event-related potentials elicited by occasional changes in sound location

    Brain Research

    (2006)
  • M. Tervaniemi et al.

    Functional specialization of the human auditory cortex in processing phonetic and musical sounds: a magnetoencephalographic (MEG) study

    NeuroImage

    (1999)
  • P.S. Ahlfors et al.

    Sensitivity of MEG and EEG to source orientation

    Brain Topography

    (2010)
  • J. Ahveninen et al.

    Task-modulated “what” and “where” pathways in human auditory cortex

    Proceedings of the National Academy of Sciences

    (2006)
  • K. Alho et al.

    Processing of novel sounds and frequency changes in the human auditory cortex: magnetoencephalographic recordings

    Psychophysiology

    (1998)
  • R. Bakeman

    Recommended effect size statistics for repeated measures designs

    Behavior Research Methods

    (2005)
  • Y. Benjamini et al.

    Controlling the false discovery rate: a practical and powerful approach to multiple testing

    Journal of the Royal Statistical Society: Series B (Statistical Methodology)

    (1995)
  • I. Bornkessel et al.

    The extended argument dependency model: a neurocognitive approach to sentence comprehension across languages

    Psychological Review

    (2006)
  • L.Y. Deouell et al.

    Spatial location is accurately tracked by human auditory sensory memory: evidence from the mismatch negativity

    The European Journal of Neuroscience

    (2006)
  • S. Dikker et al.

    Early occipital sensitivity to syntactic category is based on form typicality

    Psychological Science

    (2010)
  • B. Fischl et al.

    High-resolution intersubject averaging and a coordinate system for the cortical surface

    Human Brain Mapping

    (1999)
  • Cited by (29)

    • Acupoint-brain (acubrain) mapping: Common and distinct cortical language regions activated by focused ultrasound stimulation on two language-relevant acupoints

      2021, Brain and Language
      Citation Excerpt :

      The within-modality neuromodulatory effects of acupoints observed in our FUS study may implicate different types of language processing. Previous studies have identified specific brain regions engaged in different types of language processing, for example, bilateral superior temporal gyri during prelexical phonemic speech perception (Leff et al., 2009), BA45 and BA47 during semantic processing (Thompson-Schill, D’Esposito, Aguirre, & Farah, 1997; Wagner, Paré-Blagoev, Clark, & Poldrack, 2001), dorsal BA44 and anterior portion of superior temporal gyrus during syntactic processing (Friederici, 2018; Herrmann, Maess, Hahne, Schröger, & Friederici, 2011), and bilateral superior temporal sulci during sentence comprehension (Kroczek, Gunter, Rysop, Friederici, & Haetwigsen, 2019). The specific brain regions activated by FUS on different acupoints may be related to different types of language processing but our tasks were unable to differentiate between them.

    • Language and action in Broca's area: Computational differentiation and cortical segregation

      2021, Brain and Cognition
      Citation Excerpt :

      The findings replicated the early error-detection component found in Maffongelli et al. (2015), and they further showed a facilitatory effect reflected in faster RTs reported for the error detection task. So far, the ELAN component for language has been localized in the temporal cortex and in the inferior frontal cortex or in the temporal cortex only, using MEG and dipole source localization (Groß et al., 1998; Herrmann, Maess, Hahne, Schröger, & Friederici, 2011; Herrmann, Maess, Hasting, & Friederici, 2009; Knösche, Maeß, & Friederici, 1999). On the other hand, the localization of the early component reported for action processing—recorded from a wide centro-frontal region of interest—still awaits further investigation.

    View all citing articles on Scopus
    View full text