The level of audiovisual print–speech integration deficits in dyslexia

Highlights

• Investigation of the nature of audiovisual deficits in dyslexia.

• Sample of dyslexic adolescents is unprecedented.

• Multimodal data acquisition, including EEG, fMRI, and behavioural data.

• Impaired phonetic but intact basic audiovisual integration in dyslexia.

• Deficits are most pronounced in a naturalistic, word-like audiovisual context.

Abstract

The classical phonological deficit account of dyslexia is increasingly linked to impairments in grapho-phonological conversion, and to dysfunctions in superior temporal regions associated with audiovisual integration. The present study investigates mechanisms of audiovisual integration in typical and impaired readers at the critical developmental stage of adolescence.

Congruent and incongruent audiovisual as well as unimodal (visual only and auditory only) material was presented. Audiovisual presentations were single letters and three-letter (consonant–vowel–consonant) stimuli accompanied by matching or mismatching speech sounds. Three-letter stimuli exhibited fast phonetic transitions as in real-life language processing and reading. Congruency effects, i.e. different brain responses to congruent and incongruent stimuli were taken as an indicator of audiovisual integration at a phonetic level (grapho-phonological conversion). Comparisons of unimodal and audiovisual stimuli revealed basic, more sensory aspects of audiovisual integration. By means of these two criteria of audiovisual integration, the generalizability of audiovisual deficits in dyslexia was tested. Moreover, it was expected that the more naturalistic three-letter stimuli are superior to single letters in revealing group differences. Electrophysiological and hemodynamic (EEG and fMRI) data were acquired simultaneously in a simple target detection task. Applying the same statistical models to event-related EEG potentials and fMRI responses allowed comparing the effects detected by the two techniques at a descriptive level.

Group differences in congruency effects (congruent against incongruent) were observed in regions involved in grapho-phonological processing, including the left inferior frontal and angular gyri and the inferotemporal cortex. Importantly, such differences also emerged in superior temporal key regions. Three-letter stimuli revealed stronger group differences than single letters. No significant differences in basic measures of audiovisual integration emerged. Convergence of hemodynamic and electrophysiological signals appeared to be limited and mainly occurred for highly significant and large effects in visual cortices.

The findings suggest efficient superior temporal tuning to audiovisual congruency in controls. In impaired readers, however, grapho-phonological conversion is effortful and inefficient, although basic audiovisual mechanisms seem intact. This unprecedented demonstration of audiovisual deficits in adolescent dyslexics provides critical evidence that the phonological deficit might be explained by impaired audiovisual integration at a phonetic level, especially for naturalistic and word-like stimulation.

Introduction

Developmental Dyslexia (DD) is a learning disability of neurobiological origin with substantial genetic risk (Pennington and Olson, 2008, Schulte-Körne et al., 2006). It is characterized by specific impairments in the acquisition of efficient reading and emerges despite normal intelligence, no obvious sensory deficits, and adequate instruction (Lyon et al., 2003, WHO, 1992). Dyslexia is one of the most widespread developmental disorders, affecting around 5% of school-aged children in German speaking countries (Schulte-Körne, 2010, Schulte-Körne and Remschmidt, 2003). The most commonly accepted cause of DD is thought to be a phonological processing deficit (Bradley and Bryant, 1978, Goswami, 2000, Ramus, 2003, Shaywitz and Shaywitz, 2005, Snowling, 2000, Tree, 2008, Vellutino et al., 2004). It has been suggested that this deficit is characterized by impairments in converting print (graphemes) into corresponding sounds (phonemes; Snowling, 1980). This hypothesis is appealing, considering that such conversions are obviously required in order to acquire efficient reading (Bradley and Bryant, 1983, Ehri, 2005). It also receives strong support from intervention studies in dyslexic or pre-school children that demonstrate improvements in reading skills through audiovisual (AV) training programmes (Brem et al., 2010, Kujala et al., 2001, Lovio et al., 2012, Törmänen and Takala, 2009). Given the importance of AV conversion for reading, it is surprising that neurobiological origins of DD have until lately mainly been investigated by means of unimodal paradigms, both in the fMRI and EEG literature. Pekkola et al. (2006) were the first to use bimodal AV stimulations in order to probe phonological deficits in dyslexic adults. They found increased activation to incongruent stimulation in motor speech regions (Broca, left premotor cortex), which was more pronounced in dyslexic compared to typical adult readers. This was interpreted as increased use of subvocal motor-articulatory strategies during AV speech processing.

More recently, reduced congruency effects in impaired compared to nonimpaired readers were reported for superior temporal sulci (STS) and auditory cortices (planum temporale, PT; Blau et al., 2010, Blau et al., 2009). Specifically, stronger activation to congruent than incongruent letter–speech sound pairings was observed in these regions for typically reading children (Blau et al., 2010) and adults (Blau et al., 2009) but not for their dyslexic peers. STS regions have repeatedly been associated with multisensory integration and it has been argued that a distinction between congruent and incongruent pairings can only emerge after the unisensory inputs have been integrated successfully (Blomert, 2011, van Atteveldt et al., 2007). Thus, Blau et al., 2009, Blau et al., 2010 reasoned that letter–sound integration is an emergent property of learning to read which develops inadequately in dyslexic readers, presumably as a result of lacking specialization at the neuroanatomical level. This has eventually been interpreted as a specific crossmodal binding deficit and as an impairment in the automated formation of unique grapho-phonological objects (Blomert, 2011). Note that Pekkola et al. (2006) had observed stronger activation to incongruent rather than congruent stimulation in the left STS of both groups. This difference to Blau et al., 2009, Blau et al., 2010 might be explained by attentional factors given that participants in Pekkola et al. (2006) but not in Blau et al., 2009, Blau et al., 2010 had to actively monitor congruency state (see van Atteveldt et al., 2007 for active versus passive processing). The significance of superior temporal regions for dyslexia is also supported by observations of reduced grey matter volume (Richlan et al., 2012, Welcome et al., 2011).

According to a range of other criteria, the STS and adjacent gyral regions have repeatedly been related to multisensory integration: The “super-additivity” criterion, for instance, requires the response to crossmodally congruent stimuli to be stronger than the summed unisensory responses. In addition, the response to incongruent multisensory stimuli should be sub-additive compared to the summed unisensory stimulation, as has been shown for single cells of the superior colliculi in mammals (Kadunce et al., 1997, Stein, 1998). These criteria are based on the general concept that sensory brain responses to stimulus elements that are processed independently should be additive. Accordingly, violations of additivity can indicate the presence of multisensory interactions, or of (non-additive) attentional or cognitive processes. Calvert, Campbell, and Brammer (2000) reported that BOLD signals in the ventral part of the human left STS also fulfilled both super- and sub-additivity criteria. Subsequent fMRI studies, however, could not replicate this rather strict conjunction of criteria (Hocking & Price, 2008, give an overview). Considering that the BOLD signal originates from large clusters of neurons containing not only multisensory integration cells, subsequent studies accepted more liberal criteria of AV integration for fMRI. van Atteveldt, Formisano, Goebel, and Blomert (2004) found super-additivity (without corresponding sub-additivity) in bilateral PT and Heschl’s sulci, but nowhere in the STS. However, the STS of both hemispheres showed stronger responses to congruent stimulation than to the maximum of the unisensory responses. This “response enhancement” (Beauchamp, 2005) was met in several studies (Beauchamp et al., 2004, van Atteveldt et al., 2007). Electrophysiological studies have also made use of the super-additivity criterion and supported the role of the STS in AV integration (e.g., Besle et al., 2004, Klucharev et al., 2003, Raij et al., 2000). Interestingly, all of these electrophysiological studies found sub- rather than super-additive effects for congruent AV stimulation.

In contrast to effects of congruency mode, criteria on the interplay of uni- and multimodal responses do not require grapho-phonological knowledge or conversion when probing AV integration. Comparisons of phonetically matching versus conflicting conditions therefore probe phonetic features of AV integration (Hocking and Price, 2008, Klucharev et al., 2003, Ojanen et al., 2005), whereas super-additivity effects probe general features of AV integration, including more basic and sensory aspects because phonological processing is no premise for the super-additivity criterion (e.g., Klucharev et al., 2003).

As mentioned above, reduced congruency effects have been reported for dyslexia (Blau et al., 2010, Blau et al., 2009). However, criteria of super-additivity have never been tested in dyslexia, although they could clarify whether reading impairments originate from a more general AV deficit. The present study therefore includes both congruency as well as super-additivity approaches of testing AV integration. Because previous studies had found reduced AV integration in DD during passive (Blau et al., 2010, Blau et al., 2009) but not active congruency matching (Pekkola et al., 2006), the design of Blau et al. (2010) was adopted to simulate AV integration during reading. Three further important aspects go beyond previous studies: First, by using consonant–vowel–consonant sequences (CVCs) in addition to single letter stimuli, we intended to go one step further towards realistic reading. CVCs are more speech-like than mere letters given their rapid acoustic changes (formant transitions) that are an immanent property of natural speech. A recent study used uni- and bimodal word and pseudoword stimuli and found reduced overall activation in the right STS of dyslexic adults, indicating less proficient engagement of circuits involved in AV processing (Kast, Bezzola, Jäncke, & Meyer, 2011). However, no contrasts to specifically test for AV integration were computed. The presently used CVCs were all without semantic content in order to avoid the engagement of confounding higher-order psycholinguistic processing. Of central interest was whether integrative effects behave differently in the CVC compared to the single letter condition. Second, participants were at an adolescent age by the end of compulsory schooling, i.e. at a critical developmental stage where reading proficiency is still trained at a high level before literacy experiences diverge due to different professional tracks. Audiovisual deficits have so far not been investigated in dyslexic adolescents. Third, EEG was recorded simultaneously with fMRI data in order to investigate temporal properties of AV integration and of group differences. Previous electrophysiological studies have confirmed AV integration deficits in dyslexic children (Froyen et al., 2011, Widmann et al., 2012) and adults (Mittag, Thesleff, Laasonen, & Kujala, 2012). Widmann et al. (2012), for instance, probed congruency effects in an oddball paradigm. Congruent symbol–sound pairings were interspersed with incongruent pairings. A component of the event-related potential (ERP) that is associated with the orienting response to rare events was significantly less pronounced in dyslexic compared to control subjects. Considering the nonverbal stimulus material, the authors concluded that AV deficits could be more fundamental in nature, not restricted to impairments in grapho-phonological integration. The present study therefore aimed at clarifying the level of AV deficits by including and comparing the two approaches to test AV integration: super-additivity on the one hand and congruency effects on the other. Moreover, it was expected that CVC stimuli are more sensitive in revealing group differences due to their higher ecological validity with respect to reading.