Article Figures & Data
Figures
- Figure 1.
Experimental design. The experimental design involved eight conditions: two sensory modalities, tactile and visual; two types of stimuli (dimensions), temporal and spatial; and two levels of task versus no-task (i.e., doing a task vs passive observation).
- Figure 2.
Experimental design and behavioral results. A, Experimental tasks. Temporal sequences: series of six flashes (white circles) presented on the screen, or a series of six tactile stimuli presented on four fingers of both hands for longer or shorter periods of time (50 or 200 ms with 100-ms blank interval between). The temporal sequences control condition involved presentation of a temporally regular series of stimuli (visual or tactile). Each stimulus was presented for the same duration (125 ms with a 100-ms blank interval between). B, Behavioral results. Performance in the fMRI (the accuracy of the same/different decision in the experimental task). Behavioral results were all significantly above chance level. There were significant differences between groups (F = 12.57, p < 0.001), but there were no significant differences between modalities or between types of task (p > 0.05). We found a significant effect of interaction of dimension and modality (F(1,142) = 10.02, p < 0.01), and a significant effect of interaction between group and dimension (F(1,142) = 11.53, p < 0.01); *p < 0.05, **p < 0.01, ***p < 0.001. Error bars represent SEM.
- Figure 3.
A, Visual and tactile processing lead to auditory cortex recruitment in the deaf and hard of hearing but not in the hearing. The effect of group (deaf/hard of hearing ≥ hearing) peaks in the right auditory cortex MNI = 60, −20, 1, t = 6.11. B, Interaction group × dimension in the right superior and middle temporal gyrus (MNI = 66, −32, −12, t = 6.32).
- Figure 4.
A, Task-specific effect for temporal stimuli (temporal sequences task > temporal sequences no-task) for both modalities is reflected in an activation within the right auditory cortex MNI = 58, −22, −4, t = 5.43). B, Visual and tactile task activation peaks in a very similar region within the right auditory cortex. The overlap (yellow) between visual (red) and tactile (green) temporal sequences (for task–no-task condition) contains 87% of the tactile activation cluster. Temporal sequences and tactile temporal sequences (relative to no-task condition, conjunction) peaking in the superior temporal gyrus (MNI = 52, −18, −6, t = 3.85). Visually induced activation is more spread out across the auditory cortex than tactile activation. C, Comparison between temporal and spatial sequences discrimination task (relative to no-task) in both modalities revealed an activation cluster within right auditory cortex in deaf and hard of hearing subjects (temporal sequences > spatial sequences × task > no task) peak MNI = 60, −22, 4, t = 4.18. SVC analysis, threshold: p < 0.05, voxel-wise, FWE. All effects (A–C) were also equally significant as corrected within an alternative, anatomically defined ROI: the right superior temporal gyrus mask (neuromorphometrics); p < 0.005 voxel-wise, p < 0.05 cluster-wise, FWE corrected.
- Figure 5.
Results of the functional ROI analyses. The ROI was defined around the peak of activation (as revealed in the previous study) for the contrast: the visual rhythms condition versus the visual control condition. A, ROI analysis revealed a significant task-specific activation for temporal sequences versus temporal control condition in both the visual and tactile modalities. This effect is specific for the temporal task but not for the spatial task. The analysis revealed a significant effect of interaction: temporal sequences relative to spatial sequences (task vs no-task). B, No significant activation within the ROI in the hearing subjects. Significance: *p < 0.05, **p < 0.01, ***p < 0.001. Error bars represent SEM.
- Figure 6.
Results of the ROI analysis in which activations in the auditory cortex induced by visual temporal sequences (visual rhythms) and tactile temporal sequences were used as independent localizers for each other. A, ROIs for comparison between visual conditions were defined based on tactile temporal sequences versus regular visual stimulation (no-task) contrast. ROI analysis revealed a significant difference between the temporal visual sequence discrimination task and the temporal visual control condition. B, ROIs for comparison between tactile tasks were defined based on visual temporal sequences versus regular visual stimulation (no-task) contrast. ROI analysis revealed a significant difference between the temporal tactile sequence discrimination task and the temporal tactile control condition: *p < 0.05, **p < 0.01, ***p < 0.001. Error bars represent SEM.
Tables
General Deafness Hearing aids / cochlear implants 1 2 3 4 5 6 7 8 9 10 11 Age Sex Level of deafness Diagnosis Cause of deafness Did you ever use hearing aids in the past? Are you using hearing aids use now? How well do you understand speech when using hearing aids? How well are you able to understand speech without seeing the person talking? Cochlear implant 1 37 F 90 dB Profound deafness Medical mistake Yes Yes Well Poor No 2 30 F 90–119 dB Profound deafness Unknown Yes Yes Moderate No No 3 19 F 90–119 dB Profound deafness Genetic defect Yes No Moderate No No 4 26 F 90–119 dB Profound deafness Unknown Yes Yes Moderate No No 5 41 M > 120 dB Absolute deafness Unknown No No – No No 6 26 M 90–119 dB Profound deafness Genetic defect Yes No Poor No No 7 42 F 90–119 dB Profound deafness Unknown Yes Yes Poor No No 8 25 F 90–119 dB Profound deafness Unknown Yes No Well Poor No 9 24 M > 120 dB Absolute deafness Genetic defect No No – No No 10 35 F 115 dB Profound deafness Genetic defect Yes Yes Poor No No 11 30 M 90–119 dB Profound deafness Unknown Yes Yes Poor Moderate No 12 21 M 90–119 dB Profound deafness Unknown Yes Yes Well Moderate No 13 23 M 90–119 dB Profound deafness Unknown Yes No Well No No 14 21 F 100 dB Profound deafness Unknown Yes Yes Well Poor No 15 34 M > 120 dB Absolute deafness Genetic defect No No – No No 16 23 M 85–90 dB Profound deafness Genetic defect Yes No Well Well No 17 23 F 90–119 dB Profound deafness Genetic defect Yes No Moderate No No 18 31 F 90–119 dB Profound deafness Genetic defect Yes No Well Poor No 19 40 F 90–119 dB Profound deafness Unknown Yes Yes Poor No No 20 34 F 90–119 dB Profound deafness Genetic defect No No – No No 21 29 M 90–119 dB Profound deafness Genetic defect Yes Yes Poor No No Options for question 9: poor, moderate, good, very good. For question 10: not at all, poor, moderate, good, very good. scale of possible answers for question 12: none, poor, moderate, good, fluent. For questions 13 and 15: never, rarely, sometimes, often, all the time.
Language 12 13 14 15 16 17 18 19 20 lp. How well do you know PJM? How often did you use PJM in your childhood? In your childhood, did you speak in PJM with deaf adults? How often do you use PJM now? What is your main way of communicating now? Is your mother deaf? Is your father deaf? From whom did you learn PJM? Since when have you used PJM? 1 Fluent All the time Yes All the time PJM No No Teachers Since primary school 2 Fluent All the time Yes All the time PJM No No Teachers Since elementary school 3 Fluent All the time Yes All the time PJM Yes Yes Parents Always 4 Fluent All the time Yes All the time PJM No No Teachers Since primary school 5 Fluent Often Yes Often PJM Yes Yes Parents Always 6 Fluent All the time Yes All the time PJM No No Teachers Since school 7 Well All the time Yes All the time PJM No No Teachers Since primary school 8 Fluent All the time Yes All the time PJM No No Teachers Since primary school 9 Fluent All the time Yes All the time PJM Yes Yes Parents Always 10 Fluent All the time Yes All the time PJM Yes Yes Parents Always 11 Fluent All the time Yes All the time PJM No No Teachers Since primary school 12 Fluent All the time Yes All the time PJM Yes Yes Parents Always 13 Fluent All the time Yes All the time PJM Yes Yes Parents Always 14 Fluent All the time Yes All the time PJM No No Teachers Since school 15 Fluent All the time Yes All the time PJM Yes Yes Parents Always 16 Fluent All the time Yes All the time PJM No No Teachers Since primary school 17 Well All the time Yes All the time PJM No No Teachers Since primary school 18 Moderate Often Yes Often PJM Yes Yes Teachers Since primary school 19 Moderate All the time Yes All the time PJM No No Teachers Since primary school 20 Fluent All the time Yes All the time PJM Yes Yes Parents Always 21 Fluent All the time Yes All the time PJM No No Parents Always Scale of possible answers for question 12: none, poor, moderate, good, fluent. For questions 13 and 15: never, rarely, sometimes, often, all the time.
Contrast Region Hemisphere Cluster size F value MNI coordinates Main effect of modality (visual vs tactile) Inferior occipital gyrus Right 5653 408.86 30 −86 8 18 −84 −10 20 −94 4 Inferior occipital gyrus Left 4864 326.81 −16 −94 2 −24 −88 6 −18 −84 −12 Central operculum Right 371 56.60 48 −16 18 52 −24 20 Central operculum Right 463 53.64 44 8 30 Central operculum Left 330 49.17 −56 −20 14 −48 −24 20 Main effect of task vs no task Superior frontal gyrus Left 722 76.97 −22 −4 60 −24 −6 52 −36 0 58 Middle frontal gyrus Right 641 64.02 28 2 60 26 −6 52 Superior parietal lobule Right 329 62.24 14 −68 58 Superior parietal lobule Right 366 54.93 38 −40 44 Superior parietal lobule Left 438 41.58 −32 −58 56 −14 −70 52 −30 −50 42 Main effect of type of stimuli (temporal vs spatial) Supplementary motor area Left/right 280 83.37 −4 0 62 6 16 48 Middle frontal gyrus Left 2583 71.93 −28 24 54 −18 56 32 −36 20 50 Precentral gyrus Left 212 65.29 −46 −4 54 Middle occipital gyrus 280 54.46 50 −68 24 38 −80 26 Angular gyrus Left 1020 50.82 −48 −68 26 −42 −68 32 −34 −74 46 Posterior cingulate gyrus Left 2171 50.40 −8 −46 36 0 −44 36 Middle frontal gyrus Right 358 45.97 32 0 54 52 6 46 38.22 44 2 54 Supramarginal gyrus Right 290 42.73 46 −34 48 48 10 24 Fusiform gyrus Left 200 37.62 −28 −42 −12 Superior frontal gyrus 237 34.14 26 34 50 18 46 44 26 24 38 Main effect of group (deaf and hard of hearing vs hearing) Superior temporal gyrus Right 98 37.30 60 −20 0 66 −30 10 Supplementary motor area Left 6 27.86 −6 −2 68 Contrast Region Hemisphere t stat Cluster size MNI coordinates Whole-brain effects (between group) Effect of group: deaf (hard of hearing) > hearing Superior temporal gyrus Right 6.11 145 60 −20 2 66 −30 10 Interaction: dimension × group Spatial > temporal × deaf > hearing Superior temporal gyrus Right 6.32 236 66 −31 12 Middle temporal gyrus 56 −38 10 Effects in the auditory cortex of the deaf and hard of hearing: temporal sequences (task > no-task, control) Superior temporal gyrus Right 5.43 456 58 −22 −4 48 20 1 64 −30 4 Interaction effect of dimension (temporal > spatial) × (task > no-task) Superior temporal gyrus Right 4.18 210 60 −22 −4 66 −16 −8 54 −12 6 Effect of task > no-task Superior temporal gyrus Right 3.52 129 56 −20 −4 Whole-brain effects in the deaf and hard of hearing: effect of dimension (spatial > temporal sequences) Superior frontal gyrus Left 6.81 93 −22 −6 60 −24 −6 52 −36 −6 48 Superior frontal gyrus Right 6.78 491 26 −6 52 32 2 50 32 −2 48 Middle temporal gyrus Right 6.77 183 52 −54 6 Superior parietal lobule Right 6.21 285 34 −38 38 Superior temporal gyrus Right 5.51 171 63 −38 14 Middle temporal gyrus Left 5.42 145 54 −38 12 Temporal task > no task (Fig. 4A) Conjunction: visual and tactile temporal task > no task (Fig. 4B) Interaction: temporal > spatia × task > no task (Fig. 4C) Peak MNI coordinates Peak MNI coordinates Peak MNI coordinates Participant excluded from the analysis t value x y z t value x y z t value x y z 1 5.33 56 −20 −6 3.76 52 −18 −6 4.32 60 −22 −4 2 5.43 58 −22 −4 3.93 52 −18 −6 3.89 60 −22 −4 3 5.55 58 −22 −4 3.66 52 −18 −6 4.13 60 −22 −4 4 5.22 58 −22 −4 3.67 52 −18 −6 4.35 58 −22 −6 5 5.14 56 −20 −6 3.71 52 −18 −6 4.23 60 −22 −4 6 5.34 56 −20 −6 3.8 52 −18 −6 3.88 60 −22 −4 7 5.72 58 −22 −4 3.9 52 −18 −6 3.88 60 −22 −4 8 5.47 56 −20 −6 4.15 52 −18 −6 4.05 60 −22 −4 9 5.32 56 −20 −6 3.79 52 −18 −6 3.95 60 −22 −4 10 5.29 58 −22 −4 3.57 52 −18 −6 4.37 60 −22 −4 11 5.05 56 −20 −6 3.48 52 −18 −6 4.3 60 −22 −4 12 5.37 56 −20 −6 4.09 54 −16 −4 4.04 60 −22 −4 13 5.07 56 −20 −6 3.65 52 −18 −6 4.39 58 −22 −6 14 5.18 56 −20 −6 3.76 54 −16 −4 4.28 60 −22 −4 15 5.19 58 −22 −4 3.74 52 −18 −6 4.2 60 −22 −4 16 5.42 58 −22 −4 3.84 52 −18 −6 4.15 58 −22 −6 17 5.21 58 −22 −4 3.52 52 −18 −6 4.18 58 −22 −6 18 5.48 58 −22 −4 3.92 54 −16 −4 4.07 60 −22 −4 19 5.44 58 −22 −4 3.83 52 −18 −6 3.6 60 −22 −4 20 5.75 56 −20 −6 3.71 52 −18 −6 4.19 60 −22 −4 21 5.64 56 −20 −6 4.09 52 −18 −6 4.1 60 −22 −4 Whole group analysis 5.43 58 −22 −4 3.85 52 −18 −6 4.18 60 −22 −4 From each of the linear models, one participant (from the group of deaf and hard of hearing individuals) has been left out.
