Abstract
The role of the motor cortex in perceptual and cognitive functions is highly controversial. Here, we investigated the hypothesis that the motor cortex can be instrumental for translating foreign language vocabulary. Human participants of both sexes were trained on foreign language (L2) words and their native language translations over 4 consecutive days. L2 words were accompanied by complementary gestures (sensorimotor enrichment) or pictures (sensory enrichment). Following training, participants translated the auditorily presented L2 words that they had learned. During translation, repetitive transcranial magnetic stimulation was applied bilaterally to a site within the primary motor cortex (Brodmann area 4) located in the vicinity of the arm functional compartment. Responses within the stimulated motor region have previously been found to correlate with behavioral benefits of sensorimotor-enriched L2 vocabulary learning. Compared to sham stimulation, effective perturbation by repetitive transcranial magnetic stimulation slowed down the translation of sensorimotor-enriched L2 words, but not sensory-enriched L2 words. This finding suggests that sensorimotor-enriched training induced changes in L2 representations within the motor cortex, which in turn facilitated the translation of L2 words. The motor cortex may play a causal role in precipitating sensorimotor-based learning benefits, and may directly aid in remembering the native language translations of foreign language words following sensorimotor-enriched training. These findings support multisensory theories of learning while challenging reactivation-based theories.
SIGNIFICANCE STATEMENT Despite the potential for sensorimotor enrichment to serve as a powerful tool for learning in many domains, its underlying brain mechanisms remain largely unexplored. Using transcranial magnetic stimulation and a foreign language (L2) learning paradigm, we found that sensorimotor-enriched training can induce changes in L2 representations within the motor cortex, which in turn causally facilitate the translation of L2 words. The translation of recently acquired L2 words may therefore rely not only on auditory information stored in memory or on modality-independent L2 representations, but also on the sensorimotor context in which the words have been experienced.
Introduction
A wealth of sensorimotor information is available for learning in natural environments. Sights, sounds, kinesthetic signals, and proprioceptive information may all be taken into account by learners as they acquire knowledge and skills. There is growing consensus that the brain appears optimized to function based on input arising across multiple sensorimotor modalities (Röder and Wallace, 2010; Pasqualotto et al., 2016; Spence, 2018), and that training protocols incorporating sensorimotor functions can enhance learning (Meltzoff et al., 2009; Shams et al., 2011). We here refer to the presence of complementary sensory information during learning as sensory enrichment and the presence of complementary sensory and motor information during learning as sensorimotor enrichment (Mayer et al., 2015; Mathias et al., 2021). Behavioral evidence that enriched training yields stronger learning outcomes relative to unisensory training has accumulated in several domains (MacLeod et al., 2010; Mathias et al., 2015; Andrä et al., 2020; for review, see Sheffert and Olson, 2004; Shams and Seitz, 2008; von Kriegstein et al., 2008).
Despite the potential for sensorimotor enrichment to serve as a powerful tool for learning (see, e.g., Freeman et al., 2014), its underlying brain mechanisms remain largely unexplored. Research in the domain of foreign language (L2) learning has shown that cross-modal cerebral cortex responses occur following sensorimotor-enriched training: The translation of auditorily presented L2 words into one's native language (L1) elicits pre-/motor cortex responses following gesture-based (sensorimotor-enriched) L2 learning (Macedonia et al., 2011; Mayer et al., 2015; Macedonia and Mueller, 2016). Moreover, behavioral benefits of sensorimotor-enriched L2 training have been shown to correspond to classification accuracy of a multivariate pattern analyzer trained to dissociate motor cortex responses to sensorimotor-enriched stimuli (Mayer et al., 2015). The cross-modal responses did not depend on the type of vocabulary that that was learned; that is, they occurred for both concrete and abstract nouns and did not differ across word types. These findings suggest that motor cortex responses that occur during the translation of auditorily presented L2 words may be functionally linked to behavioral benefits of sensorimotor-enriched L2 training.
There are two opposing explanations regarding the relationship of responses within the motor cortices to behavioral benefits of sensorimotor-enriched training. On one hand, responses to unimodal stimuli within the motor cortices following sensorimotor-enriched learning may causally enhance stimulus recognition. This is the view taken by the predictive coding theory of multisensory learning (von Kriegstein, 2012), which proposes that sensory and motor cortices build up sensorimotor forward models during perception that simulate missing input (compare Friston, 2012). On the other hand, responses to unimodal stimuli within sensory and motor brain regions following sensorimotor-enriched training may be viewed as epiphenomenal, a view taken by reactivation-based theories (Wheeler et al., 2000; Nyberg et al., 2001; Fuster, 2009; Danker and Anderson, 2010). Reactivation theories suppose that motor brain responses engender a mere representation of a memorized stimulus, and therefore serve effectively no functional role in the recognition of the incoming stimulus. For example, if the sight of a bicycle triggered the recall of a motoric memory, reactivation theories assume that the recollected memory may arise from the reactivated motor cortex, but that the motor cortex responses would not aid in making the visual experience of the bike more precise.
Here we tested the hypothesis that motor cortex causally contributes to benefits of sensorimotor-enriched training using transcranial magnetic stimulation (TMS). We perturbed a site within the motor cortex whose response patterns were previously found to correlate positively with the magnitude of sensorimotor-enriched learning benefits (Mayer et al., 2015). The site was located in the vicinity of the arm functional compartment of the primary motor cortex (Brodmann area [BA] 4) (Meier et al., 2008; Strother et al., 2012), which was expected to contribute to sensorimotor-enriched learning that involved the performance of gestures primarily with the arms. Noninvasive brain stimulation methods, such as TMS, permit tests of whether stimulated brain areas casually contribute to ongoing behavioral outcomes, usually evidenced by increased response latencies (Day et al., 1989; Pascual-Leone et al., 1996; Siebner et al., 2009; Hartwigsen et al., 2017).
Materials and Methods
Overview of the experimental design
The study consisted of a training phase followed by four TMS sessions: two immediately following the training phase and two as a long-term follow-up. During the training phase, native German speakers completed a 4 d training program in which they learned L2 words and their L1 translations (Fig. 1a). L2 vocabulary was learned in two conditions. In a sensorimotor-enriched learning condition, individuals viewed and performed gestures while L2 words were presented auditorily (gesture performance enrichment; Fig. 1b, left). In a sensory-enriched learning condition, individuals viewed pictures while L2 words were presented auditorily (picture viewing enrichment; Fig. 1b, right). We included these learning conditions in the current study for two reasons: First, of four learning conditions previously tested (gesture performance, gesture viewing, picture performance, picture viewing) (Mayer et al., 2015), only these two conditions benefitted post-training L2 translation compared with auditory-only learning. Second, learning in these two conditions was associated with responses in distinct areas of the cerebral cortex (the arm functional compartment of the motor cortex and the biological motion superior temporal sulcus [bmSTS] for gesture performance enrichment and the lateral occipital complex for picture viewing enrichment) (Mayer et al., 2015), which allowed us to control for potential nonspecific TMS effects: that is, the activation of other brain regions by TMS that cause similar effects on behavioral performance. For succinctness, we refer to the gesture performance enrichment simply as gesture enrichment, and picture viewing enrichment as picture enrichment.
Study timeline and foreign language learning procedure. a, Participants learned foreign language (L2) vocabulary over 4 consecutive days (“learn”) in groups to emulate a classroom setting. Paper-and-pencil free recall and translation tests (“test”) were administered on days 2-4 and ∼5 months post-training. Following learning, participants completed a translation task during which TMS was administered and response times were measured (“TMS translation task”). The TMS translation task sessions took place on days 5 and 6 as well as on 2 consecutive days ∼5 months post-training. Effective TMS was applied during one of the two sessions that occurred on days 5 and 6 and one of the two sessions that occurred in month 5, and ineffective sham (placebo) stimulation was applied during the other sessions, with session orders counterbalanced across participants. b, In both the gesture and picture learning conditions, participants heard an L2 word, followed by the translation in their native language (L1) and a repetition of the L2 word. Videos of iconic gestures and pictures accompanied L2 words in gesture and picture trials, respectively. Participants performed the gesture along with the video during its repetition. No motor component was added to the picture learning condition as this was previously found not to benefit L2 vocabulary learning (Mayer et al., 2015).
During the TMS sessions (Fig. 2a), participants translated auditorily presented L2 words into L1 while undergoing effective or sham TMS. The bilateral motor cortices were targeted with a combination of effective or sham offline TMS (TMS before the task; Fig. 2b) and effective or sham online TMS (TMS during the task; Fig. 2c). A combination of online and offline stimulation was used, as the right and left hemisphere motor sites were anatomically too close to each other for simultaneous dual-site online stimulation (for a similar TMS design, see Hartwigsen et al., 2016). Participants performed a multiple choice translation task, in which they selected the correct L1 translation of an auditorily presented L2 word from a list of options (Fig. 2c); performance in this multiple choice task previously correlated with multivariate pattern analyzer classification accuracy within the motor cortex site stimulated in the present study (Mayer et al., 2015). We also included an exploratory recall task in which participants pressed a button as soon as the L1 translation came to their mind when hearing each L2 word. Participants did not perform gestures or view pictures during either of these tasks. Response time was used as the dependent variable because TMS typically influences response times rather than accuracy (Pascual-Leone et al., 1996; Ashbridge et al., 1997; Sack et al., 2007; Hartwigsen et al., 2017; Mathias et al., 2021). To evaluate long-term effects of motor cortex representations on enriched L2 vocabulary learning, TMS was administered both 5 d and 5 months following the start of training. Coils used in the sham condition were positioned at a 90° angle over each stimulation coil (Fig. 2d,e), resulting in no effective brain stimulation (i.e., placebo stimulation). Thus, the sham condition served as a baseline for effects of motor cortex TMS on response latencies.
Transcranial magnetic stimulation (TMS). a, Each TMS session began with the coregistration of the participant's scalp onto their previously acquired T1-weighted structural MRI scan. Individual stimulation intensities were then acquired by measuring motor-evoked potentials. Stimulation coils were then neuronavigated to the bilateral motor cortex sites of interest for the experiment. Following a practice task, offline theta burst stimulation was delivered to the right motor cortex. The experimental TMS translation task was then performed as online rTMS was delivered to the left motor cortex. Each TMS session lasted ∼1.5 h in total. b, The offline theta burst stimulation consisted of 600 pulses, represented by blue lines, which were presented in 200 ms frames containing 3 pulses each. c, On each trial of the TMS translation task, participants heard an L2 word that they had learned during the 4 d training phase and pressed a button as soon as the L1 translation came to their mind (exploratory recall task). They then selected the L1 translation by button press from a list of options presented on a screen (multiple choice task). L1 words were presented in German. Online rTMS, which consisted of trains of 7 TMS pulses at 10 Hz, was applied to the left motor cortex 50 ms following each L2 word onset. d, Offline stimulation was applied over the right motor cortex. e, Online stimulation was applied over the left motor cortex.
Our study included both concrete and abstract nouns. A word's concreteness indicates the extent to which its referent can be perceived by the body's sensory systems (Paivio et al., 1968; Hoffman, 2016). Whereas the referents of concrete words, such as bottle, are highly tangible and visible and can be efficiently conveyed using gestures or pictures, the referents of abstract words, such as democracy, are typically more difficult to convey. Despite these differences, sensorimotor-enriched learning has been shown to benefit the learning of variety of word classes, including nouns, verbs, adverbs, adjectives, and prepositions (Saltz and Donnenwerth-Nolan, 1981; Macedonia and Knösche, 2011; Macedonia et al., 2011; Macedonia and Klimesch, 2014).
Aims and hypotheses
The study had three aims. The first and primary aim was to test whether the motor cortex causally contributes to the translation of recently learned, sensorimotor-enriched L2 vocabulary. Based on the predictive coding theory of multisensory learning (von Kriegstein, 2012), we hypothesized that motor cortex stimulation would disrupt (i.e., slow down) the translation of sensorimotor-enriched L2 stimuli. Reactivation-based theories of learning (Fuster, 2009) do not consider differential effects of motor cortex stimulation on the translation of auditorily presented L2 words. We tested this hypothesis by comparing the influence of effective TMS and a sham stimulation control condition on responses to gesture-enriched L2 words. The inclusion of a picture learning control condition, which is associated with responses in another region of the cerebral cortex (the lateral occipital complex) (Mayer et al., 2015), allowed us to control for potential nonspecific TMS effects: that is, the modulation of other brain regions by TMS that cause similar effects on behavioral performance.
The second aim was to evaluate contributions of the motor cortex to the translation of sensorimotor-enriched L2 words over long post-training durations (e.g., several months). We expected greater effects of TMS on the translation of sensorimotor-enriched L2 words compared with picture-enriched vocabulary at later post-training time points. This hypothesis was based on evidence that effects of sensorimotor enrichment on L2 translation are stronger than effects of sensory enrichment half a year after learning the L2 vocabulary (Mayer et al., 2015). We tested this hypothesis by comparing effects of effective and sham stimulation on responses to gesture-enriched L2 words at both post-training time points.
Our third aim was to compare effects of motor cortex stimulation on the translation of two different L2 vocabulary types: abstract and concrete nouns. We hypothesized that motor cortex stimulation would similarly influence the translation of both vocabulary types, based on previous results showing that sensorimotor enrichment can benefit the learning of both concrete and abstract words (Macedonia and Knösche, 2011; Macedonia, 2014; Mayer et al., 2017). We tested this hypothesis by comparing effects of effective and sham stimulation on responses to concrete and abstract gesture-enriched words.
In addition to testing our three main hypotheses, the design allowed us to test the reliability of the previously reported finding that benefits of gesture performance enrichment exceed those of picture viewing enrichment (Mayer et al., 2015). To this end, we examined accuracy outcomes in the multiple choice task, and predicted that overall accuracy would be greater for gesture-enriched words compared to picture-enriched words. We additionally expected an accuracy advantage for concrete words compared with abstract words, based on the previous finding that concrete words tend to be translated more accurately than abstract words (Macedonia and Knösche, 2011; Macedonia and Klimesch, 2014).
Participants
Twenty-two right-handed native German speakers completed the study (15 female; mean age = 27.2 years, SD = 3.6 years). The sample size was based on two previous experiments (n = 22 per experiment) that investigated beneficial effects of gesture and picture enrichment on L2 learning outcomes (Mayer et al., 2015) (Experiments 1 and 2), as well as a previous TMS experiment (n = 22) demonstrating effects of bmSTS stimulation on recently learned gesture-enriched L2 vocabulary (Mathias et al., 2021). A repeated-measures power analysis conducted in G*Power (Faul et al., 2007, 2009) based on the TMS effect (
Before the experiment, each participant was approved for TMS by a physician based on a safety screening questionnaire and physical examination. No participants reported a history of head injury, neurologic, psychiatric, or psychological disorders, or contraindications for TMS. On average, the participants had studied one or more foreign languages for 14.3 years (SD = 6.9 years) and resided in a country in which a foreign language was spoken for 0.43 years (SD = 0.85 years). None of the participants was raised in a bilingual home. All participants reported normal or corrected-to-normal visual acuity; 1 participant who was weakly farsighted (right eye: +1.75 diopter [dpt], left eye: +2.25 dpt) reported being able to see the stimuli on a computer screen (∼1 m in front of the participant) during the TMS sessions without wearing glasses. No participants reported any hearing problems. Participants were compensated 212€ for completing the study. Written informed consent was obtained from each participant before beginning the experiment. The study was approved by the Ethics Committee of the University of Leipzig (Aktenzeichen 118/16-ek).
Stimuli
Vocabulary
Participants learned 90 words selected from a corpus of pseudowords intended for experiments on foreign language learning and referred to as “Vimmi” (Table 1) (Macedonia et al., 2010, 2011). The 90 words were randomly matched with 45 concrete and 45 abstract German nouns. Pseudowords were used for the purpose of avoiding potential associations with participants' previous L2 knowledge. The use of an artificial language also allowed us to control for several factors that potentially influence vocabulary learning, such as word length and frequency, which are difficult to control in a natural language. All L2 words consisted of three syllables and followed Italian phonotactic rules. Concrete and abstract German words were equivalent in terms of the number of syllables they contained (concrete mean = 2.40 syllables, SD = 0.84; abstract: mean = 2.69 syllables, SD = 0.90). Additionally, the frequency of concrete and abstract nouns in written German was roughly equivalent, according to the lexical database “Wortschatzportal” (http://wortschatz.uni-leipzig.de/de; concrete frequency scores: mean = 11.00, SD = 1.18, range 9-13; abstract frequency scores: mean = 10.96, SD = 0.98, range: 9-13).
Vocabulary used in the experimenta
Audio files
Audio recordings of the Vimmi and German words featured an Italian-German bilingual speaker who spoke with an Italian accent. Audio recording was conducted in a sound-damped chamber with a RØDE NT55 microphone (RØDE Microphones). Audio stimuli ranged in length from 654 to 850 ms (mean = 819.7 ms, SD = 47.3 ms). Word lengths did not significantly differ between the subsets of words assigned to the two learning conditions: t(88) = 1.30, p = 0.20, or across the vocabulary types, t(88) = 0.86, p = 0.39. Examples of the audio stimuli are available at http://kriegstein.cbs.mpg.de/mayer_etal_stimuli/.
Videos and pictures
To conduct the language training, each L2 word was paired with a video or picture that illustrated the word's meaning (Fig. 1b). Each video depicted an actress performing an iconic or symbolic gesture that embodied the meaning of a specific word. The actress was the same individual featured in the Vimmi and German audio recordings. Concrete words were enacted by iconic gestures that imitated their use or function (e.g., window was represented by the actress opening an imaginary window). Abstract words were embodied by symbolic gestures that illustrated their meaning (e.g., plea was represented by the actress taking a bow with her hands folded as in prayer). These symbolic gestures were previously agreed on by three independent raters (Mayer et al., 2015). Each colored video was recorded by a Canon Legria HF S10 camcorder (Canon) and lasted for 4 s. Videos always started and ended with the actress standing still with a neutral face in the center of the screen. All gestures consisted of head, unilateral and bilateral arm, leg, or finger movements, or a combination of these movements. Movements were limited to a radius of one meter around the actress. The actress in the gesture videos did not speak or mouth the L1 or L2 words. The full set of video stimuli is available at https://figshare.com/collections/Gesture_video_corpus/5296396.
A professional graphic artist (https://www.klaus-pitter.com/) created a black-and-white line drawing for each of the 90 L2 words (Fig. 1b). Abstract words were often illustrated by complex scenes, while concrete words were usually pictured as single objects. The complexity of visualization was not balanced between conditions to conform to differences that occur in natural learning settings. Examples of the picture stimuli are available at http://kriegstein.cbs.mpg.de/mayer_etal_stimuli/.
Design
The study had a 2 × 2 × 2 × 2 repeated-measures design. Within-participant independent factors were learning condition (gesture, picture), stimulation type (effective TMS, sham stimulation), testing time point (day 5, month 5), and L2 vocabulary type (concrete, abstract). The dependent variable was response time. We examined accuracy in the multiple choice TMS task to evaluate whether differences in response times between conditions could be attributed to speed-accuracy tradeoffs.
Procedure
Training phase
Participants learned 45 L2 words in each of the two learning conditions. In the gesture learning condition, individuals viewed and performed gestures while L2 words were presented auditorily (Fig. 1b). In the picture learning condition, individuals viewed pictures while L2 words were presented auditorily. To ensure that each of the 90 total L2 words was equally represented in both learning conditions, half of the participants learned one set of 45 words with gesture enrichment and the other 45 words with picture enrichment. The other half of participants received the reverse assignment of words to learning conditions to counterbalance the stimuli between participants and across learning conditions.
Each day of training comprised four 33 min learning blocks. Each block consisted of 45 L2 words presented in a random order. Each word was repeated 4 times per block, yielding a total of 180 trials per block. In two of the blocks, L2 words were enriched with their associated gestures; and in the other two blocks, L2 words were enriched with associated pictures. The order of learning blocks was counterbalanced across days and participant groups. Between blocks, the participants had breaks of 10-15 min. Snacks and drinks were provided for participants to consume during breaks.
Participants were instructed before the start of training that the goal was to learn as many L2 words as possible over the 4 d of training. Participants received no further instruction during the training except to be informed about which learning condition would occur next (i.e., gesture or picture enrichment). Since the L2 vocabulary learning took place in groups of up to 5 individuals, training sessions occurred in a seminar room with a projector and sound system. Audio recordings were played via speakers located on each side of the screen. The volume of the playback was adjusted so that all participants could comfortably hear the words. Participants were asked to stand during all learning blocks. The physical locations of individuals in each group within the training room were counterbalanced across training days.
Each gesture trial started with the presentation of a video showing a gesture for 4 s combined with the L2 word recording, which began 0.2 s after the video's onset (Fig. 1b). The video was followed by a black screen paired with the audio presentation of the L1 translation for 1.7 s. Finally, the gesture video and corresponding L2 word were presented again for 4 s. The picture trials were similarly constructed. Each picture trial began with the presentation of the picture enrichment for 4 s combined with the recording of the L2 vocabulary, which began 0.2 s after the picture's onset. A black screen paired with the audio presentation of the L1 translation followed. Finally, the picture and corresponding L2 word were presented again. In the gesture-enriched blocks, participants were asked to perform the gestures along with the actress when the video was shown for the second time. A motor component was not included in the picture learning condition as a previous study found that the addition of a motor component to picture enrichment does not yield beneficial learning effects compared with nonenriched (auditory-only) learning (Mayer et al., 2015).
On days 2, 3, and 4 of the training phase, participants were asked to complete three paper-and-pencil vocabulary tests before beginning the training blocks. We included these tests to maintain the same L2 training procedure used by Mayer et al. (2015). Participants were given 15 min to complete each test. The first test was always a free recall test in which participants were asked to write down all L1 and L2 words, single or in combination, that they could remember. The second and third tests were L1-L2 and L2-L1 translation tests, in which participants were asked to translate lists of the 90 L1 or L2 words. The order of the two translation tests was counterbalanced across participants and training days. The order of words in the translation tests was randomized each day. The participant in each group with the highest score summed across the paper-and-pencil vocabulary tests administered on days 2, 3, and 4 was rewarded with an additional 21€ beyond the total study compensation. Participants were informed about this monetary incentive on day 1 before the start of the learning blocks.
Paper-and-pencil tests were independently scored for accuracy by two raters. Three points were given for each correct translation (German-Vimmi or Vimmi-German word pair), and one point was given for each correctly recalled German word that was missing a corresponding Vimmi translation and vice versa. Points were summed to determine a total score for each participant, learning condition, and vocabulary type.
L1-L2 and L2-L1 translation tests were scored in terms of the percent correct translations recalled for each participant, learning condition, and vocabulary type. A Vimmi word was considered correct if the two independent raters agreed that the word that was written down was valid for the sound pronounced in the audio file according to German sound-letter-mapping. A German word was considered correct if a participant wrote down the German word that was assigned to the Vimmi word during learning or if a participant wrote down a synonym of the German word, according to a standard German synonym database (http://www.duden.de). Paper-and-pencil vocabulary test results are shown in Table 2.
Paper-and-pencil vocabulary test performancea
Before beginning the training phase, all participants completed three psychological pretests that assessed their concentration ability (Concentration test, mean score = 208.9, SD = 34.3) (Brickenkamp, 2002), speech repetition ability (Nonword Repetition test, mean score = 101.5, SD = 7.3) (Korkman et al., 1998), and verbal working memory (Digit Span test, mean score = 18.2, SD = 4.4) (Neubauer and Horn, 2006). None of the participants were outliers (2 SDs above or below the group mean) with respect to their scores on any of the three tests, and all participants performed within the norms of the Concentration test for which norms were available.
TMS sessions
Following completion of the training phase, participants took part in the TMS sessions (Fig. 2a) at two time points: 5 d and 5 months following the start of the training. At each time point, participants completed two TMS sessions. During one of these sessions, effective stimulation was delivered to the right and left motor cortices; and during the other session, sham stimulation was delivered to the same sites.
Multiple choice and exploratory recall tasks were completed during each of the four TMS sessions occurred in 4 blocks, and each block contained 45 trials, yielding a total of 180 trials per TMS session. Each of the 90 L2 words was therefore presented 2 times during each TMS session. Word orders were randomized within each block and for each participant. Participants responded during the translation test on a custom-made four-button response box.
Each test trial began with the visually presented instruction “Press the button as soon as you know the translation” (“Drücken Sie den Knopf, sobald Sie die Übersetzung wissen”), which was shown for 1.5 s on an EIZO 19 inch LCD monitor (EIZO Europe; white letters, font: Arial, font size: 32 pt; black background). The screen was placed ∼1 m in front of the participants. Next, one of the L2 words was presented over in-ear noise-cancelling headphones (Shure SE215, Shure Europe), combined with a black screen. The participants had 3 s, beginning at the L2 word's onset, to indicate by button press if they knew the correct L1 translation of the auditorily presented L2 word (exploratory recall task; Fig. 2c). They were instructed to press the first button on the response pad with their index finger if they knew the L1 translation, and to not press any button if they did not know the L1 translation. After 3 s, three possible L1 translations appeared on the screen, and participants selected the correct translation by pressing one of the four buttons on the response box (multiple choice task; Fig. 2c). A fourth response option (“unknown/different word,” “unbekannt/anderes Wort”) was included in every trial. Participants were told to select this option if they did not know the correct L1 translation or if they had thought of a different L1 translation before the appearance of the three answer choices. Participants responded by pressing one of four buttons on the response pad with their index, middle, ring, or little fingers. The response options remained on the screen for 2 s, followed by a jittered intertrial interval of 0.5-1 s (mean = 0.75 s). Participants were instructed to respond as quickly and as accurately possible.
The TMS sessions that were conducted 5 months after the start of the training phase followed the same procedure as the TMS sessions that occurred on days 5 and 6. The orders of effective and sham TMS sessions were counterbalanced across participants, both within each time point and between time points. The two TMS sessions were separated by a period of 18 weeks (mean = 18.2 weeks, SD = 1.4 weeks).
Before beginning the translation task, participants completed a practice task with 20 common English words instead of Vimmi words. This practice task was administered during the first TMS session at each time point (i.e., 5 d and 5 months following the start of the training phase).
Finally, participants returned 1-6 d (mean = 3.7 d, SD = 1.2 d) after their final TMS session, to complete again the three pencil-and-paper vocabulary tests (free recall, L1-L2 translation, and L2-L1 translation). Test orders were again counterbalanced across participants.
Participants had no knowledge of the month 5 follow-up TMS and behavioral sessions until they were contacted a few weeks before these month 5 testing dates. This was done to avoid potential rehearsal of the vocabulary during the 5 month interval between testing time points.
TMS
Neuronavigation
T1-weighted structural MRI scans were acquired for each participant using a 3 Tesla MAGNETOM Prisma-fit (Siemens Healthcare) with an MPRAGE sequence in sagittal orientation (inversion time = 900 ms, TR = 2300 ms, TE = 2.98 ms, flip angle = 9°, voxel size = 1 × 1 × 1 mm). During the TMS sessions, the scalp of each participant was coregistered to their T1 scan using the neuronavigation software Localite (Localite; Fig. 2a).
Right and left motor stimulation sites were based on fMRI results from a group of 22 participants (Mayer et al., 2015). These participants completed a similar sensorimotor-enriched L2 vocabulary training paradigm, and a multivariate pattern classifier was trained to classify brain BOLD responses to sensorimotor-enriched versus nonenriched auditorily presented L2 words. TMS was applied over the coordinates that showed the highest positive correlation of the behavioral sensorimotor-enriched learning benefit with classifier accuracy (left hemisphere: x, y, z = −18, −43, 61 mm; right hemisphere: x, y, z = 21, −41, 73 mm, MNI space) within the motor ROI used by Mayer et al. (2015). This ROI included the sum of the (bilateral) maps for BA 4a, 4p, and 6 implemented in the Anatomy toolbox (Eickhoff et al., 2005) for SPM8 (Wellcome Trust Center for Neuroimaging, University College London; http://www.fil.ion.ucl.ac.uk/spm/). The stimulation coordinates were located in BA 4, which corresponds to the primary motor cortex (for review, see Chouinard and Paus, 2006), and were located in the vicinity of the arm, elbow, forearm, and wrist functional compartments of the motor cortex (Meier et al., 2008; Strother et al., 2012). MNI coordinates were transformed to individual space for each participant using SPM8.
TMS parameters
TMS pulses were generated by a MagPro X100 stimulator (MagVenture A/S) and four focal figure-of-eight coils (C-B60; outer diameter = 7.5 cm). The pulses were triggered with Signal software version 1.59 (Cambridge Electronic Design). Presentation software (Neurobehavioral Systems; www.neurobs.com) was used for stimulus delivery and response recording. Coils were held in position using fixation arms (Manfrotto 244).
For offline TMS, we used continuous theta burst stimulation (cTBS), which consisted of 600 pulses divided into 200 frames (Huang et al., 2005; Valchev et al., 2017). Each frame had a duration of 200 ms and consisted of 3 separate square pulses with 5 ms durations and an interpulse interval of 15 ms (Fig. 2b). The cTBS frequency was 50 Hz and was delivered to the right hemisphere before the task. The online repetitive TMS (rTMS) protocol consisted of seven single pulses delivered at 10 Hz beginning 50 ms after each stimulus onset (Fig. 2c). Both TMS protocols are in line with the published safety guidelines (Rossi et al., 2009). Coils used in the sham TMS condition were positioned at a 90° angle over each stimulation coil (Fig. 2d,e), resulting in no effective brain stimulation. To avoid any potential carryover effects between TMS sessions, the sessions (i.e., the effective stimulation session and the sham stimulation session) were conducted on separate days (Rossi et al., 2009).
We combined cTBS and online stimulation rather than using dual-site online stimulation because the right and left hemisphere motor sites were anatomically too close to each other for simultaneous dual-site online stimulation given the TMS coil size. The combination of cTBS and online TMS (condition-and-perturb approach) has been shown to be well suited for the disruption of processing in several brain areas (e.g., Hartwigsen et al., 2016). Combining cTBS and rTMS is also preferable to conducting dual site cTBS as inhibitory effects of cTBS are known to diminish over time following stimulation, which is not the case for rTMS as it is applied on every trial and is therefore generally regarded as having stronger inhibitory effects than cTBS (Huang et al., 2005). Compared with online perturbation of other frontal regions, TMS of the motor cortex usually induces very mild side effects that are unlikely to influence behavior (Meteyard and Holmes, 2018). Finally, a prestudy pilot experiment revealed no significant effects of unilateral left hemisphere cTBS stimulation without rTMS, in line with several recent studies that demonstrated null effects of unilateral compared with bilateral TMS (Ritzinger et al., 2012; Jelić et al., 2017; Park et al., 2017; Yang et al., 2018). The pilot experiment results are available at osf.io/tnfa3.
Motor threshold measurement
Motor thresholds were measured for each participant to determine individual stimulation intensities (Fig. 2a). Single TMS pulses were applied to the left primary motor (M1) area controlling the first dorsal interosseous (FDI) muscle located on the right hand. A meta-analysis by Mayka et al. (2006) provided mean stereotactic coordinates of the left M1 (x, y, z = −37, −21, 58 mm, MNI space), which were used as a starting point to locate the M1 FDI area.
Each participant's active motor threshold was defined as the lowest stimulus intensity that caused at least 5 MEPs of 150-200 µV peak-to-peak amplitude out of 10 consecutive TMS pulses in the tonically active FDI muscle. A 50 µV peak-to-peak amplitude criterion was used to define the resting motor threshold while the FDI muscle was relaxed. Following standard intensities used in previous studies, the cTBS stimulation intensity was set to 90% of active motor threshold (Chistyakov et al., 2010; Brückner et al., 2013), and rTMS intensity was set to 90% of the resting motor threshold (Halawa et al., 2019; Mathias et al., 2021). The same cTBS and rTMS intensities were used for each individual across all four TMS sessions (cTBS mean intensity = 36.4% of maximum stimulator output, SD = 5.9%; rTMS mean intensity = 41.4% of maximum stimulator output, SD = 6.5%).
Data analysis
Exploratory recall task
Participants (n = 22) indicated that they recalled the L1 translation before the appearance of the four response options during roughly half of all trials across the two TMS sessions (mean = 58.5% of trials, SE = 32.3%), leaving an insufficient number of trials for a meaningful analysis of the exploratory recall task. For the multiple choice task, participants selected a translation from the options presented during mean = 78.8% (SE = 21.3%) of trials across the two TMS sessions. We nevertheless performed an exploratory analysis of recall response times using a four-way ANOVA with the factors learning condition (gesture, picture), stimulation type (effective, sham), time point (day 5, month 5), and vocabulary type (concrete, abstract). Response time was defined as the time elapsed between the start of the auditory L2 word presentation and the participant's indication by button press (before the appearance of the four response options) that they knew the L1 translation of the presented L2 word. We analyzed trials in which participants indicated by button press that they recalled the L1 translation and subsequently selected the correct translation from the list of response options presented on the screen. The ANOVA yielded no significant main effects or interactions involving the stimulation type factor (TMS-Gesture: mean = 1410 ms, SE = 30 ms; Sham-Gesture: mean = 1409 ms, SE = 28 ms; TMS-Picture: mean = 1431 ms, SE = 32 ms; Sham-Picture: mean = 1405 ms, SE = 24 ms). Additionally, the usually robust difference between concrete and abstract vocabulary types (Paivio, 1965; Atkinson and Juola, 1973; Walker and Hulme, 1999; Macedonia and Knösche, 2011) was also not significant (F(1,21) = 1.15, p = 0.30, two-tailed,
Multiple choice task
Response times in the multiple choice task were computed as the time interval from the appearance of the multiple choice options on the screen until the response. Trials in which participants did not respond following the appearance of the multiple choice options, selected the incorrect translation, or selected the fourth response option (“Unknown/Other word”) were excluded from the response time analyses.
To test our first hypothesis (see Aims and hypotheses), we ran a two-way repeated-measures ANOVA with the factors learning condition (gesture, picture) and stimulation type (TMS, sham) on the multiple choice task response times.
To test our second hypothesis on differential contributions of the motor cortex at early and later time points, we ran a three-way repeated-measures ANOVA on multiple choice task response times with factors learning condition (gesture, picture), stimulation type (effective, sham), and time point (day 5, month 5). We followed up the three-way ANOVA with two-way repeated-measures ANOVAs on multiple choice task response times with factors learning condition (gesture, picture) and stimulation type (effective, sham) for each time point.
To test our third hypothesis regarding influences of vocabulary type on multiple choice task response times, we ran a four-way repeated-measures ANOVA with factors learning condition (gesture, picture), stimulation type (effective, sham), time point (day 5, month 5), and vocabulary type (concrete, abstract) on response times.
Pairwise comparisons were conducted using Tukey HSD post hoc tests. Hedge's g and
Translation accuracy
We analyzed translation accuracy to assess potential speed-accuracy trade-offs in the observed TMS effects and to test previously reported effects of enrichment and vocabulary type on L2 learning (Macedonia and Knösche, 2011; Macedonia and Klimesch, 2014; Mayer et al., 2015). Inaccurate response trials consisted of trials in which participants did not respond following the appearance of the multiple choice options, selected the incorrect translation, or selected the fourth response option (“Unknown/Other word”).
To evaluate whether the observed patterns of response times in the multiple choice task were because of speed-accuracy tradeoffs, we correlated response times in the multiple choice task with accuracy (percent correct) for each learning and stimulation condition.
To test the previously reported effect that benefits of gesture performance enrichment on L2 translation exceed those of picture viewing enrichment over the long-term (Mayer et al., 2015), as well as the effect that concrete words tend to be translated more accurately than abstract words (Macedonia and Knösche, 2011; Macedonia and Klimesch, 2014), we evaluated translation accuracy scores in the multiple choice task (percent correct) in the absence of TMS (sham condition only) using a three-way ANOVA with the factors learning condition, time point, and vocabulary type.
We did not expect any significant effects of TMS on accuracy, as TMS tends to influence response time rather than accuracy (Pascual-Leone et al., 1996; Ashbridge et al., 1997; Sack et al., 2007; Hartwigsen et al., 2017; Mathias et al., 2021). For information purposes, we additionally report analyses on TMS effects on accuracy. We ran a four-way repeated-measures ANOVA on accuracy in the multiple choice task, with the factors learning condition (gesture, picture), stimulation type (effective, sham), testing time point (day 5, month 5), and vocabulary type (concrete, abstract) and examined all interactions involving the stimulation type factor.
Results
Motor cortex stimulation slows the translation of gesture-enriched L2 vocabulary
Our first and primary hypothesis (see Aims and hypotheses) was that motor cortex integrity would contribute to L2 translation following gesture-enriched L2 learning, but not picture-enriched L2 learning. We therefore first tested whether motor cortex stimulation modulated L2 translation, regardless of testing time point and vocabulary type, but depending on learning condition. The results confirmed our hypothesis. A two-way ANOVA on response times in the multiple-choice task revealed a learning condition × stimulation type interaction (F(1,21) = 4.15, p = 0.04, two-tailed,
Effects of motor cortex stimulation on L2 translation in the multiple choice task. Bilateral motor cortex stimulation slowed the translation of L2 vocabulary learned using gestures compared with sham stimulation. There was no such effect on L2 vocabulary learned using pictures. The mean of each condition across time points (5 d and 5 months following the start of learning) is shown (n = 22 participants). Error bars indicate SEM. *p < 0.05. n.s. = not significant.
In a control analysis, we tested whether differences in response times observed under effective stimulation compared with sham stimulation conditions could be because of tradeoffs between translation speed and accuracy. Response times for correct responses in the multiple choice translation task were compared with accuracy (percent correct) for each learning and stimulation condition. Response times correlated negatively with translation accuracy: Slower responses were associated with lower accuracy for all conditions (gesture-TMS, r(20) = −0.62, p = 0.002, Bonferroni-corrected; gesture-sham, r(20) = −0.54, p = 0.009; picture-TMS, r(20) = −0.60, p = 0.003; picture-sham, r(20) = −0.55, p = 0.008; Fig. 4). The correlations for TMS and sham conditions did not significantly differ (all p values > 0.36). These results suggest that participants did not trade speed for accuracy in the multiple choice task.
Speed-accuracy relationships in L2 translation in the multiple choice task. Slower response times correlated with lower translation accuracy, indicating that there was no speed-accuracy trade-off in the multiple choice TMS task. All p values <0.01, Bonferroni-corrected; df = 20 for all correlations.
Thus, our first hypothesis, that motor cortex integrity would contribute to L2 translation following gesture-enriched L2 learning, but not picture-enriched L2 learning, was supported by the results: TMS applied to the motor cortex disrupted the translation of words learned with gestures relative to sham stimulation, and the response time differences were not explained by speed-accuracy tradeoffs in translation performance. Next, we tested whether this pattern of TMS effects on the translation of gesture- and picture-enriched L2 words differed across time points.
Effects of motor cortex stimulation occurred 5 d following the start of L2 training
We next tested our second hypothesis (see Aims and hypotheses) that effects of motor cortex stimulation on the translation of gesture-enriched L2 words would be greater at the later post-training time point (month 5) compared with the earlier post-training time point (day 5). Contrary to our hypothesis, a three-way ANOVA with the factors learning condition, stimulation type, and time point on translation response times in the multiple-choice task yielded no significant three-way interaction. There was a significant main effect of time point (F(1,21) = 62.21, p < 0.001, two-tailed,
To explore whether the learning condition and stimulation type variables interacted significantly within each time point, we performed separate two-way ANOVAs with the factors learning condition and stimulation type on response times at each time point. The ANOVA on day 5 response times yielded a significant learning condition × stimulation type interaction (F(1,21) = 4.59, p = 0.04, two-tailed,
In sum, significant effects of motor cortex stimulation on L2 translation occurred 5 d following the start of the L2 training period; and although effects of stimulation were not significant at month 5, there was no significant interaction of TMS effects between the two time points (Fig. 5).
Effects of motor cortex stimulation on L2 translation by time point in the multiple choice task. Effects of motor stimulation occurred 5 d following learning (n = 22 participants). There was no significant three-way interaction between time point, stimulation type, and learning condition factors. Error bars indicate SEM. *p < 0.05. n.s. = not significant.
Motor cortex integrity influences L2 translation independent of vocabulary type
Finally, we tested our third hypothesis (see Aims and hypotheses) that the motor cortex integrity would influence L2 translation independent of vocabulary type. We conducted a four-way ANOVA on translation response times in the multiple choice task with the factors learning condition, stimulation type, time point, and vocabulary type. The four-way ANOVA yielded a two-way learning condition × stimulation type interaction (F(1,21) = 5.20, p = 0.03, two-tailed,
L2 vocabulary translation response times in the multiple choice task at the day 5 TMS session (left) and month 5 TMS session (right) by learning condition, stimulation type, and vocabulary type (n = 22 participants). Error bars indicate SEM.
There was a main effect of vocabulary type (F(1,21) = 43.52, p < 0.001, two-tailed,
Gesture-enriched learning enhances L2 translation accuracy
We conducted a three-way ANOVA on translation accuracy scores in the multiple choice task (percent correct) under sham stimulation only with the factors learning condition, time point, and vocabulary type. The ANOVA revealed significant main effects of learning condition (F(1,21) = 4.15, p = 0.05, two-tailed,
Accuracy of L2 translation following learning while undergoing sham TMS. Overall, participants translated gesture-enriched L2 words more accurately than picture-enriched L2 words, and translation accuracy was higher at day 5 than month 5. Learning type and time point did not significantly interact (n = 22 participants). *p < 0.05. ***p < 0.001.
The ANOVA also yielded a main effect of vocabulary type on translation accuracy (F(1,21) = 42.67, p < 0.001, two-tailed,
Effects of motor cortex stimulation on foreign vocabulary translation accuracy
We additionally evaluated effects of TMS on multiple choice task accuracy by conducting a four-way ANOVA on translation accuracy scores in the multiple choice task (percent correct) with the factors learning condition, stimulation type, time point, and vocabulary type. As expected, there were no significant effects of stimulation type on accuracy for either vocabulary type or learning condition at either time point (Fig. 8).
Accuracy of L2 translation in the multiple choice task depending on learning condition, stimulation type, time point, and vocabulary type. As expected, no significant effects of TMS on L2 translation accuracy were observed at either time point (n = 22 participants). n.s. = not significant.
Discussion
We used inhibitory TMS to test the causal relevance of motor cortex function following sensorimotor-enriched L2 vocabulary learning. There were three main findings. First, motor cortex stimulation disrupted the translation of recently learned, gesture-enriched vocabulary relative to sham stimulation. This result supports our principal hypothesis that motor cortex integrity causally contributes to the translation of sensorimotor-enriched L2 vocabulary. Motor cortex stimulation had no effect on the translation of picture-enriched vocabulary. Second, motor cortex integrity benefitted L2 translation in the short-term but not, contrary to our hypothesis, in the long-term. Third, in congruence with our hypothesis, effects of motor cortex stimulation on the translation of sensorimotor-enriched L2 vocabulary were not influenced by whether the L2 word was concrete (e.g., ball) or abstract (e.g., thought). Together, these findings suggest that sensorimotor-enriched training constructs associations between L2 words and their L1 translations by way of representations arising from the motor cortex.
Motor cortex integrity influenced task performance in another sensory modality, depending on associations forged during learning. This finding differentiates predictive coding (Friston, 2012; von Kriegstein, 2012) and reactivation-based (Fuster, 2009; Danker and Anderson, 2010) theories of multisensory learning. The causal relation observed between motor cortex integrity and auditory L2 translation is predicted by multisensory predictive coding theories, but not by reactivation theories. Predictive coding theories suggest that, to recall the meaning of a newly acquired L2 word, one may internally reenact or simulate the perceptual and motor processes that were involved in learning that word via a generative model (Rao and Ballard, 1999; von Kriegstein et al., 2008; Friston and Kiebel, 2009). Numerous behavioral studies support the notion that perceptual and motor processes involved in learning are reenacted during recognition (Zwaan and Yaxley, 2003; Estes et al., 2008; Witt et al., 2010; Dudschig et al., 2013; Meade et al., 2017; Matheson et al., 2019). During reading, for example, concrete nouns, such as plane and foot, elicit upward and downward saccades, respectively, suggesting that visuomotor simulations are involved in accessing semantic knowledge (Estes et al., 2008; replicated in Dudschig et al., 2013). In the present study, the motor cortex may have supported the internal simulation of movement-related information during L2 translation, thereby benefitting response times. This, however, remains speculative, as the present study was not designed to conclusively demonstrate that a predictive coding mechanism underlies the causal relevance of motor cortex representations in L2 translation.
Several alternative theories cannot fully account for the causal engagement of motor cortex in L2 translation following sensorimotor-enriched training. Levels-of-processing theory (Craik and Lockhart, 1972) attributes enhanced memory performance in word learning tasks to the processing of word meaning across a greater number of levels (e.g., a semantic level and an orthographic level), associated with increased activity in prefrontal and temporal areas (Nyberg, 2002). Other lines of work have pointed toward the efficacy of emotion- or attention-based interventions for performance on cognitive tasks, such as the positive effects of background music on IQ tests (Schellenberg et al., 2007). However, if behavioral performance following enrichment-based learning was determined solely by levels of processing or emotion-based mechanisms, then stimulation of a motor area would not have disrupted L2 translation. Another explanation is that sensorimotor enrichment affects learning outcomes by influencing brain responses corresponding to the sensory modality experienced at test (e.g., the auditory modality for auditory tests) (for review, see Shams and Seitz, 2008). This account also would have predicted no influence of motor cortex integrity on L2 translation following gesture-enriched learning. In sum, these alternative mechanisms are unable to explain how motor cortex integrity contributed to post-enrichment performance.
It is likely that neural representations of enrichment-related information during L2 translation are not limited to only motor-related information, but rather extend to additional relevant input received during enriched L2 learning. Support for this inference comes from a recent study showing that inhibitory stimulation of a region specialized in the processing of visual biological motion (Grossman and Blake, 2001; Jastorff et al., 2006) also disrupted the auditory translation of gesture-enriched (but not picture-enriched) L2 words (Mathias et al., 2021). Neural representations of gesture-based input experienced during learning may also include a somatosensory component. Incidentally, the motor area stimulated in the current study, which was identified by Mayer et al. (2015) as the region correlating most strongly with behavioral performance following sensorimotor-enriched L2 learning, was spatially proximal to the somatosensory cortex. Similar patterns of activity occurring at the boundary of motor and somatosensory cortices have been observed following children's learning of symbols through handwriting (Vinci-Booher et al., 2016). Given that action-based enrichment techniques are consistently accompanied by sensory feedback, and that perceptual and motor learning appear reciprocally linked and generally occur together (for review, see Ostry and Gribble, 2016), it is likely that neural representations of enrichment-related information include both motor and associated somatosensory components. The magnetic field used to stimulate the motor cortex may have also impacted some part of nearby somatosensory cortex, as the effective area of figure-of-eight coil stimulation is ∼1-2 cm (Walsh and Rushworth, 1999; Sandrini et al., 2011); increasing the focality of stimulation to a smaller cortical target is not currently possible, except through significantly more invasive methods, such as the placement of electrodes on the cortical surface.
The perception of familiar vocabulary in the native language activates an experience-dependent network of sensory and motor areas (for review, see Kiefer and Pulvermüller, 2012; Lambon Ralph et al., 2017) and enhances motor cortex excitability (Watkins et al., 2003). While some have argued that motor activity during L1 perception is merely a downstream byproduct of L1 processing in “core” language areas (Lotto et al., 2009), other studies have provided evidence that motor structures contribute to speech perception (Rogers et al., 2014; Stokes et al., 2019; for review, see Skipper et al., 2017) or have argued that perceiving speech is essentially perceiving gestures (motor theory of speech perception) (Liberman and Mattingly, 1985; for review, see Galantucci et al., 2006). Motor structures may functionally support L1 comprehension by representing conceptual aspects of language processing (for review, see Barsalou, 2005; Kemmerer et al., 2012; Pulvermüller, 2012; Kemmerer, 2015); concepts may be implemented via a predictive coding mechanism (for review, see Matheson and Barsalou, 2018). Hearing L1 words that refer to body movements (Hauk et al., 2004; Marino et al., 2017) can trigger responses in pre-/motor cortices. Neurostimulation evidence has pointed to the functional relevance of motor areas in behavioral responses to L1 words referring to body movements (Vukovic et al., 2017). Our findings add a fundamentally novel line of research to these previous results by demonstrating the causal relevance of motor responses in the representation of recently acquired sensorimotor-enriched stimuli, and, in particular, nonaction words, such as concrete and abstract nouns. Unfamiliar words became associated with information represented within the motor cortex through short-term sensorimotor-enriched learning, and this information was able to causally influence the auditory translation of those words.
Contributions of the motor cortex to the translation of L2 vocabulary arose following a relatively brief period of gesture-enriched L2 training. This finding suggests that sensory and motor elements of sensorimotor-enriched learning experience can be integrated rapidly with vocabulary representations, and that lifelong experience with words and their semantic associations is not a prerequisite for establishing representations of words within motor cortex. The relatively rapid acquisition of multisensory representations has been demonstrated in some previous studies. Thelen et al. (2012) report effects of single-trial audiovisual learning on evoked potentials elicited during subsequent visual-only recognition. In another study, von Kriegstein and Giraud (2006) demonstrated that 2 min of exposure to a voice paired with a face enhanced functional coupling between voice and face brain areas during subsequent perception of the trained voices. Effects of motor cortex stimulation on the translation of sensorimotor-enriched words in the current study were limited to the day immediately following the 4 d training program. This result was not expected, as effects of sensorimotor enrichment on L2 translation have been shown to exceed those of sensory enrichment up to several months following enriched L2 vocabulary training (Mayer et al., 2015). One explanation for the differential effects of stimulation at the two time points tested here could be that participants in the current study received a lesser amount of training compared with those in the study conducted by Mayer et al. (2015), potentially diminishing the magnitude of enrichment-based effects at the later time point.
The aim of the current study was to examine whether the motor cortex causally contributes to L2 translation following sensorimotor-enriched learning. Addressing this aim hinged critically on a difference in outcomes under effective and sham TMS applied to the motor cortex. One possible explanation for the difference in effects of effective and sham stimulation could be the presence of nonspecific TMS effects (e.g., the stimulation of nearby brain regions that generally disrupt auditory word perception or that cause similar generalized TMS effects). However, if that were the case, we would have also expected a difference between effective and sham stimulation conditions for picture-enriched words, which did not occur. Thus, the inclusion of a picture learning control condition, which is associated with responses in another region of the cerebral cortex (the lateral occipital complex) (Mayer et al., 2015), allowed us to control for nonspecific TMS effects. Nevertheless, the current findings do not preclude the possibility that even more cortical regions, in addition to the motor cortex and bmSTS (Mathias et al., 2021), causally contribute to the translation of sensorimotor-enriched L2 words. The finding that only gesture-enriched words were influenced by TMS is also inconsistent with a potential placebo effect of TMS, as a placebo effect would induce a change in performance under effective compared with sham stimulation for both gesture- and picture-enriched words. Picture-enriched learning, rather than auditory-only learning, was selected to serve as a control for nonspecific TMS effects because picture enrichment has been shown to enhance learning outcomes in a manner similar to gesture enrichment, although this enhancement is likely because of a reliance on visual, rather than motor, cortices (Mayer et al., 2015).
L2-L1 translation likely involves multiple subprocesses, including the sensory processing of the L2 word, semantic and lexical retrieval, and L1 activation (Dijkstra and Rekké, 2010). It is an open question whether the motor cortices provide relatively greater or lesser contributions during some stages of processing compared with others. The motor cortex presumably also plays a causal role not only in the translation of gesture-enriched L2 words, but also in the learning of those words if the learning involves the performance of gestures. Drijvers et al. (2018), for example, showed that simply viewing iconic gestures engaged the hand area of the motor cortex when the gestures disambiguated incoming speech. However, viewing gestures without performing them has shown to be no more effective for L2 vocabulary learning than learning L2 words without enrichment (Mayer et al., 2015). Another open question is whether learning by viewing pictures that incorporate an action component might, like the performance of gestures, recruit the motor cortex during subsequent auditory L2 translation. Forty-three of the 45 complex scenes associated with abstract words in the current study depicted situations involving one or more person(s) doing something. For example, the word courage was depicted by a drawing of a person walking into a tiger's cage, and the word purpose was depicted by a drawing of a razor and alongside it a person using a razor to shave their facial hair. If the presence of action content (for abstract word pictures) versus absence of action content (for concrete word pictures) led to differential motor responses during the translation of picture-enriched L2 vocabulary, then we would have expected differential effects of TMS on abstract and concrete picture-enriched words. This was, however, not the case. Previous fMRI work showed that adding a motor component to pictures (i.e., by tracing the outline of the picture in the air with an index finger) during learning also did not lead to significant classification accuracy based on motor cortex responses (Mayer et al., 2015). Thus, current and previous findings suggest that the inclusion of a motor component in picture-enriched learning does not recruit the motor cortices to the same extent as gesture-enriched learning during subsequent auditory L2 translation.
In conclusion, behavioral performance in vocabulary translation following sensorimotor-enriched training is supported at least in part by representations in the motor cortex. The translation of recently acquired L2 words may therefore rely not only on auditory information stored in memory or modality-independent L2 representations, but also on the sensorimotor context in which the words have been experienced. The causal relation observed between motor cortex stimulation and behavioral performance contributes to the broader neuroscientific debate on the role of motor brain structures in human cognitive functions; specialized sensorimotor brain mechanisms contribute beneficially to sensorimotor-enriched learning.
Footnotes
This work was supported by German Research Foundation Grant KR 3735/3-1, a Max Planck Research Group Grant to K.v.K, and an Erasmus Mundus Postdoctoral Fellowship in Auditory Cognitive Neuroscience to B.M. B.M. was also supported by the ERC-Consolidator Grant SENSOCOM 647051 to K.v.K.
The authors declare no competing financial interests.
- Correspondence should be addressed to Brian Mathias at brian.mathias{at}tu-dresden.de
