Abstract
Context plays a key role in coding high-level components of others' behavior, including the goal and the intention of an observed action. However, little is known about its possible role in shaping lower levels of action processing, such as simulating action kinematics and muscular activity. Furthermore, there is no evidence regarding the time course and the neural mechanisms subserving this modulation. To address these issues, we combined single-pulse transcranial magnetic stimulation and motor-evoked potentials while healthy humans watched videos of everyday actions embedded in congruent, incongruent, or ambiguous contexts. Video endings were occluded from view and participants had to predict action unfolding. Transcranial magnetic stimulation was delivered at 80, 240, and 400 ms after action onset. An earlier selective facilitation of motor resonance occurring at 240 ms was observed for actions embedded in congruent contexts, compared with those occurring in incongruent and ambiguous ones. Later on, at 400 ms, a selective inhibition of motor resonance was found for actions embedded in incongruent contexts, compared with those taking place in congruent and ambiguous ones. No modulations were observed at 80 ms. Together, these findings indicate that motor resonance can be modulated by contextual information with different timings, depending on the (in)congruency between the different levels of action representation. Furthermore, the different time course of these effects suggests that they stem from partially independent mechanisms, with the early facilitation directly involving M1, and the later inhibition recruiting high-level structures outside the motor system.
SIGNIFICANCE STATEMENT Previous studies indicate that, when we observe other people's actions, the context in which actions take place influences intention understanding. However, little is known about the precise mechanisms involved in the contextual modulation of action representation (i.e., inhibition vs facilitation) and how they unfold in time. The present study sheds light on these aspects. Specifically, we show an early top-down facilitation (at ∼240 ms) and a later inhibition (at ∼400 ms) of motor resonance in response to actions observed in congruent and incongruent contexts, respectively.
Introduction
When observing someone performing an action, our brain does not perceive movements in isolation but rather in context. Single-pulse transcranial magnetic stimulation (spTMS) studies show that observing others' movements covertly facilitates the observers' motor system in a selective manner, mapping the effector that would be actually involved in the execution of the perceived movement (Fadiga et al., 2005; Alaerts et al., 2009; Urgesi et al., 2010). Most previous studies, however, have used movements detached from background as visual stimuli. While this approach allows controlling for the multiple factors that can contribute to action representation, it is blind to an intrinsic property of human cognition: its contextual and situated nature (Ibañez and Manes, 2012).
Predictive coding models of action processing (Kilner et al., 2007; Kilner, 2011) suggest that, when observing someone performing an action, our brain generates top-down predictions (i.e., intention-related expectations) to explain away the sensory input (i.e., perceptual kinematics). Interestingly, this approach also highlights the importance of context in shaping the predictive process, by proposing that contextual cues establish which intention is more likely to drive an action (Kilner et al., 2007; Press et al., 2011). Hence, action recognition might be easily achieved within those contexts that are congruent with the observed kinematics, since they should constrain expectations in a facilitatory way. Conversely, incongruent contexts might mislead expectations and thus interfere with action processing.
Neuroimaging studies suggest that contextual effects pervade high-level components of action, such as goal and intention understanding (Iacoboni et al., 2005; Wurm and Schubotz, 2012). Furthermore, even the representation of lower-level aspects, such as action kinematics and muscular activity, are not completely impervious to context. Indeed, the observation of actions embedded in incongruent contexts interferes with action representation in the observer's motor system (Amoruso and Urgesi, 2016). However, the possibility that both inhibitory and facilitatory mechanisms may coexist in contextual modulation of action representation and how they unfold in time are important issues that remain unexplored. Recent TMS studies have shown that top-down factors, such as newly learned visuomotor associations (Barchiesi and Cattaneo, 2013; Cavallo et al., 2014; Ubaldi et al., 2015), task instructions (Bardi et al., 2015), sequencing learning (Candidi et al., 2014), and the moral content of an intention (Liuzza et al., 2015), modulate motor facilitation only at later stages of action processing (∼200–300 ms after action onset). These results align with the proposal of a two-stage model of how action observation modulates corticospinal excitability (CSE) in humans (Naish et al., 2014). In this view, an early activation of the motor system that is automatic and not muscle-specific is followed by a later phase (∼200 ms onwards), which is muscle-specific and is thought to reflect a deeper level of action processing, sensitive to high-level factors.
Here, we aimed to track the time course of contextual modulation on muscle-specific motor facilitation responses. We delivered spTMS at different delays (80, 240, and 400 ms) after video onset while recording motor-evoked potentials (MEPs) from forearm [flexor carpi radialis (FCR)] and hand [first dorsal interosseus (FDI)] muscles. Videos were interrupted before action ending (when the hand closed to grasp the object) and participants were requested to predict action unfolding. Contextual information was manipulated so that observed kinematics could be congruent, incongruent, or ambiguous with respect to the context in which actions were embedded. Based on the predictive coding views of action processing and the two-stage model of motor facilitation, we predicted the following: (1) no contextual modulations at early phases of motor facilitation followed by (2) an enhancement of motor facilitation for the observation of movement kinematics embedded in congruent contexts and (3) a decrease of motor facilitation for movement kinematics embedded in incongruent contexts, compared with those occurring in ambiguous ones. Overall, based on the fact that the FDI muscle is specifically involved in the execution and observation of reaching-to-grasping movements and that the FCR is involved in wrist and arm movements (Fadiga et al., 1995), we expected a selective modulation only in the former muscle.
Materials and Methods
Participants
Fifteen individuals (10 women; mean age, 25.86 years; SEM = 2.06) recruited at the University of Udine took part in the experiment. The sample size required for our 2 * 3 * 3 (muscle * context * time) repeated-measures (RM) ANOVA design was determined using the G*power software (Faul et al., 2009), setting expected effects size at 0.25 based on previous studies (Amoruso and Urgesi, 2016), α-level at 0.05, and desired power (1 − β) at 80%. Participants were all right-handed according to the Standard Handedness Inventory (Briggs and Nebes, 1975), had normal or corrected-to-normal visual acuity, and were free from any contraindication to TMS (Rossi et al., 2009). They gave their written informed consent before experimentation and received course credits for their participation in the study. The experimental procedures were approved by the Ethics Committee of the Istituto di Ricovero e Cura a Carattere Scientifico Eugenio Medea and were carried out in accordance with the ethical standards of the 1964 Declaration of Helsinki. None of the participants reported a history of neurological, psychiatric, or other major medical problems. No discomfort or adverse effects during TMS and MEP acquisitions were reported or noticed.
Stimuli
Video clips were recorded with a Canon EOS 550D camera and were further edited with the Adobe Premiere Pro CS3 3.0 software so that they had an equivalent duration (video length was 400 ms, presented at a rate of 30 frames per second, for a total of 15 frames). All videos depicted a female model (aged 31 years), with only her torso and right arm and hand visible, performing everyday-life actions with the same effector (right hand). The actions involved the reach-to-grasp movement of seven different objects (a bottle, a cup, a spoon, a glass, a spray cleaner, a hammer, and a screw) that could be manipulated to perform either one of two possible actions. For example, in the case of the object “bottle,” the two potential actions were (1) to pour and (2) to place, and each action could be performed with their correspondent kinematics: reaching to grasp and pour using a whole-hand prehension grip, and reaching to grasp and lift using a precision grip, respectively (De Stefani et al., 2012).
In addition, actions were recorded in three different contextual settings: congruent, incongruent, and ambiguous. It is worth noting that contexts were not congruent or incongruent per se but with regards to action movement kinematics. In the congruent condition, the contextual constraints were in accordance with the observed kinematics. In keeping with the above-described example of pouring versus placing a bottle, the context congruent with the first action (i.e., pouring) depicted an empty glass, while the context congruent with the second action (i.e., placing) depicted a glass already filled with water. In the incongruent condition, the contextual information interfered with the perception of the observed kinematics by cueing to the opposite action. Specifically, the context that was incongruent with the action “pouring” depicted a glass already filled with water and the one that was incongruent with the action “placing” was an empty glass. Finally, in the ambiguous condition, both actions were equally suggested by the contextual constraints present in the environment (e.g., the glass was half full of water). Figure 1 illustrates the stimuli and conditions used in the study. For a complete description of objects, action labels, grip types, contexts, and their possible combinations, refer to Amoruso and Urgesi (2016). Video recording and manipulation, together, resulted in the creation of 42 video clips (14 actions embedded in three different contexts). Stimuli were validated in a previous study (Amoruso and Urgesi, 2016), which confirmed the appropriate manipulation of the plausibility of each action when embedded within congruent, ambiguous, and incongruent contexts (respectively, with parametrically decreasing levels of plausibility). It is also worth noting that this validation revealed that actions were less likely to occur in incongruent than in congruent and ambiguous contexts, but were not impossible or necessarily erroneous.
A, Stimuli. Fourteen actions were used in the experiment. Depending on the action, the reach-to-grasp movement kinematics suggested a precision or a whole-hand grip (e.g., Kinematics A: precision grip use for drinking; Kinematics B: whole-hand grip use for cleaning). B, Experimental conditions. Actions could be performed in three different contexts: congruent, incongruent, and ambiguous. C, Event-related design of the TMS experiment. In a two-alternative forced-choice task, participants watched the videos and predicted which action the model was going to perform. Before making their predictions, participants were requested to pay attention to the kinematic information of the model's hand movements and the contextual information in which these movements were embedded. Each trial began with a frame with the word “attention” on the screen lasting 5000 ms, followed by the video clip, which lasted 400 ms. A single TMS pulse was delivered at three different time delays: 80, 240, and 400 ms after video onset. After the video clip, a frame with the verbal descriptors of two possible goals (e.g., “to drink,” “to clean”; one located up and the other located down) was presented. This frame remained on the screen until a response was recorded.
Task
In a two-alternative forced-choice task, participants were requested to watch the video clips and predict the unfolding of the observed action. A temporal occlusion paradigm was used, with the video clip being stopped two frames before the model made contact with the object. Note that, since actions were stopped as the hand closed to grasp the object, participants observed the preshaping of the hand configuration during the reaching-to-grasp phase of the movement and not the grasping movement itself. Before being asked to make predictions, participants were instructed to carefully pay attention to both sources of information: the kinematic information of the model's hand movements and the contextual information in which these movements were embedded. The inclusion of ambiguous trials, in which context was not informative on how the observed action would unfold, prevented participants from focusing their attention only on the contextual cues when giving their responses.
Electromyography recording and TMS
spTMS was applied to the left M1 using a Magstim 200 stimulator (maximum output, 2 T at coil surface; pulse duration, 250 μs; rise time, 60 μs; Magstim) connected to a 70 mm figure-of-eight coil (Magstim polyurethane-coated coil). MEPs were recorded simultaneously from the FDI and from the FCR of the right hand. While both muscles are involved in reaching to grasp movements, only the FDI, but not the FCR, is involved in determining the grasping kinematics that differentiates the two alternative actions in each pair (Fadiga et al., 1995). Surface Ag/AgCl disposable electrodes (1 cm diameter) were placed in a belly-tendon montage for each muscle. The electromyographic (EMG) signal was amplified, filtered (bandpass 5 Hz to 20 kHz), and recorded with the Biopac MP-36 system (Biopac Systems) at a sampling rate of 50 kHz.
The coil was positioned tangentially on the scalp, with the handle pointing backward and ∼45° lateral from the midline, perpendicular to the line of the central sulcus (Di Lazzaro et al., 1998). This orientation was chosen based on the finding that the lowest motor threshold is achieved when the induced electric current in the brain is flowing perpendicular to the central sulcus (Brasil-Neto et al., 1992; Mills et al., 1992). The optimal scalp position for inducing MEPs in the right FDI and FCR muscles was detected by moving the coil in 1 cm steps over the left M1 and by delivering TMS pulses at constant intensity until the largest MEPs for both muscles were found. Then, the position was marked with a pen on a tight-fitting bathing cap worn by the participants. The coil was held on the scalp by a coil holder with an articulated arm, and its position with respect to the mark was checked continuously to compensate for small movements of the participants' head during data collection.
The TMS intensity during the recording sessions was adjusted to 120% of the motor threshold at rest (rMT), which was defined as the minimum intensity able to evoke MEPs with ≥50 μV peak-to-peak amplitude in the lowest threshold muscle (namely, the FDI), in 5 of 10 consecutive pulses. The rMT ranged from 41 to 68% (mean, 50.4%; SD = 8.39%) of the maximum stimulator output. To ensure that there was no unwanted background EMG activity before the magnetic pulse, the signal from both muscles was continuously monitored, and when voluntary contractions were detected, participants were encouraged to fully relax their muscles. Furthermore, during the preliminary session, EMG signals were sent to loudspeakers to provide participants with an auditory feedback of their muscle activity. MEPs' peak-to-peak amplitudes (in millivolts) were collected and stored in a computer for off-line analysis.
Procedure
Each participant was tested in a single experimental session lasting ∼60 min. They each sat in a comfortable armchair in a dimly lit room ∼1 m away from a 21 inch CRT monitor (resolution, 1024 × 768 pixels; refresh frequency, 75 Hz) with prone hands resting on a pillow. They were instructed to pay attention to the displayed stimuli and to avoid moving their right hand. Before starting the experiment, participants received acoustic (via loudspeakers) and visual (via oscilloscope) feedback of their muscle activity to help them to get relaxed. Videos appeared at the center of the screen on a neutral background (subtending ∼15.96° × 11.97° of visual angle).
Before video presentation, baseline CSE was assessed by acquiring 10 MEPs while participants passively watched a fixation cross. Another series of 10 MEPs were recorded at the end of the experimental session. Trials started with a visual warning cue lasting for 5000 ms (the word “attention”) and it was followed by the video-clip presentation lasting 400 ms. It is worth noting that, in a previous study (Amoruso and Urgesi, 2016), we used a similar paradigm to investigate contextual modulations on motor resonance responses. In that study, the focus was not on the timing of these modulations and, hence, video onset was not matched with movement onset, so that participants had a reliable prior exposure to context (∼400 ms) before the action began. Here, we adopted a different strategy. Since our aim was to track the time course of these modulations, video and action onset were matched so that participants perceived both sources of information (contextual cues and action kinematics) at the same time. This allowed us to determine “when” both sources of information actually interact. After the video clip, a frame was presented with the verbal descriptors of the two possible goals (e.g., “to pour,” “to place”; one located up and the other located down) written in black on a white background. This frame remained on the screen until a response was recorded. Participants were requested to verbalize their response (by saying “up” or “down”) and the experimenter recorded the answer by pressing a computer key. The location of the two descriptors was counterbalanced, ensuring that in half of the trials one of the descriptors was presented up and, in the other half, it was presented down. This procedure enabled us prevent participants from planning their response in advance on the basis of the descriptors' location. Verbal responses were used to prevent peripheral muscular contraction artifacts resulting from button press to contaminate the MEPs. Importantly, verbal responses were required only after the TMS pulse was delivered, thus reducing the possibility of interfering with the hand MEPs (Gentilucci et al., 2006). The interval between TMS pulses was at least 7500 ms to avoid cumulative effects of the stimulation. A total of 42 video clips (14 actions embedded in three different contexts) were created. Each action video was presented four times. This resulted in a total of 168 stimuli randomly presented in four blocks of 42 trials each. A brief break was allowed between blocks.
After running the TMS session, participants performed a behavioral session using the same procedure and visual stimuli as in the previous one but without TMS. During the behavioral session, participants were requested to observe the videos and predict the unfolding of the observed actions by providing button-press responses. However, while videos in the TMS task were always presented fully (400 ms), videos in the post-TMS behavioral task were occluded according to the timing at which TMS pulses were delivered in the original task; therefore, they had different lengths and could last 80, 240, or 400 ms, respectively. This was to rule out the possibility of response patterns being affected by the TMS procedure.
Stimulus-presentation timing, EMG recording, and TMS triggering, as well as randomization of stimuli in a block, were controlled using E-prime V2 software (Psychology Software Tools) running on a PC.
Kinematic analysis of the stimuli
To ensure that the observed CSE modulations were actually due to the manipulation of the contextual information and not to differences in the kinematic profiles of the same action across scenes, we performed a kinematic analysis of the visual stimuli (Table 1; Aglioti et al., 2008; Makris and Urgesi, 2015). Specifically, we checked, for each action pair, that movement-kinematics differences remained the same across contexts (ambiguous, congruent, and incongruent) by conducting a frame-by-frame joint-angle analysis in seven frames of interest in each video. The definition of the joint-angle profile was performed using dedicated software for motion analysis (Dartfish Connect v. 4.0, Dartfish). For each frame, we defined the amplitude of two joint angles on the right hand of the model accounting for the activity of the two muscles of interest: one index finger flexion angle associated with the activity of the FDI muscle and one wrist flexion angle associated with the activity of the FCR muscle (Figure 2, black line, FDI; white line, FCR). The raw angle values of the 14 actions were submitted to an RM ANOVA with time frame (F1: 80 ms; F2: 240 ms; F3: 293 ms; F4: 319 ms; F5: 346 ms; F6: 373 ms; F7: 400 ms), angle (index finger flexion, wrist flexion), and context (congruent, incongruent, and ambiguous) as within-videos variables. To specifically test whether the discriminability of the two alternative actions in each pair changed according to context, for each of the seven pairs of actions we performed an additional analysis subtracting the angles obtained for each kinematics (|A − B|) and comparing them by series of nonparametric Friedman ANOVAs on ranks across time. Nonparametric testing was used for this analysis considering the low number of cases (<10).
Kinematics analysisa
Data analysis
Behavioral analyses.
Individual performance values expressed as percentage of correct responses and as d′, a bias-corrected measure of sensitivity in discriminating between two categories (Macmillan and Kaplan, 1985), were estimated for each experimental condition, in both the TMS and the behavioral sessions (Table 2). In the d′ analysis, videos with “kinematics A” identified as “kinematics A” were considered “hits” and videos with “kinematics B” identified as “kinematics A” were considered “false alarms.” More specifically, correct responses were defined by the kinematics, not by the context (e.g., when the participant observed a whole-hand grip of the bottle in a context where the glass is full, “pour” was the correct answer). The d′ values were calculated by transforming the response proportion to z-scores, and then subtracting the z-score that corresponds to the false-alarm rate from the z-score that corresponds to the hit rate (Stanislaw and Todorov, 1999). Furthermore, we calculated a measure of response criterion (c) for each participant and condition, which reflects the existence of a bias in providing a specific response. The c values were calculated by averaging the z-score corresponding to the hit rate and the z-score corresponding to the false-alarm rate, and then multiplying the result by −1 (Stanislaw and Todorov, 1999). The d′ and c values were subjected to separate RM ANOVAs with time (80, 240, 400 ms) and context (congruent, incongruent, ambiguous) as within-subjects variables.
Participants' performance in action recognitiona
Finally, to check whether overall recognition performance changed according to the specific kinematics of the grasping movements, we performed an item-wise analysis (Clark, 1973) by averaging across subjects the percentage of correct responses for each action video. These data were submitted to an RM ANOVA with grip type (precision, whole-hand) as a between-item factor and context (ambiguous, congruent, incongruent) and time (80, 240, 400 ms) as within-item factors.
MEP analyses.
Individual mean peak-to-peak amplitudes of MEPs recorded from the FDI and FCR muscles were calculated separately for each condition. EMG activity was visually monitored during the experiment to ensure full muscle relaxation. Since background EMG is known to modulate the MEP amplitude, pre-TMS EMG was assessed in each participant by calculating the mean rectified signal across a 150 ms interval before TMS. MEPs with preceding background EMG deviating from the mean by >2 SD were removed from further analysis. Trials were also excluded if the peak-to-peak amplitude of the MEP was ±2 SDs beyond the participant's mean calculated in each experimental condition. The total percentage of excluded MEPs (FDI, 80 ms: congruent, 8.77%; incongruent, 7.36; ambiguous, 9.47%; FDI, 240 ms: congruent, 11.92%; incongruent, 7.01%; ambiguous, 8.42%; FDI, 400 ms: congruent, 12.98%; incongruent, 7.71%; ambiguous, 11.92%; FCR, 80 ms: congruent, 8.77%; incongruent, 10.87%; ambiguous, 8.42%; FCR, 240 ms: congruent, 11.57%; incongruent, 6.66%; ambiguous, 9.82%; FCR, 400 ms: congruent, 11.92%; incongruent, 9.12%; ambiguous, 13.33%) was not significantly different across conditions and muscles (all F's < 2.42; all P's > 0.10).
The individual mean MEP amplitudes during the experimental task were normalized by using the following formula: (Condition MEP − Baseline MEP)/Baseline MEP * 100%. Normalized data were submitted to an RM ANOVA with muscle (FDI, FCR), context (congruent, incongruent, ambiguous), and time (80, 240, 400 ms) as within-subjects variables. In addition, we performed a supplementary RM ANOVA that included the factor grip (precision, whole-hand) to test for a differential excitability of the FDI and FCR muscles for the two types of grasping movements.
Estimates of the effect size were obtained using the partial η-squared and Cohen's d (Cohen, 1988). Post hoc pairwise comparisons were carried out using the Newman–Keuls test. The α value for all statistical tests was set at 0.05. The analyses were implemented in Statistica software (Statsoft).
Results
Kinematics results
The RM ANOVA on angle values showed neither significant main effect of context (F(2,26) = 0.119, p = 0.88) nor significant interactions between context, time, and angle (all F's < 2.235, all P's > 0.06), indicating that the kinematic profile of each action did not vary across the different contextual settings (ambiguous, congruent, incongruent). In addition, when testing the angle differences between paired actions, none of the Friedman ANOVAs yielded significance (index finger flexion: all χ2 < 6.33, all p's > 0.18; wrist flexion: all χ2 < 2.90, all p's > 0.23), ensuring that differences between the two actions in each pair remained comparable across context types (Fig. 2).
Joint-angle differences (expressed in absolute values, |A-B|°) between actions across contexts (congruent, incongruent, ambiguous) for the index finger flexion (top) and the wrist flexion (bottom). Error bars represent SEM. The video-clip frames below the graph show examples of kinematic measurements; black and white lines denote the measured joint angles.
Behavioral results
Figure 3 shows participants' performance in each context and time point for the behavioral session carried out at the end of the experiment and for the TMS session. The RM ANOVA performed on the d′ values obtained during the behavioral session yielded a main effect of context (F(2,28) = 31.46, p < 0.001, ηp2 = 0.69), nonsignificant effect of time (p = 0.59), and a significant context-by-time interaction (F(4,56) = 5.331, p = 0.001, ηp2 = 0.27). Post hoc comparisons [mean square error (MSE), 0.089; df = 56] performed on the interaction indicated that all context conditions differed (Congruent > Ambiguous > Incongruent) at all time points (all p's < 0.001). When considering conditions over time, congruent (80 ms: mean, 1.24; SEM = 0.09; 240 ms: mean, 1.45, SEM = 0.08; 400 ms: mean, 1.37; SEM = 0.1) and ambiguous actions (80 ms: mean, 0.72; SEM = 0.12; 240 ms: mean, 0.66; SEM = 0.14; 400 ms: mean, 0.85; SEM = 0.15) were similarly recognized within the different time windows (all p's > 0.14). However, for those actions embedded in incongruent contexts (80 ms: mean, −0.08; SEM = 0.2; 240 ms: mean, −0.16; SEM = 0.24; 400 ms: mean, −0.49; SEM = 0.26), a decrease in action recognition performance was observed at the longest viewing time (80 vs 400 ms, p = 0.001; 240 vs 400 ms, p = 0.003).
A, B, Participants' performance in predicting the course of the observed actions (expressed as d′) when these were embedded in congruent, incongruent, or ambiguous contexts at the different time points and response criterion (c) for each type of context, within the behavioral (A) and the TMS (B) sessions. Asterisks indicate significant comparison (p < 0.05). Error bars represent SEM.
A similar pattern of results was obtained for the TMS session, with significant effects of context (F(2,28) = 23.72, p < 0.0001, ηp2 = 0.62) and of the interaction between time and context (F(4,56) = 8.03, p < 0.0001, ηp2 = 0.36), but a nonsignificant trend for time (p = 0.06). Again, post hoc comparisons (MSE, 0.071; df = 56) showed that all context conditions differed (Congruent > Ambiguous > Incongruent) at all time points (all p's < 0.001). The only comparison that did not reach the significance threshold was that between congruent and ambiguous contexts at 80 ms (p = 0.22).
Finally, the analysis performed on the response criterion (c) revealed neither main effect of context or time, nor an interaction between the two factors for the responses provided either during the behavioral session (all F's < 1.84, all p's > 0.13) or the TMS session (all F's < 2.17, all p's > 0.13). Thus, the TMS procedure did not interfere with the pattern of responses obtained in the task. However, the fact that time interacted with context in affecting performance also during the TMS session, in which videos were not occluded and time corresponded to TMS delay, suggests that the peripheral effects of TMS (e.g., blinking) might have interfered with viewing the part of videos after the pulse. Although this modulation by time in the TMS session may be due to spurious effects of peripheral TMS effects (and we did not explore it further), it is possible that the responses obtained during TMS are likely to reflect the processing of the initial part of videos before the TMS pulse.
Grip analysis
The item-wise analysis showed a main effect of context in both the behavioral (F(2,24) = 49.9, p < 0.0001, ηp2 = 0.8) and the TMS session (F(2,24) = 31.06, p < 0.0001, ηp2 = 0.72), suggesting that actions were differently recognized, depending on the context in which they were embedded (Congruent > Ambiguous > Incongruent; all p's < 0.05). No other effects were observed in the TMS session; however, in the behavioral one, a context-by-grip interaction was found (F(2,24) = 5.39, p = 0.01, ηp2 = 0.31). Post hoc comparisons on the interaction indicated that precision and finger grips differed only in the ambiguous condition (p = 0.008). This effect might reflect that, at least when context is neutral, it is easier to detect a precision grip than a whole-hand grip since the former deviates more and earlier from the hand starting position compared with the latter. Importantly, no differences were observed for the two types of grip in the congruent and the incongruent conditions (all p's > 0.48). Overall, these results suggest that both grips were similarly recognized across time points and that context congruency affected their recognition in a comparable fashion. Table 3 depicts the percentage of correct responses for precision and whole-hand grips within each context and time point in the TMS and the behavioral sessions.
Grip analysisa
MEP results
The raw mean amplitudes of MEPs recorded from the FDI and the FCR muscles in the three observation conditions at the different time points are reported in Table 4.
Mean raw MEP amplitudes for each observation conditiona
The ANOVA performed on the mean raw MEP amplitudes recorded during the two baseline blocks run at the beginning (FDI: mean, 0.230; SEM = 0.03; FCR: mean, 0.269; SEM = 0.06) and at the end (FDI: mean, 0.239; SEM = 0.03; FCR: mean, 0.215; SEM = 0.04) of the experimental session revealed no main effects of muscle (F(1,14) = 0.027, p = 0.87) or block (F(1,14) = 0.552, p = 0.46), and a nonsignificant trend for their interaction (F(1,14) = 4.093, p = 0.06), which indicates that, for the FCR muscle, MEPs recorded during the first block tended to be slightly, but nonsignificantly, higher than those recorded during the second one. Together, these results indicate that MEPs recorded from the two muscles were comparable and that TMS per se did not induce any changes in CSE during the experimental session. Thus, the MEPs collected in the two baseline conditions were collapsed and used to normalize MEPs during the action observation trials.
To test how the intensity of motor facilitation of the two muscles was modulated by context, we entered normalized MEP amplitudes into a 2 * 3 * 3 (muscle * context * time) RM ANOVA. A significant intercept (F(1,14) = 39.24, p < 0.0001, ηp2 = 0.73) was found, ensuring that the overall motor facilitation index was different from zero. Furthermore, a one-sample t test (one-tailed) revealed that normalized MEPs were >0% in all conditions (all t's > 2.25, all p's < 0.04), thus showing reliable motor facilitation responses recorded from both muscles at all time points independently of context congruency.
When testing the single ANOVA effects, results revealed main effects of context (F(2,28) = 22.102, p < 0.0001, ηp2 = 0.61) and time (F(2,28) = 8.897, p = 0.001, ηp2 = 0.38), and significant time-by-muscle (F(2,28) = 3.856, p = 0.03, ηp2 = 0.21), time-by-context (F(2,28) = 3.796, p = 0.008, ηp2 = 0.21), and muscle-by-context (F(2,28) = 8.417, p = 0.001, ηp2 = 0.37) interactions, which were further qualified by a significant three-way interaction of all factors (F(4,56) = 2.679, p = 0.04, ηp2 = 0.16). Thus, the omnibus ANOVA showed that the pattern of contextual modulation on the muscle-specific facilitation of the motor cortex during action observation was different at the three time points at which it was recorded (Fig. 4).
Amplitudes of MEPs recorded from the FDI and FCR muscles during the three action observation conditions (Congruent, Incongruent, and Ambiguous) at the three time points (80, 240, and 400 ms) are expressed as value percentages (%) of the MEPs recorded during baseline. Asterisks indicate significant comparison (p < 0.05). Error bars represent SEM.
Post hoc comparisons (MSE, 3696.4; df = 56) performed on the three-way interaction showed that, at 80 ms, no differences across conditions were observed for either the FDI or the FCR muscles (all p's > 0.08). However, at 240 ms, motor facilitation for the FDI was enhanced for actions observed within a congruent context (mean, 273%; SEM = 55.75%) compared with those observed in ambiguous (mean, 177.35%; SEM = 44.11%; p < 0.001; Cohen's d = 0.51) and incongruent ones (mean, 118.%; SEM = 29.89%, p < 0.001; Cohen's d = 0.92), which in turn did not differ (p = 0.16; Cohen's d = 0.41). No modulations were observed for the FCR muscle (all p's > 0.43). Finally, at 400 ms, an inhibition of motor facilitation was observed for the FDI, with lower normalized MEP amplitudes for actions observed within an incongruent context (mean, 79.8%; SEM = 35.31%) compared with those observed within congruent (mean, 195.4%; SEM = 46.95%; p < 0.001; Cohen's d = 0.74) and ambiguous (mean, 169.9%; SEM = 38.19%; p = 0.006; Cohen's d = 0.65) contexts, which in turn did not differ (p = 0.77; Cohen's d = 0.16). No modulations were again observed for the FCR muscle (all p's > 0.32).
Grip analysis
To test whether the contextual modulation of motor resonance for the FDI muscle was different for precision versus whole-hand grips, we performed a complementary analysis that included the factor grip. The analysis for the FDI muscle yielded significant main effects of context (F(2,28) = 18.68, p = 0.00001), grip (F(1,14) = 5.13, p = 0.03), and time (F(2,28) = 11.54, p = 0.0002), and a significant grip-by-time interaction (F(2,28) = 3.9452, p = 0.03). Post hoc comparisons on the interaction (MSE, 5751.3; df = 28) indicated that, regardless of context, the two types of grip elicited a differential CSE within the last time window (400 ms), with a decrease in MEP amplitudes for precision grip (mean, 131.79; SEM = 34.92) compared with whole-hand grip actions (mean, 177.33; SEM = 49.27; p = 0.008). In addition, the CSE associated with the precision grip in the 400 ms window was lower compared with all the other conditions (all p's < 0.01). Notably, the last time window of the video clips depicted the closing phase of the movement, when the hand aperture was minimal. It has been previously shown that mirror-like motor facilitation of the FDI muscle decreases during observation of the end posture of an action, where the hand has maximal finger closure (Urgesi et al., 2006, 2010). These findings might reflect this phenomenon, showing that the time course of motor activation triggered by action observation parallels the dynamics of movement execution (Gangitano et al., 2001, 2004; Montagna et al., 2005). This interpretation is also supported by the fact that no main effects or interactions involving the factor grip were observed in the analysis performed on the FCR muscle (all F's < 1.72, all P's > 0.15). More importantly for the purpose of the present study, context comparably affected motor resonance during observation of precision and whole-hand grips.
Discussion
The aim of this study was to investigate the time course of facilitatory and/or inhibitory mechanisms underlying the contextual modulation of an observer's representation of others' action kinematics. We tested motor resonance and tracked the time course of its modulation while participants observed occluded videos of everyday actions embedded in congruent, incongruent, or ambiguous contexts and predicted their unfolding. Behaviorally, we found that congruent or incongruent contexts improved or impaired, respectively, accuracy in action prediction compared with ambiguous contexts. Interestingly, both effects were mirrored in a muscle-specific fashion during action observation but with different timings. Specifically, at 240 ms after video onset, we observed a selective facilitation of motor resonance (higher MEP amplitudes) for actions embedded in congruent contexts, compared with those occurring in incongruent and ambiguous ones. Later on, at 400 ms, we found a selective inhibition of motor resonance (lower MEP amplitudes) for actions embedded in incongruent contexts, compared with those taking place in congruent and ambiguous ones. No modulations were observed at 80 ms. Importantly, both effects were found in a time window (>240 ms) where, we can assume, contextual information had already been fully processed (Biederman et al., 1974; Thorpe et al., 1996; Bar et al., 2006; Kveraga et al., 2007).
In all action observation conditions, CSE was higher than baseline, suggesting that incongruent contexts reduce the extent of motor facilitation but do not suppress it completely. According to the two-stage model of motor facilitation (Naish et al., 2014), the muscle specificity and late timing of the contextual effects suggest that they reflect high-level modulation of specific action representation in M1 rather than nonspecific motor activation. Furthermore, the different time course of facilitatory and inhibitory effects suggests they may at least in part stem from independent mechanisms.
Converging evidence indicates that the brain is a proactive organ that benefits from context to predict future behaviors (Bar, 2004; Amoruso et al., 2014). From a predictive coding perspective (Kilner et al., 2007; Kilner, 2011), given an expectation about the intention of the person we are observing, we can use our own motor system as a model to compute sensory input (i.e., kinematics). The comparison between the predicted and the observed kinematics generates an error signal, which is sent back up to higher levels and can be used to adjust the model. By minimizing the error signal at the different levels of action representation, the most likely cause of the observed action can be inferred. But where is the prior intention generated? Interestingly, this perspective suggests that intentions are estimated from contextual information in higher areas outside the mirror neuron system (MNS; Press et al., 2011).
On this view, it could be that, during our experimental task, the prior intention was estimated at a contextual level and modulated MNS activity and action prediction in a top-down fashion. Indeed, the behavioral pattern of results obtained in the incongruent condition indicates that participants were more inclined to decide on the basis of the intention motivated by the context. The top-down signal conveying information about the most probable intention, as estimated by context processing, modulated M1 activity in distinct ways, depending on time and information congruency. Interestingly, at 240 ms, the matching between the predicted and the observed kinematics resulted in a facilitation of motor resonance and, later on, at 400 ms, the mismatching between them resulted in an inhibition. Notably, these modulations were specifically observed for the FDI muscle, which is known to be involved in reaching-to-grasp movement observation and execution (Fadiga et al., 1995) and is differently activated for precision and whole-hand grips; no modulation was obtained for the FCR, which is involved in wrist movements (Borroni et al., 2005; Tidoni et al., 2013) and is comparably activated in the two types of grip.
The distinct timing of facilitatory and inhibitory processes suggests that they stem, at least partially, from different mechanisms. On the one hand, the early muscle-specific facilitatory effect may reflect the convergence in M1 of two matching predictions: those based on the context and those based on the observed kinematics. The timing of this effect aligns with previous spTMS studies showing not only that muscle-specificity arises from 200 ms after action onset (Naish et al., 2014; Catmur, 2015), but also that it's sensitive to high-level factors, such as movement correctness (Candidi et al., 2014), associative learning (Cavallo et al., 2014), and object-related affordances (Villiger et al., 2011) at this latency and not before. In line with these studies, we show that mirror-like responses are sensitive to high-level contextual representations and are facilitated in a muscle-specific fashion at 240 ms, supporting the notion that motor resonance is not an entirely automatic process.
On the other hand, the inhibitory muscle-specific effect might reflect the mismatch between the two predictions and the correspondent “muting” of action representation in the motor system. In keeping with the predictive-coding view (Kilner et al., 2007), the motor system becomes active during action observation because it is offering the best model of the observed action. In a similar vein, it could be that, under conflicting situations, the motor system stops its mirroring because it is not offering the best model of the observed action anymore. In addition, other brain regions (i.e., mentalizing areas) might be called into play to solve the conflict.
A key question here concerns whether MEP modulations actually reflect prediction errors or prediction signals. A classic signature of predictive coding is the observation of reduced activity to predictable stimuli (Kilner et al., 2007; Kilner, 2011). Hence, if MEP modulations reflect prediction error, one might expect higher amplitudes in the mismatching condition (incongruent context) and lower amplitudes in the matching one (congruent context). Some previous studies align with this view, reporting higher MEP amplitudes in response to the observation of erroneous (unexpected) actions compared with correct ones (Aglioti et al., 2008; Tidoni et al., 2013; Candidi et al., 2014). However, it is worth noting that movement kinematics in our experiment were not erroneous per se, but rather incompatible with the intention predicted from the context. Indeed, other studies that have compared predicted versus observed kinematics found comparable patterns of motor responses as those found here. For instance, Senot et al. (2011) showed that inducing a mismatch between actual and expected kinematics by introducing a high-level cue (i.e., a semantic label) generates an inhibition of the corticospinal system. Similarly, Janssen et al. (2015) found that, when there is a mismatch between an action specified by a prior symbolic cue (i.e., an arrow indicating a particular grip) and the observed kinematics, motor resonance is attenuated. Furthermore, previously (Amoruso and Urgesi, 2016) we found a similar suppression of CSE during the observation of actions embedded in incongruent contexts. Current results extend these previous findings by describing the direction (facilitation vs inhibition) and the timing (early vs late) of motor resonance modulation.
A more parsimonious explanation can be found by moving beyond this dichotomy: it could be that both prediction and prediction errors can be captured with the MEP methodology depending on the timing and task demands. Indeed, M1 can be conceived as a recipient of processes occurring elsewhere and, hence, MEPs would be a measure of input into rather than output from M1 (Bestmann and Krakauer, 2015). If what we assess in M1 reflects connectivity with other areas, these areas may interface and transiently affect M1 activity with different timings (Borgomaneri et al., 2014, 2015). In line with this idea, Borgomaneri et al. (2015) found that, when probed at 150 ms, M1 activity reflected the emotional content of body actions, with inhibition for emotional versus neutral actions. However, at 300 ms, such inhibition was not present anymore, and rather M1 reflected the dynamicity of the actions, with higher activity for emotional and neutral actions compared with static bodies. Similarly, Ubaldi et al. (2015) found that early activity of M1 (∼150 ms) reflected mirror properties independently from a counter-mirror training, and only later on (∼300 ms) counter-mirror responses were noticeable. Altogether, these findings reflect the transient modulation of M1 activity, which might be explained by signal modulation from processes occurring elsewhere.
Although the present spTMS study is not informative regarding the specific neural substrates of the observed effects, we can speculate that top-down contextual signals might have originated in the prefrontal cortex (PFC) and exerted indirect modulations on M1. The PFC has a role in rapidly encoding contextual information (Kveraga et al., 2007) and output from this region targets specific premotor areas (Lu et al., 1994) that represent hand-action kinematics and are strongly connected to M1 (Prabhu et al., 2009). In addition, the premotor cortex is known to be involved in the facilitation and inhibition of motor responses in M1 (Koch et al., 2007; Buch et al., 2010). Therefore, it could be that M1 responses were gathered by inputs received from premotor areas, which were, in turn, modulated by top-down signals coming from the PFC.
In conclusion, we showed that motor resonance can be modulated by context-based expectations with different timings, depending on the (in)congruency of the information conveyed by the predicted and the observed kinematics. The different time course of facilitatory and inhibitory effects suggests that they stem from partially independent mechanisms involving different cortical areas. Further studies are necessary to understand the neural signatures of contextual regulation of action coding, giving special attention to where and how prior intentions are generated.
Footnotes
The research was supported by grants from the European Commission (MCSA-H2020-NBUCA, Grant No. 656881), the Ministero Istruzione Università e Ricerca (Futuro In Ricerca, FIR 2012, Prot. N. RBFR12F0BD; to C.U.), and from Istituto di Ricovero e Cura a Carattere Scientifico Eugenio Medea (Ricerca Corrente 2014, Ministero Italiano della Salute; to C.U.).
The authors declare no competing financial interests.
- Correspondence should be addressed to Lucia Amoruso, PhD, Laboratory of Cognitive Neuroscience, University of Udine, Via Margreth, 3, I-33100 Udine, Italy. lucia.amoruso{at}uniud.it
