Abstract
The activity of mirror neurons in macaque ventral premotor cortex (PMv) and primary motor cortex (M1) is modulated by the observation of another's movements. This modulation could underpin well documented changes in EEG/MEG activity indicating the existence of a mirror neuron system in humans. Because the local field potential (LFP) represents an important link between macaque single neuron and human noninvasive studies, we focused on mirror properties of intracortical LFPs recorded in the PMv and M1 hand regions in two macaques while they reached, grasped and held different objects, or observed the same actions performed by an experimenter. Upper limb EMGs were recorded to control for covert muscle activity during observation.
The movement-related potential (MRP), investigated as intracortical low-frequency LFP activity (<9 Hz), was modulated in both M1 and PMv, not only during action execution but also during action observation. Moreover, the temporal LFP modulations during execution and observation were highly correlated in both cortical areas. Beta power in both PMv and M1 was clearly modulated in both conditions. Although the MRP was detected only during dynamic periods of the task (reach/grasp/release), beta decreased during dynamic and increased during static periods (hold).
Comparison of LFPs for different grasps provided evidence for partially nonoverlapping networks being active during execution and observation, which might be related to different inputs to motor areas during these conditions. We found substantial information about grasp in the MRP corroborating its suitability for brain–machine interfaces, although information about grasp was generally low during action observation.
Introduction
The observation of another's movements influences the activity of some neurons in cortical motor areas. This special class of mirror neurons modulate their firing activity during both action execution and observation. First discovered in macaque ventral premotor cortex (PMv), area F5 (Gallese et al., 1996; Rizzolatti et al., 1996), more recent reports indicate that they also exist in the primary motor cortex (M1; Tkach et al., 2007; Dushanova and Donoghue, 2010). We now know that even corticospinal neurons in PMv and M1 can exhibit mirror properties (Kraskov et al., 2009; Vigneswaran et al., 2013). Mirror neurons have been implicated in a variety of functions, including action understanding and motor imagery (Rizzolatti et al., 2001; Cisek and Kalaska, 2004; Raos et al., 2007).
Although studies using noninvasive recordings have been successful in revealing mirror-like neuronal activity in the human brain, the identity and function of a mirror neuron system in humans is still unclear. This is mainly due to the difficulty of relating invasive mirror studies using single neuron recording in monkeys to noninvasive mirror studies using EEG/MEG in humans. We focused on the mirror properties of the intracortical local field potential (LFP), reflecting superimposed neuronal activity (Buzsáki et al., 2012), as this signal represents an important link between these two approaches to the mirror neuron system.
Since the first evidence of EEG modulation by action observation (Faure and Cohen-Seat, 1954; Gastaut and Bert, 1954), mirror-like neuronal activity in human motor areas has been confirmed indirectly using fMRI (Buccino et al., 2001; Gazzola and Keysers, 2009), functional near-infrared spectroscopy (Koehler et al., 2012), and TMS (Fadiga et al., 1995). Although one study has reported mirror activity of single neurons in humans (Mukamel et al., 2010), human mirror studies mainly used extracortical electrophysiological techniques and commonly reported an amplitude decrease in mu and beta frequencies during action observation (Hari et al., 1998; Tremblay et al., 2004; Silas et al., 2010; Quandt et al., 2012). These modulations can vary with kinematics (Press et al., 2011; Avanzini et al., 2012) or type of hand movements (Streltsova et al., 2010) and might be further accompanied by increased activity in the gamma (Perry et al., 2010; Wriessnegger et al., 2013) and decreased activity in the theta band (Frenkel-Toledo et al., 2013). However, precise spatial localization or a direct comparison of different motor areas is inherently challenging using noninvasive recordings.
Previous studies have shown that the intracortical LFP is modulated during observation in the beta range in monkey M1 (Tkach et al., 2007) and PMv (Kilner et al., 2014), in the delta/beta/gamma range in the human STN (Alegre et al., 2010) or, if trial-averaged and filtered 1–100 Hz, in monkey F5 (Caggiano et al., 2013).
In this study, we analyzed the intracortical LFP recorded in both PMv and M1 in two monkeys during action execution and observation of different grasping movements. Simultaneously with the LFP, we sampled the electromyogram of digit/hand/arm muscles to control for covert muscle activity during observation. LFPs up to 100 Hz, with a focus on low-pass filtered neuronal activity, were investigated and compared across different grasps and areas.
Materials and Methods
Subjects
All experimental procedures were approved by the Local Ethical Procedures committee and performed in accordance with the UK Animals (Scientific Procedures) Act. Experiments involved two purpose-bred, male, adult Macaca mulatta (M46, 7.8 kg; M47, 4.9 kg). The monkeys were housed in a unit with other Rhesus monkeys, which had natural light and access to an exercise pen and forage area. Monkeys were pair-housed at all times, apart from brief periods after surgeries. Both monkeys gained weight regularly throughout the procedure.
Task
The same task as described by Vigneswaran et al., 2013 was used. Three different objects (ring, sphere, precision grip) were mounted on a carousel device. The monkey sat in front of the carousel. In execution trials, one of the objects was presented to the monkey and the monkey performed a reach-to-grasp movement toward it. In observation trials, the monkey sat still and observed a human experimenter sitting on the opposite side of the carousel performing the same actions. Execution and observation trials and grasp types were interleaved using a pseudorandom process.
Execution trials.
Each trial began with the monkey's hands resting on the respective homepads in front of the monkey. The object remained hidden by an opaque screen placed in the monkey's line of sight with the object. After a short delay (∼0.8 s) the screen was electronically switched to become transparent (Fig. 1a; trigger LCDon) and a white LED illuminated the object. A green LED came on after a variable delay (0.5–1.5 s) and acted as the start signal (trigger GO). The monkey released his active hand (left hand for both M46 and M47) from the homepad (trigger HPR), reached out, grasped, and displaced the object [displacement onset (DO) trigger], held the object for 1 s (trigger Hon to Hoff), and released it and replaced his hand on the homepad (trigger HPN). The other (inactive) hand had to remain on the homepad throughout the trial.
Observation trials.
The roles were reversed. The carousel was now used to present an object to the experimenter. A second opaque screen placed in the monkey's line of sight hid the experimenter's object until the screen became transparent. The monkey then looked at the object (confirmed using a noninvasive eye tracker, results not presented here). The location of the object allowed the monkey to identify both which object was presented and whether it was an execution trial (monkey grasp) or an observation trial (observe the experimenter's grasp). Throughout observation trials, the monkey's object remained hidden and the monkey was required to sit still with both hands on his respective homepads. Again, monitoring of eye movements showed that the monkeys routinely watched the experimenter's actions.
The monkey received a reward after every successful trial (execution and observation). The analog object displacement signal, generated by a Hall-effect sensor, was continuously sampled at 5 kHz and recorded simultaneously with digital events (LCD, homepads, LED) and neuronal data to the computer.
Neuronal recordings
When the monkeys were fully trained, a Thomas Recording GmbH multiple electrode system was used to record intracortical LFPs. Broadband data were recorded with a sampling frequency of 25 kHz per channel. In each recording session, one to six microelectrodes were targeted at varying depths within area F5 (rostral division of the PMv) and/or the hand area in the M1, both contralateral to the grasping hand. LFPs were recorded from every electrode inserted into cortical tissue. At the majority of these recording sites, mirror neuron activity was detected.
EMG recordings
In both monkeys and in most recording sessions we were able to record the EMG (sampled at 5 kHz) simultaneously with the LFP. EMG electrodes were chronically implanted in multiple arm, hand, and digit muscles in the arm used for grasping [muscles sampled: deltoid, brachioradialis, extensor carpi ulnaris, extensor carpi radialis (long head), extensor digitorum longus, flexor carpi ulnaris, flexor digitorum profundus, palmaris, abductor pollicis longus, thenar muscles, first dorsal interosseous (1DI), abductor digiti minimi].
Data analysis
Data were analyzed using MATLAB R2014a (MathWorks).
Preprocessing.
Raw LFP data were bandpass filtered (0.1–300 Hz, third order Butterworth, noncausal), resampled at 1 kHz and cut into trials ranging from 6 s before to 6 s after the event DO. Raw EMG data were high-pass filtered (0.1 Hz), rectified, low-pass filtered (10 Hz), resampled at 50 Hz, and cut into trials.
Analysis of low-frequency LFPs.
To analyze the low-pass filtered neuronal population activity in cortical motor areas (the movement related potential, MRP), LFP data were low-pass filtered using a cutoff of 5 Hz (unless other cutoff mentioned; third order Butterworth, noncausal).
Strength of LFP modulations.
To assess the strength of modulation of the low-pass filtered LFP, artifact-free trials (see below) were averaged for each recording site, grasp type, and execution and observation. This resulted in six MRPs per recording site (compare Fig. 1c,d). The strength of modulation was defined as the integral (area) of the MRP between 400 ms before and 600 ms after DO (i.e., the period of interest: reach, grasp, and displacement; see Fig. 3a, inset, gray area). This measure provides integrated information about the period of largest MRP modulation and is more robust than maximum/minimum amplitude values (the latter measure was also tested and produced qualitatively the same results).
Frequency analysis.
The frequency composition of the recorded LFPs was estimated every 30 ms using a time-resolved short-time FFT with zero-padded tapering (Fig. 5). The total window length was 4.096 s, of which the middle 500 ms reflected a Hann taper; thus providing simultaneously high frequency granularity and temporal resolution for an overall illustration of amplitude modulations. Spectrograms were calculated for each trial separately (only artifact-free trials, as defined below). Then, the mean across trials was calculated for each recording site individually, followed by averaging these means across all recording sites for each condition separately (e.g., all execution trials for ring recorded at sites in M1 of M46). Finally, the grand-averaged spectrogram was normalized by dividing the amplitude modulation in each frequency bin by the frequency bin specific average amplitude at baseline (−60 to 60 ms around LCDon). Each spectrogram was scaled to range from 0.7 to 1.3.
For Figures 6 and 8, the frequency composition of the recorded LFPs was estimated every 30 ms using a time-resolved short-time FFT with multitapering (Slepian, time-bandwidth product 2.5) for reduced leakage across frequency bins and a window length of 1024 ms for a frequency resolution of ∼1 Hz.
For Figure 7, we used the same procedure as in Figures 6 and 8 but a 256 ms long window, shifted in 20 ms steps, to extract beta amplitude modulations with a better temporal resolution.
Beta: individual trigger alignment.
Beta amplitude modulations (∼20 Hz; see Fig. 7) were extracted from frequency analyses (as defined above) and artifact-free trials (as defined below). Differences in behavioral performance (such as reaction or movement time) across trials, grasp types, execution, and observation change the temporal spacing of events on a trial-by-trial basis. As neuronal activity and behavior are locked, such differences can result in inaccurate estimates of averaged neuronal activity. For accurate temporal presentation of amplitude modulations, we averaged trials piecewise around each event individually as follows: for each trial, we separately analyzed the LFP in a window ranging from 2 s before to 2 s after each trial-specific event (seven events). This resulted in seven partially overlapping segments for each trial.
For each recording site separately, we then averaged these segments trigger-wise across trials, resulting in seven segments for each of the six task conditions (execution and observation of the three grasp types) and both brain areas (PMv and M1). Next, we averaged across recording sites for each segment and condition individually and propagated the SEM (calculated across trials) of each recording site into the SEM of the grand-averages using Gaussian error propagation (error might be larger as possible noise correlations were neglected). This step resulted in 7 · 3 · 2 · 2 partly overlapping segments.
For presentation, we aligned respective segments of all conditions to the median temporal occurrence of events during execution (and relative to DO). Overlapping segments for execution were cut in the middle between events. For observation, a gap was inserted if for one condition the temporal spacing between successive events was shorter than during execution (i.e., the experimenter was faster); an overlap was left to indicate the spacing was longer (i.e., the monkey was faster).
This method produced averaged profiles, which are piecewise linear in time, and allowed to investigate accurate temporal alignment of neuronal activity around each individual behavioral event (Fig. 7).
We have not applied this approach to the MRP as for this signal component we were mainly interested in the signal around a single event (DO).
Strength of modulation during task versus baseline across frequencies up to 100 Hz.
We averaged segments trigger-wise across trials as described for Figure 7 (resulting in 7 · 3 · 2 partly overlapping segments per recording site and frequency bin). We then fitted (MATLAB: fit) the seven segments to obtain one “continuous” amplitude modulation per condition (3 · 2), recording site, and frequency bin.
Next, the fitted amplitude modulations were normalized to their frequency bin-specific average amplitude at baseline (−60 to 60 ms around LCDon). To assess the strength of modulation during the task versus baseline (see Fig. 6), three steps were necessary: (1) offset correction: subtract one from each baseline-normalized frequency bin to bring the baseline to zero and amplitudes to modulate around zero, (2) estimate strength of modulation: calculate area under these offset-corrected amplitude modulations (comparable to Fig. 3a, inset) between the events GO and Hoff for each frequency bin independently, and (3) normalize to account for task-independent modulation: divide these frequency-specific areas by the frequency-specific area around baseline (800 ms before to 200 ms after LCDon). The areas “baseline” versus “GO to Hoff” were normalized to account for their different integral lengths. Finally, the normalized strength of modulations were averaged across grasp type and recording sites.
Decoding.
Artifact-free trials (see below) were decoded using a regularized linear discriminant analyses and cross-validation with mutually exclusive training and test data (MATLAB: fitcdiscr, cvshrink, predict). Figure 8a: For each frequency bin and recording site separately, amplitude modulations between −100 and +100 ms around 0, 0.5, and 1 s relative to displacement onset were used as input to the decoder (note: due to the window used in frequency analysis, the information provided to the decoder at 0.5 s partly overlapped with that used at the other two time points, whose respective information is independent). Figure 8b: The input to the decoder depended on the analyzed signal. For MRPs, the time points approximately −150, 0, 100, 200, 300, 400 ms relative to DO were used. For beta, amplitudes estimated using frequency analysis (see above) with windows centered approximately at −125, −100, −75, −50, −25, 0, 25, 50 ms, and 850, 875, 900, 925, 950, 975, 1000, 1025 ms relative to DO were used to assess beta oscillations during the reach-grasp-displacement and hold period, respectively.
The reported decoding accuracies reflect the fraction of correctly decoded trials based on the average performance across 20 randomized tenfold cross-validation runs. Based on the results obtained for observation trials, decoding was performed only within modality, i.e., both training and test data were either exclusively execution trials or observation trials.
Significance of decoding results.
We used the binomial cumulative distribution and α = 0.05 to assess the significance level (Waldert et al., 2008) for each recording site individually (i.e., based on the respective number of trials that were available for decoding). Instead of using an average significance level or correction for multiple testing, we presented the range of significance levels across recording sites. The highest level of this range corresponds to the site with the lowest number of trials and reflects a stringent significance level for all recording sites.
Which trials were automatically excluded from analyses?
The results presented here are based on trials without muscle activity during observation (if EMG was recorded) and artifact-free trials and trial-averaged MRPs that exhibited substantial modulation as defined below.
Trial rejection: LFP.
Trials were classified as containing artifacts if the rectified MRP exceeded five times the SD of the low-pass filtered LFP or 350 μV at any time between −2 s and +2 s around DO. As this study was focused on the low-pass filtered activity, these trials were excluded from all analyses (i.e., also frequency decomposition, decoding, etc).
Trial rejection: EMG during observation.
The EMG recorded during observation trials was analyzed for possible muscle activity. Trials were labeled as such if the EMG between GO and Hoff exceeded nine times the 95 percentile of the EMG around LCDon (baseline). Although this sensitive threshold enabled us to detect all EMG fluctuations in the period when the monkey observed the experimenter's movements, it might have partly produced false positives.
To test whether the trial-averaged MRP during observation differed significantly between trials without any muscle activity and trials with detected EMG fluctuations, we compared the difference between the two corresponding average MRPs with 1000 bootstrapped differences on an α level of 0.05 (two-sided) for all recording sites and within a window of 1 s before to 1.5 s after DO (Fig. 1a). Correction for multiple testing was not necessary due to insignificant findings in almost all cases already for 0.05.
Substantial modulation.
To prevent analysis of MRPs simply reflecting noise during both execution and observation (noise–noise analysis), we tested each trial-averaged MRP for substantial modulation (quantities reported in Results). Substantial modulation was confirmed if, for execution or observation or both, the SD and area during the period of interest was two times larger than each measure within a window shortly before LCDon (baseline: 2 s before DO until LCDon). The length of the two windows was normalized.
Results
Database
The data used in this study were recorded in 78 and 71 sessions for M46 and M47, respectively, and from 310 and 251 intracortical recording sites, respectively. The details are shown in Table 1. After rejection of artifactual trials (see Materials and Methods), the average number of trials available for each recording site and for each experimental condition (grasp type, execution, and observation) was 35; the minimum number of trials available was 10.
Dataset used in this study
EMG activity during execution and observation
For most of the recording sites we not only measured neuronal activity but also simultaneously recorded the EMG of digit, hand, and arm muscles (M46: 303 of 310 sites; M47: 141 of 251).
Although EMG activity during movement execution was high and differed across grasps (Fig. 1b left; 1DI activity), the majority of observation trials exhibited no detectable muscle activity (Fig. 1b, right). However, in a few trials fluctuations in the EMG signal were detected during movement observation. Sensitive detection of fluctuations in the recorded EMG revealed that approximately one-tenth of all trials during observation exhibited some fluctuation in the recorded EMG although many of these may have been artifactual rather than genuine muscle activity. These trials were excluded from consecutive analysis.
Temporal comparison of behavior, EMG, and intracortical low-pass filtered LFPs during action execution (left; monkey) and observation (right; monkey observed experimenter performing movements) as recorded in M47. a, Normalized object displacement signals for the three objects. Averages of ∼37 trials for each grasp type. The peak of the signal represents a displacement of ∼17 mm. b, Single-trial EMGs (gray) of all 12 finger, hand, and arm muscles during execution and observation. The EMG of each trial was normalized to the muscle-specific 99th percentile of all execution trials. Left, Single-trial (gray) and grasp-specific trial-averaged EMGs of muscle 1DI (red, blue, and green traces) during execution, to show that muscles were differentially active according to the grasp used. Right, Single-trial EMGs of the monkey's finger, hand, and arm muscles during observation. c, Low-pass filtered LFP (<5 Hz) as recorded at a site in M1. Signals were trial-averaged (n ≈ 37 trials) for each grasp; thickness of each curve reflects SE of the mean. Gray lines reflect two filtered, single trials for illustration. d, As in c but for a recording site in PMv. Left, Gray bar depicts window used for decoding. Events: LCDon, screen becomes transparent and object visible; GO, green LED indicates go signal; HPR, monkey released his active hand from the homepad, reached out, grasped, and displaced the object (DO); Hon, beginning of object hold period; Hoff, end of hold period and release of object; HPN, return and replacement of hand on homepad.
Modulation of intracortical low-frequency LFP activity
Analysis of the intracortical low-pass filtered LFP activity revealed that this signal component was not only modulated during action execution but also by action observation (Figs. 1c,d, 2). Note: We will interchangeably use the terms “low-pass filtered LFPs” and “MRP” throughout the paper.
During movement execution, the MRP in both PMv and M1 was clearly present during the dynamic periods of the task (Figs. 1c,d, 2; activity around time 0 s), i.e., during reach, grasp, and displacement of the object, and again during release of the object and return to the starting position (activity ∼1.5 s). During static periods of the task, i.e., during holding of the object (Hon to Hoff), the signal remained around baseline level.
Importantly, the same pattern of MRP modulation was found during action observation (Fig. 1c,d, right), i.e., when the monkey observed the same actions performed by a human experimenter. However, while the average MRP during observation was weak (Figs. 2, 3), the temporal pattern of the MRP was significantly and strongly correlated for execution versus observation (correlation coefficient for M46: M1 0.55 and PMv 0.89; M47: M1 0.83 and PMv 0.83).
Grand-average low-pass filtered LFPs for action execution and observation (a, M46; b, M47). Curves reflect the mean across all recording sites, shaded areas the SEM propagated through from trial-by-trial variations into the grand-average (averages across 115–190 recording sites and ∼35 trials per grasp and site). Note: The SEM is comparatively large due to averaging across different grasps, i.e., different levels of neuronal activity. Event times are presented as the average event time during execution and observation.
Strength of low-pass filtered LFP modulations, calculated as the area under the trial-averaged LFP (a, M46; b, M47) (inset, gray area between the two vertical gray lines). The position of each dot reflects the strength of LFP modulation at one recording site during movement execution and observation of a particular grasp type (color-coded). Only recording sites exhibiting substantial LFP modulation during either execution, observation or both are shown (see Materials and Methods). Gray dots depict the area as calculated during baseline (from one second before LCDon to LCDon; Fig. 1a) for the same recording sites. The gray ellipse covers 90% of all gray dots; ellipse axes (eigenvectors of covariance matrix) scaled according to eigenvalues. Gray vertical lines mark same values for ‘area observation’ (per monkey). b, The gray lines connect the three dots corresponding to the M1 and PMv LFPs as shown in Figure 1c and d, respectively.
Using the sessions with EMG recordings, we found that excluding trials with small but detectable EMG fluctuations did not significantly (p > 0.05, bootstrap) change the trial-averaged MRP during observation.
During execution, the strength of MRP modulation (as measured from the area of the MRP between 400 ms before and 600 ms after DO; Fig. 3a, inset) was generally higher in M1 than PMv (Fig. 1c,d, left; note gain difference). This is documented for both subjects in Figures 2⇑–4a,c. During observation, the relationship for the two cortical areas differed between the two subjects: in M46 there were many recording sites showing a stronger modulation in PMv than in M1 (Figs. 3a, 4a) while in M47 it was the other way around (Figs. 3b, 4c). Yet, although for both subjects the strength of modulation during action observation was on average significantly (p < 10−6) weaker than during action execution (Fig. 4b,d: ratios of ‘area execution’ divided by ‘area observation’ larger than one), the strength of modulation during execution and observation was more similar in PMv than in M1 (Fig. 4b,d: ratios in PMv closer to one and smaller than in M1). Furthermore, the ratios were larger than those obtained during baseline (‘area execution’ divided by ‘area observation’ for signals around LCDon; Fig. 4b,d, gray vertical line). These results were independent of the type of grasp performed.
Comparison of the strength of modulation of the low-pass filtered LFP during execution and observation. a, c, Minimum, 25th percentile, median, 75th percentile, and maximum strength of modulation in the indicated conditions. c Whiskers for execution M1 were cut at the top for better comparability. b, d, For each recording site and grasp type, a ratio was calculated by dividing area execution by area observation. The curves reflect the approximated distributions of these ratios. The gray vertical line indicates the median ratio of the corresponding baseline values (Fig. 3, gray dots). e, f, Each dot corresponds to one recording site in M1 or PMv. The 3D position (and color) of each dot is determined by the three grasp-specific ratios: area execution divided by area observation for ring, sphere, and precision grip. Large cube, Only recording sites exhibiting substantial LFP modulation during either execution, observation or both are shown (see Materials and Methods). Encircled dots reflect recording sites for which the recorded low-pass filtered LFP allowed for significant decoding in both execution and observation. Right, 2D projections of the cube. Only recordings sites with significant decoding for execution and observation (encircled dotes in cube) are shown; color of each dot as in the cube. The deviation of the dots from the diagonal could not be explained by noise (see Results).
The distribution of ratios (Fig. 4b,d) was dominated by the nominator ‘area execution’ because the strength of LFP modulations was generally higher and more variable during execution. To show that the denominator ‘area observation’ substantially influenced this distribution, we compared the original distribution of ratios with the distribution of ratios obtained by dividing the individual values ‘area execution’ by the mean across ‘area observation’. The two distributions were significantly different in both M1 and PMv and in both subjects (p < 0.05, Kolmogorov–Smirnov).
Overall, the MRP was substantially modulated at the majority of recording sites during execution (M46: M1 116/120, PMv 186/190; M47: M1 92/136, PMv 91/115). The number of recording sites exhibiting substantial modulation during action observation was lower (M46: M1 14/120, PMv 23/190; M47: M1 79/136, PMv 16/115).
Modulation across different grasps
We found that at most sites in both M1 and PMv, the MRP was most strongly modulated when the monkey grasped the sphere, followed by the ring, and least for the precision grip (Fig. 4a,c). No such overall pattern in the MRP was apparent when the monkey observed the experimenter grasping different objects.
Next, we calculated the ratio of the MRP area during grasp of a given object for execution versus observation. As grasps of three different objects were studied, three ratios could be calculated for each recording site. Hence, each recording site can be identified by a point in a three-dimensional space spanned by the ratios for the three objects (Fig. 4e,f).
We found that using the ratios of any two grasps could explain only ≈41% of the variance in the ratios of the third grasp (average linear adjusted r2 value). Moreover, if there was, across all objects, a fixed relationship between the strength of MRP modulation during execution versus observation, all points would be located on the diagonal. This is not what was found (Fig. 4e,f). In contrast, the relationship between neuronal activity during execution and observation seemed to be grasp-dependent at many recording sites.
We also tested whether the deviation of dots from the diagonal could be explained by noise instead of different, grasp-specific ratios between execution and observation. To this end, we compared the distances of the original dots to the diagonal (Fig. 4e,f) with the distances of the corresponding dots reflecting ratios during the baseline (around LCDon, dots not shown; Fig. 4e,f). The latter is an overestimation of the influence of noise on the position of the original dots as noise influences the area (Fig. 3a, inset; used for calculation of the ratios) strongest if the signal fluctuates around zero. In addition, the noise level was similar for low (baseline) and high signal amplitudes (task; Figs. 1c,d, 2).
Despite this overestimation of the influence of noise, we found the distances of the original dots to the diagonal to be significantly larger (p < 10−6, Wilcoxon one-tailed signed rank test), suggesting the ratios between execution versus observation for the three grasps to be genuinely different.
Amplitude modulations of the intracortical LFP across different frequencies
Spectrograms were visually inspected up to 100 Hz (Fig. 5). Amplitude modulations in low frequencies were described above in the time-domain (the MRP). Figure 5 allows comparison of amplitudes over time and for different grasps and conditions. We also assessed the modulation in amplitude of LFP components up to 100 Hz to estimate their involvement during the task (Fig. 6; normalized area between events GO and Hoff, see Materials and Methods).
Comparison of grand-average LFP spectrograms (a, M46; b, M47). All spectrograms were normalized to the frequency-specific baseline (−60 to 60 ms around LCDon) and share the same color bar ranging from 0.7 (blue) to 1.3 (red, see Materials and Methods). Averages calculated across 115–190 recording sites and ∼35 trials per grasp and site, see Results, Database. Timings as indicated by vertical lines: black line, DO; short lines: red, LCDon; green, GO; blue, HPR; black, DO; magenta, Hon; cyan, Hoff; yellow, HPN (end of trial). Events described in caption of Figure 1.
Comparison of the strength of amplitude modulations in LFP components up to 100 Hz (a, M46; b, M47). During the task and relative to baseline (event LCDon), strongest amplitude modulations were detected in low-frequency LFPs recorded in M1 during execution and in beta frequencies ∼20 Hz in both areas and during both execution and observation. Modulations in frequencies >50 Hz could reflect genuine high-gamma LFP activity, multiunit spiking activity or a mixture of both. The curves reflect the mean and the SEM across recording sites (averages across 115–190 recording sites and ∼35 trials per grasp and site). Values were normalized across frequencies and to the baseline (see Materials and Methods). Note that the strength of MRP modulations here (data from the entire trial) cannot be directly related to the results presented in Figures 3 and 4 (data from reach-to-grasp period only).
In addition to the MRP described above, across all frequencies up to 50 Hz, LFP activity ∼20 Hz was most strongly modulated during both execution and observation, in both M1 and PMv (Figs. 5, 6). Although this beta activity was centered at a similar frequency for both monkeys, the beta band was broader in M46 than in M47. We further observed increased activity in LFP frequencies >50 Hz during execution. This finding might either reflect a genuine increase of neuronal oscillations in the high-gamma range or possible spike contamination of the LFP or a mixture of both (Waldert et al., 2013).
Unlike the MRP, beta was modulated during both dynamic and static periods of the task: we found a decrease in beta power during the reach, grasp, displacement, release, and return-to-homepad (dynamic periods), and an increase during hold (static period) in both M1 and PMv (Fig. 5). Modulation in the amplitude of LFP differed between PMv and M1 and between monkeys.
For more detailed analysis, in M46, we compared beta activity (∼20 Hz) for all grasps, both areas and execution and observation (Fig. 7). In both PMv and M1, beta power deviated from baseline level as soon as information about the type of grasp and who is going to perform the movement was available to the monkey (Fig. 7; trigger LCDon). Although beta power slightly decreased in execution trials, it slightly increased in observation trials.
Amplitude modulations in the beta frequency band (∼20 Hz) during movement execution and observation in M46. Signals were calculated trial-wise around each trial-specific trigger and then averaged and plotted against each trigger individually (see Materials and Methods). Signals averaged across all recordings sites. The thickness of the traces reflects the SEM propagated through from trial-by-trial variations into the average across recording sites. Events described in caption of Figure 1. Gray horizontal lines indicate window used for decoding (note: the lines indicate the time points of the centers of the windows used for the frequency analysis).
After the “go” signal, beta power immediately decreased in execution trials but remained elevated in observation trials until the actual onset of the experimenter's movement (Fig. 7; trigger HPR). In both conditions beta power reached its minimum shortly after movement onset, increased above baseline level during the hold period (trigger Hon to Hoff), decreased during object release (trigger Hoff) and return-to-homepad (trigger HPN), and then declined to baseline level before the next trial. During the hold period, beta power (∼20 Hz) was higher in PMv than in M1.
M47 showed qualitatively similar pattern of beta modulations (data not shown).
For both monkeys, the strength of modulation (not normalized across frequencies) during execution and observation was more similar for beta than for low-frequency LFPs (beta median ratios execution versus observation across all recording sites and grasp types: M46: M1 2.49, PMv 1.41; M47: M1 1.26, PMv: 1.05; compare with MRP ratios, Fig. 4b,d).
Decoding of grasp-type
LFPs recorded during movement execution contained substantially more information about grasp-type than LFPs recorded during movement observation (Fig. 8). M1 LFPs were much more informative about grasp type than LFPs recorded in PMv.
Amount of information (decoding accuracy) about grasp-type in different LFP components during execution and observation. a, For each frequency bin and recording site separately, amplitude modulations between −100 and +100 ms around 0, 0.5, and 1 s relative to displacement onset were used as input to the decoder. Curves reflect the mean across all recording sites in the indicated area and monkey; the overall average and maximum SEM was 0.7 and 1.45%, respectively (averages across 115–190 recording sites and ∼35 trials per grasp and site). b, Direct comparison of decoding accuracies for selected signal components, i.e., <5 Hz, 5–9 Hz, <9 Hz: low or bandpass filtered LFP, window used for decoding indicated in Figure 1d (gray line); β grasp and hold: amplitude modulation in the beta frequency band (∼20 Hz), window used for decoding indicated Figure 7 (gray lines). Boxplots reflect minimum, 25th percentile, median, 75th percentile, and maximum decoding accuracies.
We first used amplitude modulations of individual frequency bins between 1 and 100 Hz as input to the decoder. The average decoding accuracies across recording sites in M1 and PMv indicated that low-frequency and beta-range LFP activity were most informative about grasp type during action execution and in M1 (Fig. 8a).
Second, we investigated decoding performances obtained by using the low-pass filtered LFP (the MRP) as input to the decoder (Fig. 8b). For LFPs recorded during movement execution, the MRP allowed for significant decoding at nearly all recording sites in both M1 and PMv. The LFP between 5 and 9 Hz carried substantially less, though mostly significant, information about grasp type. Using LFP modulations <9 Hz resulted in still lower performance than using the LFP <5 Hz alone.
Thirdly, using the amplitude of the beta range frequencies as input to the decoder showed that: (1) a similar amount of information about grasp type was present during both the dynamic and static period of the task, and (2) beta provided a lower decoding performance than using the LFP in the time domain in frequencies up to 9 Hz (Fig. 8b).
For LFPs recorded during movement observation, some recording sites allowed for decoding of grasp type (Fig. 8). However, apart from the MRP recorded in M47, the majority of recorded LFPs resulted in decoding accuracies below significance level for all signal components we have analyzed (LFP in the time-domain up to 9 Hz and amplitudes of oscillations up to 100 Hz).
Qualitatively the same results as reported above were obtained if the MRP and amplitude modulations in all frequencies up to 45 Hz were combined and used simultaneously as input to the decoder.
Decoding accuracies for amplitude modulations in frequencies >50 Hz were generally lower than accuracies obtained for frequencies <5 Hz and might be influenced by grasp-specific multiunit spiking activity (Waldert et al., 2013).
Discussion
The intracortical LFP in the hand areas of PMv and M1 is modulated at different frequencies during both action execution and observation. We showed for the first time that the low-frequency LFP can be modulated during action observation in both cortical areas and that temporal modulations in LFP are correlated for execution versus observation. No covert muscle activity was detected in the majority of observation trials.
EMG activity during movement observation
In most mirror neurons studies no simultaneous monitoring of EMG has been carried out. Our EMG recordings showed that even in highly trained monkeys covert muscle activity can be present during some action observation trials. For our LFP data, we found that inclusion of trials exhibiting EMG fluctuations resulted in small quantitative but no qualitative differences (results not presented).
However, spiking activity could be more sensitive to covert muscle activity, especially in M1 where the generation of such activity might be expected to yield a deeper modulation of spike rate than seen during action observation uncontaminated by EMG (Vigneswaran et al., 2013). As the LFP reflects population activity, it might be insensitive to the influence of those neurons, whose firing is directly associated with covert muscle activity and therefore should not be considered as mirror neurons. Our findings show that mirror studies should include EMG monitoring to confirm the existence of genuine mirror activity, and EMG should be assessed using an impartial, automated procedure.
Modulation of intracortical low-frequency LFPs
Several studies have described modulations of the low-pass filtered LFP during movement execution. The intracortical MRP is, for example, tuned to movement parameters such as movement direction (Mehring et al., 2003) or grasp aperture (Zhuang et al., 2010). Similar tuning, though on different scales, has been found in extracortical recordings using ECoG/EEG/MEG (cf. Waldert et al., 2009).
We found that this signal component is modulated during movement observation and in the absence of covert EMG. Hence, observing movements changes not only spiking (Gallese et al., 1996) and beta activity (Tkach et al., 2007; Kilner et al., 2014), but also low-frequency population activity, in both PMv and M1.
Temporal MRP changes during execution versus observation were highly correlated, suggesting that MRPs during observation genuinely reflect neuronal activity of the mirror system. These findings also refute speculations about MRPs being related to artifacts due to head movements, which can occur even during small hand movements (Waldert et al., 2012).
However, compared with movement execution, MRP modulation was weaker and detected at fewer recordings sites (Figs. 2, 3, 4a–d, 6). This generally weak modulation might be the reason why low-frequency brain activity during action observation has not been reported in mirror studies using extracortical recordings.
The MRP was modulated only during the dynamic periods of the task, suggesting that it is involved in altering as opposed to maintaining motor output. The two temporally separated modulations (Fig. 2) might merge and appear as a single, complex waveform in tasks without a hold period.
Amplitude modulations of the intracortical LFP
Modulations ∼20 Hz were prominent in both cortical areas and both conditions (Figs. 5, 6), which possibly explains why beta rhythms have been often reported in mirror studies using noninvasive recordings (Hari et al., 1998). We detected the well known beta decrease during movements (Pfurtscheller, 1989; Donoghue et al., 1998) and increase during the hold period (isometric muscle contraction; Spinks et al., 2008). The function of this rhythm is debated (Kilavik et al., 2013). Importantly, we identified this pattern of desynchronization and resynchronization in both M1 and PMv, and in the absence of covert muscle activity during action observation.
The level of beta activity was dependent on the area and task condition (Figs. 6, 7). The temporal modulation of beta amplitudes (Fig. 7) was strongly correlated across recording sites in both cortical areas, speaking for widespread and synchronized activity in this frequency range.
High gamma activity was generally much more pronounced during execution than observation (Fig. 6); further analysis is required to differentiate between the contribution of genuine neuronal oscillations and multiunit spiking activity to this LFP component (Waldert et al., 2013).
Comparison of LFPs during execution and observation
We observed a weak positive relationship between modulation strength during execution versus observation (Fig. 3); modulation strength was generally more comparable for beta than for MRPs (Fig. 6; and smaller ratios execution vs observation for beta).
The differences between LFPs recorded during execution and observation might result from the inherent coupling of synaptic and spiking activity. The latter differs between the two conditions because: (1) a smaller number of active neurons in M1 and PMv during action observation than execution (Kilner and Lemon, 2013), (2) the larger extent of the neuronal network for action execution, and (3) the activity patterns of individual mirror neurons being different for observation versus execution (Dushanova and Donoghue, 2010; Vigneswaran et al., 2013). The larger ratios between execution versus observation for M1 than PMv MRPs (Fig. 4b,d) could, for example, be related to the finding that, compared with execution, PMv mirror neurons show a smaller reduction in spiking activity than M1 mirror neurons (Maranesi et al., 2012; Vigneswaran et al., 2013).
Such differences in local spiking activity might be explained by differences in haptic, proprioceptive and visual inputs during execution versus observation (Riehle et al., 2013; Galán et al., 2015). However, even without the need to argue for differences in local processing, different (synaptic) inputs alter the LFP directly.
Partially nonoverlapping functional networks for execution and observation
For the majority of recording sites the ratio between the strength of neuronal activity during execution versus observation was different for the three grasps (Fig. 4e,f; for MRPs). The LFP measured at different recording sites could be interpreted as reflecting the relative activity of cortical “subnetworks” (e.g., a local cluster of neurons functionally relevant for a particular task) engaged in grasp control. The lower level of LFP activity for action observation (Figs. 3, 4a,c, 6) could mean that some subnetworks, although recruited during action execution, are less active or inactive during observation. Our results suggest that the engagement of the different subnetworks does vary (distribution of ratios, Fig. 4b,d) and is grasp-specific (Fig. 4a,c).
Different, grasp-specific ratios for one and the same recording site, as found here, mean that the involvement of a particular subnetwork during execution of a grasp is weakly correlated with the involvement of that subnetwork during observation, with both additionally being dependent on the particular grasp itself (Fig. 4e,f). Two possible interpretations of this finding are as follows: (1) the same network is used during execution and observation but in different ways because the input to the network is different during execution and observation (as discussed above), and (2) nonidentical but partially overlapping networks are involved during execution and observation. The latter would mean that the neuronal resources used to actively control a particular movement are different to those responding to the same movement during observation.
However, LFPs inherently have the disadvantage of reflecting superimposed/averaged neuronal activity. In an extreme case, the same LFP could be recorded on one electrode although two mutually exclusive groups of neurons were involved. Repetition suppression (Grill-Spector et al., 2006), a concept applied in fMRI mirror studies (Dinstein et al., 2007) and recently also for beta (Kilner et al., 2014), cannot completely resolve this ambiguity. Only analysis of mirror neuron spiking activity across different grasps can provide unambiguous evidence for nonoverlapping functional networks during execution and observation. For example, the activity of broadly congruent mirror neurons (Gallese et al., 1996) during execution versus observation could be similar for some grasp types and different for others, whereas the activity of strictly congruent mirror neurons could be more similar.
More quantitative approaches must now be incorporated in task design to test these ideas more formally, e.g., using the framework suggested here (Fig. 4e,f).
Decoding of grasp during execution and observation
Most information about grasp could be inferred from low-pass filtered LFPs recorded during movement execution in M1 (Fig. 8), which is in-line with single neuron studies (Carmena et al., 2003) and reflects M1's role in direct motor control. Decoding of beta amplitudes revealed some information about grasp, possibly due to coherences between beta and muscle activity (Witte et al., 2007). Decoding accuracies for high-gamma amplitude modulations might be explained by grasp-specific spiking activity (Waldert et al., 2013).
With the exception of some recording sites, the signal components analyzed here generally carried little or no information about the observed grasp. Several reasons might account for this finding: (1) weaker signal modulations during observation than execution, i.e., lower signal-to-noise ratios; (2) the salient event for the monkey is the occurrence of a reach-to-grasp action (signaling upcoming reward), but the particular type of grasp performed was of little importance; (3) the majority of mirror neurons are “broadly congruent”, and do not show grasp specific activity during observation (Gallese et al., 1996; Rizzolatti et al., 1996); and (4) the LFP is a population signal, potentially obscuring grasp-specific spiking activity of individual neurons during observation.
Although the MRP is a suitable input signal for brain–machine interfaces (BMIs) because it carries substantial information about movement parameters in LFP/ECoG/EEG/MEG (Waldert et al., 2009), calibrating BMIs using the MRP during action observation, as for spiking activity (Wahnoun et al., 2006), would appear to be impractical unless the task design is optimized for this purpose.
Footnotes
This work was supported by a Marie Curie Postdoctoral Fellowship, the Wellcome Trust, NC3Rs and the UCL Grand Challenge Scheme. We thank Tabatha Lawton, Lianne McCombe, Sam Shepherd, Spencer Neal, Dan Voyce, Victor Baller, Jonathon Henton, Dave Thomas, Xavier Golay and Martin Lawton for expert advice and support.
The authors declare no competing financial interests.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License Creative Commons Attribution 4.0 International, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
- Correspondence should be addressed to Dr Stephan Waldert, Sobell Department of Motor Neuroscience and Movement Disorders, UCL Institute of Neurology, London WC1N 3BG, UK. s.waldert{at}ucl.ac.uk
This article is freely available online through the J Neurosci Author Open Choice option.
