Abstract
Brain activity can be used as a control signal for brain–machine interfaces (BMIs). A powerful and widely acknowledged BMI approach, so far only applied in invasive recording techniques, uses neuronal signals related to limb movements for equivalent, multidimensional control of an external effector. Here, we investigated whether this approach is also applicable for noninvasive recording techniques. To this end, we recorded whole-head MEG during center-out movements with the hand and found significant power modulation of MEG activity between rest and movement in three frequency bands: an increase for ≤7 Hz (low-frequency band) and 62–87 Hz (high-γ band) and a decrease for 10–30 Hz (β band) during movement. Movement directions could be inferred on a single-trial basis from the low-pass filtered MEG activity as well as from power modulations in the low-frequency band, but not from the β and high-γ bands. Using sensors above the motor area, we obtained a surprisingly high decoding accuracy of 67% on average across subjects. Decoding accuracy started to rise significantly above chance level before movement onset. Based on simultaneous MEG and EEG recordings, we show that the inference of movement direction works equally well for both recording techniques. In summary, our results show that neuronal activity associated with different movements of the same effector can be distinguished by means of noninvasive recordings and might, thus, be used to drive a noninvasive BMI.
Introduction
A brain–machine interface (BMI) translates neuronal signals reflecting a persons' intention into commands driving a machine (prosthesis, cursor, computer, robot, etc.). The brain signals analyzed in a BMI are recorded either with invasive or noninvasive techniques. Next to the distinction at this technical level, it is an ongoing issue which type of brain activity should be used and which paradigm or mental task the BMI should rely on.
Several approaches and brain signals have been used in the field of noninvasive BMI to drive an external device: the self-regulation of evoked brain activity [e.g., slow cortical potentials (Birbaumer et al., 1999) or amplitudes of different frequency bands (Wolpaw et al., 1991; Wolpaw and McFarland, 2004)]; imagination of movements of different parts of the body (Pfurtscheller et al., 1993; Pfurtscheller and Neuper, 2006; Blankertz et al., 2007); performance of different cognitive tasks (Penny and Roberts, 1999; Curran et al., 2004); or the visually evoked P300 potential (Farwell and Donchin, 1988; Hoffmann et al., 2007).
In invasive BMI research, a different approach has been successful: the spiking activity of cortical cells related to movements of one limb is used for equivalent control of an external effector (Serruya et al., 2002; Taylor et al., 2002; Carmena et al., 2003; Hochberg et al., 2006). Such direct BMI control is more precise and intuitive and has the potential to make extensive subject training redundant. The basis of this approach is the finding that the direction of performed arm movements can be inferred from trial-averaged firing rates of neurons in monkey motor cortex (Georgopoulos et al., 1983, 1986). Likewise, firing rates of human motor cortex neurons are tuned to the direction of intended hand movements (Hochberg et al., 2006). Invasive BMI research also revealed that movement direction can be inferred from monkey local field potentials (LFPs) (Mehring et al., 2003) and from human electrocorticograms (ECoGs) (Leuthardt et al., 2004; Mehring et al., 2004). So far, however, information about the directional tuning in noninvasive recordings is lacking.
To use the aforementioned approach also for noninvasive BMIs, we investigated MEG activity during hand movements and show that self-chosen movement directions of a single effector (the hand) can be inferred on a single-trial basis using neuronal signals measured outside the head. We determined the decoding performance that can be gained from different sensor locations and signal components, and characterized brain activity related to hand movement control. Based on simultaneous MEG and EEG recordings above contralateral motor areas, we investigated and compared the decoding performance gained from MEG and EEG covering the same brain area. Finally, we discuss the relation between our classification results (MEG and EEG) and those obtained in invasive studies using single-unit activity (SUA), LFP, and ECoG.
Materials and Methods
Recording systems
MEG system.
The magnetic component of brain activity was measured using a 151-channel whole-head MEG system (VSM MedTech, Vancouver, British Columbia, Canada) consisting of first-order axial gradiometer with 5 cm baseline and ∼2.5 cm intersensor spacing. The system was installed in an electromagnetically shielded room (Vakuumschmelze, Hanau, Germany); the noise level was 10 fT/√Hz.
EEG system.
The electrical component of brain activity was measured using an AC-coupled EEG system (VSM MedTech). We positioned 20 recording electrodes with ∼2.5 cm spacing medially above the contralateral motor area. Ground was attached to the neck, and two electrodes were attached above mastoid processes (to allow re-referencing). During recordings, the reference electrode was Cz. Impedance for all electrodes was <5 kΩ.
MEG and EEG were passed simultaneously through an analog low-pass filter with a cutoff frequency of 208 Hz and sampled at 625 Hz.
Experiment
Nine right-handed subjects participated voluntarily in this study, which was approved by the ethics committee of the University of Tübingen Medical Faculty. Subjects were instructed to move a joystick from a center position toward one of four targets located at 90° intervals (center-out paradigm) using right hand and wrist only. In each trial, the target was self-chosen by the subject. The subject's elbow rested on a pillow to prevent upper arm and shoulder movements (Fig. 1a); the head was stabilized by small pillows. Deflection of joystick was 4.5 cm (19.5°). Targets were arranged in the form of a square with corners pointing left, right, up, and down relative to the subject's frame of reference. The frictionless joystick (without return springs) very slightly routed the subject's movement to support the subject in conceiving the task (reaching in four perpendicular directions). Visual trigger signals were presented on a screen ∼65 cm in front of the subject. These signals were used exclusively to start a trial or to indicate possible errors. In addition, a red cross was continuously presented for fixation.
a, Photograph of the experimental setup showing the joystick and the positioning of the subject. b, Trial sequence with time constraints.
The experiment contained three blocks, each containing several trials, with each trial consisting of the sequence depicted in Figure 1b. Within the shown time constraints, the sequence was self-paced. Each trial started with the joystick in the center position and was initiated by presenting a gray circle (diameter: 3 cm) on the screen. After a variable delay between 1 and 2 s, the disappearance of the circle indicated the “go” signal. A movement was assigned as a correct trial if the joystick exceeded an angle of 12° in a certain direction. The position of the joystick was sampled at 500 Hz. A dark gray circle was displayed if time constraints (Fig. 1b) were violated; such trials were invalid and not used for further analysis. To obtain approximately the same number of trials per target, the subject was told which directions were underrepresented after each of the first two blocks.
Furthermore, the electrooculogram (EOG) was recorded. Head movements, which may cause artifacts and shifts in the source-sensor mapping, were monitored using three head coils (left and right ear preauricular point and nasion); data were analyzed only if head movement was <1 cm. Subjects were instructed to sit with their head in a centered position, which legitimizes averaging in sensor space across subjects (Kaiser et al., 2000).
Data analysis
Preprocessing: whole-head MEG and contralateral EEG.
EEG data were re-referenced to linked mastoid electrodes (linked ears). Data were high-pass filtered (0.5 Hz, Butterworth, third-order, zero phase shift) to remove offset and trend. We then redefined a trial as the time window from 1000 ms before to 750 ms after movement onset. Movement onset was defined as the time point when the subject started to move the joystick (determined as the beginning of a directed movement extracted from the joystick's positional information) to leave the center position and reach a target (determined as 70% of maximal deflection). Eye artifacts were detected in the EOG by threshold detection. Trials containing eye artifacts were discarded.
After this preprocessing, on average 68 trials per target and subject (minimum: 52, 53, 54, and 55 for left, right, up, and down, respectively) were used for further data analysis.
Time-resolved power spectrum: whole-head MEG.
Temporal evolution of the power of different frequencies was determined by time-resolved spectral analysis using fast Fourier transform with a sliding Hamming window of 250 data points (400 ms) width, i.e., at a frequency resolution of 2.5 Hz. The center of the window was shifted in steps of 10 data points (16 ms). To reveal power modulations within a trial, the power spectra were normalized by dividing the time-resolved power by a baseline power separately for each frequency bin (Rickert et al., 2005). The baseline power was defined as the mean power of the respective frequency bin in the interval ranging from 814 to 300 ms before movement onset.
Decoding: whole-head MEG data.
MEG activity was decoded on a single-trial basis by regularized linear discriminant analysis (RLDA) (Friedman, 1989). The percentage of correctly decoded trials, termed decoding accuracy (DA), was used to quantify the decoding performance. For each subject, the DA was calculated by 10 times 10-fold cross-validation (Efron and Tibshirani, 1994), where the set of trials used for training the RLDA and the set of trials used for decoding were mutually exclusive.
As inputs to the RLDA classification, we used either the MEG signals in the time domain or the normalized power of different frequency bands. In both cases, the input reflected a time-varying signal gathered from a certain period of time (different time windows were used in the following and are described in the respective paragraphs). For decoding in the time domain, MEG activity of each trial was low-pass filtered (Butterworth, third-order, zero phase shift) using cutoff frequencies from 2 to 12 Hz and resampled at the four-fold of the cutoff frequency. Cutoffs >12 Hz were not applied because DA decreased with increasing cutoffs. For decoding in the time-frequency domain, time-resolved normalized power spectra were computed according to the procedure described above, with spectra computed using a window of 400 ms width shifted in time steps of 80 ms. Then, the normalized power was averaged across all frequency bins belonging to a selected band.
Decoding: contralateral MEG/EEG data.
MEG and EEG activity was decoded in the time domain according to the procedure described above. We defined a group of 20 contiguous MEG sensors covering the medial contralateral motor cortex, i.e., approximately the same area as the EEG electrodes. Thereby, our analyses of the contralateral MEG and EEG activity were based on brain activity originating from approximately the same brain area, and we used the same number of sensors in both MEG and EEG.
Tuning characteristics.
Tuning curves were calculated separately for the low-pass filtered (3 Hz cutoff frequency) MEG signals of each MEG sensor above motor-related areas and separately for each subject. We distinguished four time windows (pre: −150 to −50 ms; early: 0–100 ms; late: 300–400 ms; and post: 650–750 ms; all times relative to movement onset). The tuning curves reflected the 3 Hz low-pass filtered MEG activity averaged across all trials of the corresponding direction. Significance of tuning was assessed by a one-way ANOVA. Cosine fits (Georgopoulos et al., 1982) were calculated using least-square sum. The squared correlation coefficient (r2) between the cosine fit and the measured values was used to assess the degree of cosine tuning.
Test of significance.
The binomial cumulative distribution was used to assess the statistical significance of the decoding accuracy. With n being the number of trials and t the number of targets, the probability to predict the correct target at least k times by chance is calculated as follows:
We used the lowest total number of trials (n = 233) across subjects to obtain a statistically conservative significance level l = k/n valid for all subjects.
The Wilcoxon signed-rank test was used (1) to test for statistical differences in the decoding performance of MEG and EEG and (2) to assess the statistical significance of the normalized power modulation in different frequency bins. Here, for each subject separately, an n-dimensional vector was created and used as input to the test. The vector contained n pairwise power differences (calculated from each trial) obtained between rest and movement.
Decoded information.
For a direct comparison of our decoding results with those obtained in other related invasive/noninvasive studies, and to quantify the amount of information about movement direction extracted from the MEG and EEG signals, we computed the Shannon mutual information (Cover and Thomas, 1991) between the decoded and real movement directions:
where Td and Tr are the sets of decoded and real targets (movement directions), p(td) and p(tr) the marginal probability distributions of decoded and real targets, and p(td,tr) the joint probability distribution of decoded and real targets, respectively. We will refer to this measure as decoded information (DI). It yields a lower bound (Cover and Thomas, 1991) on the mutual information between the MEG/EEG signals and the movement directions. However, the mutual information estimated from limited experimental data are biased (Treves and Panzeri, 1995). This sampling bias was corrected here using bootstrap resampling (Efron and Tibshirani, 1994).
In addition to the direct calculation of DI, a function DI(DA) can be derived under the assumptions of (1) equal probabilities for correct predictions in each class and (2) equal distributions of false predictions across all classes. Under these assumptions and because of
the following equations hold:
Finally, combining Equation 2 with Equations 4–7 and replacing |Tr| by 100/DAchance results in the following equation:
This function allows to calculate the dependence of DI on DA across the whole range of DA given the above assumptions.
Results
Power and signal modulations of MEG activity
Our analysis of the MEG activity revealed clear task-related power modulation (Fig. 2). We found significant (p < 0.05) power modulation between rest and movement for sensors over motor-related areas in three distinct frequency bands: an increase for frequencies ≤7 Hz (low-frequency band) and 62–87 Hz (high-γ band) and a decrease for frequencies of 10–30 Hz (β band) during movement. The modulation was strongest at motor areas.
Normalized power spectra averaged across all subjects, targets, and trials for each sensor from 630 ms before to 700 ms after movement onset (abscissa) and from 0.5 to 100 Hz (ordinate) at a frequency resolution of 2.5 Hz. The sensor field covers the whole brain and was projected on a two-dimensional plane for the sake of clarity. The group of sensors above motor-related areas is indicated by the black solid line. The dotted black line surrounds the position of the contralateral EEG and MEG sensor group. Graph in the top right corner, Normalized power modulation of sensor * for the frequency bands ≤7 Hz (low-frequency band, red), 10–30 Hz (β, blue), and 62–87 Hz (high-γ, green) along with the SEM; abscissa, time relative to movement onset in seconds.
With regard to the low-frequency band, a power increase occurred bilaterally ∼100 ms before movement onset, spanning from parietal to prefrontal areas, and mainly contralaterally at motor areas during the movement, followed by a decrease around movement end (Fig. 3a).
Topography of activation patterns averaged across all subjects, targets, and trials around the indicated time points relative to movement onset. a–c, Power modulation in the three frequency bands (normalized power). d, Three hertz low-pass (LP) filtered MEG activity (magnetic field strength, in fT). The go signal and movement end were approximately −240 and 500 ms, respectively. Sensor field is shown from above (in approximation to the head, nose pointing upward); all sensors (black dots) are visible. The group of sensors above motor-related areas is surrounded by a black line in the top left plot.
In the β band, the power decrease started ∼100 ms before movement onset at contralateral motor areas and became bilateral (almost symmetrical) at motor areas after movement onset (Fig. 3b).
In the high-γ band, an early (∼150 ms before movement onset) contralateral prefrontal increase in power was found, followed by an increase mainly at contralateral central motor areas during the movement, which vanished around movement end (Fig. 3c).
Investigating the low-pass filtered (3 Hz cutoff) magnetic field strength directly in the time domain, we found a power increase at the contralateral medial parietal areas at movement onset (Fig. 3d). Shortly after movement onset, this increase spread rostrally and reached contralateral motor areas. After the movement, a decrease was observed. This signal type is shown because, in the end, it yielded the highest DA.
Decoding of movement direction
We decoded low-pass filtered MEG activity and the normalized power modulations of the three aforementioned frequency bands using signals from different groups of sensors and from different time windows. For the low-pass filtered MEG activity, we will focus on the 3 Hz cutoff in the following paragraphs, because cutoff frequencies between 2 Hz and 12 Hz were tested additionally, but none yielded higher DA than the 3 Hz cutoff frequency.
First, we investigated the DA that can be obtained if only the information provided by a single sensor was used (Fig. 4a). For the 3 Hz low-pass filtered MEG signal, a DA above chance level of 25% was gained from medial sensors reaching from ventral parietal to prefrontal areas ∼100 ms before movement onset. Around and after movement onset, sensors showing significant (p < 0.001) DA cumulated above medial contralateral motor areas. A maximum single sensor-based DA of 36% was reached for a sensor located above contralateral motor-related areas. Furthermore, sensors above medial ipsilateral and contralateral prefrontal areas provided significant DA as well. The normalized power modulations of the three frequency bands provided significant (p < 0.05) DA only for the low-frequency band (Fig. 4a). Here, a DA of up to 30% was gained from few sensors exclusively above contralateral motor-related areas after movement onset. No information about movement directions could be extracted from the power modulation in the β and high-γ bands.
a, Topography of DA for single sensor-based decoding and different time windows. All DA values are averages across subjects. The sensor field is shown from above (in approximation to the head, nose pointing upward); all sensors (black dots) are visible. The top three panels show the DA for the three frequency bands, and the bottom shows the DA for the 3 Hz low-pass (LP) filtered MEG activity. Sensors showing significant DA are marked with an “x” [top three panels, 29.4% (p < 0.05); bottom, 33.7% (p < 0.001)]. Black lines surround selected regions of interest (for later decoding) for the low-frequency band (first panel) and the 3 Hz LP filtered activity (last panel). b, Three hertz low-pass filtered MEG activity from two subjects along with DA provided with each sensor (sensor locations identical for both subjects). The curves depict the trial-averaged signal for each target (blue, right; green, up; red, left; cyan, down), and the transparent color bands depict the corresponding SEM. The DA topography in the middle is taken from a: 3 Hz LP filtered, 50–300 ms.
The DA topographies shown in Figure 4a were obtained using signals from single sensors; i.e., no correlation information was used. To investigate the effect of local correlations on the spatial distribution of DA, we decoded the signals of contiguous sensor triplets and quintuplets: increased DA values with a spatiotemporal pattern of DA similar to that for the single sensor decoding were found (data not shown) (results summarized in Fig. 5).
DA (averaged across all subjects, decoded time window: 0–500 ms) using signals from different groups of sensors ranging from single sensor-based decoding to decoding of all sensors above motor-related areas. The DA values for both the 3 Hz low-pass (LP) filtered MEG activity and the three frequency bands with significant power modulations are shown. Error bars indicate SEM. MRA, Motor-related area; ROI, regions of interest derived from the single sensor-based decoding topographies in Figure 4a (for the 3 Hz LP filtered activity: black line in last panel; for the relative power modulation: black line in first panel of Fig. 4a). The three horizontal lines indicate the chance, p < 0.05, and p < 0.001 levels.
The trial-averaged 3 Hz low-pass filtered MEG activity of single sensors was markedly different for the movements to the different targets (Fig. 4b). The difference was stronger for sensors with a higher DA value.
Next, we decoded the signal components that provided significant directional information (i.e., the 3 Hz low-pass filtered activity and the power modulation of the low-frequency band) using different groups of sensors: a region of interest defined on the basis of the DA topographies individually for both signal components (Fig. 4a, 3 Hz: black line in last panel; power modulation: black line in first panel) and all sensors from contralateral, ipsilateral, or bilateral motor-related areas as defined in Figures 2 (black solid line) and 3 (black line in the top left plot). The decoding results are summarized in Figure 5.
In general, the DA gained by decoding the 3 Hz low-pass filtered MEG activity increased with the number of sensors used. The highest DA on average across subjects was 67%, obtained by using the signals of all sensors above motor-related areas from a time window ranging from 0 to 500 ms relative to movement onset (DA remained at this level when the activity during the 100 ms before movement onset was additionally included). For this signal component, the DA of single subjects was in the range of 54–85% (Table 1, condition C1). The calculated DI was 0.63 bit on average across all subjects, ranging from 0.27 to 1.20 bit for individual subjects (Table 1, condition C2).
DA and DI values for each subject separately
Decoding the power modulation of the low-frequency band yielded significant (p < 0.001) DA after movement onset if the region of interest or all sensors above motor-related areas were used. However, the DA for the power modulation of low-frequency band was very low (34%) and could also not be improved by varying the exact frequency range of this band (approximately the same DA for the frequency ranges ≤2.5 Hz and ≤5 Hz). As already shown above for single sensor decoding, the DA gained by decoding the power modulation of the β and high-γ bands never provided significant (p < 0.05) DA, even if the signals of multiple sensors were used.
With the 3 Hz low-pass filtered component, we assessed whether MEG signals from a subject can be decoded if the classifier was trained exclusively on data from other subjects. We tested this by decoding each of the nine subjects using the RLDA classifier trained on the data from the remaining eight subjects. The DA was 39% on average across all subjects and for the single subjects in the range of 35–47% (Table 1, condition C3).
To verify that the directional information found in the MEG can also be decoded from the EEG, we decoded the 3 Hz low-pass filtered, contralateral MEG and EEG activity in a time window ranging from 0 to 500 ms relative to movement onset. On average across all subjects, the DAs obtained from the corresponding 20 EEG, 20 MEG, and 10 + 10 MEG/EEG sensors were 55, 60.2, and 59.3%, respectively (Table 1, condition C4/C6/C8), and the DIs were 0.35, 0.46, and 0.43 bit (Table 1, condition C5/C7/C9). None of these differences was significant (p > 0.05).
To investigate the temporal evolution of the decoding performance, we calculated the time-resolved DA. Here, the signal, which was decoded, was 3 Hz low-pass filtered until the end of the corresponding decoding window. By this, we ensured that exclusively data before the current time point were used. Using the sensor configuration that provided the highest DA for MEG (i.e., all sensors above motor-related areas), the time-resolved DA (Fig. 6a) averaged across all subjects (black line) revealed significant (p < 0.05) DA 100 ms before movement onset. Using MEG and EEG activity of sensors above contralateral motor-related areas, the time-resolved DA (Fig. 6b) revealed significant (p < 0.05) DA ∼100 ms before movement onset for MEG (red) and MEG+EEG (blue) and around movement onset for EEG (green). These differences in DA onset of ∼100 ms were significant (p < 0.05) for EEG versus MEG and for EEG versus MEG+EEG [p < 0.05 if all 20 MEG + 20 EEG sensors (data not shown) and p < 0.10 if 10 MEG + 10 EEG sensors were used]. With regard to all signals and sensor configurations shown in Figure 6, DA increased continuously up to ∼500 ms after movement onset, which corresponds to the average movement end, and decreased after that time point.
Time-resolved decoding accuracy using 3 Hz low-pass filtered signals. a, Result of decoding MEG signals from sensors above bilateral motor areas, averaged across all subjects (black) and for each subject separately (gray). b, Results of decoding EEG, MEG, and MEG+EEG signals from sensors above contralateral motor areas, along with corresponding SEM (transparent color bands). Graphs show DA of epochs of 250 ms length immediately before the time indicated on the abscissa; temporal resolution is 25 ms. The time points of the go signal and target reached along with SDs are indicated by gray diamonds. The three horizontal lines indicate the chance, p < 0.05, and p < 0.001 levels.
On average across all subjects, 6.3, 17.6, 35.9, and 7.6% of the MEG sensors above motor-related areas showed significant (p < 0.001, one-way ANOVA) tuning within the pre-, early-, late-, and postmovement phase, respectively. We also investigated the tuning curves with respect to a possible cosine tuning (Georgopoulos et al., 1982) and found that 55.6, 40.8, 26.5, and 24.2% of the significantly tuned MEG sensors exhibited a tuning curve close to a cosine function with r2 > 0.96. Such cosine-like tuning curves might allow for generalization of directional decoding to movement directions not contained in the training set, e.g., movement directions between the four directions used here. Across all subjects and movement phases, the six sensors showing significant tuning curves most frequently were not only located above the contralateral motor area, but, as expected, were also identical to those showing the highest single sensor-based decoding performance (Fig. 4a, last panel).
We further decoded the EOG signal and found that the DA for EOG was very low (≤30.7%) and not significant (p < 0.01) throughout the whole trial (the significance level of p < 0.05 was crossed very briefly (for ∼90 ms) around movement end).
Discussion
We showed that the direction of small hand movements can be inferred from noninvasive MEG recordings on a single-trial basis with a reliability of 67%. This demonstrates that different movements of the same extremity yield different cortical activation patterns at the level of neuronal mass activity. Moreover, our equivalent results from simultaneous MEG and EEG recordings extend the applicability of directional tuning and the center-out paradigm from BMIs based on invasive to BMIs based on noninvasive recording techniques.
Noninvasively measured brain activity related to movements
Most previous EEG, MEG, and fMRI studies investigating the neuronal correlates of movements examined the differences in neuronal activity associated with the use of different parts of the body (Obermaier et al., 2001; Blankertz et al., 2003; Pfurtscheller et al., 2003; Naeem et al., 2006) or neuronal activity associated with one extremity but different parts of it (Deng et al., 2005).
Here, we demonstrate that it is possible to distinguish four brain activity patterns related to four distinct reaching movements of one and the same body part on the basis of both single-trial MEG and EEG recordings. So far, only very few MEG and fMRI (Kelso et al., 1998; Georgopoulos et al., 2005; Schaechter et al., 2006; Jerbi et al., 2007) and no EEG studies investigated the neuronal coding of kinematic parameters of a single effector by means of noninvasive recordings. Kelso et al. (1998) and Jerbi et al. (2007) found a relationship between MEG activity and movement speed but did not predict movements from neuronal activity. Georgopoulos et al. (2005) inferred movement trajectories from the MEG, recorded while subjects continuously copied a pentagon. Our study distinguishes from this investigation by a different movement paradigm, simultaneous MEG and EEG recordings, and an analysis of the topography of directional information: sensors providing relevant information cumulated above contralateral motor areas and the decoding performance decreased with distance to this area. The early (−200 to 50 ms) (Fig. 4a) increase of decoding performance at medial motor areas might reflect the activation of the supplementary motor area before primary motor cortex at which the directional information increases later (50–300 ms) (Fig. 4a) (cf. Amo et al., 2007).
Inferring movement directions in the time domain and comparison to other recording techniques
We gained the highest DA (67%) from the 3 Hz low-pass filtered MEG activity at bilateral motor areas. In Figure 7, the DA and DI for center-out movements are compared between different recording techniques.
Comparison of MEG, EEG, ECoG, LFP, and SUA with respect to DI about movement direction. The dotted curve reflects the function DI(DA) valid under certain assumptions (see Materials and Methods) and for a four-target paradigm. Decoding performances of MEG/EEG from this study, of ECoG (T. Ball, A. Schulze-Bonhage, A. Aertsen, and C. Mehring, unpublished observations), and of LFP and SUA (Mehring et al., 2003) are shown. DI values of LFP/SUA were computed from DA values obtained for an eight-target center-out movement paradigm (Mehring et al., 2003) by using Equation 8 (see Materials and Methods), because DA/DI values for a four-target center-out movement were not available from the literature.
The decoding accuracies for MEG were lower but still surprisingly close to those obtained from invasive recordings (Fig. 7). For the decoded information, however, the differences between invasive and noninvasive recordings were larger: the DI of ECoG was nearly twice and the DI of SUA and LFP was approximately thrice as high as for MEG. In addition, the DA for SUA (and to a lesser extent also for LFP and ECoG) might be increased by using additional recording sites, thanks to relatively little redundancy between channels. Most likely, the DA for MEG/EEG will profit less from a higher number of sensors. Based on the technologies currently available, these results demonstrate the superiority of invasive measurements for the development of BMIs. Nevertheless, our findings demonstrate that the clinically favorable noninvasive recording techniques, along with their inherent advantages, could be applied in BMIs using directional decoding.
In addition, the found DA of 67% for MEG might possibly be increased by combining different signal components (Rickert et al., 2005) and automatically selecting an optimal subset of sensors (Lal et al., 2004) and/or features (Garrett et al., 2003; Dornhege et al., 2004). Neuronal adaptation and habituation together with automatization during BMI control can increase classification performance further (Taylor et al., 2002; Carmena et al., 2003). With direct cuing of the four responses required after extensive training, components of the MEG/EEG pattern related to goal-directed motivational factors irrelevant for the specific movement direction might decrease and “pure” response-correlated components encoding directional tuning remain (Wood and Neal, 2007) and, hence, effectively become amplified.
A direct comparison under identical conditions (simultaneous recording from the same brain area) revealed that the DA and the DI for MEG and EEG did, effectively, not differ.
Interestingly, before movement onset significant directional information was available in the MEG but not in the EEG. Possible explanations might be the higher signal-to-noise ratio for MEG or directionally unspecific brain activity affecting EEG (noise contamination) but not MEG. The latter might occur if the corresponding brain activity originates from radial sources to which MEG is less sensitive (Hämäläinen et al., 1993).
Applying a classifier trained exclusively on data from known subjects to a novel, unknown subject without any further adaptation provided significant but very low DA (39%). This underpins the necessity of additional training of the classifier on data from the individual subject. In this respect, adaptive classification algorithms (Li and Guan, 2006; Shenoy et al., 2006; Vidaurre et al., 2006; Blumberg et al., 2007) initialized with training data from other subjects could be beneficial.
The substantial information about movement kinematics in the 3 Hz low-pass filtered MEG activity might reflect a general feature of neuronal population signals as corroborated by findings in LFP (Mehring et al., 2003; Rickert et al., 2005), ECoG (Schalk et al., 2007; Pistohl et al., 2008), and MEG (Jerbi et al., 2007) recordings. As can be seen in Figure 4b (subject 8), the 3 Hz low-pass filtered MEG signals differ with movement direction in peak amplitude, peak latency, and onset time of the evoked potential (all these parameters can be used by the RLDA classifier). The premovement part of this directionally sensitive signal might be related to the Bereitschaftspotential (BP) or readiness potential (Kornhuber and Deecke, 1965; Brunia and Van Boxtel, 2000), a term mostly used for self-paced but also for paced movements (Rockstroh et al., 1989). Previously, it has been shown that the BP can vary considerably depending on parameters of the movement (Birbaumer et al., 1990), and, thus, it might also be directionally modulated as the premovement signal observed here. Slow cortical potentials such as the BP occurring before and after a motor response consist of eight components with different neuronal sources and latencies. The lateralized component of the BP begins 500 ms before movement onset, originates in the cerebellar-thalamo-cortical loop [the “lateral system” of Goldberg (1985)], and provides context dependent adjustments of movement parameters as required in this experiment.
Despite the observed modulations at parietal areas, little directional information could be extracted from MEG sensors above this region. In contrast, previous studies obtained good directional decoding results using SUA/LFP recorded in the parietal reach region (PRR) (Andersen et al., 2004). The following reasons might account for this difference: First, no movement selective visual cues were presented in our study. The PRR exhibits several features of a movement-related area; however, with the PRR receiving direct visual input, directional- or target-dependent activity in this area might only be present if visual information about possible directions is provided. Second, the spatial organization of directional tuning in the PRR might prevent the detection of directionally dependent activity with the weaker spatial resolution of MEG.
The movement-related MEG signal in the time–frequency domain
We observed a power increase for low frequencies, which was also found in LFP recordings during center-out movements (Rickert et al., 2005). The observed decrease of β oscillations at sensorimotor areas shortly before up to the end of the movement is consistent with previous studies [e.g., LFP (Donoghue et al., 1998); ECoG (Crone et al., 1998a); EEG (Pfurtscheller, 1989); MEG (Salmelin et al., 1995)]. Furthermore, we observed a high-γ band MEG power increase during movement centered at contralateral motor cortex, consistent with previous ECoG and LFP studies (Arroyo et al., 1993; Crone et al., 1998b, 2006; Pfurtscheller et al., 2003; Mehring et al., 2004; Rickert et al., 2005) and a recent EEG study using visually cued finger movements (Gonzalez et al., 2006). However, there are differences in the exact frequency ranges in previous LFP/ECoG studies (Mehring et al., 2004; Rickert et al., 2005) using the same paradigm (center-out) as here and in contrast to the invasive studies, the MEG activity investigated here did not reveal any modulation in frequencies >90 Hz. A possible explanation for this effect might be that the recording situation in noninvasive measurements resembles that of a low-pass filter.
Inferring movement directions in the time–frequency domain
Decoding the MEG power modulations of the low-frequency band provided only little DA (34%), much less than the low-pass filtered activity in the time domain (67%). The reason for this difference might be the missing phase information.
Power modulations in the β and high-γ bands did not allow extracting any information about movement direction. In contrast, in similar studies using LFP (Rickert et al., 2005) and ECoG (Leuthardt et al., 2004), the high-γ band was informative. If the γ band modulations observed in the different studies originate at least partially from the same neuronal sources, one reason for the difference could be the lower signal-to-noise ratio for noninvasive recordings in combination with attenuated amplitudes for higher frequencies. Another reason might be that the directional tuning of high-frequency activity is averaged out if measured noninvasively: if the tuning changes spatially on a small scale, it might not be accessible anymore with the weaker spatial resolution of noninvasive recordings. In any case, the different properties of invasively and noninvasively measured γ band activity deserve further investigation.
In summary, our results show that single-trial decoding of hand movement direction can be achieved using noninvasive measurements of cortical activity. This indicates that also for noninvasive BMIs, neuronal signals related to specific movement parameters of hand control might be usable for equivalent controls of an external effector.
Footnotes
-
This work was supported by the WIN-Kolleg of the Heidelberg Academy of Sciences and Humanities, German Federal Ministry of Education and Research Grant 01GQ0761 TP 1, German Research Foundation Grants 550/B5 and C6, and the German Scholarship Foundation.
- Correspondence should be addressed to Stephan Waldert, Institute of Biology I, University of Freiburg, Hauptstrasse 1, 79104 Freiburg, Germany. waldert{at}bccn.uni-freiburg.de
