Decoding Complete Reach and Grasp Actions from Local Primary Motor Cortex Populations

The dynamic grasping task elicits a wide range of upper limb movements. a, Movement was recorded by tracking 29 reflective markers attached to the monkey with mild water-soluble adhesive. b, Model of the hand and arm fit to the 3D position of the markers to calculate the joint angles for each frame. c, Distribution of measured grip apertures (top) and wrist angles (bottom). d, Projection of the position of the arm endpoint (proximal wrist marker) onto the coronal, sagittal, and transverse planes (dataset C1). Color saturation indicates the density of the points. Marginal distributions of density are shown along each axis. Crosses denote the mean and quartiles. The highest density of points (highest color saturation) was noted at the location where the monkeys typically rested their arm after each reach and grasp action.

The dynamic grasping task essentially decouples the DoFs of the hand, wrist, and arm. a, Correlations display the relative coupling between DoFs, which are weaker as distance between joints increases. Values for eight representative parameters averaged across monkeys are shown. s., Shoulder; e., elbow; w., wrist; int./ext. rot., internal/external rotation; pron./sup., pronation/supination; uln./rad., ulnar/radial deviation; flex./ext., flexion/extension. b, Dimensionality estimation for reach and grasp actions. This plot shows the cumulative percentage of variance accounted for as a function of the number of PCs used to represent the joint angles for each of four sessions. c, The firing rate of individual M1 neurons was only moderately correlated with any given joint angle. Correlation coefficients (CC) between the firing rate of each neuron and each measured joint angle are divided into three groups: those related to the arm (shoulder and elbow, 4 DoFs), the wrist (3 DoFs), and the hand (18 DoFs).

Neural correlations with multiple sets of kinematic parameters cannot be accounted for by correlations among the kinematics themselves. In all histograms, results are shown in solid gray for monkey C and in black outlines for monkey G. a, Coefficients of multiple correlation (R²) for each of the recorded neurons. These values can be interpreted as the fraction of variance in firing rates accounted for by linear relationships with the full set of kinematics (joint angles and joint angle velocities). Note that in all cases neural activity preceded movement, because the analysis was limited to predictive lags to approximate online decoding conditions. b–d, Semipartial correlation coefficients (sr²) for three sets of kinematic variables. These values can be interpreted as the amount of firing rate variance accounted for by one set of kinematic variables (arm, wrist, or hand) beyond that accounted for by the other two. Thus, b shows the variance in neuronal firing rates explained exclusively by arm kinematics after removing the variance explained by hand and wrist variables. c and d show similar distributions for wrist and hand joint angles. e, Overlap between groups of neurons that show statistically significant semipartial correlations >0.5 (representing >5% of the variance in firing rate) to arm, wrist, or hand kinematics. Values for monkeys C and G are shown separately in each partition. Overall, 86.7% of the neurons recorded in monkey C and 75.4% of the neurons recorded in monkey G displayed a set-specific relationship with at least one set group of kinematic variables. Note that the areas in the Venn diagram are not to scale. MAD, Median absolute deviation.

Single-unit activity in M1 during naturalistic reaching and grasping movements. Raster plot showing the action potentials fired by 25 individual neurons over a span of 5.5 s. Raster plots are colored according to set specific relationships with kinematics (significant semipartial correlations >0.05; for details, see Results, Neuron tuning properties): blue, hand only; green, wrist only; red, hand only; orange, hand + wrist; magenta, hand + arm; black, hand + arm + wrist. The data shown corresponds to session G2. Of the 30 neurons recorded in M1, only the 25 shown had set-specific relationships to at least one set of kinematic parameters. During the time span shown, the monkey performed three separate reach-to-grasp movements targeting a small ball. Grip aperture measures are overlaid over the raster plot to highlight the moments when prehension occurs (troughs). The full posture of the arm is shown for four time points in the first movement.

Most neurons were not preferentially related to specific subsets of kinematic parameters, even after removing the influence of correlations between kinematics. In all panels, results are shown in solid gray for monkey C and in black outlines for monkey G. a, Index of kinematic selectivity for arm joints (for details, see Results). These values are a ratio of the variance in firing rates exclusively represented by arm joints (semipartial correlation for arm kinematics) over the total set-specific variance (the sum of semipartial correlations for the arm, wrist, and hand). b, c, Index of kinematic selectivity for wrist and hand joints, respectively. MAD, Median absolute deviation.

M1 neurons are simultaneously tuned to proximal and distal DoFs. Pairs of measured DoFs were discretized into 100 equally sized 2D bins spanning 99% of the data (after removing outliers at each extreme). Colored circles show the average firing rate for each bin (color was interpolated for the space between the circles). Each row shows the activity of a single M1 neuron. The first column shows firing rate as a function of shoulder elevation and grip aperture. The second and third columns have similar displays, using grip aperture versus wrist pronation/supination and shoulder elevation versus wrist pronation/supination.

Optimal decoding was accomplished using populations of 30 neurons to reconstruct each DoF. a, Mean r (over all variables) between measured and decoded kinematics as a function of the number of neurons used to decode each DoF. Neurons were added to the models in order from most to least correlated with the DoF being decoded (following the “greedy”' selection method described in Results, Arm and hand movement reconstruction). Based on this graph, the final set of decoding models were constructed using 30 neurons for each DoF. The total number of neurons used for decoding in each session (pooled across all DoFs) is shown next to each plot as a fraction of the total number of neurons recorded. For details on the overlap of populations used to decode arm, wrist, and hand movements, see supplemental Figure 7 (available at www.jneurosci.org as supplemental material). b, Error distribution across all joint angles reconstructed across all test datasets using sets of 30 neurons (solid gray histogram for monkey C and outlined black histogram for monkey G). Triangles along the x-axis denote the range that encompasses 95% of the data for each distribution (green for monkey G and red for monkey C). More than 50% of the reconstructed joint angles were within 5° of the measured values. MAD, Median absolute deviation.