Multielectrode array implantation sites for the three NHPs used in the study. A 10 × 10 microelectrode array (400 mμ spacing) was chronically implanted at each site (Blackrock Microsystems). In all cases, arrays were positioned just anterior to the genu of the arcuate sulcus at the (mediolateral) level of the principal sulcus. 3D rendering of the cortical surfaces was generated based on 3T MRI scans (Brown MRI Research Facility) using BrainSight software (Rouge Research). CS, Central sulcus; AS, arcuate sulcus; PS, principal sulcus.
CGID task. Task timeline, showing the onset and offset of the object light, grip cue, and go cue. There are four types of trials, representing combinations of two objects and three grips (with both objects sharing one grip in common). For analysis purposes, the task was divided into five 1 s time periods.
Object and grip related information is reflected in the firing pattern of individual neurons. The activity patterns of two simultaneously recorded PMv neurons (Monkey G) are shown for each of the four object + grip combinations. In all cases, raster histograms are aligned to object presentation. Note that one neuron presents similar activity patterns for both power grips (bottom), whereas the other responds in a different way when the same grip is used with different target objects (top). Insets near the top right corner of each set of plots show the mean and 95% confidence interval of the waveforms assigned to each unit during spike sorting. The x-axis corresponds to 1.6 ms, whereas the y-axis represents ±100 μV. The SNR calculated for these units was 1.7 and 1.6 for units 12 and 39, respectively.
Single-units displaying object- and grip-related information. SSIMS projections were used to determine whether individual neurons presented significant grip/object related information (significant difference in median within and between category SSIMS distances; KW, p < 0.05). Object information was only evaluated for trials with the same grip (power) and different objects, whereas grip information was evaluated for different grips performed on the same object (key vs power or precision vs power). Each plot shows the number of neurons displaying exclusively object information, exclusively grip information, or both. The dashed line shows the number of false-positives expected for the given p value (5%). Data from each monkey is presented separately.
Clustering of ensemble states according to object and grip strategy in the CGID task: single-session examples. Each panel clustering of ensemble states in SSIMS for a given task phase (arranged horizontally) and monkey (arranged vertically). The five task phases are shown in chronological order from left to right. There are four types of trials, representing combinations of two objects and three grips (with both objects sharing one grip in common. Colors denote type of trial, as noted in the key. Each point represents the ensemble state for a single trial, incorporating 1 s spike trains from each recorded neuron. The distance between the points denotes the relative similarity between the ensemble states. Note the progression from a random mix of trial types (colors), to clustering according to object, and then object/grip in the later stages of the task. Each SSIMS plot was generated independently to emphasize the relationship between individual trials during each task phase (axes are therefore not consistent between plots). Neural trajectories showing the transition between phases in a single unified space are shown in Figure 7.
Decoding object and grip type. A nearest-neighbor classifier using leave-one-out cross validation was used to classify both grip type and object strategy at each time point based on SSIMS projections generated using partially overlapping 1 s windows (50 ms shifts). Results for each monkey are shown separately (mean over 4 sessions in each animal, with shading denoting SD). The upper bounds for the 99% confidence intervals (dashed lines) for object/grip classification were determined using 10,000 random shuffles of the trial labels (values differ because there are 3 grips and 2 objects).
SSIMS trajectories: linking objects to actions. The temporal evolution of ensemble activity is tracked by generating SSIMS projections from sliding time windows (1 s duration, sliding by 50 ms). The resulting space captures the relationship between activity patterns observed at different task phases. The mean trajectories for the four different types of trials are shown in different colors for one session in Monkey G (two viewpoints are provided to display 3D structure). Dots along the lines mark the 50 ms shifts in the time bins. Trajectories were smoothed with a Gaussian kernel (400 ms SD). Object presentation events, grip cues, and go cues, are highlighted by circles, stars, and squares, respectively. The graph in the boxed region shows a summary of the general pattern observed. Similar results were obtained for the other two monkeys.