In grasping, the CNS controls a particularly large number of degrees of freedom. We tested the idea that this control is facilitated by the presence of muscle synergies. According to the strong version of this concept, these synergies are invariant, hard-wired patterns of activation across muscles. Synergies may serve as modules that linearly sum, each with specific amplitude and timing coefficients, to generate a large array of muscle patterns. We tested two predictions of the synergy model. A small number of synergies should (1) account for a large fraction of variation in muscle activity, and (2) be modulated in their recruitment by task variables, even in novel behavioral contexts. We also examined whether the synergies would (3) have broadly similar structures across animals. We recorded from 15 to 19 electrodes implanted in forelimb muscles of two rhesus macaques as they grasped and transported 25 objects of variable shape and size. We show that three synergies accounted for 81% of the electromyographic data variation in each monkey. Each synergy was modulated in its recruitment strength and/or timing by object shape and/or size. Even when synergies were extracted from a small subset of object shape and size conditions and then used to reconstruct the entire dataset, we observed highly similar synergies and patterns of modulation. The synergies were well conserved between monkeys, with two of the synergies exceeding chance structural similarity, and the third being recruited, in both animals, in proportion to the size of the object handled.
This study concerns the manner in which the nervous system controls hand movements. The primate hand possesses many degrees of freedom, which give it flexibility in interacting with the environment but complicate its control. Motor primitives are a possible heuristic solution to this problem (Sherrington, 1906). Such primitives may take the form of muscle synergies, or invariant patterns of muscle activity, that could serve as modular building blocks for more complex motor plans (Tresch et al., 1999).
Evidence for muscle synergies has been demonstrated in limb control. In lower vertebrates, movements elicited by cutaneous or spinal stimulation appear to have synergistic patterns of muscle recruitment (Tresch et al., 1999; Saltiel et al., 2001). A synergistic organization has also been observed in natural behaviors of the frog (d'Avella et al., 2003; d'Avella and Bizzi, 2005). Even in humans, combinations of synergies have been shown to capture much of the activity in proximal arm muscles during reaching (d'Avella et al., 2006), and in trunk and leg muscles during responses to postural perturbations (Torres-Oviedo and Ting, 2007).
This investigation extends the synergy model to a particularly complex system, the hand. electromyographic (EMG) activity was recorded from arm and hand muscles of two monkeys as they interacted with a variety of objects. The optimization procedure of d'Avella et al. (2003) was applied to test the degree to which the EMG patterns could be reconstructed by linear combinations of synergies. Here, each synergy represented a set of time-varying activation profiles across muscles, and each was recruited with an amplitude and timing coefficient specific to each condition.
We tried to elicit diverse patterns of muscle activation by sampling from the large variety of postures which the primate hand affords (Napier, 1956). In particular, we included a wide array of objects, and had the monkeys perform a sequence of naturalistic reach, grasp, and transport movements with each. Although we did not train the animals to produce distinct hand postures (Klein Breteler et al., 2006), our design enabled us to systematically investigate the modulation of muscle synergies as a function of object shape and size.
We tested two main hypotheses, each a prediction of the synergy idea: that time-varying synergies would account for a significant amount of the EMG variation, and that the synergy coefficients would be modulated by object properties. We investigated whether these predictions would hold when synergies were extracted from only a subset of object conditions, and were then used to reconstruct data from all object conditions.
Studies have also suggested that muscle synergies may be partially conserved across animals (d'Avella and Bizzi, 2005; d'Avella et al., 2006). However, such conservation may not hold for systems more complicated than the limb, or for other definitions of muscle primitives. When grasping EMG data were decomposed into principal components, Brochier et al. (2004) found them to be generally dissimilar between monkeys. We thus tested a third hypothesis, consistent with previous literature but not a requirement of the synergy idea, that the synergies would be conserved across animals.
Materials and Methods
We collected EMG and behavioral data from two rhesus macaques (Macaca mulatta) while they reached for, grasped, transported, and released objects of various sizes and shapes. In the text that follows we refer to these monkeys as G1 and G2 (a 5.9 kg, 8-year-old female and a 6.5 kg, 4-year-old male, respectively). Throughout the recordings, the monkeys were head-fixed by means of an implanted cranial post, but able to see a tray in front of them on which surface they performed the task. All procedures were approved by the Massachusetts Institute of Technology Committee on Animal Care.
Each animal was trained to use its left hand (1) to press a start button located on the left side of its workspace, then (2) within 1.0 s to reach for and remove a clearly visible object in one of two 3-cm-deep, 7-cm-wide wells on the tray, and finally (3) to transport and, within 1.0 s, release it in the opposing tray well centered 20 cm away, where it had to remain stably for at least 0.1 s without bouncing out (see Fig. 1). During the transport, the monkey's hand had to clear a plastic barrier 5 cm high that divided the left and right halves of the tray. The monkey's right hand was not able to participate in the task given another plastic barrier. After receiving its reward (0.2–0.3 ml of water or apple juice) the monkey was able to initiate a new trial at will by pressing the start button. After every 10 successful trials, we removed the object and replaced it with another (or, during training, with the same, canonical sphere). Within such 10-trial blocks, objects were presented for as many trials as were required to achieve 10 successful reach plus transport movements in either direction. In the analysis that follows, we consider only trials that delivered the object to the left target well, to limit systematic data variation to the object shape and size dimensions described below. The selection of objects across blocks was pseudorandom except that objects were balanced in frequency within each day's object list (including either all the objects or a subset including at least 13 of these objects), and that the same object could not be presented for two or more blocks in a row. The reach and transport movements emerged spontaneously and early in training, and were generally performed well within the time limits. Qualitatively, it appeared that most of the training was directed instead toward reminding the monkey to press the start button, which we used to trigger EMG recording while the monkey's hand was at a common starting location. The recordings from monkey G1 comprised 12 recording sessions spanning 30 d; those from G2 comprised nine sessions spanning 15 d.
We attempted to explore a wide space of grasp behaviors by presenting the monkeys with a rich set of objects. Among these objects we systematically varied object parameters (displayed schematically in Fig. 1F), in particular shape and size. The 25 objects were custom-designed shapes fashioned of Delrin plastic (density 1.4 g/cm3) and weighing between 2.8 and 102 g. The canonical sphere used in training was 34 g in mass and 3.6 cm in diameter The 25-object set included this training object and four other spheres of variable diameter (denoted Sx; range, 1.6–3.6 cm), five cubes of variable width (Cx; range, 1.5–3.6 cm), and five cylinders that spanned each of three dimensions (height Hx, 0.6–5.7 cm; uniform diameter Dx, 1.3–3.8 cm; and inner diameter Ix, 0.6–3.2 cm). One of the cylinders (D5) was actually common to each of the three cylinder dimensions, but for simplicity is treated here as belonging only to the Dx class. (The number in the object code denotes the relative magnitude of the variable dimension according to a linear scale, e.g., cube width incremented by 0.5 cm.) Given each monkey's exposure to a subset of 13 of the objects before these recordings, and the lack of any measurable adaptation to any of the objects during these recordings, we consider each animal's behavior to be reasonably stable over the presentation of these objects [as in the study by Mason et al. (2004)].
The base of each well was a conical surface, with a 40° slope down to its center that caused the objects to settle to the same location over trials. In the case of the cylinders, one end of the object would typically settle near the base of the well, with upper end of the object resting beyond the lip of the well. (Such a configuration was invariably the case for the longest cylinders, Dx and Ix, and with a lesser frequency for Hx cylinders.) For trials with these objects, the experimenter would manually standardize the orientation of the object before each trial, so that the upper end pointed toward the monkey's right (see Fig. 1C). We used digital video footage taken by an overhead video camera (G1, DCR-TRV17; G2, DCR-HC46; Sony, Tokyo, Japan) to quantify the consistency of this object orientation. Included in this footage were all 40 hits per condition for monkey G2, and 34 ± 8 hits per condition for monkey G1 (because for three of the days of recording, only partial footage was available). We found that the object direction (mean ± circular SD) (Fisher, 1993) across all these cylinders was indeed nearly rightward, specifically 10 ± 9° (monkey G1) and 2 ± 5° (G2), where 0° refers to the rightward direction viewed from above the well and angles increase in a counterclockwise direction. There were still significant interobject differences in rotation (G1, Y(9) = 129.32, p < 0.0001; G2, Y(9) = 24.70, p < 0.01), as shown by circular tests for common mean direction (Fisher, 1993). But circular–linear rank correlation tests (Mardia, 1976) revealed no significant correlation between object rotation and the coefficients of any of the synergies (see below, Synergy extraction) for either monkey (p > 0.01, corrected for multiple comparisons). Thus, the muscle synergies evident when the monkeys handled these cylinders appeared to be unaffected by the initial placement of the object.
Basic behavioral information was collected in the form of event times, such as the time of button release and reward dispensation. Two photosensors (E3T-SR12; Omron, Kyoto, Japan) mounted orthogonally and at staggered heights in each well provided information on times of object removal and object deposit in the target well. We used the button, object removal and reward event markers to divide each trial into the initial reach component followed by the object transport time (see Fig. 1A).
The 15 (monkey G1) or 19 (G2) EMG electrodes included 12 implanted in the same muscle of each monkey, and three or seven implanted in unique muscles or muscle divisions in either monkey. Both muscle sets included muscles that were intrinsic to (i.e., originated within) the hand as well as extrinsic hand and wrist muscles with either a primarily flexor or extensor action, and more proximal muscles acting on the elbow or shoulder. Two muscles (G1's flexor digitorum profundus and G2's triceps) were also implanted in multiple locations given known anatomical and functional divisions (Howell and Straus, 1971). The following abbreviations were used. Proximal muscles acting on the shoulder and wrist included the deltoideus (Del), pectoralis major (Pec), triceps brachii ulnar short head (TriU), triceps brachii radial short head (TriR), biceps brachii longus (Bic), and brachioradialis (BR). Wrist and extrinsic hand extensors included the extensor carpi radialis brevis (ECRB), abductor pollicis longus (AbPL), extensor digitorum communis (EDC), extensor digiti secundi and tertii proprius (ED23), extensor digiti quarti and quinti proprius (ED45), and extensor carpi ulnaris (ECU). Wrist and extrinsic hand flexors included the flexor carpi radialis (FCR), flexor digitorum superficialis (FDS), ulnar flexor digitorum profundus (FDPU), radial flexor digitorum profundus (FDPR), and flexor carpi ulnaris (FCU). Intrinsic hand muscles included the abductor pollicis brevis (AbPB), adductor pollicis (AdP), opponens pollicis (OpP), flexor digiti quinti brevis manus (F5B), and opponens digiti quinti manus (Op5).
EMG electrode implantation.
We implanted intramuscular bipolar EMG electrodes in each monkey over three surgical sessions. Surgeries were performed under anesthesia (0.05 mg/kg atropine and 10 mg/kg ketaset injected intramuscularly, followed in G1 by 5 mg/kg sodium pentobarbital intravenously or in G2 by inhalation of 1–2% isoflurane with 2 L of O2). We incised the skin overlaying each group of muscles to be implanted. We generally identified muscles based on size, fiber direction, and relative anatomical location. For intrinsic hand muscles, we used either a hand-held bipolar needle electrode or the implanted intramuscular electrode itself to electrically stimulate the muscles to make functional identification. The electrodes were made of two Teflon-coated 50 μm stainless steel wires anchored together under the muscle belly by means of a ∼2 mm wax ball covered by a thin layer of epoxy resin for strength. A short length of each wire was stripped of 1–3 mm of insulation where it was to pass through the muscle belly. We led the electrode wires subcutaneously (Park et al., 2000) to a cranially mounted circular connector (Cooper Industries, Salem, NJ).
The EMG data from all the muscles implanted in each monkey were recorded between button press and reward signals. The data were bandpass-filtered (10 Hz high-pass and 1000 Hz low-pass), notch filtered (60 Hz), and amplified (5000×) in a differential manner using a programmable signal conditioner (CyberAmp 380; Molecular Devices, Sunnyvale, CA) controlled by software (CyberControl; Molecular Devices). Data were then digitized at 2 kHz using a data acquisition board (NI PCI-6035E; National Instruments, Austin, TX) controlled by custom-written software (LabVIEW; National Instruments).
Muscle cross talk.
Each monkey's muscle set excludes additional muscles whose signal quality, as assessed by off-line analyses of muscle cross correlation, had degraded sufficiently by the time of these recordings to warrant their exclusion from the dataset. Such cross correlation can indicate artifactual cross talk between channels, as well as synchronous firing (Kilner et al., 2002). The cross correlation of any given pair of EMG channels was calculated as the cross correlation between their unfiltered EMG data at a lag of 0 s, averaged across all 1000 trials. For each monkey, we rejected as many channels as necessary, and in whatever combination possible, to bring the maximum cross correlation between any pair of channels below 0.3 (Brochier et al., 2004). Of 24 EMG electrodes implanted in each monkey, this required exclusion of nine channels in monkey G1 and five in G2. The maximal remaining cross correlation among monkey G1's channels was 0.24, and among G2's was 0.26.
The EMG data in each trial were centered by median subtraction within each channel, high-pass filtered (50 Hz) to remove low-frequency movement artifacts, full-wave rectified, and low-pass filtered (20 Hz). The data were further bin-averaged over 9 ms (G1) or 11 ms (G2), thereby equalizing the quantity of data points contributed by each monkey given the difference in their trial window durations (see below, Trial alignment). The amplitude of each channel was then normalized to the maximum integrated EMG in that channel over all object conditions.
We analyzed data only from leftward-directed trials that were rewarded according to the task criteria (quoted above, in Task). Among these trials, we selected an equivalent number of trials in each object condition (defined by object shape and size), to balance the contribution of each object condition to the dataset. We chose to use 40 trials in each object condition, the minimum number performed by either monkey in any condition, with reach and transport times that were typical for that object condition. Specifically, the trials were those 40 having , where dreach was the reach duration on the trial and was the mean reach duration over all trials in that object condition, and similarly for dtransport and . We expected this selection to reduce EMG variation because of nonsystematic variability in movement times (cf. systematic manipulation of task variables).
The EMG signals were time aligned on the time of object removal from the “origin” well (see Fig. 1A). To equalize the overall quantity of samples contributed by each object condition beyond the trial selection described above, we restricted the EMG data to a fixed window around the time of object removal. We chose this window to be either [−0.35 0.55] s (G1) or [−0.5 0.6] s (G2) around object removal (i.e., 0.9 or 1.1 s total). These durations were set at the median of each monkey's distribution of reach durations (before object removal) and transport durations (after removal). These windows excluded the majority of the muscle activity related to button press or reward events. The difference in windows between the animals reflects the fact that monkey G1 was slightly faster than monkey G2, particularly in its reach and grasp movements. For the purpose of synergy extraction (but not data reconstruction), each monkey's 40 selected trials within each object condition were averaged together (see Fig. 3). Reach or transport durations that exceeded the 0.9 s (G1) or 1.1 s (G2) total window were only included in the average for the duration they spanned.
We used an iterative decomposition algorithm to identify a common set of time-varying synergies underlying the generation of each monkey's muscle patterns for reaching to and transporting the entire set of objects. According to the time-varying synergy model, the time sequence of muscle activation vectors for the jth object condition [mj(tk), where tk is the kth time sample in a trial] is generated by the combination of N sequences of S synergy vectors, or time-varying synergies [wi(ts); where i = 1,…,N, and ts represents the sth time sample in the sequence]. These synergies are each scaled in amplitude (cij) and recruited with an onset delay (tij) dependent on the object condition: In matrix form, the time-varying synergy vectors wi(ts) define synergy matrices Wi (as depicted in Fig. 5). The decomposition algorithm (d'Avella et al., 2003), implemented in Matlab (MathWorks, Natick, MA), finds a set of N time-varying synergies, and (for each synergy and object condition) one non-negative amplitude scaling coefficient and one time-shift coefficient, by iteratively minimizing the total reconstruction error: The three steps performed by the algorithm on each iteration (time-shift coefficient selection, amplitude scaling coefficient update, and synergy update) were repeated until the reconstruction R2 increased by <0.001 over 10 iterations. (Across extractions and monkeys, this required 96 ± 65 repetitions, mean ± SD.) The entire algorithm was itself repeated five times for each extraction. Although we selected only the set with highest EMG variation explained for further analysis, the synergies from each repetition tended to be highly similar, with a 0.98 ± 0.02 scalar product between synergies from different repetitions (computed at whichever time difference maximized the product of the synergies, across muscles). For the selected extractions, we also segregated EMG variation explained according to approximate muscle origin (proximal, wrist/extrinsic, or intrinsic hand) and applied a one-way ANOVA to assess the effect of this muscle grouping on reconstruction fidelity.
Synergy parameter selection.
The two free parameters of the decomposition, the number of time samples for each synergy S (synergy duration) and the number of synergies N (synergy set dimensionality), were chosen as follows. The synergy duration was chosen to be equal to 50 samples of data (i.e., 0.45 s for monkey G1 and 0.55 s for monkey G2), approximately the duration of the longer phasic bursts we observed in the data (see Fig. 3). As single synergies alone were insufficient to span the duration of each trial, we only consider extractions with at least two synergies. To arrive at an estimate of appropriate dimensionality of the synergy set, we adapted the method of Cheung et al. (2005). We divided the dataset of each monkey into two subsets (here, three-quarter and one-quarter portions), and then fit two through eight synergies derived from the former dataset to the latter dataset. This procedure was repeated five times (d'Avella et al., 2003), each time using a different, random division of the dataset (subject to the constraint that each dataset be based on an equal number of trials from each object condition, i.e., either 30 or 10). We then took each monkey's average R2 curve explained by two through eight synergies across these five repetitions (see Fig. 4), and fit the mean values to a straight line. We iteratively fit this line between the R2 curves extending from two to eight synergies, three to eight, four to eight etc. until the mean squared error remaining after the fit fell below a threshold of 1 × 10−5.
In addition to visual comparison of synergies extracted from the two monkeys, we also used a more quantitative method to determine their similarity (d'Avella et al., 2003, 2006). In particular, the scalar product of each synergy in monkey G1's set was computed in relation to each synergy of G2's set (within the subset of corresponding muscle channels, and then averaged over these channels). This scalar product was computed at every possible time difference between the two synergy structures, and the final similarity score was taken as the maximum of these products. Once the best match had been found between any synergy of G1 and any synergy of G2, the synergies involved were removed from the sets, and the procedure was repeated among the remaining synergies. The three or seven muscles implanted uniquely in either monkey were not included in comparisons of synergies from the two animals. The minimal scalar product one might expect between different synergies would not be zero, particularly given the non-negative average activation profile ascribed to each muscle within the synergies. To find a chance level of similarity, we took the synergy set extracted from each monkey, and randomized the order of muscles within each synergy before calculating its “auto-similarity.” We repeated this procedure five times for each synergy shown in Figure 5, and by doing so found an average scalar product across synergies and repetitions of 0.78 ± 0.05 (G1) and 0.79 ± 0.13 (G2). We consider the more conservative of these values, 0.79, to be the threshold for deeming similarity between the two monkeys' synergies significant.
To determine the covariations of each synergy's amplitude and timing coefficient with object shape and size, we used an analysis of covariance (ANCOVA) in which object mass was the (continuous) predictor and shape was the (discrete) grouping variable of the coefficient data. All effects are reported, with a significance threshold of p < 0.01. The coefficients were derived from the fit of each monkey's 1000 (5 × 5 × 40) nonaveraged trials by the synergies extracted from their within-condition-averaged data (see “Synergy extraction”, above). We qualitatively compared the coefficient modulations observed in this reconstruction with those in reconstruction of the same data by synergies derived from only three, rather than all 25, object conditions. Specifically, we fit the data using synergies derived from trials performed with the three midsized cylinders H3, D3, and I3.
As described in Materials and Methods, two monkeys (G1 and G2) reached for, grasped, and transported plastic objects between two wells located in front of them on a tray. After multiple days of practice with a training sphere, the monkeys were exposed to a set of 25 objects, presented in a pseudorandom manner over 12 (monkey G1) or 9 (G2) additional days. The monkeys were required to reach each object within 1 s and then bring it to rest in the target well within an additional 1 s, and to complete a block of 10 such successful trials or “hits” before being given a new object (Fig. 1A).
The objects presented to the monkey included the original training sphere and a number of other spheres and cubes of variable size, and cylinders of variable height, diameter, or concavity (Fig. 1F). In Figure 2, we give traces of each monkey's hand posture at the time it removed objects from the origin well. The figure includes sample trials with the training sphere, a midsized cube, and the largest cylinder (common to each of the three cylinder classes), as well as the smallest sphere and cube and narrowest cylinder (Fig. 2D). Although we do not attempt a quantification of hand kinematics here, grasp posture appeared to vary with object shape and size, and with animal. At least the smallest sphere and cube (Fig. 2D) appeared to elicit precision grips in which the object was suspended between the monkey's thumb (occluded under its palm and the object) and index finger.
The monkeys could choose to alternate leftward and rightward transport movements as they wished (i.e., they were not required to treat the target well of the previous trial as the origin well of the current trial). In the analysis that follows, we focus on trials involving leftward transport movements, to limit task variability to the more richly sampled object shape and size dimensions. Of the 9278 and 2799 hits performed by monkey G1 and G2, 49 and 47%, respectively, involved leftward transportation. As measured by the fraction of trials which met the timing requirements above, the monkeys performed with 89% (G1) and 76% (G2) accuracy. Monkey G1's better performance may be attributed to its greater number of days and trials per day to practice.
To equalize the contribution of each object to the data pool, we selected 40 trials in each object condition defined by object shape and size (40 × 5 × 5 = 1000), and reduced each trial to an equivalent time window. Note that the 1-s time limits we had imposed on both the reach and transport movements were well in excess of the measured movement latencies. For instance, the median of the reach and transport durations were for 0.35 s and 0.55 s for monkey G1, respectively, and 0.5 s and 0.6 s for monkey G2. Nevertheless, monkey G2's trials, and in particular its reach-to-grasp movements, were longer than those of monkey G1. To select equivalent behavioral windows for each monkey and object, we balanced the duration of the data windows (0.9 s and 1.1 s, respectively, based on the percentiles quoted above) and the data integration bins (9 and 11 ms).
While each monkey was performing the task, we recorded EMG activity from 15 to 19 bipolar intramuscular electrodes implanted in the hand, arm, and shoulder. While three (G1) or seven (G2) of the electrodes in each monkey were implanted in muscles or muscle divisions unique to that animal, 12 of the implantations were common to the two monkeys. In Figure 1B we plot EMG data from a sample trial of one monkey, G2, wherein it handled a cylindrical object. The behavioral event times superimposed on the EMG data include the monkey's release of the start button (Fig. 1C), object removal from the origin well (Fig. 1D), and arrival at the target well (Fig. 1E), as shown by the accompanying tracings. We aligned all trials on the time of object removal before restricting the data to the fixed time windows described above.
In the sample trial shown in Figure 1B and in plots that follow, muscles have been vertically sorted into groups of proximal shoulder and elbow muscles (top), extensors and flexors of the wrist or extrinsic to the hand (middle), and then muscles intrinsic to the hand (bottom). In the analysis that follows we focus on the reach and transport phases of each trial, excluding most of the button press and reward related activity outside of these phases. The wrist and extrinsic extensors and flexors were each active in multiple, staggered phases of activity, whereas the proximal and intrinsic muscles appeared to be modulated somewhat independently of these muscles. The wrist and extrinsic hand flexors were most active as the hand approached and grasped the object, whereas the extensors were most active as the hand approached the target well and released the object. Apart from button-related activity, the intrinsic hand muscles as well as the proximal muscles tended to be most active while the hand was in contact with the object.
Similar trends were evident in the EMG activity after filtering, median subtraction, rectification, integration, normalization, alignment on object removal, and averaging over trials in each object condition. In Figure 3 we show such activity from the muscle channels common to both monkeys, for several sample task conditions. In the conditions of Figure 3A, monkey G1 handled the medium-sized object in each object shape subset. In Figure 3B, monkey G2 handled cylinders of variable scale but related shape (including the midsized cylinder plotted in Fig. 3A). At the level of individual muscle recruitment, and to the degree that two implants could be equated across animals, it was apparent that the two monkeys did not use identical muscle recruitment strategies, although the profiles of several muscles appeared moderately conserved (e.g., AbPL, FDPU, and OpP). The variation in muscle activity between monkeys was considerably greater than the variation between object conditions performed by the same monkey. Indeed, object properties appeared to exert only a modulating influence on the underlying pattern of muscle recruitment.
From each monkey's EMG activity in all 25 object conditions (each as shown in Fig. 3) we extracted between two and eight time-varying muscle synergies. In Figure 4, we plot the amount of EMG variation within this dataset that was accounted for when the same data were reconstructed by a linear summation of the individually weighted and timed muscle synergies. Three synergies appeared to define an appropriate dimensionality for each monkey, and explained 81% of the EMG variation within each animal's dataset. This dimensionality agreed with a cross-validation analysis (Cheung et al., 2005) in which we extracted synergies from datasets containing a random selection of three quarters of the trials in each object condition, and then fit these synergies to the dataset derived from the remaining trials. We repeated this procedure five times, and averaged the resulting R2 curves. The right side of the mean cross-validation curve, not shown but similar to that in Figure 4, could be fit by a straight line (with <1 × 10−5 mean squared error) only if this line began at or to the right of the three-synergy point of the curve. Three such synergies could explain 78 ± 3% (G1) or 80 ± 1% (G2) of the variance among the fit data.
We investigated whether the EMG variance among relatively proximal and relatively distal muscles was explained equally well by three synergies. After grouping reconstruction errors over the entire dataset by approximate muscle origin (proximal, extrinsic to the hand and wrist, or intrinsic to the hand) (see Materials and Methods), we found no relation between such muscle grouping and reconstruction fidelity. Averaging over both monkeys, the EMG variation explained among proximal muscles was 71 ± 22%; among wrist and extrinsic hand muscles, 83 ± 12%; and among intrinsic muscles, 81 ± 11%. A one-way ANOVA revealed no significant effect of muscle grouping on reconstruction error for either monkey (p ≥ 0.2).
The synergies that we extracted from the entire dataset are shown in Figure 5. Each synergy appeared to capture a different set of monophasic or biphasic relationships between the muscles. Each of the three synergies of monkey G1 could be matched uniquely to one of monkey G2, with scalar products (from left to right) of 0.90, 0.68, and 0.81. The first and third scalar products exceeded the threshold for significant similarity (see Materials and Methods), defined as the average degree of “auto-similarity” for each synergy after randomization of its muscle order (0.79). The first and third pairs of synergies were most similar to each other with a 0 s relative lag; the second pair of synergies was most similar when monkey G1's was shifted −0.02 s with respect to monkey G2's.
Although these similarity values do not imply that any of the synergy structures were identical between monkeys, the paired synergies did appear to possess broad similarities in the relative phase and combinations of chief muscles. The first pair of synergies shown in Figure 5 for each animal, W1, was dominated in both monkeys by tonic or biphasic activity across the proximal, wrist and extrinsic hand flexor, and intrinsic thumb abduction and fifth-digit muscles. Although the second synergy pair, W2, did not meet the similarity threshold, it was nevertheless dominated in both monkeys by two phases of wrist and extrinsic hand muscle activity. However, in monkey G1 these phases involved coactivation of most of these muscles, whereas in monkey G2 flexors defined the first phase while extensors defined the second, with proximal and intrinsic muscles coactive to a less distinct degree in one phase or the other. Finally, the third pair of synergies, W3, involved a phase of activation of the wrist, extrinsic hand extensor, and most intrinsic hand muscles accompanied by relatively tonic activity in proximal muscles, particularly triceps (and in the distal flexors sampled in monkey G1).
Modulation of synergy coefficients
The synergies shown in Figure 5 were extracted from datasets including 25 trial averages representing task conditions defined by variable object shape and size. Figure 6 depicts the reconstruction of several of these conditions experienced by monkey G1, in which it transported variously shaped, midsized objects (as in Fig. 3A). Figure 7 presents averaged trials in which monkey G2 transported cylinders of variable height (as in Fig. 3B). Variation in the averaged EMG activity of each trial condition appeared qualitatively to be well captured by small but systematic changes in these coefficients.
Although the underlying pattern of muscle activation was similar for each of the trial conditions performed by a given monkey, inspection of the synergy amplitude coefficients shown in Figure 7 indicates that these coefficients could vary with object properties. Most notably, synergy W2 appeared to be recruited with greater amplitude for larger objects. We investigated such relationships after reconstructing each monkey's full set of 1000 nonaveraged trials by linear combination of the three synergies. Although object size can be defined along a continuous dimension like mass, effects of object shape are less easy to parameterize because of the categorical manner in which shapes are defined. Nevertheless, we were able to confirm effects of both of these variables by applying a univariate ANCOVA to each monkey's coefficients, in which object mass was treated as a predictor of the coefficient data, grouped by object shape. [A similar pattern of significant results was obtained when coefficients were regressed not to object mass but to distance along the critical dimension of each object shape class, e.g., the diameter of the sphere (data not shown).] Because of the large number of results, we report the statistics in Table 1 and discuss a subset of the results in the following text.
Figure 8 summarizes each synergy's modulation of amplitude coefficients with object properties in this reconstruction. As confirmed by the ANCOVA results (Table 1), there was a significant effect of object size on the c2 coefficients for both monkeys, reflecting greater recruitment of this synergy for larger objects. The remaining scaling coefficients, c1 and c3, displayed a weaker dependence on object size, and to a different extent in each monkey. For each synergy there were also main effects of object shape on the scaling coefficients, and in the case of c1 and (for G1) c2, also significant shape by size interactions. For instance, the direction of the c1 versus mass relationship was different for spheres and for cylinders of variable inner diameter (Fig. 8), although the actual direction of such c1 and c3 covariations differed between monkeys.
In Figure 9 we depict the relationship between the timing coefficients and object properties. These coefficients (i.e., the synergy onset times) are given relative to the time of object removal from the origin well. (Note that these times refer to the onset of the synergies shown in Fig. 5, and not to particular phases of muscle activity within these structures which could follow the onset times with some delay, e.g., in the case of W2.) For both monkeys, there was a consistent relationship in the timing of the three synergies. In particular, with its early t1 the first synergy was largely limited to the reach phase of the trials (G1, −0.31 ± 0.11 s; G2, −0.45 ± 0.16 s). The second synergy spanned the time of object contact, given its intermediate timing coefficient t2 (G1, −0.10 ± 0.07 s; G2, −0.19 ± 0.10 s). The onset time t3 followed immediately after object removal, and thus the third synergy was restricted to the transport phase (G1, 0.04 ± 0.16 s; G2, 0.01 ± 0.17 s). Note that the relatively smaller variability in t2 may be attributable to this synergy's temporal proximity to the time of object removal from the origin well (i.e., the time used to align the trials from which the synergies were derived).
Compared with the amplitude coefficients, the onset times were generally less sensitive to object properties, with modulations on the order of 0.1 s (Fig. 9). Statistically, the second synergy's onset times registered the strongest effects of object shape, whereas the onset times of the first synergy and monkey G1's third synergy were more sensitive to object size (Table 1). The relationships between t3 and object mass generally tended to mirror those of t1 (Fig. 9). Thus, for both monkeys, when the first, reach-related synergy was observed relatively early (e.g., for large spheres), the third, transport-related synergy was observed relatively late after object removal.
The modulation of the coefficients with object properties was not a trivial consequence of the covariations between muscle activity and object variables that existed within the data used to extract the synergies (and that are evident in Fig. 3). We repeated the above analysis using only the data from an arbitrary subset of similarly sized objects (the midsized cylinders H3, D3, I3; mass 46–52 g) to extract three synergies. We then used these synergies to reconstruct the entire dataset (where mass ranged 3–102 g). In doing so we found that the synergies (data not shown) were highly and unambiguously similar in structure to those presented in Figure 5, with scalar products of 0.94–0.99 (G1) or 0.91–0.94 (G2). From this it was apparent that the same dimensions of EMG variation were present in both the full and reduced datasets (i.e., with or without the full range of object sizes and shapes present). More importantly, we found a similar pattern of modulation among the synergy coefficients when these synergies were fit to the entire 1000-trial datasets. For simplicity we show only the amplitude coefficients, in Figure 10. Qualitative comparison with Figure 8 suggests that the same relationships with object properties existed when these trials were fit by synergies derived from trials performed with only a few object shapes and limited size variation. That is, EMG patterns observed when the monkeys handled novel object shapes and sizes were reconstructed with an appropriate coordination of these synergies.
We have identified a small number of synergies in EMG patterns observed in natural movements of the primate hand. Three time-varying synergies (far fewer than the number of object conditions or muscle channels defining the data) could account for 81% of the EMG variation, with only minimal additional variation explained by additional synergies (Fig. 4). Each of the synergies was significantly modulated in its scaling and/or timing coefficients by object size and/or shape (Table 1). Many of these modulations could be represented as linear relationships between the coefficient and object mass, in some cases dependent also on object shape. Despite differences between monkeys in grasp postures (Fig. 2) and underlying muscle patterns (Fig. 3), the animals were similar not only in the appropriate number of synergies but in the spatiotemporal identity of two of the synergies (Fig. 5), the dependence on object size of the remaining synergy's recruitment (Fig. 8), and the temporal ordering of all three (Fig. 9). In the discussion that follows we evaluate in more detail the support for each of the three hypotheses we raised in the Introduction, concerning data reconstruction, modulation, and conservation.
Data reconstruction by time-varying synergies
At present there is no consensus about either the appropriate algorithm for reducing the dimensionality of muscle data, or the appropriate dimensionality at which to reduce the data. In this study we extracted muscle synergies using the decomposition algorithm of d'Avella et al. (2003). This optimization algorithm exploits data non-negativity to identify synergies through a multiplicative update rule (Lee and Seung, 1999). Dimensionality reduction can also be obtained with principal component analysis (PCA) (Santello et al., 1998; Brochier et al., 2004), singular value decomposition (Mason et al., 2001, 2004), and independent components analysis (Bell and Sejnowski, 1995; Hyvärinen and Oja, 2000; Kargo and Nitz, 2003; Hart and Giszter, 2004). A previous analysis suggests that several of these factorization techniques produce comparable synergy structures and degrees of variation explained (Tresch et al., 2006).
However, the majority of the techniques above consider synergies to be time-invariant patterns of coactivation between kinematic or muscle dimensions, although the recruitment of these vectors can vary over time. The algorithm used here is “time-varying” in that the synergies encode a short, temporal profile of activity for each muscle (d'Avella and Tresch, 2001; d'Avella et al., 2003; d'Avella and Bizzi, 2005). This allowance incorporates a physiological assumption that muscle coordination can involve invariant but nonsynchronous relationships between muscles, as were clearly a feature of the synergies here (Fig. 5). A previous adaptation of PCA to human kinematics (Santello et al., 2002) and EMG (Klein Breteler et al., 2006) similarly solves for PCs involving nonsynchronous relations between joints or muscles, but does not allow for independent temporal shifting of the synergies themselves. Such independent delays appear to be a feature of human reaching (d'Avella et al., 2006) and of the results presented here, as evidenced by modulation of synergy onset times as a function of object properties (Fig. 9).
In whatever way muscle synergies may be defined, their utility is often justified by showing that the amount of EMG variation explained by a small number of the components is “relatively” high. We quantified this notion using the straight-line fitting procedure of Cheung et al. (2005), which confirmed that minimal EMG variation was accounted for by more than three synergies (Fig. 4). Our synergies also compare reasonably well with the PCs found by Brochier et al. (2004) in primate grasping, three of which accounted for 81% of the variation across 10–12 muscles, 20 time points, and five repetitions of each of six object conditions. In comparison, three synergies in our study accounted for 81% of the variation across 15–19 muscles, 100 time points, and 25 object conditions. As additional support for choosing a dimensionality of three synergies, we note that they remained structurally (Fig. 5) and functionally (Figs. 8, 9) distinct from one another. Ting and Macpherson (2005) and d'Avella et al. (2006) have similarly shown that postural responses of cats and reaching of humans can be decomposed into a small number of muscle synergies with differentiated directional tuning.
Modulation of synergy recruitment across behavioral conditions
Given the primate hand's capacity for individuated actions (Bennett and Lemon, 1994) and its direct monosynaptic projections from an extensive representation in sensorimotor cortex (Lemon, 1993), it might appear to be the system least likely to yield any evidence for modular motor control. We approached the system by selecting behaviors that systematically spanned hand postures. In particular, we presented the monkeys with a wide range of object shapes and sizes. Although our task involved reach, grasp, and transport movements, neither the task nor the objects were explicitly designed to target particular postures, even if postures such as precision grips were evident for certain objects (Fig. 2). Our object conditions may not have been as distinct from each other as in previous studies of monkey grasping (Brochier et al., 2004) and human fingerspelling (Klein Breteler et al., 2006). But because they were defined by systematic variation of object properties [as in the study by Mason et al. (2004)], the conditions allowed parameterization of the resulting muscle activity patterns.
In our analysis we did not focus on the task specificity of EMG activity (Brochier et al., 2004), but rather on how a spectrum of muscle patterns could be created by combining a small number of common, invariant modules. We found that one of these modules was recruited with a particularly strong dependence on object size, as well as shape. As such it resembled kinematic PCs that have been described previously in human grasping (Santello and Soechting, 1997). The remaining synergies appeared to be modulated in their recruitment amplitude and/or timing to a smaller and more monkey-specific extent by object shape and size properties. Perhaps like the eigenposture characterizing reach-to-grasp kinematics documented by Mason et al. (2004) in their analysis of monkey prehension, these synergies were relatively independent of the force requirements present during object grasp.
As we discovered, both the structure of the synergies and their covariations with object properties were preserved when synergies were extracted from only a small subset of available muscle patterns rather than from all object conditions. Thus the principal dimensions of EMG variation were not strictly dependent on the task variables used to define the dataset. Instead, we suggest that the covariations observed between synergy coefficients and object properties may result from recruitment of invariant muscle synergies, as directed by task parameters. These modules and simple rules governing their selection and combination could provide a substrate for complex motor behaviors.
Conservation of synergies between animals
A strong interpretation of the synergy concept holds that synergies are “hard-wired” into the nervous system. As such they might be expected to be conserved between animals. Previous studies investigating limb behaviors in frogs have indeed shown that most time-varying synergies are structurally similar across animals (d'Avella and Bizzi, 2005). In human reaching, too, d'Avella et al. (2006) found that they could group four to five synergies extracted from each of nine subjects into just six clusters marked by common, distinctive features.
Here, we observed a reasonable degree of correspondence in the structure (Fig. 5) and relative timing (Fig. 9) of the two monkeys' synergies, as well as in their modulation by object properties. To summarize the identity of each of the three synergies, the first captured reach-related muscle activity, principally among proximal muscles and more distal flexors. Conversely, the third synergy captured transport-related muscle activity, principally among proximal muscles and more distal extensors (although distal flexors appeared to be coactivated in the case of monkey G1). The second synergy was the least well matched between the monkeys but was nevertheless characterized by bimodal activation among distal muscles. Its timing was intermediate between the first and third synergies, such that it spanned the time the monkey was in contact with each object.
We found these similarities between monkeys despite differences between their grasp postures (Fig. 2). Their muscle patterns differed, too (Fig. 3), although our equation of most (12) muscles in each monkey is necessarily tentative. Even within an individual muscle, let alone across muscles of different animals, one can record single EMG units with very different activity profiles (Weiss and Flanders, 2004). Such variability may be especially pronounced for muscles extrinsic to the hand with anatomical or functional subdivisions (Schieber, 1993).
One might also not expect these similarities given that the task involved relatively naturalistic grasping movements of the hand. Brochier et al. (2004) extracted PCs from EMG data recorded during grasping behaviors of two macaques, and found these PCs to be generally dissimilar between animals. Although we cannot directly compare this level of correspondence with ours, we suggest that the time-varying synergy model may be well suited for capturing conservation of temporal coordination of muscles, and indeed of synergies, between animals. These results are consistent with the idea that the nervous system is constrained to use invariant spatiotemporal patterns of muscle activation to build complex motor behaviors.
This work was supported by National Institutes of Health, National Institute of Neurological Disorders and Stroke Grant NS44393 (E.B.). This work benefited from the technical assistance of Margo Cantor, Sylvester Szczepanowski, Steve Marchetti, and from discussions with Matthew Tresch and Andrew Richardson.
- Correspondence should be addressed to Dr. Simon A. Overduin, Department of Brain and Cognitive Sciences and McGovern Institute for Brain Research, Massachusetts Institute of Technology, 46-6177, 43 Vassar Street, Cambridge, MA 02139.