Robotic training paradigms that enforce a fixed kinematic control might be suboptimal for rehabilitative training because they abolish variability, an intrinsic property of neuromuscular control (Jezernik et al., 2003). In the present study we introduce “assist-as-needed” (AAN) robotic training paradigms for rehabilitation of spinal cord injury subjects. To test the efficacy of these robotic control strategies to teach spinal mice to step, we divided 27 adult female Swiss–Webster mice randomly into three groups. Each group was trained robotically by using one of three control strategies: a fixed training trajectory (Fixed group), an AAN training paradigm without interlimb coordination (Band group), and an AAN training paradigm with bilateral hindlimb coordination (Window group). Beginning at 14 d after a complete midthoracic spinal cord transection, the mice were trained daily (10 min/d, 5 d/week) to step on a treadmill 10 min after the administration of quipazine (0.5 mg/kg), a serotonin agonist, for a period of 6 weeks. During weekly performance evaluations, the mice trained with the AAN window paradigm generally showed the highest level of recovery as measured by the number, consistency, and periodicity of steps during the testing sessions. In all three measurements there were no significant differences between the Band and the Fixed training groups. These results indicate that the window training approach, which includes loose alternating interlimb coordination, is more effective than a fixed trajectory paradigm with rigid alternating interlimb coordination or an AAN paradigm without any interlimb constraints in promoting robust postinjury stepping behavior.
It has been shown that adult spinal mice can be trained to step on a moving treadmill belt with the use of a robotic device (Fong et al., 2005). In addition, there is a positive interaction effect between robotic training and the administration of quipazine, a broad serotonin agonist. To date, the algorithms that have been used for locomotor training with robotic devices have focused primarily on repeated movements of the limbs via fixed kinematic trajectories. These types of training, however, abolish the cycle-to-cycle variation in the kinematics and the sensorimotor pathways, a fundamental feature of the neural control of repetitive movements such as stepping (Hausdorff, 2005). Appropriately recreating this feature of neural control may be critical to the development of effective robotic control algorithms for assisting the post-spinal cord injury (post-SCI) neuromuscular system to learn a motor task. A robotic orthosis driven in a fixed pattern effectively limits the degrees of freedom of the motion of the leg as compared with naturally occurring muscle activation patterns (Hidler and Wall, 2005). Thus fixed trajectory training may produce habituation to sensory input, resulting in markedly reduced sensory responses associated with weight-bearing locomotion. As a consequence, the training could become counterproductive, resulting in a decrease in the activity of sensorimotor systems that are normally highly active. In turn, this is likely to reduce the activity of the spinal neural control circuits that control locomotion (Wirz et al., 2005). Additionally, we believe that a rigid training paradigm may lead to “learned helplessness,” a condition in which the lower spinal cord habituates to repetitive activation of the same sensory pathways during a training session (Skinner, 1979; Wool et al., 1980).
In the present study we tested the hypothesis that the post-SCI spinal cord can relearn to step more effectively if it is challenged constantly during locomotor training by an adaptive training pattern. Using complete spinal mice, we examined two paradigms of assist-as-needed (AAN) hindlimb gait training. Similar robotic control algorithms have been used for rehabilitative training of stroke patients (Hogan and Krebs, 2004; Patton and Mussa-Ivaldi, 2004; Emken and Reinkensmeyer, 2005; Patton et al., 2006). Our AAN algorithms mostly allowed the mice to control their own motions when they were performing well. These approaches also facilitated variability in the trained stepping trajectory, promoting stepping conditions comparable to normal locomotion.
We compared the efficacy of two AAN robotic training algorithms and a fixed trajectory robotic training paradigm on the recovery of locomotor ability in complete spinalized adult mice that were administered quipazine daily, immediately before each training session. The results indicate that mice undergoing AAN robotic training with loose control of interlimb coordination exhibit faster and more pronounced recovery of stepping ability than mice trained by using a fixed robotic paradigm or an AAN robotic training paradigm without interlimb coordination constraints.
Materials and Methods
Animals and animal care
Adult female Swiss–Webster mice (mean body weight of 25.3 ± 1.3 g on the day of spinal cord transection) obtained from Charles River Laboratories (Wilmington, MA) were used. The mice were housed individually, had access to food and water ad libitum, and were kept on a 12 h light/dark cycle for the duration of the study.
Surgical procedures and postsurgical care
Surgeries were performed at approximately postnatal day 60 (P60). Surgical procedure was the same as in Fong et al., (2005). Briefly, the mice were maintained in a deep anesthetic state throughout the surgery with isoflurane gas (2–5% isoflurane mixed with 0.4% O2 via face mask). All procedures were performed under aseptic conditions. The mice were placed on a heating pad to maintain body temperature. A skin incision was made along the dorsal midline from ∼T6 to T9 to expose the musculature overlying the vertebrae. The paravertebral musculature was retracted to expose the vertebral column, and a laminectomy was performed from ∼T7 to T8. Gelfoam then was inserted into the gap to ensure complete separation of the proximal and distal stumps (de la Torre and Goldsmith, 1990). The musculature and fascia were repositioned, and the wound was closed by using 5-0 Dexon internal and 5-0 Ethilon external sutures (Fong et al., 2005). All animal procedures used in this study were conducted in accordance with the Animal Care Guidelines of the American Physiological Society and were reviewed and approved by the Animal Research Committee at the University of California, Los Angeles.
The mice were placed in an incubator maintained at ∼29 ± 1°C and observed until fully recovered from anesthesia after surgery. The mice were returned to their cages and given Baytril (40 μg/g body weight), a broad spectrum antibiotic, via drinking water for 14 d. Postsurgical care and maintenance procedures were similar to those described previously for SCI rats and cats (Roy et al., 1992; Ellegala et al., 1996). The bladders of the spinal mice were expressed manually twice daily to minimize the risk of bladder infection and related complications. After bladder expression the hindlimbs of the mice were stretched lightly once through a full range of motion to help sustain joint mobility. Food rewards were given to stimulate positive interaction between the mouse and handler.
Serotonin agonists are shown to modulate and/or induce locomotion in rats (Antri et al., 2003) and mice (Landry and Guertin, 2004). In this study quipazine, a broad spectrum serotonin agonist, was used to facilitate stepping in all mice. Based on our own dose–response tests, as well as dose–response studies reported for rats (Orsal et al., 2002; Fong et al., 2005), a dose of 0.5 mg/kg body weight was administered intraperitoneally 10 min before each training or testing session.
Robotic step training and measurement system
A four-axis robotic system (stepper) was developed for both active training and data acquisition (see schematic drawing at http://robotics.caltech.edu/jneurosci) of mouse limb movements. The mice were placed into the stepper by using a cone-shaped cloth harness. Once in the harness, the mouse was secured with a binder clip, which was attached magnetically to a body weight support system. The legs of the mouse were connected to the robotic arm by using a drawstring loop attachment. During training the speed of the treadmill was adjusted to match the speed of the robotic device. Typically, this speed was ∼3 cm/s. A detailed description of the robot and mouse attachment has been published previously (Fong et al., 2005).
So that the AAN algorithms could be implemented, precise control of the robotic linkages was required. Therefore, a four-axis dedicated controller board (DMC-2240, Galil Motion Control, Rocklin, CA) was used. The control output was sent to the motors via an interconnection module with an integrated amplifier (ICM/AMP 1900; Galil Motion Control). The control algorithms were written with LabVIEW (National Instruments, Austin, TX), and their output commands were sent to the controller board via an Ethernet connection. The feedback commands were updated at 200 Hz.
Robotic training algorithms
The mice were divided randomly and equally into three groups (n = 9/group). Each group received a different robotic training algorithm, i.e., a repetitive training algorithm with a fixed and tightly controlled trajectory or one of two AAN training algorithms. The two AAN algorithms differed in the amount of interlimb coordination that was imposed during training. The AAN training algorithms were implemented by using a velocity field approach in which the velocity of the distal tip of the linkage was commanded to a specific speed defined by a velocity field. Each mouse was trained for 10 min/d, 5 d/week, for 6 weeks.
Mice in the Fixed group received a rigid robotic training algorithm. A proportional–integral–derivative (PID) controller was used continuously to track the desired training pattern. When attached to the mice, the robotic arms actively moved the ankles along this fixed trajectory, which enforced alternating interlimb coordination. Because neonatally transected mice spontaneously can recover functional stepping without pharmacological or mechanical assistance (Fong et al., 2005), the imposed trajectory of each ankle in the sagittal plane was obtained from a neonatally transected mouse that recovered and stepped well at approximately the same age and weight as the adult mice used in the present study.
The Band group of mice received an AAN strategy that implements two fixed boundaries, an inner bound and an outer bound, forming a band surrounding the desired trajectory. When the mouse moves its ankle to sagittal plane regions exterior to the band, an outward-spiraling (or inward-spiraling) convergent velocity field drives the ankle back into the band region (Fig. 1a). Within the band the ankle is guided by a small constant velocity field tangent to the desired trajectory. Thus the robot provides gentle guidance at a constant rate but does not enforce specific timing of leg movements, nor does it force the ankle to be at a specific location (soft control). In this way the mouse mostly dictates its own motions inside the band, with only a small bias provided by the robot. Note that this particular instantiation of the AAN paradigm does not impose an interlimb coordination constraint.
The last group of mice (the Window group) received an AAN training paradigm analogous to the second group but based on a moving window geometry. In this approach a circular window moves along the desired trajectory with a predetermined speed (Fig. 1b). The 4-mm-diameter window size, which was fixed throughout the experiment, was chosen because it was close to the maximum variation observed during the stepping of a neonatally transected mouse at an early stage of recovery (Fig. 2). Within the window, a small constant velocity field tangent to the desired trajectory biases the robot's motion but without spatial or temporal enforcement of specific ankle position. Outside the window boundary, the robotic movement is guided by a radial velocity field that points inward with a magnitude proportional to the distance from the center of the circle as follows: v = k (d − r), where v is the velocity field magnitude, d is the distance between the ankle and the center of the moving window, r is the window radius, and k is a constant. Hence when the ankle of the mouse deviates from the window, it is returned quickly to the window. Within the window, the ankle is guided gently in the direction of the trajectory, providing loose timing control. The same strategy was used on both hindlimbs, and the control systems for each leg were timed to provide alternating gait coordination that is consistent with weight-bearing stepping.
Data analysis and evaluation methods
Testing was performed on the fifth day of each training week. The mice were given a 2 min “warm-up” period before each testing session, using the same training algorithm associated with that particular group. The mouse stepper was used in a passive recording mode to track the ankle position of each leg for 2 min at a treadmill speed of 3 cm/s. Position data were recorded at 200 Hz, using a custom acquisition program written in LabVIEW (National Instruments). In addition to the robot data, video footage of both the left and right sides of each mouse was captured during testing, and a log of qualitative observations was maintained. Using these data, we assessed the quality of stepping in terms of (1) the number of steps performed; (2) the periodicity of the steps, i.e., the ability to maintain a regular stepping frequency; and (3) the shape regularity of the stepping patterns. The following analyses were used for these assessments of stepping ability.
Number of steps.
Video footage and plots of ankle position data were used to identify steps and the number of steps performed by each mouse during the 2 min testing period. Steps were identified on the basis of predetermined criteria for step length (minimum 5 mm), height (minimum 5 mm), duration (minimum 0.5 s/maximum 1.5 s), and degree of interlimb coordination (Fong et al., 2005). On each testing day the 12 s stepping interval containing the most steps was recorded for subsequent analyses.
Fast Fourier transform (FFT) analysis was applied to the horizontal component of the stepping trajectories to quantify step periodicity (Fong et al., 2005). Mice that stepped rhythmically exhibited a sharp and distinct fundamental peak in the FFT spectrum of their ankle trajectories. The location of the predominant peak corresponds to the fundamental stepping frequency. Conversely, mice with poor stepping periodicity either exhibited a very broad fundamental peak in the FFT spectrum or, in extreme cases, failed to demonstrate a fundamental peak. To quantify these observations, we measured the full width at half-maximum (FWHM) of the fundamental peak. A low FWHM value corresponds to temporally consistent, rhythmic stepping, whereas a high value typically indicates erratic stepping, consistent with stumbling and foot dragging.
To measure consistency of stepping systematically, principal components analysis (PCA) was used. The principal components (PC) of the data set were calculated as well as the percentage of the total variance captured by the first PC. The first PC can be interpreted as the “nominal” trajectory from the test data set, and the PCA percentage score measures the consistency of the trajectories. A high PCA percentage score corresponds to a consistent execution of the same stepping trajectory. A minimum PCA percentage of 45% was assigned to all mice that could not step on the test day, because this was the lowest PCA score that was encountered (Fong et al., 2005).
Locomotor performance scores are reported as the mean ± SEM. One-way ANOVA analysis was used to compare the three training groups within each training day. A p value less than 0.05 was used to define statistical significance, which corresponded to a critical F value of 3.44. To measure statistical difference between groups, we calculated the least significant difference (LSD).
All groups showed improvement over the 6 week training period, based on the average number of steps taken in the best 12 s interval during each testing session (Fig. 3a). The mice in the Window group, however, had a faster rate of recovery than the other two groups. The average number of steps taken by the Window group was greater than in the Fixed group from weeks 1 through 3 and higher than in the Band group at weeks 1 and 3. There were no significant differences between the Band and Fixed groups at any time point, and the average number of steps was similar in the three groups after 6 weeks of training.
Inverse FWHM scores for the mice in the Window group were significantly higher than in the Band and Fixed groups after 4, 5, and 6 weeks of training (Fig. 3b). There were no significant differences between the Band and Fixed groups at any time point. The maximum level of step rhythmicity was achieved after 6 weeks of training, with the Window group (8.3) exhibiting better step rhythm than the Band (6.4) and Fixed (6.3) groups.
All three groups showed progressive improvement in step shape consistency, based on PCA analyses, throughout the first 5 weeks of training and then a slight decrease at 6 weeks (Fig. 3c). There were no significant differences among the three groups at any time point.
All step training algorithms improved stepping beyond the level that is achieved without any step training
We have demonstrated previously that a combination of quipazine administration and robotic training can improve the locomotor performance of adult spinal mice significantly. Without pharmacological and/or mechanical intervention the average number of steps performed in a full 2 min interval was 16.0 ± 5.1, and no mouse was able to perform the minimum of three consecutive steps in a 12 s interval that were required for FFT and PCA analysis (Fong et al., 2005). All of the mice in the current experiment received quipazine treatment and robotic training. Consequently, all training paradigms improved stepping, and the level of improvement was greater in this study than the initial test at week 0 and the level of performance reported previously (Fong et al., 2005). Even after only 1 week of training, the lowest average number of steps for the best 12 s interval among the three groups was 3.7 ± 0.7 steps. These results demonstrate the effectiveness of robotic systems in enhancing locomotor training after a SCI, even when suboptimal training algorithms are used.
Permitting an intrinsic network solution facilitates stepping more effectively after a SCI than imposing an extrinsic motor solution
A key objective of this study was to ascertain whether permitting variability during step training enhances stepping recovery after a complete spinal cord transection in adult mice. We hypothesized that a fixed trajectory training strategy would drive the spinal circuitry toward a state of learned helplessness (Wool et al., 1980; Grau et al., 1998). This occurs when the spinal cord is not permitted to explore potential solutions to stepping patterns and defers to the fixed training pattern, causing the relevant neural circuits to habituate. Figure 3b shows that, although the fixed training paradigm is one in which the periodicity of stepping is controlled most tightly, the animal failed to produce a consistent stepping rhythm during testing. In contrast, when the training paradigm allowed the stepping period to vary, the mice were able to adapt to a consistent stepping period. These results, combined with the number of steps performed, suggest that the window training paradigm significantly improved the stepping ability of the mice as compared with the band and the fixed training paradigms.
Sensory information is critical to motor learning. The pattern and timing of assistance provided during step training seems to play a critical role in specific sensorimotor pathways that become reinforced after a SCI (de Leon et al., 1998). Regardless of the level of practice of a task, some variability in the patterns and levels of activation of motor units within a motor pool persists even during the simplest repetitive actions. Thus there also must be variation in the efficacy of the ensemble patterns of sensory input to the spinal circuitry from step to step. Given this intrinsic variability, the subsequent motor output resulting from sensory inputs is highly unlikely to match the imposed mechanically fixed pattern. From this perspective it appears that a continuous incongruity between the input and output signals will occur when a fixed trajectory is imposed. Thus a training algorithm that is incompatible with this basic feature of variability during stepping ultimately seems likely to hinder the ability of the spinal cord to learn to step after a SCI.
Conversely, a training algorithm that “permits” the intrinsic variability in the activation of motor pools may allow the spinal circuitries to explore multiple patterns of activation and thereby optimize training effectiveness. In this experiment, we tested only two such algorithms. Spinal mice recovered stepping ability more effectively with the window AAN algorithm than the band AAN or fixed training paradigms. Even the window algorithm, however, is unlikely to be the optimum solution, as demonstrated by the peak in locomotor performance reached after 4 weeks of training in the current study. One can imagine many variations of the AAN training algorithm, but it will be difficult to test experimentally all of these variations in attempts to find an optimum rehabilitative training strategy. One approach will be to develop a learning model for the plasticity within the spinal cord derived from machine learning theories. Having such a model will allow us to explore many more parameters, such as window size and shape, than otherwise would be feasible experimentally. Once candidate training algorithms are identified analytically, these algorithms can be validated experimentally by using an experimental paradigm as in the present study.
An imposed interlimb coordination pattern facilitates learning to step
Another observation from this study is that control of interlimb coordination can improve locomotor recovery. Although it had been shown that spinal cats can adapt to different walking speeds on a split treadmill (Barbeau and Rossignol, 1987), in examining the cause of failure to continue stepping in chronic spinal cats that had been trained to step, de Leon and colleagues (de Leon et al., 1999) found that the most consistent contributing factor was a gradual loss of appropriate interlimb coordination. Rarely was failure attributable to poor intralimb kinematics. Similarly, the current experiment suggests that maintaining interlimb coordination plays an important role in training adult spinal mice to step. In mice trained with the band algorithm, the steps were typically arrhythmic and frequently interrupted by dragging, which is emphasized by their low inverse FWHM values (Fig. 3b). In contrast, stepping executed by the mice in the Window group was rhythmic and of appropriate duration. In many cases, the mice were able to step throughout the entire 2 min testing period. This was reflected by the average number of steps taken in the best 12 s interval by the mice, which showed that stepping in the Window group converged to a frequency near 1 Hz as the study progressed, and is consistent with constant speed treadmill locomotion at 3 cm/s (Fig. 3a, week 4).
Distinction between shape consistency and quality of stepping
There were no significant differences in step shape consistency among the three groups throughout the study, based on PCA scores (Fig. 3c). Even after 1 week of training the average PCA scores of the Band and Window groups were >70%. By week 3 the average PCA score of all three groups was >80%, indicating that all of the mice could perform consecutive rhythmic movements that were similar in shape within that animal. Therefore, the PCA results reflected consistency in cyclic movements but did not differentiate the quality of stepping among the different training groups. Combined with previous results, which reported an average PCA score of 77 ± 4% with quipazine administration alone (Fong et al., 2005), one reasonable hypothesis from these results is that quipazine may have a greater effect on the shape consistency of the stepping than robotic training, especially when the training itself is not rigid.
The present results provide strong evidence that a fundamental strategy of the neural control of a given motor task (stepping) is to incorporate a degree of variability in the sensorimotor pathways. These data suggest that when the intrinsic variability is overridden, e.g., when a “fixed” pattern is imposed, learning of a task is suboptimal relative to the condition when the training is assist-as-needed. Beyond the insight provided by these results on the strategy for neural control of movement, the practical implications may be highly significant for future efforts to develop robotic devices that can be used to facilitate recovery from neuromotor impairments.
This work was supported by grants from the Christopher Reeve Foundation, the National Institute of Child Health and Human Development (HD44830), and the National Institute of Neurological Disorders and Stroke (NS16333). We thank Hui Zhong for her assistance with animal surgeries; Maynor Herrera for providing excellent animal care; Niranjala Tillakaratne, John Hodgson, Rebekah Molyneux, and Sharon Zdunowski for their constructive comments and valuable discussions; and an incredible team of undergraduate students, Vivek Agarwal, Michael Andrew, Nikhil Daga, Armen Derian, Annie Kao, Dan Popa, Liliana Lacayo, and Traci Shiraishi, for all of their hard work.
- Correspondence should be addressed to V. Reggie Edgerton, Department of Physiological Science, University of California, Los Angeles, 621 Charles E. Young Drive South, Los Angeles, CA 90095.