Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
    • Special Collections
  • EDITORIAL BOARD
    • Editorial Board
    • ECR Advisory Board
    • Journal Staff
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
    • Accessibility
  • SUBSCRIBE

User menu

  • Log out
  • Log in
  • My Cart

Search

  • Advanced search
Journal of Neuroscience
  • Log out
  • Log in
  • My Cart
Journal of Neuroscience

Advanced Search

Submit a Manuscript
  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
    • Special Collections
  • EDITORIAL BOARD
    • Editorial Board
    • ECR Advisory Board
    • Journal Staff
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
    • Accessibility
  • SUBSCRIBE
PreviousNext
Research Articles, Systems/Circuits

Hypothalamic Control of Forelimb Motor Adaptation

Dane Donegan, Christoph M. Kanzler, Julia Büscher, Paulius Viskaitis, Ed F. Bracey, Olivier Lambercy and Denis Burdakov
Journal of Neuroscience 10 August 2022, 42 (32) 6243-6257; https://doi.org/10.1523/JNEUROSCI.0705-22.2022
Dane Donegan
1Neurobehavioral Dynamics Laboratory, Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Schwerzenbach 8603, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christoph M. Kanzler
2Rehabilitation Engineering Laboratory (RELab), Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Zürich 8008, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julia Büscher
1Neurobehavioral Dynamics Laboratory, Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Schwerzenbach 8603, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Julia Büscher
Paulius Viskaitis
1Neurobehavioral Dynamics Laboratory, Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Schwerzenbach 8603, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Ed F. Bracey
1Neurobehavioral Dynamics Laboratory, Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Schwerzenbach 8603, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Olivier Lambercy
2Rehabilitation Engineering Laboratory (RELab), Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Zürich 8008, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Denis Burdakov
1Neurobehavioral Dynamics Laboratory, Department of Health Sciences and Technology, Eidgenössische Technische Hochschule Zürich, Schwerzenbach 8603, Switzerland
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Abstract

The ability to perform skilled arm movements is central to everyday life, as limb impairments in common neurologic disorders such as stroke demonstrate. Skilled arm movements require adaptation of motor commands based on discrepancies between desired and actual movements, called sensory errors. Studies in humans show that this involves predictive and reactive movement adaptations to the errors, and also requires a general motivation to move. How these distinct aspects map onto defined neural signals remains unclear, because of a shortage of equivalent studies in experimental animal models that permit neural-level insights. Therefore, we adapted robotic technology used in human studies to mice, enabling insights into the neural underpinnings of motivational, reactive, and predictive aspects of motor adaptation. Here, we show that forelimb motor adaptation is regulated by neurons previously implicated in motivation and arousal, but not in forelimb motor control: the hypothalamic orexin/hypocretin neurons (HONs). By studying goal-oriented mouse-robot interactions in male mice, we found distinct HON signals occur during forelimb movements and motor adaptation. Temporally-delimited optosilencing of these movement-associated HON signals impaired sensory error-based motor adaptation. Unexpectedly, optosilencing affected neither task reward or execution rates, nor motor performance in tasks that did not require adaptation, indicating that the temporally-defined HON signals studied here were distinct from signals governing general task engagement or sensorimotor control. Collectively, these results reveal a hypothalamic neural substrate regulating forelimb motor adaptation.

SIGNIFICANCE STATEMENT The ability to perform skilled, adaptable movements is a fundamental part of daily life, and is impaired in common neurologic diseases such as stroke. Maintaining motor adaptation is thus of great interest, but the necessary brain components remain incompletely identified. We found that impaired motor adaptation results from disruption of cells not previously implicated in this pathology: hypothalamic orexin/hypocretin neurons (HONs). We show that temporally confined HON signals are associated with skilled movements. Without these newly-identified signals, a resistance to movement that is normally rapidly overcome leads to prolonged movement impairment. These results identify natural brain signals that enable rapid and effective motor adaptation.

  • hypothalamus
  • motor adaptation
  • motor learning
  • movement
  • orexin/hypocretin
  • upper limb

Introduction

Regulation of skilled arm movements has been a subject of intense research (Franklin and Wolpert, 2011). In human experiments, robotic manipulandums produce controlled and reproducible velocity-dependent force field perturbations, that allow fundamental elements of sensorimotor control to be dissected (Shadmehr and Mussa-Ivaldi, 1994; Wolpert and Miall, 1996; Shadmehr et al., 2010; Franklin and Wolpert, 2011). These elements include the evaluation of sensory expectations based on motor commands and internal models of sensorimotor transformations that determine motor outputs (Kawato, 1999; Thoroughman and Shadmehr, 2000; Shadmehr et al., 2010; Wolpert et al., 2011; Wolpert and Flanagan, 2016). Internal models can be seen as libraries of motor commands. They implement a model of limb and task dynamics which transforms a desired movement trajectory into motor commands (Wolpert and Kawato, 1998). In unfamiliar motor tasks, the internal model uses the discrepancy between the desired and actual movement trajectory (“sensory prediction error”) to iteratively update motor commands across attempts and thus reduce error (“sensory error-based motor adaptation”; Wolpert et al., 1998, 2011; Kawato, 1999; Thoroughman and Shadmehr, 2000; Shadmehr et al., 2010; Wolpert and Flanagan, 2016). Alternatively, when a reward is given after each successful attempt, adaptation can also occur according to changes in this feedback (“reward-based motor adaptation”; Izawa and Shadmehr, 2011). The relative contributions of sensory versus reward-based learning to motor adaptation depend on the experimental paradigm, and on whether motor adaptation is primarily guided by sensory errors rather than reward rate (Mazzoni and Krakauer, 2006; Schaefer et al., 2012; Mathis et al., 2017).

Error-based motor adaptation is impaired in common disorders, e.g., after strokes (Takahashi and Reinkensmeyer, 2003; Smith and Shadmehr, 2005; Krakauer, 2006; Scheidt and Stoeckmann, 2007; Butcher et al., 2017). While the behavioral consequences of lesions in certain brain regions have been well characterized, we know little about how genetically-defined neurons influence specific elements within the frameworks of error-based motor control. In human studies, this knowledge is limited by the inability to record and manipulate genetically-defined neurons. In rodent models, it has been limited by a lack of robot-assisted paradigms that provide reproducible perturbations. In particular, no studies have been performed using velocity-dependent force fields, which enable dynamic remapping of internal models and quantification of neural data in the context of adaptation frameworks (Butler et al., 2000; Gandolfo et al., 2000; Li et al., 2001; Franklin and Wolpert, 2011; Mathis et al., 2017).

Among the multiple central nervous system modules implicated in adaptive motor control, the cortex, basal ganglia, cerebellum, brainstem, and spinal cord have been intensively studied (Bear et al., 2015; Arber and Costa, 2018). In contrast, hypothalamic orexin/hypocretin neurons (HONs; de Lecea et al., 1998; Sakurai et al., 1998) have received much less attention. Although HONs were originally studied in the context of arousal and motivation (Mahler et al., 2014; Sakurai, 2014), abundant evidence implicates them in central motor control (Hu et al., 2015). HONs directly innervate and activate the classic motor systems, including the motor and somatosensory cortices, the basal ganglia, the cerebellum, and the spinal cord (Peyron et al., 1998; Cutler et al., 1999; Nambu et al., 1999; van den Pol, 1999; Date et al., 2000; Korotkova et al., 2002; Bayer et al., 2004; Yamuy et al., 2004; Biswabharati et al., 2018). At the functional level, however, HONs have only been studied in relation to innate locomotion and vestibular control (Chemelli et al., 1999; Kiyashchenko et al., 2001; Zhang et al., 2011; Feng et al., 2020; Karnani et al., 2020), and the roles of HONs in acquired, refined motor skills remain undefined.

Here, we applied robotics inspired by classic human upper limb studies to a mouse model for studying HON activity. During a skilled forelimb task, we exposed mice to velocity-dependent force fields designed to require dynamic motor adaptation, measuring analogs of behavioral elements typically analyzed in human studies (Schwarz et al., 2019; Kanzler et al., 2020). Recording and control of HONs in this context allowed examination of their involvement in skilled motor performance.

Materials and Methods

Genetic targeting of optical sensors and actuators of neural activity

Animal experiments were performed in accordance with the Animal Welfare Ordinance (TSchV 455.1) of the Swiss Federal Food Safety and Veterinary Office, and approved by the Zurich Cantonal Veterinary Office. Subjects were adult male C57BL/6 mice over the age of seven weeks. Mice were housed in a reversed 12/12 h light/dark cycle. Experiments were performed during the dark phase. The specific targeting of GCaMP6s and ArchT to orexin neurons was performed using the same surgical and genetic tools described and histologically validated in our previous study (Karnani et al., 2020). Briefly, to target the optogenetic inhibitory actuator ArchT to orexin neurons, we injected AAV1-hORX.ArchT.TdTomato (1:20 dilution, titer: 1.03 × 1013 GC/ml; prepared by Vigene Biosciences) into the lateral hypothalamus. To target GCaMP6s to orexin neurons, we injected AAV1-hORX.GCaMP6s (titer: 2.0 × 1013 GC/ml; prepared by Penn Vector Core) into the lateral hypothalamus. To control for possible heating artifacts because of laser manipulation, a subset of mice were implanted with optic fibers, with no actuators expressed. For stereotaxic brain injections, mice were anesthetized with isoflurane (1% at 1 l/min), injected with meloxicam (10 µl/g of body weight, s.c.) for analgesia, and placed into a stereotaxic frame (Kopf Instruments). A craniotomy was performed, and a glass pipette mounted on a Nanoject II injector (Drummond SCI) was used to inject the AAV vectors into the hypothalamus; 400 nl of ArchT AAV vector or 150 nl of GCaMP AAV vector was injected per hemisphere at the following coordinates: bregma −1.55 mm, midline ± 1 mm, depth 5.4 mm skull surface (González et al., 2016a; Kosse and Burdakov, 2019). Mice recovered from surgery for at least 10 d before experiments.

Mouse-robot interaction

Mice used their left forelimb to interact with a robotic arm to obtain rewards. The robot was based on the ETH Pattus (Vigaru et al., 2011, 2013; Lambercy et al., 2015), and re-developed to enable forelimb-motor-adaptation analysis in mice (Fig. 1A). It comprises a two-degrees-of-freedom robotic manipulandum operated by electromagnetic motors coupled with high-resolution rotary encoders (Vigaru et al., 2011, 2013; Lambercy et al., 2015). Mice were trained to interact with the robot via a horizontally oriented joystick handle fixed at the tip of the robotic arm, 5 mm in width and 1 mm in diameter. The handle could be manipulated to perform a specific motor task [in this study, to pull 9 mm in the y-dimension from starting point with lateral (x) deviations not exceeding ±5 mm; Fig. 1A,B]. Upon successful task execution, the robot automatically delivered a milkshake reward (see behavior training, below). A blue LED cue was placed near the mouse to signify the trial state. For each trial, the handle was unlocked and a cue (blue LED turning off) informed the mouse that it could begin to pull. After the start cue, pulls were performed only in infra-red (IR) light, so sensory feedback was limited to somatosensation (Fig. 1A). The dynamic and kinematic components of the planar movements in the x-y direction, and their responses to precisely-defined force fields, were extracted in an automatically controlled and reproducible manner. During movements, the robot continuously recorded handle position and velocity at a sampling frequency of 1 kHz. During force field trials, the robot perturbed this movement by generating a force field that pushed the handle from left to right in the x-direction in a y-velocity-dependent manner (Fig. 1B). The y-velocity was linearly converted to x-direction force as described previously (Vigaru et al., 2013), producing velocity-dependent force fields similar to those used to study forelimb adaptation in humans (Shadmehr and Mussa-Ivaldi, 1994). The gain of the linear velocity to force conversion was adjusted empirically and kept constant throughout the study. Kinematic and dynamic data of the robot were recorded in LabView and transferred to MATLAB for analysis.

Optical recording and control of neural signals

Fiberoptic implants (200 µm in diameter, 0.39 NA, Thorlabs) were placed stereotaxically with the fiber tip above the lateral hypothalamus (bregma −1.55 mm, midline ±1 mm, depth 5.1 mm from skull surface), and attached to the skull with dental cement (Karnani et al., 2020). A custom printed head plate (Protolabs Inc) was fitted for head fixation to ensure the head was placed at a natural angle (basi-occipital plane ∼22° from horizontal; Vidal et al., 2004).

To record the activity of GCaMP6s-expressing HONs, fiber photometry was conducted using the lock-in mode of a Doric fiber photometry system (405- and 465-nm excitation, modulated at 334 and 471 Hz, respectively, with an excitation light power of ≈100 µW at the fiber tip). Demodulated signals and TTL outputs from the ETH Pattus robot were then aligned for subsequent analysis. GCaMP6s fluorescence emitted in response to 405-nm excitation gave a real-time control for movement artefacts (Kim et al., 2016). For each recording session, a convex hull function was fitted to the raw 405 nm- excited signal and raw 465-nm excited signal, then each were divided by this fit to produce a normalized ΔF/F. The normalized 405-nm excited signal was then subtracted from the normalized 465 nm-excited signal, then z-scored across the session. Before combining signals from different sessions (Fig. 2C,D), the mean z-scored value of −2 to −1 s before pull onset was subtracted from the signal to produce a “Δ z score” value. In Figure 2E,F, “Δ z score” values, representing the change in pull-associated signal amplitude after trial 50 (i.e., when force field was introduced in some sessions), were calculated from signal z scores as follows: (1) pull-associated signal amplitude was calculated by subtracting the lowest signal between cue and pull onset from the signal at end of pull, before the reward was given; (2) the mean value of pull-associated signals during the baseline trials 1–50 was subtracted from each pull-associated signal amplitude.

To control the activity of opsin-expressing HONs, green lasers (532 nm, LaserGlow) were connected to the bilateral fiber implants to yield ≈10-mW light power output at each fiber tip. Since photometry recordings showed that pull-associated endogenous orexin signals started to activate between the cue and pull onset (Fig. 2C,D), laser illumination was started at the cue and terminated at pull offset. Importantly, this enabled selective temporal targeting of premotor and pull-associated orexin cell activity, without targeting orexin signals associated with reward delivery status after the pull offset, or orexin signals between trials.

Behavioral training

After recovery from surgery, mice were food restricted to bring them to 90% bodyweight. For the first 2 d of training, they were habituated to handling and head-fixation on the robotic apparatus for around 15 min. The head plate was held by custom-built holders that positioned the mouse in the same place on a stage relative to the robotic handle across days. A Plexiglas tube that loosely encased the mouse's body also helped to keep body position similar across mice and sessions, yet allowed the mouse to fully extend its forelimb without hindrance. A small metal bolt was attached to the right side of the stage, which the mouse could hold on to with its right forepaw for stability while interacting with the robot.

In the habituation phase, the robot handle was positioned 1 mm from the stage (at the natural resting position of the mouse's left forepaw). A reward spout, gated by a TTL-operated solenoid valve (NR research), was positioned ∼2 mm from the mouth. Throughout all experiments, the spout delivered 6 µl of milkshake 500 ms after the animal made a correct forelimb movement. Reward delivery lasted 500 ms and was accompanied by an auditory tone of the same duration. Over the first 2 d, rewards were delivered automatically any time the mouse moved the robot handle >0.2 mm and for a duration longer than 40 ms. If necessary, rewards were delivered manually by the experimenter using a switch. During this phase, mice rapidly learnt to associate touching the handle with reward, typically performing 50–100 touches per day.

The next phase taught the mice to wait for a cue, and to reach for the robot handle. First, the robot handle was retracted to 2 mm from the left forepaw, and locked in place by the motors until the encoders detected no movement above 0.2 mm for 2 s. After this, the handle was unlocked and a cue (blue LED turning off) informed the mouse that a trial could be started. During this period, any time the mouse touched or moved the handle that resulted in displacement of >0.2 mm for >40 ms resulted in a reward. After a successful touch, the cue LED was turned back on to prompt the mouse not to move the handle. The robot then moved the handle back to the start position and locked it until the next trial. If the mouse did not initiate a trial 10 s after the cue, the handle was again locked in place and the above sequence restarted. After 10 rewards had been delivered, the handle was then moved away from the mouse by 1 mm, and again any movement of the handle after the cue resulted in reward delivery. This step was repeated until the handle reached the final position of 15 mm from the stage. These sessions lasted up to 50 min and this training phase typically took 2–3 d.

Mice were next taught to pull the handle 3 mm in the y-dimension. Pulls shorter than 3 mm in the y-dimension, that moved outside the ±10 mm in the x-dimension, or stopped movement of >0.2 mm for >50 ms, resulted in no reward and the robot being reset to the start position. Once mice successfully pulled ≥6 out of 10 consecutive trials, the y length of the pull was increased by 3 mm. After a further 6/10 successful trials, the y length of the pull was increased a further 3 mm, to the final pull distance of 9 mm. This phase typically took 3–4 d to learn.

In the next phase, the reward “tunnel” was narrowed to ±5 mm from the x-midline. Mice had to perform above 70% successful pulls per day for two consecutive days, which typically took two to three weeks. All sessions mentioned above were terminated after 50 min, and typically allowed the mouse to perform 200–400 trials.

Sessions with force field trials were then introduced, in which a velocity-dependent force field was generated by the robot motors 0.5 mm to 9 mm along the y-dimension of the pull trajectory. Mice performed 50 preforce field trials during which they had to perform above 65% success, with the pull criteria mentioned above, else trials continued without force field until mice reached 250 trials or 50 min in the apparatus. Data from these days was excluded from further analyses because of low baseline success rate. In sessions where mice reached above 65% in the 50 preforce field (baseline) trials, the force field was then imposed until animals had completed an additional 150 pulls. After this, a further 50 “washout” trials were performed without the force field. Mice typically performed all 250 trials within 35 min. Force field and nonforce field days were randomly interleaved, avoiding session order artefacts caused by mice predicting force field onset, as well as serving as an additional control for baseline motor execution. Optosilenced and light control mice performed 3 d of control and optosilencing sessions in the force field, randomly interleaved, along with interleaving nonforcefield days. Data were discarded if it did not meet the steering angle criteria (see below, Steering angle).

Task performance metrics

Pull kinematics and dynamics

Pull velocity signals were low-pass filtered using a second-order Butterworth filter with a cutoff frequency of 30 Hz. Position and velocity traces were resampled to 201 points using b-spline interpolation, to obtain a constant number of samples per trial and allow for calculation of mean trajectories and mean velocity profiles over an entire session. Trajectory plots (Figs. 1D,E, 3C) were calculated by taking the average of the x and y coordinates of the 201 interpolated points within 10 trial blocks, and then averaged across sessions. The adaptation analyses were focused on velocity and displacement metrics that are typically extracted in human studies (Schwarz et al., 2019; Kanzler et al., 2020).

Max y-velocity

The velocity metric (max y-velocity) was measured at the peak of the y-component of velocity for each pull.

Lateral displacement end x-position

The lateral displacement (end x-position) was measured at the final sample of x in the trajectory.

Probability of counterforce correction

For the probability of counterforce correction, trials were classified into two classes: with or without counterforce correction (Fig. 6B). The criteria for assigning a trial as a counterforce correction was as follows: if at any point in the trajectory the x-velocity component was higher than a threshold of 10 mm/s in the direction of the force field (empirically determined) and then the x-velocity later switched direction against the force field, this was considered a counterforce correction.

Steering angle

The initial steering angle was calculated from the x-position (X0.5, in mm) corresponding to y position of 0.5 mm by finding the arctan of X0.5/0.5, and shown in degrees from y-axis (Figs. 1C,G, 6C–E). Trials in which the total distance covered during the pull was <2 mm were discarded from analyses. Since preoptointerference behavioral state could affect the impact of optointerference, we ensured that control and optointerfered sessions had comparable preoptointerference motor outputs by discarding from analysis any sessions where the mean steering angle in the first 50 trials (i.e., preforce field, preoptointerference) was above or below 25°. In the analyzed sessions, individual data points of end x-position and angle were then baselined to the mean of the first 50 trials, for clearer visualization of the change from preforce field trials.

Success

Success was defined as a correctly executed trial/pull which was rewarded with milkshake. The % success (Figs. 1I, 4A) was computed from the sliding average of 10 trials in a session.

Missed trials, force field session time, and reaction times

Missed trials were defined as the % of trials during which the mouse did not attempt to pull for 10 s after the cue. Force field session time was defined as the total time taken to complete the 150 force field trials. Within-trial reaction time was defined as the time between the cue onset and time when the mice initiated the pull (pull start). All variables plotted as a function of trial number were smoothed using a 10 trial moving mean within a session before averaging sessions (Figs. 1F–I, 2E, 3D,F, 4A,C, 6A–F). This was done for visualization only; for statistical data analysis, unsmoothed data were used.

Number of trials to reach 80% max adaptation

Force field trials in individual sessions were analyzed. Each session was smoothed using the 10 trial moving average, then the 80% value of the maximal end x-position was noted. Then, the trial number needed to reach the 80% value was determined by finding the first trial above the 80% criterion.

Computational models of sensory prediction error-based and reward prediction error-based motor adaptation

Based on previous work (Todorov and Jordan, 2002; Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Mathis et al., 2017), we used a sensory prediction error-based model and a reward prediction error-based model to quantify the influence of those two factors on motor adaptation.

Sensory prediction error-based model

Specifically, the end x-position xk at trial k was modeled as being dependent on the motor command uk, the perturbation pk, and motor noise ηm ∼ N(0, σm): xk= uk + pk + ηm.

Further, the approach assumes that mice receive an estimate of their paw position through proprioception that is influenced by sensory noise ηs ∼ N(0, σs): yk= xk + ηs.

Given the efference copy of the motor command, mice can provide an internal estimate of their expected paw position: x̂k=p̂k + uk relies on an internal model of the perturbation p̂k that is obtained over trial to trial and is updated based on a retention factor r (0 ≤ r ≤ 1) and noise ηp ∼. N(0, σp): p̂k+1=rp̂k + ηp.

These relationships can be expressed in matrix form within a Kalman filter framework (Kalman, 1960): Xk+1=AXk + B uk + η, with the states Xk=(pkxk), the state-transition matrix A=(r010), the control-input vector B=(01), and noise ηp ∼ N(0, Q) with Q=(σp200σm2). In addition, the observation equation is defined as: yk=HXk+ηs, with the observation model H =(01). Based on this Kalman filter framework (details in Todorov and Jordan, 2002; Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Mathis et al., 2017), an a-priori prediction of the paw position can be made, which is compared with the obtained sensory feedback and then used to update the internal model of the perturbation (i.e., sensory prediction error-based learning). Given this internal model of the perturbation, the optimal motor command to counteract the perturbation is: uk=−p̂k.

Reward prediction error-based model

In addition, it is assumed that reward might influence motor adaptation. Hence, reward prediction error-based motor adaptation was modeled using an actor-critic reinforcement learning approach based on the temporal difference rule (Todorov and Jordan, 2002; Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Mathis et al., 2017). Specifically, the optimal action ak according to this model was defined as: ak+1= ak+αaδ, with the learning rate αa (αa>0) and the reward prediction error δ being defined as: δ=fk+γV̂k+1−V̂k, with a discounting factor γ (0 ≤ γ ≤ 1), the value function V̂k, V̂k+1=V̂k+αvδ, and the reward function fk: if trial k is rewarded: fk=1− βuk2, if trial k is rewarded: fk=− βuk2, with the cost parameter β.

Fitting of models to experimental data

We fit two separate models to the data, the first purely relying on the reward prediction error to update motor command, and the second purely relying on the sensory prediction error: Reward prediction error-based model: uk= ak + ηe, Sensory prediction error-based model: uk=−p̂k with exploratory noise ηe ∼ N(0, σe). In each case, the model was fitted to minimize the root mean square error between the estimated and the actual end x-position of mice in the control condition during the force field. For each mouse, the mean of the baseline data were subtracted. Further, a perturbation pkwas defined for each mouse based on the average value in end x-position during the first three trials in the force field. In addition, the end x-position was used for the proprioception-based estimate of the paw position yk.

The free parameters of the model were optimized through a pattern search algorithm (MATLAB function pattern search). Given that this optimization procedure requires the random initialization of free parameters that affects the optimization results, the optimization process was repeated 50 times with different initial parameter values. For the model purely based on the sensory prediction error, the free parameters were optimized on the following intervals: r ∈ [0,1], σm ∈ [0, τ], σs ∈ [0, τ], σp ∈ [0, τ], where τ was defined as the mean interquartile range of end x-position during the baseline period. For the model purely based on the reward prediction error, the free parameters were randomly initialized and optimized on the following intervals: σm ∈ [0, τ], σe ∈ [0, τ], αv∈ [0, 1], αa∈ [0, 1], γ ∈ [0, 1], β ∈ [0, 1]. For both models, all parameters were randomly initialized on the interval [0, 1]. For the 50 optimizations of the reward prediction error-based model, 11 optimization runs had to be removed because of the occurrence of extreme values (infinity) in the model output. No optimization runs had to be removed for the sensory prediction error-based model. In Figure 5, we visualized the motor commands averaged across these realizations. For Figure 5B,D, we report the strength of the command uk from the sensory-based and reward-based models separately. Additionally, we normalized uk relative to the sensory prediction error yk−HXk and the reward prediction error δ for the sensory-based and reward-based models, respectively. This allowed to capture the magnitude of model contributions relative to the changes in sensory perception or reward rate, thereby allowing better distinction between the contribution of different modalities.

Statistical analyses

Statistical analyses were performed using Prism 8.4.3 (GraphPad Inc.). To test the hypothesis that optogenetic manipulations affect adaptation rate of various parameters (end x-position, angle, velocity, reward rate, within-trial reaction times), we first examined the slopes of least-squares linear fits of the plots of the parameter and trial number in the first 20 force field trials, where the trajectory error-based adaptation was expected to be the greatest because of the greatest error at the start of the force field, and before adaptation of control sessions reached a plateau (from visual examination of control end x-position adaptation in Fig. 3D). If the difference in these slopes in control and optomanipulated sessions was significant (Figs. 3E, 6C), we then assessed adaptation rates of the full 150 force field trials using exponential fit statistics; and if it was not significant (Figs. 3G, 4B,D, 6A,B,E,F), the rates were not analyzed further. Where exponential fit comparisons were performed (Figs. 3D,E, 6C,D), the adaptation rates were estimated using a least-squares fit on an exponential plateau equation to all data points including all sessions and all trials in the 150 force field trial block (trials 50–200) of each dataset. The equation was: Y = YM – (YM – Y0) *exp (-k *x), where Y is end x-position or angle, YM is the plateau value, Y0 is the initial value, and k is adaptation rate. Y0 values were estimated from the mean of the first five force field trials and fixed as shared parameters between control and optosilenced datasets, and YM and k were free parameters during the fits. An extra sum-of-squares F test was then used to assess whether the best-fit values of k differed between datasets. Group data are presented as mean ± SEM unless indicated otherwise.

Results

Reproducing assessment of human upper limb adaptation using mouse-robot interactions

We first explored whether mice can perform complex and precisely quantifiable forelimb tasks that mimic those used in robot-assisted studies of human upper limb adaptation (Shadmehr and Mussa-Ivaldi, 1994). A total of 18 mice were trained to pull a robotic arm on a defined trajectory (9-mm straight pull with <5-mm lateral deviation was rewarded with milkshake, Fig. 1A,B; see Materials and Methods). The robot tracked pull dynamics and kinematics at a high temporal resolution, and used this information to compute and impose precisely defined velocity-dependent force fields perpendicular to the pulling motion. This perturbed the pull trajectory in real-time (Fig. 1B–E). The pull was performed in the dark, so sensory feedback was limited to somatosensation. After the mice performed 50 pulls without a force field (baseline trials), the robot introduced a velocity-dependent force field for the next 150 pulls (force field trials), followed by 50 pulls without the force field (washout trials; Fig. 1B–E). During initial force field trials, the unpredicted force field produced significant lateral displacement of the pull trajectory, quantified using the lateral endpoint position (“end x-position,” explanation in Fig. 1C,D–F; statistical analysis in Fig. 1F, right panel). During subsequent trials, mice progressively compensated for the force field (Fig. 1D–F), reducing lateral displacement by ∼50% by the end of the 150 force field trials [Fig. 1F, right panel, one-way repeated measures (RM) ANOVA across force field trials, F(2,64) = 13.53, p < 0.0001]. In the washout trials, the unpredicted removal of the force field resulted in a significant x-overshoot in the direction opposing the force field (Fig. 1D–F; statistical analysis in Fig. 1F, right panel).

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Mouse-robot interactions reveal forelimb motor adaptations to defined force fields. A, Left, Diagram of the ETH Pattus robotic manipulandum with top-down view of a head-fixed mouse, pulling the robot handle. Right, Diagram of the skilled motor task. Mice had to pull 9 mm in the y-direction within a ±5 mm x “tunnel” to obtain a reward. On trials 51–200, a velocity-dependent force (cyan arrows) pushed the animal's forelimb in the x direction. B, Top row, Diagram of the sequence of a single trial. Bottom row, Diagram of a single force field session. The mice performed 50 trials with no force field to get a baseline (black bar), then 150 trials of force field exposure (cyan to magenta bar), followed by 50 trials of no force field to determine the washout (red bar). C, Graphical explanation of end x-position and steering angle. D, Left panel, Typical example of trajectory evolution across a force field session. The force field was present in trials 51–200, individual lines represent average of 25 blocks. Right panel, Magnification of the first 0.5 mm of the left trajectories, demonstrating preemptive steering adaptation. E, Averages of five trajectories taken from preforce field (black), force field (blue, purple, magenta), and postforce field (washout, red) trial blocks. Data from 33 sessions from 12 mice. A, Left panel, Lateral displacement at the end of pulls (end x-position), expressed as mean ± SEM, versus trials (n = 33 sessions from 12 mice). The shared x-axis for panels F, H is given in panel H. Right panel, End x-position within indicated trials (mean ± SEM). RM ANOVA F(4,128) = 86.28, p < 0.0001; Sidak's multiple comparison tests ****p < 0.0001, ***p < 0.001. G, Left panel, Same as in F, left panel, for predictive steering angle. Right panel, Same as in F, right panel, for predictive steering angle. RM ANOVA F(4,128) = 18.07, p < 0.0001; Sidak's multiple comparison tests: ****p < 0.0001, ns = p > 0.05. H, Left panel, Same as in F, left panel, showing max y-velocity, ns = p > 0.05. Right panel, Same as in F, right panel, for max y-velocity. RM ANOVA F(4,128) = 3.578, p < 0.01; Sidak's multiple comparison tests: ns = p > 0.05. I, Left panel, Same as in F, left panel, for percentage of successful trials. Right panel, Same as in F, right panel, for percentage of successful trials. RM ANOVA F(4,128) = 5.532, p < 0.001; Sidak's multiple comparison tests: ns = p > 0.05, ****p < 0.0001.

In theory, mice had several ways to reduce the perturbation induced by the velocity-dependent force field, including reducing pull velocity and/or adapting the initial pull angle to counteract the force field. Across force field trials, mice progressively changed their initial pull angle, measured during the first 0.5 mm of the pull, i.e., before force field onset (Fig. 1G, right panel; one-way RM ANOVA across force field trials, F(2,64) = 25.09, p < 0.0001; steering angle explanation in Fig. 1C). This preemptive steering indicates that mice learnt to expect the force field from recent force field trials, and predictively deployed compensatory motor commands. The delay in resuming preforcefield pull angles in washout trials confirmed this (Fig. 1G, right panel). In contrast, we found no significant within-force field adaptation in y-velocity (Fig. 1H, right panel; one-way RM ANOVA across force field trials, F(2,64) = 3.021, p > 0.05).

Importantly, the reward rate did not change significantly during force field trials (Fig. 1I, right panel; one-way RM ANOVA across force field trials, F(2,64) = 1.895, ns = p > 0.05). Such absence of a strong relationship between trial-to-trial changes in motor output and reward feedback is thought to indicate that the observed motor adaptation is driven primarily by sensory errors rather than reward errors (Izawa and Shadmehr, 2011; Mathis et al., 2017).

Cumulatively, these data indicate that mice adapted to the velocity-dependent force field by updating the sensory-error-based internal model, manifested by (1) change in the initial pull angle, (2) progressively corrected x-displacement across the force field trials, (3) significant after effects when the force field was removed.

HON activity associated with task execution and motor adaptation

The development of the human-like adaptation task for mice (Fig. 1) allowed us to apply neural sensing technologies to explore HON correlates of skilled forelimb movements and motor adaptation. Genetic targeting of the calcium sensor GCaMP6s to HONs can be used to observe their activity at high temporal resolution (Karnani et al., 2020; Fig. 2A). Measuring emitted GCaMP6s fluorescence in these cells is a good proxy for activity, because it is linearly related to HON action potential firing frequency (González et al., 2016b). Anatomically, HONs show ∼80% ipsilateral preference in projecting to motor-related targets such as the cortex and spinal cord (van den Pol, 1999; Jin et al., 2016). However, these targets control contralateral and ipsilateral limbs, respectively, and so unilateral HONs may in theory influence both forelimbs. Consistent with this, when we recorded HON activity bilaterally, we found no clear difference between movement-associated signals between hemispheres (Fig. 2C), and therefore combined data from both hemispheres in subsequent analyses. Recording HON-GCaMP6s activity during pull execution revealed that each pull was associated with a distinct rise in GCaMP6s fluorescence that became apparent ∼400 ms before the pull onset, persisted throughout the pull, and lasted for a few seconds afterward (Fig. 2B,C). To control for movement artifacts in these recordings, we examined the raw calcium-independent HON-GCaMP6s fluorescence signal evoked by isosbestic 405-nm excitation (Kim et al., 2016). During the pulls, we did not observe similar increases in 405 nm-evoked fluorescence (Fig. 2B). This confirms that the HON-GCaMP6s signals are not movement artefacts, but reflect endogenous hypothalamic HON activity waves temporally-associated with execution of skilled forelimb movements (HONWEMs).

Figure 2.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2.

HON signals associated with skilled forelimb movements and motor adaptation. A, Targeting schematic (left) and typical expression (right) of AAV1-hORX-GCaMP6s in HONs, and example fiber placements in two additional mice (bottom). Dashed yellow box indicates fiber placement. V3, third ventricle; VMH, ventromedial nucleus of the hypothalamus; LH, lateral hypothalamus. B, Typical example of raw Ca2+-dependent (470 = 470 nm-excited fluorescence) and Ca2+-independent (405 = 405 nm-excited fluorescence) components of HON-GCaMP6s signal during trials. Epochs of the single trial are labeled: light blue line indicates the cue onset, blue bar indicates the pull, and magenta line indicates reward delivery. C, Z-scored HON-GCaMP6s activity of left (black) and right hemispheres (blue) during nonforce field trials with an intertrial interval of >15 s. Cue bar (gray) indicates the range of trial cue onsets across animals. Blue bar indicates the range of pull durations across animals, magenta bar indicates the range of reward delivery times. Data are mean ± SEM for n = 50 trials from 4 mice. D, Top, Diagram explaining the window of analysis of premotor and motor-related HON-GCaMP6s signals of a single trial (green bar). Bottom, Evolution of pull-associated HON-GCaMP6s signals across control nonforce field trials (left), and force field trials (right), aligned to pull completion. Traces are baseline subtracted from −2 to −1 s before pull start and averaged across sessions (NForce field = 15 sessions, NNo Force field = 9 sessions, from 4 mice). E, Average GCaMP6s signal amplitude, expressed as Δ Z – score, within indicated trial blocks from D of no force field (black) and force field (green) sessions. Two-way mixed-effects model analysis, interaction F(4,56) = 8.082, p < 0.0001; Sidak's multiple comparison tests: ****p < 0.0001, ns = p > 0.05. Signal trends across the force field trials (cyan shading) and equivalent no force field trials (yellow shading) are compared in Results. F, Average GCaMP6s signal amplitude versus trial number across force field (green) and no force field sessions (black), Δ Z – score represents signal amplitude (see Materials and Methods; NForce field = 15 sessions, NNo Force field = 9 sessions, from 4 mice).

We next examined whether HONWEMs change during motor adaptation. Formal frameworks of motor adaptation envision that motor commands are updated to reduce sensory errors (Wolpert et al., 1998; Izawa and Shadmehr, 2011). During force field adaptation, we thus reasoned that a neural signal representing sensory errors would initially be high, then decay during error-reducing adaptation. In turn, a signal representing components of motor adaptation would gradually increase during force field adaptation, whereas a signal not involved, such as a permissive signal for movement, would stay constant.

To differentiate between these possibilities, we tracked HONWEM amplitude (quantified at movement end) across baseline, force field, and washout trials. We found that HONWEM amplitude gradually increased during force field trials (Fig. 2E, one-way RM ANOVA across force field trials, F(2,58) = 20.74, p < 0.0001; Fig. 2F). In contrast, when the same experiment was performed in the same mice without the force field, HONWEM amplitude was unchanged across the trial epoch that corresponded to force field trials in force field sessions (Fig. 2E, one-way RM ANOVA across trials 51–200, F(2,34) = 1.182, p > 0.05; Fig. 2F). The latter finding confirms that the gradual increase in HONWEM amplitude during force field trials was related to responses to force field, rather than to nonspecific factors such as task duration or satiation. Overall, our data therefore suggest that HONWEMs represent some aspect(s) of updated motor command, rather than a sensory error signal.

Role of movement-associated HON signals in forelimb motor adaptation

To explore whether HONWEMs influence adaptation rather than simply represent it, we used optogenetics to inhibit them during force field adaptation. We targeted the optogenetic inhibitory opsin ArchT to HONs (Garau et al., 2020; Karnani et al., 2020), and bilaterally implanted optic fibers in the lateral hypothalamus (Fig. 3A). We confirmed effective HON-ArchT optosilencing using patch-clamp recordings in acute brain slices (Fig. 3B). On each force field trial, lateral hypothalamic illumination (bilateral 532-nm laser) started at the cue and stopped at the end of the pull before reward delivery (see Materials and Methods), thus targeting premotor and movement epochs of each pull, since both of these epochs are implicated in movement control (da Silva et al., 2018; Karnani et al., 2020; Inagaki et al., 2022).

Figure 3.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 3.

HON optosilencing impairs motor adaptation. A, Top, Targeting schematic (left) and typical expression of ArchT (right) in HONs. Bottom, Single trial diagram depicting when laser stimulation was applied (magenta bar). B, Whole-cell patch-clamp brain slice recording confirming optosilencing of an HON-ArchT neuron by green laser (representative example of n = 20 cells). C, Average trajectories of baseline, force field, and washout trajectories in control and optosilencing sessions. Blocks of 10 trials were averaged within a session and then averaged across sessions. Means ± SEM of 19 control and 14 optosilencing sessions from 7 mice. D, Average end x-position value versus trial number. Means ± SEM of 19 control (black) and 14 optosilencing (magenta) sessions from 7 mice. Inset, Exponential fits to the data (thick lines) and their confidence intervals (thin lines) demonstrating reduced adaptation rate in control versus optosilenced sessions (extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,4801) = 14.08, p < 0.001). Smoothed means are shown for visualization only, the fit was performed using individual trial points of unsmoothed data. E, For data shown in D, initial adaptation rates from individual sessions, obtained using least-squares linear fits of the first 20 force field trials of each session. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 73, *p < 0.05. F, Average end x-position value versus trial number of light control mice. Means ± SEM of 10 control (black) and 13 light on (orange) sessions from 6 mice. Inset, Exponential fits to the data (thick lines) and their confidence intervals (thin lines), demonstrating no difference in adaptation rate because of brain heating (extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,3378) = 3.330, p > 0.05) G. Same as in E, for light controls. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(NLightOff = 10, NLightOn = 13) = 58, p > 0.05.

The HONWEM-optosilenced force field sessions displayed significantly slower rates of motor adaptation compared with control force field sessions in the same mice [trajectory examples, Fig. 3C,D, extra sum-of-squares F test comparison of exponential rate constants across all 150 force field trials: F(1,296) = 18.33, p < 0.0001, Fig. 3E; comparison of linear slopes of the initial 20 force field trials, chosen as the region with the steepest adaptation before control sessions reached plateau, see Materials and Methods, Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 73, p = 0.0288; Fig. 3E]. Additionally, during the force field trial block, we quantified the number of trials to reach 80% of maximal adaptation in each session (see Materials and Methods), and found that optosilencing increased this number, indicating slower adaptation (optosilenced: 100.4 ± 12.8 trials, control: 62.1 ± 8.8 trials, two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 68.5, p = 0.017).

Of note, the different adaptation rates in HONWEM-optosilenced and control force field sessions involved similar initial trajectory displacements, and thus similar sensory errors, during the first force field trial (Fig. 3D; trial 51: optosilenced end x-position: −5.8 ± 1.17 mm, control: −5.62 ± 1.37 mm, n = 14 and 19 sessions from 7 mice, respectively; two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 132, p > 0.05). Importantly, this indicates that the force field initially perturbed limb movements to a similar extent during control and optosilenced trials, but that optosilencing impaired motor updating in response to the sensory error. Despite this slower updating of motor commands in HONWEM-optosilenced sessions, at the end of the 150 consecutive force field trials the mice reached similar levels of motor output to control force field sessions (Fig. 3D). Thus, optosilencing did not abolish the ability of the motor system to eventually generate control-like trajectories, but selectively affected the rate of trajectory updating. The same optomanipulation in control mice (without inhibitory opsins) did not influence forelimb adaptation, confirming that motor adaptation impairment (Fig. 3C–E) was not because of artefacts such as tissue heating (Fig. 3F, extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,3378) = 3.330, p > 0.05; Fig. 3G, two-tailed Mann–Whitney test, U(NLightOff = 10, NLightOn = 13) = 58, p > 0.05.)

Role of movement-associated HON signals in general task engagement

In contrast to these differences in motor adaptation rate, in HONWEM-optosilenced and control force field sessions, mice displayed similar reward success rates (Fig. 4A,B), similar cue-to-pull-initiation times (Fig. 4C,D), similar numbers of missed trials (% of trials where pull was not initiated in >10 s after cue: control = 5.26 ± 3.10; optosilenced = 6.43 ± 2.43, two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 87.50, p > 0.05), and similar times to complete the 150 force field trials (control = 20.14±1.209 min; optosilenced = 20.34 ± 1.000 min, two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 115, p > 0.05). Collectively, these results suggest that the HONWEMs were not required for the general motivation or attentiveness to perform the task.

Figure 4.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 4.

HON optosilencing does not impair reward rates or motivational metrics. A, Average percentage of successful trials versus trial number of control and HON optosilencing sessions. Means ± SEM of 19 control and 14 optosilencing sessions from 7 mice. B, For data shown in A, initial adaptation rates from individual sessions, obtained using least-squares linear fits of the first 20 force field trials of each session. Violin plot of density, dashed line at median and dotted line at the quartiles, Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 131, ns = p > 0.05. C, As in A for reaction time, calculated as the time between cue onset and pull onset. D, As in B for data in C, Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 105, ns = p > 0.05.

HON optosilencing results in impaired sensory prediction error-based adaptation

Recent evidence has shown that motor adaptation can be driven by both sensory-prediction errors and reward prediction errors (Izawa and Shadmehr, 2011). From previous studies in motor adaptation, internal models are updated to minimize sensory prediction errors, i.e., the error between the sensory prediction and sensory outcome (Oh and Schweighofer, 2019). Alternatively, reward prediction error-based models suggest the internal model of a motor plan would be adapted strictly based on whether actions resulted in receiving a reward. Based on previous work (Todorov and Jordan, 2002; Berniker and Kording, 2008; Izawa and Shadmehr, 2011; Mathis et al., 2017), we used a sensory prediction error-based model and a reward prediction error-based model to quantify the influence of those two factors on motor adaptation. We first tested each model on data from control mice to determine the relative contribution of reward prediction error and sensory prediction error to adaptation across the force field (Fig. 5A,B). Consistent with the incremental increase in the reward rate across the force field, a reward prediction error-based learning model failed to fit the behavioral data, in contrast to a sensory prediction error-based model which accounted well for the observed pattern of motor adaptation (Fig. 5A,B; two-tailed Mann–Whitney test, U(Ncontrol = 152, Noptosilenced = 156) = 737, p < 0.0001). Based on the previous results, we further evaluated the role of HON optosilencing in the sensory prediction error-based model. We performed the same analysis on optosilenced sessions and found that HON inactivation resulted in impaired sensory prediction error-based adaptation (Fig. 5C,D; two-tailed Mann–Whitney test, U(Ncontrol = 152, NOptosilenced = 156) = 6295, p < 0.0001).

Figure 5.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 5.

Forelimb adaptation and effects of HON optosilencing follow a sensory prediction error-based model. A, Orange line shows the average end x-position value versus trial number for control force field data from 19 sessions. The shaded orange area depicts the SEM. Black line depicts a sensory prediction error-based model that was fitted to the force field data of each session. The blue line depicts the reward prediction error-based model. The average of 50 realizations of the model with different random initial parameters is visualized. The sensory prediction error-based model shows population-level root-mean-square error of the fit is 1.12 cm, and the reward prediction error-based model shows population-level root-mean-square error of the fit is 1.98 cm. B, Normalized command strength contributions of the reward prediction error-based model (blue) and the sensory prediction error-based model (black) of control sessions. The command strength of the sensory and reward-based models were normalized with respect to the sensory and reward prediction error, respectively. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(Ncontrol = 152, NOptosilenced = 156) = 737, ****p < 0.0001. C, As in A for only the sensory prediction error-based model for both control and optosilencing sessions. D, As in B for the normalized contributions of sensory prediction error-based model for both control and optosilencing sessions. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(Ncontrol = 152, NOptosilenced = 156) = 6295, ****p < 0.0001.

Effect of HON silencing on subprocesses of motor adaptation

We next focused on trying to dissect subprocesses of limb movement that could explain the impairment of the sensory prediction error-based model in HON optosilencing, shown in Figures 3C–E and 5. A few different factors could explain why HON optosilencing impairs the model: (1) reduced strength of motor output; (2) impairment of online sensory perception and thus online motor correction; (3) improper sensorimotor representation of the force field. We performed experiments and analysis aimed at investigating whether such factors may explain the motor adaptation impairment observed in Figure 3C–E.

First, we wanted to determine whether a reduction in strength was responsible for the impaired adaptation to the force field during HON silencing. We looked at the max y-velocity of a trajectory as a proxy for strength, and found no significant difference between control and optosilenced sessions, suggesting HON inhibition did not significantly impair mouse forelimb strength (Fig. 6A; two-tailed Mann–Whitney test, U(Ncontrol = 19, NArchT = 14) = 129, ns = p > 0.05). Furthermore, since the force field is velocity-dependent, this suggests that in both control and opto-silencing sessions, mice experienced a similar amount of perturbation force from the robot on a per trial basis, so differences in force field strength cannot account for the differences in the end x-position. Finally, because end x-position in optosilencing trials reached similar levels to controls in the last forcefield trials, this suggests strength needed to create a correction was not impaired (Fig. 3D).

Figure 6.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 6.

HON optosilencing disrupts sensory prediction error-based model through impaired sensorimotor integration. A, Left panel, Average max y-velocity versus trial number. Means ± SEM of n = 19 force field sessions and 14 optosilencing sessions from 7 mice. Right panel, Initial adaptation rates from individual sessions for data shown in left panel, obtained using least-squares linear fits of the first 20 force field trials of each session. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(NControl = 19, NOptosilenced = 14) = 129, ns = p > 0.05. B, Average probability of a trial having a counterforce feedback correction versus trial number. Inset describes an example trial with counterforce feedback correction (for calculation, see Materials and Methods). C, Left panel, Same as A, but for steering angle. Inset, Exponential fits to the data (thick lines) and confidence intervals (thin lines), illustrating reduced adaptation rate (extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,4797) = 13.49, p < 0.001). Smoothed means are shown for visualization only, the fit was performed using individual trial points of unsmoothed data. Right panel, Initial adaptation rates from individual sessions for data shown in A, obtained using least-squares linear fits of the first 20 force field trials of each session. Violin plot of density, dashed line at median and dotted line at the quartiles, two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 14) = 70, *p < 0.05. D, Left panel, Same as in A, left panel, for light control mice, demonstrating no difference in adaptation rate because of tissue heating (extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,3372) = 0.9682, p > 0.05). Right panel, Same as (A, right panel) for data shown in left panel, two-tailed Mann–Whitney test, U(NLightOff = 10, NLightOff = 13) = 61, ns = p > 0.05. E, Left panel, Average steering angle value versus trial number, without the force field. Means ± SEM of 11 control and 13 optosilencing sessions from 7 mice. Right panel, As in A, right panel, for data in left panel, two-tailed Mann–Whitney test, U(NControl = 11, NOptosilenced = 13) = 64, ns = p > 0.05. F, Left panel, Average end x-position value versus trial number, without the force field. Means ± SEM of 11 control and 13 optosilencing sessions from 7 mice. Right panel, As in A, right panel, for data in left panel, two-tailed Mann–Whitney test, U(NControl = 11, NOptosilenced = 13) = 53, ns = p > 0.05.

Next, we examined online (i.e., within-trial) corrections, which can be detected by measuring sudden corrections in direction during a pull (example, Fig. 6B, inset; for description, see Materials and Methods), presumably arising from online sensory feedback. The probability of a mouse making such corrections against the force field was unchanged by HON optosilencing (Fig. 6B), suggesting that sensory perception of the force field and motor output (spinal loops, etc.) was not abolished by HON optosilencing.

The internal model represents a model of the neuromuscular system in combination with the external world, and consequently acts as a neural simulator that makes predictions of the effect of the current motor plan (Franklin and Wolpert, 2011). A necessary input for the model is a copy of the current motor plan, known as the efference copy. The efference copy is used as a sensorimotor template to perform sensory state estimation, prediction of sensory feedback, or purely for feedforward (predictive) control outcomes. In the forelimb adaptation task, sensory feedback loops presumably do not play a significant role during the first 0.5 mm of a pull, before the forcefield is engaged. Steering angle during this period can thus be used as a metric to estimate the response of the internal model to the efference copy input. We found that HON optosilencing reduced the rate of adaptation in the steering angle (Fig. 6C, extra sum-of-squares F test of whether k fit parameters are different between datasets, F(1,4797) = 13.49, p < 0.001; Fig. 6D two-tailed Mann–Whitney test, U(Ncontrol = 19, Noptosilenced = 1 4) = 51, p < 0.01, n = 7 mice). This effect of optomanipulation was not seen in control mice whose HONs lacked the inhibitory optoactuators (Fig. 6D; two-tailed Mann–Whitney test, U(Nlightoff = 10, Nlighton = 13) = 61, ns = p > 0.05, n = 6 mice).

These results are consistent with the hypothesis that HONWEMs may be selectively involved in emission of updated motor commands during force field adaptation. A key prediction of this hypothesis would be that HONWEM optosilencing should not affect task motor execution in the absence of force field perturbation, where errors, and thus motor command updating by the model, are minor. To test whether this prediction was correct, we repeated the optosilencing experiments without the force field, and indeed found that neither the pull angle (Fig. 6E; two-tailed Mann–Whitney test, U(NControl = 11, NOptosilenced = 13) = 64, ns = p > 0.05, n = 7 mice), nor pull trajectory (Fig. 6F; two-tailed Mann–Whitney test, U(NControl = 11, NOptosilenced = 13) = 53, ns = p > 0.05, n = 7 mice) were significantly affected.

Discussion

Our data show for the first time that mice are capable of performing a skilled forelimb task requiring motor adaptation to complex velocity-dependent force field perturbations, as previously observed in humans and nonhuman primates (Shadmehr and Mussa-Ivaldi, 1994; Perich et al., 2018). The robotic technology and experimental protocol described here can thus be used to study human-level motor control in a mouse model, allowing the dissection of underlying neural influences. We used this technology to explore a long-standing question: what is the role of endogenous activity of HONs, a type of neuron conserved between mice and humans and known to be an anatomic part of the motor system, in precisely defined elements of motor adaptation? At the correlational level, our results demonstrate that temporally-distinct endogenous waves of HON activity are associated with forelimb movements, and increase in amplitude during forelimb motor adaptation. At the causal level, we found that temporally-restricted optosilencing of these HONWEMs impaired sensory error-based adaptation. Crucially, HONWEM optosilencing affected neither the metrics of motivation such as reaction times and trial execution rates, nor forelimb motor outputs in tasks not requiring motor adaptation, indicating that the HON signals identified and studied were distinct from signals governing general motivation or muscle tone. Together, these results suggest that the endogenous HON signals facilitate motor adaptation during skilled forelimb movements.

Role of endogenous HON activity in motor adaptation

The hypothalamus has been theorized to act as a key regulatory module in mammalian motor control for at least 50 years (Wayner, 1970; Arber and Costa, 2018). However, the function of specific hypothalamic cell types in defined aspects of acquired, adaptable movement patterns remained largely unknown. Our neural recordings and behavioral observations now provide clear evidence that HONs are endogenously active during certain skilled movements and contribute to forelimb motor adaptation. In the context of motor control, prior studies only looked at HONs from anatomic or reduced (in vitro) physiological perspectives, or looked at the effect of HON disruption on innate behavior (running, eating) without distinguishing motivation and motor components. Our new experimental approach, combining robotics with neural recordings and control, enabled us to separate sensory-error based motor control from reward and motivation metrics. This revealed that HON signals associated with forelimb movements impact sensory prediction error-based motor adaptation, but not reward-based learning or motivation to do the task (Figs. 3, 4).

Thus, the lateral hypothalamus and HONs can be added to the growing list of brain regions involved in motor adaptation. It is noteworthy that brain regions implicated in motor adaptation in previous literature are innervated by HONs, specifically the cerebellum, cortical areas, and midbrain and striatal circuits (Peyron et al., 1998; Cutler et al., 1999; Nambu et al., 1999; Date et al., 2000; Korotkova et al., 2002; Bayer et al., 2004; Biswabharati et al., 2018). This suggests that HONs may implement motor adaptation both directly (via spinal cord projections; van den Pol, 1999; Yamuy et al., 2004) and/or indirectly by providing a modulatory influence to adaptation-implicated cortical and midbrain areas. Specifically, direct HON modulation of somatosensory cortex (Bayer et al., 2004) may be of particular importance, since the latter region was recently found to be an essential mediator of forelimb adaptation of a very similar type to that studied here (Mathis et al., 2017). Alternatively or in addition, HONs might indirectly assist forelimb motor adaptation through their modulation of locomotor and vestibular processes (Zhang et al., 2011; Karnani et al., 2020) that may contribute to overall postural adjustment that may help the new movement pattern to emerge. The relative functional importance of HON action on these different brain regions and processes in the context of motor control is a key subject for future work, for which our study provides an appropriate new mouse model.

Our conclusions about the role of endogenous HON activity in forelimb motor adaptation are based on similar experimental criteria performed in human studies. Velocity-dependent force fields allow for the perturbation to be dependent on the dynamics of the task, rather than applied at a fixed, arbitrary position, meaning that the internal model of task dynamics has to be remapped. Additionally, velocity-dependent force fields allow the same amount of force to be applied between long and short duration trials, i.e., total force is independent of time, whereas, with constant fields, the total force applied is dependent on the time spent in the field. Such velocity-dependent forcefields can only be implemented with advanced robotic technologies, which is a key methodological contribution of our work and advances the experimental paradigms applied in previous studies that typically rely on a fixed force field based on magnets (Mathis et al., 2017).

We found that mice follow broadly similar strategies to humans when their forelimbs adapt to perpendicular velocity-dependent force fields. They optimize for a strategy of overcompensation to the force field, rather than passive within-trial sensory feedback (Figs. 1G, 6B), suggesting sensorimotor remapping of the internal model. Our optosilenced data and sensory prediction error-based model suggests that HONs play an integral part in this remapping, and cannot be described by other features such as strength (Fig. 6A) or the complete sensory inability to perceive an external force (Fig. 6B).

Crucially, we also measured HON activity across trials where motor adaptation occurred (Fig. 2), revealing two unexpected features: activity waves associated with initiation and execution of skilled forelimb movements (Fig. 2B,C), and the increase in the amplitude of these waves across adaptation (Fig. 2D–F). This evolution of endogenous HON activity during the predictive/feedforward phase of each trial, i.e., before the mice start pulling and feel the force field, is consistent with a role of this HON activity in shaping predictive steering during motor adaptation (Fig. 6C). While our proposal about the role of HONs in emission of new motor commands seems a plausible explanation given our data, at present we cannot completely rule out other explanations such as changes in muscle coordination, although the lack of effect of HON silencing in motor tasks not involving adaptation makes this unlikely (Fig. 6E,F). Internal model updating is likely to be a complex process involving multiple computational subprocess and brain areas such as the cerebellum (Wolpert et al., 1998), and it will be important to address how HONs interact with these areas in future work.

Relation to previous studies of HONs in the context of arousal, motivation, and movement

Until now, HONs have been studied largely in the context of arousal and motivation. These studies revealed that loss of HONs in humans or mice makes wakefulness and muscle tone unstable, clinically known as narcolepsy with cataplexy, and orexin receptor antagonists stop rodents from working for reward (Nishino et al., 2000; Peyron et al., 2000; Thannickal et al., 2000; Hara et al., 2001; Boutrel et al., 2005; Harris et al., 2005; de Lecea et al., 2006; Harris and Aston-Jones, 2006; Mahler et al., 2014; Bassetti et al., 2019). Based on these previous conceptualizations of the endogenous role of HONs in goal-directed behavior, we expected HON optosilencing to reduce task execution rates. It was therefore surprising that HON optosilencing in our experiments affected neither motivation metrics such as trial rate, reward rate and reaction times, nor within-trial metrics such as movement velocity, arguing against major effects on muscle tone (Figs. 4C, 6A). However, we do not think our data contradict or dispute the earlier findings on the role of HONs in arousal and reward. Our study is fundamentally different because it targeted temporally and behaviorally defined HON signals (HONWEMs), whereas previous studies generally inhibited all HON signals for a prolonged time. It is not straightforward to draw parallels between these chronic effects of HON silencing, and acute effects studied here. Our results support the view that “motor” HON signals identified here are specifically involved in motor adaptation. This does not rule out that HON signals occurring at different times control arousal, reward, or muscle tone as previously proposed (Kiyashchenko et al., 2001; Mileykovskiy et al., 2002, 2005; Mochizuki et al., 2004; Lee et al., 2005; Mahler et al., 2014; Sakurai, 2014). Investigating candidate mechanisms that enable this HON “multitasking,” such as distinct temporal activity patterns of HONs and/or functionally different HON subpopulations, is a key subject for future work, for which the robotic tool presented here provides a suitable method that allows motivational/arousal and motor aspects to be distinguished.

In conclusion, despite numerous studies of the mechanisms of motor adaptation, genetically-defined hypothalamic signals that regulate it remained largely elusive. Our new mouse model of forelimb motor adaptation, which is directly translated from robotic paradigms in human motor adaptation studies, opens new possibilities for dissecting the neural underpinnings of motor adaptation. Our identification of a noncanonical role of hypothalamic neurons in adaptation illustrates the usefulness of this model, and provides a new genetically-defined entry point for probing neural circuits and signals that implement motor adaptation.

Footnotes

  • The labs of D.B. and O.L. are supported by Eidgenössische Technische Hochschule Zürich.

  • The authors declare no competing financial interests.

  • Correspondence should be addressed to Denis Burdakov at denis.burdakov{at}hest.ethz.ch or Olivier Lambercy at olivier.lambercy{at}hest.ethz.ch

SfN exclusive license.

References

  1. ↵
    1. Arber S,
    2. Costa RM
    (2018) Connecting neuronal circuits for movement. Science 360:1403–1404. doi:10.1126/science.aat5994 pmid:29954969
    OpenUrlAbstract/FREE Full Text
  2. ↵
    1. Bassetti CLA,
    2. Adamantidis A,
    3. Burdakov D,
    4. Han F,
    5. Gay S,
    6. Kallweit U,
    7. Khatami R,
    8. Koning F,
    9. Kornum BR,
    10. Lammers GJ,
    11. Liblau RS,
    12. Luppi PH,
    13. Mayer G,
    14. Pollmächer T,
    15. Sakurai T,
    16. Sallusto F,
    17. Scammell TE,
    18. Tafti M,
    19. Dauvilliers Y
    (2019) Narcolepsy - clinical spectrum, aetiopathophysiology, diagnosis and treatment. Nat Rev Neurol 15:519–539. doi:10.1038/s41582-019-0226-9 pmid:31324898
    OpenUrlCrossRefPubMed
  3. ↵
    1. Bayer L,
    2. Serafin M,
    3. Eggermann E,
    4. Saint-Mleux B,
    5. Machard D,
    6. Jones BE,
    7. Mühlethaler M
    (2004) Exclusive postsynaptic action of hypocretin-orexin on sublayer 6b cortical neurons. J Neurosci 24:6760–6764. doi:10.1523/JNEUROSCI.1783-04.2004 pmid:15282280
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Bear MF,
    2. Connors BW,
    3. Paradiso MA
    (2015) Neuroscience: exploring the brain. Burlington: Jones & Bartlett Publ Inc.
  5. ↵
    1. Berniker M,
    2. Kording K
    (2008) Estimating the sources of motor errors for adaptation and generalization. Nat Neurosci 11:1454–1461. doi:10.1038/nn.2229 pmid:19011624
    OpenUrlCrossRefPubMed
  6. ↵
    1. Biswabharati S,
    2. Jean-Xavier C,
    3. Eaton SEA,
    4. Lognon AP,
    5. Brett R,
    6. Hardjasa L,
    7. Whelan PJ
    (2018) Orexinergic modulation of spinal motor activity in the neonatal mouse spinal cord. eNeuro 5:ENEURO.0226-18.2018. doi:10.1523/ENEURO.0226-18.2018
    OpenUrlAbstract/FREE Full Text
  7. ↵
    1. Boutrel B,
    2. Kenny PJ,
    3. Specio SE,
    4. Martin-Fardon R,
    5. Markou A,
    6. Koob GF,
    7. de Lecea L
    (2005) Role for hypocretin in mediating stress-induced reinstatement of cocaine-seeking behavior. Proc Natl Acad Sci U S A 102:19168–19173. doi:10.1073/pnas.0507480102 pmid:16357203
    OpenUrlAbstract/FREE Full Text
  8. ↵
    1. Butcher PA,
    2. Ivry RB,
    3. Kuo SH,
    4. Rydz D,
    5. Krakauer JW,
    6. Taylor JA
    (2017) The cerebellum does more than sensory prediction error-based learning in sensorimotor adaptation tasks. J Neurophysiol 118:1622–1636. doi:10.1152/jn.00451.2017 pmid:28637818
    OpenUrlCrossRefPubMed
  9. ↵
    1. Butler EG,
    2. Bourke DW,
    3. Horne MK
    (2000) The activity of primate ventrolateral thalamic neurones during motor adaptation. Exp Brain Res 133:514–531. doi:10.1007/s002210000355 pmid:10985686
    OpenUrlCrossRefPubMed
  10. ↵
    1. Chemelli RM,
    2. Willie JT,
    3. Sinton CM,
    4. Elmquist JK,
    5. Scammell T,
    6. Lee C,
    7. Richardson JA,
    8. Williams SC,
    9. Xiong Y,
    10. Kisanuki Y,
    11. Fitch TE,
    12. Nakazato M,
    13. Hammer RE,
    14. Saper CB,
    15. Yanagisawa M
    (1999) Narcolepsy in orexin knockout mice: molecular genetics of sleep regulation. Cell 98:437–451. doi:10.1016/s0092-8674(00)81973-x pmid:10481909
    OpenUrlCrossRefPubMed
  11. ↵
    1. Cutler DJ,
    2. Morris R,
    3. Sheridhar V,
    4. Wattam TA,
    5. Holmes S,
    6. Patel S,
    7. Arch JR,
    8. Wilson S,
    9. Buckingham RE,
    10. Evans ML,
    11. Leslie RA,
    12. Williams G
    (1999) Differential distribution of orexin-A and orexin-B immunoreactivity in the rat brain and spinal cord. Peptides 20:1455–1470. doi:10.1016/s0196-9781(99)00157-6 pmid:10698122
    OpenUrlCrossRefPubMed
  12. ↵
    1. da Silva JA,
    2. Tecuapetla F,
    3. Paixão V,
    4. Costa RM
    (2018) Dopamine neuron activity before action initiation gates and invigorates future movements. Nature 554:244–248. doi:10.1038/nature25457
    OpenUrlCrossRefPubMed
  13. ↵
    1. Date Y,
    2. Mondal MS,
    3. Matsukura S,
    4. Nakazato M
    (2000) Distribution of orexin-A and orexin-B (hypocretins) in the rat spinal cord. Neurosci Lett 288:87–90. doi:10.1016/s0304-3940(00)01195-2 pmid:10876067
    OpenUrlCrossRefPubMed
  14. ↵
    1. de Lecea L,
    2. Kilduff TS,
    3. Peyron C,
    4. Gao X,
    5. Foye PE,
    6. Danielson PE,
    7. Fukuhara C,
    8. Battenberg EL,
    9. Gautvik VT,
    10. Bartlett FS 2nd.,
    11. Frankel WN,
    12. van den Pol AN,
    13. Bloom FE,
    14. Gautvik KM,
    15. Sutcliffe JG
    (1998) The hypocretins: hypothalamus-specific peptides with neuroexcitatory activity. Proc Natl Acad Sci U S A 95:322–327. doi:10.1073/pnas.95.1.322 pmid:9419374
    OpenUrlAbstract/FREE Full Text
  15. ↵
    1. de Lecea L,
    2. Jones BE,
    3. Boutrel B,
    4. Borgland SL,
    5. Nishino S,
    6. Bubser M,
    7. DiLeone R
    (2006) Addiction and arousal: alternative roles of hypothalamic peptides. J Neurosci 26:10372–10375. doi:10.1523/JNEUROSCI.3118-06.2006 pmid:17035520
    OpenUrlAbstract/FREE Full Text
  16. ↵
    1. Feng H,
    2. Wen SY,
    3. Qiao QC,
    4. Pang YJ,
    5. Wang SY,
    6. Li HY,
    7. Cai J,
    8. Zhang KX,
    9. Chen J,
    10. Hu ZA,
    11. Luo FL,
    12. Wang GZ,
    13. Yang N,
    14. Zhang J
    (2020) Orexin signaling modulates synchronized excitation in the sublaterodorsal tegmental nucleus to stabilize REM sleep. Nat Commun 11:3661. doi:10.1038/s41467-020-17401-3 pmid:32694504
    OpenUrlCrossRefPubMed
  17. ↵
    1. Franklin DW,
    2. Wolpert DM
    (2011) Computational mechanisms of sensorimotor control. Neuron 72:425–442. doi:10.1016/j.neuron.2011.10.006 pmid:22078503
    OpenUrlCrossRefPubMed
  18. ↵
    1. Gandolfo F,
    2. Li C,
    3. Benda BJ,
    4. Schioppa CP,
    5. Bizzi E
    (2000) Cortical correlates of learning in monkeys adapting to a new dynamical environment. Proc Natl Acad Sci U S A 97:2259–2263. doi:10.1073/pnas.040567097 pmid:10681435
    OpenUrlAbstract/FREE Full Text
  19. ↵
    1. Garau C,
    2. Blomeley C,
    3. Burdakov D
    (2020) Orexin neurons and inhibitory Agrp–>orexin circuits guide spatial exploration in mice. J Physiol 598:4371–4383. doi:10.1113/JP280158 pmid:32667686
    OpenUrlCrossRefPubMed
  20. ↵
    1. González JA,
    2. Iordanidou P,
    3. Strom M,
    4. Adamantidis A,
    5. Burdakov D
    (2016a) Awake dynamics and brain-wide direct inputs of hypothalamic MCH and orexin networks. Nat Commun 7:11395. doi:10.1038/ncomms11395 pmid:27102565
    OpenUrlCrossRefPubMed
  21. ↵
    1. González JA,
    2. Jensen LT,
    3. Iordanidou P,
    4. Strom M,
    5. Fugger L,
    6. Burdakov D
    (2016b) Inhibitory interplay between orexin neurons and eating. Curr Biol 26:2486–2491. doi:10.1016/j.cub.2016.07.013 pmid:27546579
    OpenUrlCrossRefPubMed
  22. ↵
    1. Hara J,
    2. Beuckmann CT,
    3. Nambu T,
    4. Willie JT,
    5. Chemelli RM,
    6. Sinton CM,
    7. Sugiyama F,
    8. Yagami KI,
    9. Goto K,
    10. Yanagisawa M,
    11. Sakurai T
    (2001) Genetic ablation of orexin neurons in mice results in narcolepsy, hypophagia, and obesity. Neuron 30:345–354. doi:10.1016/S0896-6273(01)00293-8 pmid:11394998
    OpenUrlCrossRefPubMed
  23. ↵
    1. Harris GC,
    2. Aston-Jones G
    (2006) Arousal and reward: a dichotomy in orexin function. Trends Neurosci 29:571–577. doi:10.1016/j.tins.2006.08.002 pmid:16904760
    OpenUrlCrossRefPubMed
  24. ↵
    1. Harris GC,
    2. Wimmer M,
    3. Aston-Jones G
    (2005) A role for lateral hypothalamic orexin neurons in reward seeking. Nature 437:556–559. doi:10.1038/nature04071 pmid:16100511
    OpenUrlCrossRefPubMed
  25. ↵
    1. Hu B,
    2. Yang N,
    3. Qiao QC,
    4. Hu ZA,
    5. Zhang J
    (2015) Roles of the orexin system in central motor control. Neurosci Biobehav Rev 49:43–54. doi:10.1016/j.neubiorev.2014.12.005 pmid:25511388
    OpenUrlCrossRefPubMed
  26. ↵
    1. Inagaki HK,
    2. Chen S,
    3. Daie K,
    4. Finklestein A,
    5. Fontolan L,
    6. Romani S,
    7. Svoboda K
    (2022) Neural algorithms and circuits for motor planning. Annu Rev Neurosci. Available at https://doi.org/10.1146/annurev-neuro-092021-121730.
  27. ↵
    1. Izawa J,
    2. Shadmehr R
    (2011) Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput Biol 7:e1002012. doi:10.1371/journal.pcbi.1002012 pmid:21423711
    OpenUrlCrossRefPubMed
  28. ↵
    1. Jin J,
    2. Chen Q,
    3. Qiao Q,
    4. Yang L,
    5. Xiong J,
    6. Xia J,
    7. Hu Z,
    8. Chen F
    (2016) Orexin neurons in the lateral hypothalamus project to the medial prefrontal cortex with a rostro-caudal gradient. Neurosci Lett 621:9–14. doi:10.1016/j.neulet.2016.04.002 pmid:27048712
    OpenUrlCrossRefPubMed
  29. ↵
    1. Kalman RE
    (1960) A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering 82:35–45.
    OpenUrlCrossRef
  30. ↵
    1. Kanzler CM,
    2. Rinderknecht MD,
    3. Schwarz A,
    4. Lamers I,
    5. Gagnon C,
    6. Held JPO,
    7. Feys P,
    8. Luft AR,
    9. Gassert R,
    10. Lambercy O
    (2020) A data-driven framework for selecting and validating digital health metrics: use-case in neurological sensorimotor impairments. NPJ Digit Med 3:80. doi:10.1038/s41746-020-0286-7 pmid:32529042
    OpenUrlCrossRefPubMed
  31. ↵
    1. Karnani MM,
    2. Schöne C,
    3. Bracey EF,
    4. González JA,
    5. Viskaitis P,
    6. Li HT,
    7. Adamantidis A,
    8. Burdakov D
    (2020) Role of spontaneous and sensory orexin network dynamics in rapid locomotion initiation. Prog Neurobiol 187:101771. doi:10.1016/j.pneurobio.2020.101771 pmid:32058043
    OpenUrlCrossRefPubMed
  32. ↵
    1. Kawato M
    (1999) Internal models for motor control and trajectory planning. Curr Opin Neurobiol 9:718–727. doi:10.1016/s0959-4388(99)00028-8 pmid:10607637
    OpenUrlCrossRefPubMed
  33. ↵
    1. Kim CK,
    2. Yang SJ,
    3. Pichamoorthy N,
    4. Young NP,
    5. Kauvar I,
    6. Jennings JH,
    7. Lerner TN,
    8. Berndt A,
    9. Lee SY,
    10. Ramakrishnan C,
    11. Davidson TJ,
    12. Inoue M,
    13. Bito H,
    14. Deisseroth K
    (2016) Simultaneous fast measurement of circuit dynamics at multiple sites across the mammalian brain. Nat Methods 13:325–328. doi:10.1038/nmeth.3770 pmid:26878381
    OpenUrlCrossRefPubMed
  34. ↵
    1. Kiyashchenko LI,
    2. Mileykovskiy BY,
    3. Lai YY,
    4. Siegel JM
    (2001) Increased and decreased muscle tone with orexin (hypocretin) microinjections in the locus coeruleus and pontine inhibitory area. J Neurophysiol 85:2008–2016. doi:10.1152/jn.2001.85.5.2008 pmid:11353017
    OpenUrlCrossRefPubMed
  35. ↵
    1. Korotkova TM,
    2. Eriksson KS,
    3. Haas HL,
    4. Brown RE
    (2002) Selective excitation of GABAergic neurons in the substantia nigra of the rat by orexin/hypocretin in vitro. Regul Pept 104:83–89. doi:10.1016/s0167-0115(01)00323-8 pmid:11830281
    OpenUrlCrossRefPubMed
  36. ↵
    1. Kosse C,
    2. Burdakov D
    (2019) Natural hypothalamic circuit dynamics underlying object memorization. Nat Commun 10:2505. doi:10.1038/s41467-019-10484-7 pmid:31175285
    OpenUrlCrossRefPubMed
  37. ↵
    1. Krakauer JW
    (2006) Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr Opin Neurol 19:84–90.
    OpenUrlCrossRefPubMed
  38. ↵
    1. Lambercy O,
    2. Schubring-Giese M,
    3. Vigaru B,
    4. Gassert R,
    5. Luft AR,
    6. Hosp JA
    (2015) Sub-processes of motor learning revealed by a robotic manipulandum for rodents. Behav Brain Res 278:569–576. doi:10.1016/j.bbr.2014.10.047 pmid:25446755
    OpenUrlCrossRefPubMed
  39. ↵
    1. Lee MG,
    2. Hassani OK,
    3. Jones BE
    (2005) Discharge of identified orexin/hypocretin neurons across the sleep-waking cycle. J Neurosci 25:6716–6720. doi:10.1523/JNEUROSCI.1887-05.2005 pmid:16014733
    OpenUrlAbstract/FREE Full Text
  40. ↵
    1. Li CS,
    2. Padoa-Schioppa C,
    3. Bizzi E
    (2001) Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30:593–607. doi:10.1016/S0896-6273(01)00301-4
    OpenUrlCrossRefPubMed
  41. ↵
    1. Mahler SV,
    2. Moorman DE,
    3. Smith RJ,
    4. James MH,
    5. Aston-Jones G
    (2014) Motivational activation: a unifying hypothesis of orexin/hypocretin function. Nat Neurosci 17:1298–1303. doi:10.1038/nn.3810 pmid:25254979
    OpenUrlCrossRefPubMed
  42. ↵
    1. Mathis MW,
    2. Mathis A,
    3. Uchida N
    (2017) Somatosensory cortex plays an essential role in forelimb motor adaptation in mice. Neuron 93:1493–1503.e6. doi:10.1016/j.neuron.2017.02.049 pmid:28334611
    OpenUrlCrossRefPubMed
  43. ↵
    1. Mazzoni P,
    2. Krakauer JW
    (2006) An implicit plan overrides an explicit strategy during visuomotor adaptation. J Neurosci 26:3642–3645. doi:10.1523/JNEUROSCI.5317-05.2006 pmid:16597717
    OpenUrlAbstract/FREE Full Text
  44. ↵
    1. Mileykovskiy BY,
    2. Kiyashchenko LI,
    3. Siegel JM
    (2002) Muscle tone facilitation and inhibition after orexin-a (hypocretin-1) microinjections into the medial medulla. J Neurophysiol 87:2480–2489. doi:10.1152/jn.2002.87.5.2480 pmid:11976385
    OpenUrlCrossRefPubMed
  45. ↵
    1. Mileykovskiy BY,
    2. Kiyashchenko LI,
    3. Siegel JM
    (2005) Behavioral correlates of activity in identified hypocretin/orexin neurons. Neuron 46:787–798. doi:10.1016/j.neuron.2005.04.035 pmid:15924864
    OpenUrlCrossRefPubMed
  46. ↵
    1. Mochizuki T,
    2. Crocker A,
    3. McCormack S,
    4. Yanagisawa M,
    5. Sakurai T,
    6. Scammell TE
    (2004) Behavioral state instability in orexin knock-out mice. J Neurosci 24:6291–6300. doi:10.1523/JNEUROSCI.0586-04.2004 pmid:15254084
    OpenUrlAbstract/FREE Full Text
  47. ↵
    1. Nambu T,
    2. Sakurai T,
    3. Mizukami K,
    4. Hosoya Y,
    5. Yanagisawa M,
    6. Goto K
    (1999) Distribution of orexin neurons in the adult rat brain. Brain Res 827:243–260. doi:10.1016/s0006-8993(99)01336-0 pmid:10320718
    OpenUrlCrossRefPubMed
  48. ↵
    1. Nishino S,
    2. Ripley B,
    3. Overeem S,
    4. Lammers GJ,
    5. Mignot E
    (2000) Hypocretin (orexin) deficiency in human narcolepsy. Lancet 355:39–40. doi:10.1016/S0140-6736(99)05582-8 pmid:10615891
    OpenUrlCrossRefPubMed
  49. ↵
    1. Oh Y,
    2. Schweighofer N
    (2019) Minimizing precision-weighted sensory prediction errors via memory formation and switching in motor adaptation. J Neurosci 39:9237–9250. doi:10.1523/JNEUROSCI.3250-18.2019 pmid:31582527
    OpenUrlAbstract/FREE Full Text
  50. ↵
    1. Perich MG,
    2. Gallego JA,
    3. Miller LE
    (2018) A neural population mechanism for rapid learning. Neuron 100:964–976.e7. doi:10.1016/j.neuron.2018.09.030 pmid:30344047
    OpenUrlCrossRefPubMed
  51. ↵
    1. Peyron C,
    2. Tighe DK,
    3. van den Pol AN,
    4. de Lecea L,
    5. Heller HC,
    6. Sutcliffe JG,
    7. Kilduff TS
    (1998) Neurons containing hypocretin (orexin) project to multiple neuronal systems. J Neurosci 18:9996–10015. pmid:9822755
    OpenUrlAbstract/FREE Full Text
  52. ↵
    1. Peyron C, et al
    . (2000) A mutation in a case of early onset narcolepsy and a generalized absence of hypocretin peptides in human narcoleptic brains. Nat Med 6:991–997. doi:10.1038/79690 pmid:10973318
    OpenUrlCrossRefPubMed
  53. ↵
    1. Sakurai T
    (2014) The role of orexin in motivated behaviours. Nat Rev Neurosci 15:719–731. doi:10.1038/nrn3837 pmid:25301357
    OpenUrlCrossRefPubMed
  54. ↵
    1. Sakurai T, et al
    . (1998) Orexins and orexin receptors: a family of hypothalamic neuropeptides and G protein-coupled receptors that regulate feeding behavior. Cell 92:573–585. doi:10.1016/S0092-8674(00)80949-6
    OpenUrlCrossRefPubMed
  55. ↵
    1. Schaefer SY,
    2. Shelly IL,
    3. Thoroughman KA
    (2012) Beside the point: motor adaptation without feedback-based error correction in task-irrelevant conditions. J Neurophysiol 107:1247–1256. doi:10.1152/jn.00273.2011 pmid:22157120
    OpenUrlCrossRefPubMed
  56. ↵
    1. Scheidt RA,
    2. Stoeckmann T
    (2007) Reach adaptation and final position control amid environmental uncertainty after stroke. J Neurophysiol 97:2824–2836. doi:10.1152/jn.00870.2006 pmid:17267755
    OpenUrlCrossRefPubMed
  57. ↵
    1. Schwarz A,
    2. Kanzler CM,
    3. Lambercy O,
    4. Luft AR,
    5. Veerbeek JM
    (2019) Systematic review on kinematic assessments of upper limb movements after stroke. Stroke 50:718–727. doi:10.1161/STROKEAHA.118.023531 pmid:30776997
    OpenUrlCrossRefPubMed
  58. ↵
    1. Shadmehr R,
    2. Mussa-Ivaldi FA
    (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224. pmid:8182467
    OpenUrlAbstract/FREE Full Text
  59. ↵
    1. Shadmehr R,
    2. Smith MA,
    3. Krakauer JW
    (2010) Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 33:89–108. doi:10.1146/annurev-neuro-060909-153135 pmid:20367317
    OpenUrlCrossRefPubMed
  60. ↵
    1. Smith MA,
    2. Shadmehr R
    (2005) Intact ability to learn internal models of arm dynamics in Huntington's disease but not cerebellar degeneration. J Neurophysiol 93:2809–2821. doi:10.1152/jn.00943.2004 pmid:15625094
    OpenUrlCrossRefPubMed
  61. ↵
    1. Takahashi CD,
    2. Reinkensmeyer DJ
    (2003) Hemiparetic stroke impairs anticipatory control of arm movement. Exp Brain Res 149:131–140. doi:10.1007/s00221-002-1340-1 pmid:12610680
    OpenUrlCrossRefPubMed
  62. ↵
    1. Thannickal TC,
    2. Moore RY,
    3. Nienhuis R,
    4. Ramanathan L,
    5. Gulyani S,
    6. Aldrich M,
    7. Cornford M,
    8. Siegel JM
    (2000) Reduced number of hypocretin neurons in human narcolepsy. Neuron 27:469–474. doi:10.1016/s0896-6273(00)00058-1 pmid:11055430
    OpenUrlCrossRefPubMed
  63. ↵
    1. Thoroughman KA,
    2. Shadmehr R
    (2000) Learning of action through adaptive combination of motor primitives. Nature 407:742–747. doi:10.1038/35037588 pmid:11048720
    OpenUrlCrossRefPubMed
  64. ↵
    1. Todorov E,
    2. Jordan MI
    (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5:1226–1235. doi:10.1038/nn963 pmid:12404008
    OpenUrlCrossRefPubMed
  65. ↵
    1. van den Pol AN
    (1999) Hypothalamic hypocretin (orexin): robust innervation of the spinal cord. J Neurosci 19:3171–3182. doi:10.1523/JNEUROSCI.19-08-03171.1999
    OpenUrlAbstract/FREE Full Text
  66. ↵
    1. Vidal PP,
    2. Degallaix L,
    3. Josset P,
    4. Gasc JP,
    5. Cullen KE
    (2004) Postural and locomotor control in normal and vestibularly deficient mice. J Physiol 559:625–638. doi:10.1113/jphysiol.2004.063883 pmid:15243133
    OpenUrlCrossRefPubMed
  67. ↵
    1. Vigaru B,
    2. Lambercy O,
    3. Graber L,
    4. Fluit R,
    5. Wespe P,
    6. Schubring-Giese M,
    7. Luft A,
    8. Gassert R
    (2011) A small-scale robotic manipulandum for motor training in stroke rats. IEEE Int Conf Rehabil Robot 2011:5975349. doi:10.1109/ICORR.2011.5975349 pmid:22275553
    OpenUrlCrossRefPubMed
  68. ↵
    1. Vigaru BC,
    2. Lambercy O,
    3. Schubring-Giese M,
    4. Hosp JA,
    5. Schneider M,
    6. Osei-Atiemo C,
    7. Luft A,
    8. Gassert R
    (2013) A robotic platform to assess, guide and perturb rat forelimb movements. IEEE Trans Neural Syst Rehabil Eng 21:796–805. doi:10.1109/TNSRE.2013.2240014 pmid:23335672
    OpenUrlCrossRefPubMed
  69. ↵
    1. Wayner MJ
    (1970) Motor control functions of the lateral hypothalamus and adjunctive behavior. Physiol Behav 5:1319–1325. doi:10.1016/0031-9384(70)90048-x pmid:5003005
    OpenUrlCrossRefPubMed
  70. ↵
    1. Wolpert DM,
    2. Miall RC
    (1996) Forward models for physiological motor control. Neural Netw 9:1265–1279. doi:10.1016/s0893-6080(96)00035-4 pmid:12662535
    OpenUrlCrossRefPubMed
  71. ↵
    1. Wolpert DM,
    2. Kawato M
    (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11:1317–1329. doi:10.1016/s0893-6080(98)00066-5 pmid:12662752
    OpenUrlCrossRefPubMed
  72. ↵
    1. Wolpert DM,
    2. Flanagan JR
    (2016) Computations underlying sensorimotor learning. Curr Opin Neurobiol 37:7–11. doi:10.1016/j.conb.2015.12.003 pmid:26719992
    OpenUrlCrossRefPubMed
  73. ↵
    1. Wolpert DM,
    2. Miall RC,
    3. Kawato M
    (1998) Internal models in the cerebellum. Trends Cogn Sci 2:338–347. doi:10.1016/s1364-6613(98)01221-2 pmid:21227230
    OpenUrlCrossRefPubMed
  74. ↵
    1. Wolpert DM,
    2. Diedrichsen J,
    3. Flanagan JR
    (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12:739–751. doi:10.1038/nrn3112 pmid:22033537
    OpenUrlCrossRefPubMed
  75. ↵
    1. Yamuy J,
    2. Fung SJ,
    3. Xi M,
    4. Chase MH
    (2004) Hypocretinergic control of spinal cord motoneurons. J Neurosci 24:5336–5345. doi:10.1523/JNEUROSCI.4812-03.2004 pmid:15190106
    OpenUrlAbstract/FREE Full Text
  76. ↵
    1. Zhang J,
    2. Li B,
    3. Yu L,
    4. He YC,
    5. Li HZ,
    6. Zhu JN,
    7. Wang JJ
    (2011) A role for orexin in central vestibular motor control. Neuron 69:793–804. doi:10.1016/j.neuron.2011.01.026 pmid:21338887
    OpenUrlCrossRefPubMed
Back to top

In this issue

The Journal of Neuroscience: 42 (32)
Journal of Neuroscience
Vol. 42, Issue 32
10 Aug 2022
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
  • Masthead (PDF)
Email

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Hypothalamic Control of Forelimb Motor Adaptation
(Your Name) has forwarded a page to you from Journal of Neuroscience
(Your Name) thought you would be interested in this article in Journal of Neuroscience.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
Hypothalamic Control of Forelimb Motor Adaptation
Dane Donegan, Christoph M. Kanzler, Julia Büscher, Paulius Viskaitis, Ed F. Bracey, Olivier Lambercy, Denis Burdakov
Journal of Neuroscience 10 August 2022, 42 (32) 6243-6257; DOI: 10.1523/JNEUROSCI.0705-22.2022

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Request Permissions
Share
Hypothalamic Control of Forelimb Motor Adaptation
Dane Donegan, Christoph M. Kanzler, Julia Büscher, Paulius Viskaitis, Ed F. Bracey, Olivier Lambercy, Denis Burdakov
Journal of Neuroscience 10 August 2022, 42 (32) 6243-6257; DOI: 10.1523/JNEUROSCI.0705-22.2022
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Introduction
    • Materials and Methods
    • Results
    • Discussion
    • Footnotes
    • References
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Keywords

  • hypothalamus
  • motor adaptation
  • motor learning
  • movement
  • orexin/hypocretin
  • upper limb

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

Research Articles

  • Regional Excitatory–Inhibitory Balance Relates to Self-Reference Effect on Recollection via the Precuneus/Posterior Cingulate Cortex–Medial Prefrontal Cortex Connectivity
  • Attention Alters Population Spatial Frequency Tuning
  • Sequence Diversity and Encoded Enzymatic Differences of Monocistronic L1 ORF2 mRNA Variants in the Aged Normal and Alzheimer's Disease Brain
Show more Research Articles

Systems/Circuits

  • Hippocampal Sharp-Wave Ripples Decrease during Physical Actions Including Consummatory Behavior in Immobile Rodents
  • Developmental Olfactory Dysfunction and Abnormal Odor Memory in Immune-Challenged Disc1+/− Mice
  • Functional Roles of Gastrin-Releasing Peptide-Producing Neurons in the Suprachiasmatic Nucleus: Insights into Photic Entrainment and Circadian Regulation
Show more Systems/Circuits
  • Home
  • Alerts
  • Follow SFN on BlueSky
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Issue Archive
  • Collections

Information

  • For Authors
  • For Advertisers
  • For the Media
  • For Subscribers

About

  • About the Journal
  • Editorial Board
  • Privacy Notice
  • Contact
  • Accessibility
(JNeurosci logo)
(SfN logo)

Copyright © 2025 by the Society for Neuroscience.
JNeurosci Online ISSN: 1529-2401

The ideas and opinions expressed in JNeurosci do not necessarily reflect those of SfN or the JNeurosci Editorial Board. Publication of an advertisement or other product mention in JNeurosci should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in JNeurosci.