Elsevier

Current Opinion in Neurobiology

Volume 9, Issue 6, 1 December 1999, Pages 718-727
Current Opinion in Neurobiology

Review
Internal models for motor control and trajectory planning

https://doi.org/10.1016/S0959-4388(99)00028-8Get rights and content

Abstract

A number of internal model concepts are now widespread in neuroscience and cognitive science. These concepts are supported by behavioral, neurophysiological, and imaging data; furthermore, these models have had their structures and functions revealed by such data. In particular, a specific theory on inverse dynamics model learning is directly supported by unit recordings from cerebellar Purkinje cells. Multiple paired forward inverse models describing how diverse objects and environments can be controlled and learned separately have recently been proposed. The ‘minimum variance model’ is another major recent advance in the computational theory of motor control. This model integrates two furiously disputed approaches on trajectory planning, strongly suggesting that both kinematic and dynamic internal models are utilized in movement planning and control.

Introduction

Internal models are neural mechanisms that can mimic the input/output characteristics, or their inverses, of the motor apparatus. Forward internal models can predict sensory consequences from efference copies of issued motor commands. Inverse internal models, on the other hand, can calculate necessary feedforward motor commands from desired trajectory information.

Fast and coordinated arm movements cannot be executed solely under feedback control, since biological feedback loops are slow and have small gains (Figure 1a). Thus, the internal model hypothesis (Figure 1b) proposes that the brain needs to acquire an inverse dynamics model of the object to be controlled through motor learning, after which motor control can be executed in a pure feedforward manner. In theory, a forward model of the motor apparatus embedded in an internal feedback loop can approximate an inverse model.

The internal model concept has its origin in control theory and robotics, but Ito [1] proposed almost 30 years ago that the cerebellum contains forward models of the limbs and other brain regions. More recently, internal models have attracted a broader range of specialists (e.g. neural network modelers, connectionists and neurophysiologists 2, 3, 4), and have been studied increasingly seriously as one of the major theories of motor control and learning in neuroscience and cognitive science. Accordingly, in the past few years, much more direct and convincing data than ever before have been accumulated. Such data can already show the existence, structures, learning, functions and anatomy of internal models. Of particular importance, we have seen significant theoretical advances in elucidation of the generalization, multiplicity and switching of internal models, and their possible use in trajectory planning.

In this review, I will discuss data supporting the existence of internal models. It has been shown that the behavioral paradigms in use are diverse and include adaptation to force fields, posture control, grip-force–load-force coupling, oculomanual coordination, and the vestibular system. An explanation will be given on points of controversy between the equilibrium point control hypothesis and the internal model hypothesis, and some clues towards their resolution will be presented. Recent neurophysiological and imaging studies that suggest that the cerebellar cortex is a major site of internal models will also be discussed. Furthermore, structures of internal models will be explored by ‘generalization’ experiments; modularity and multiplicity are suggested by the data obtained. Finally, two major approaches to trajectory planning will be reviewed and a new theory will be introduced to integrate them.

Section snippets

Existence of internal models

When subjects first undertake point-to-point arm reaching movements under force fields which effectively change dynamic characteristics of the arm, their hand trajectories are distorted compared with the normal, roughly straight paths; also, the end point errors are large, especially in the direction of the applied force. The force fields generated predetermined forces which depended on the state space point (position, velocity), and were produced by a robot manipulandum [5] or by a rotating

Internal models in the cerebellum

It is conceivable that internal models are located in all brain regions having synaptic plasticity, provided that they receive and send out relevant information for their input and output. We have good reason to believe that at least some internal models are acquired and stored in the cerebellar cortex. For example, there is a new computational theory [27••] that allocates supervised learning, reinforcement learning, and unsupervised learning to the cerebellum, the basal ganglia and the

Structures of internal models

The functional structures of internal models can be probed by the so-called ‘generalization experiment’ 49, 50, 51, 52, 53•, 54, 55•. Humans or animals are trained for a specific set of movement trajectories with an altered kinematic or dynamic perturbation. After sufficient learning, the organism’s ability to cope with different trajectories or movements in a different part of the workspace in which the motor apparatus can move is examined. If ‘generalization’ is considered perfect, new

Trajectory formation

Computational theories on how arm reaching trajectories are planned have been a central issue in motor control since it was shown that they involve roughly straight hand paths and bell-shaped velocity profiles [64]. Many of the different computational models can be classified into two types: kinematic models such as the minimum jerk model [65], and dynamic models such as the minimum torque-change model [66]. Because these two classes of models enable the experimental testing of qualitatively

Conclusions

The concepts concerning internal models have now been well supported by behavioral studies in the field of sensory motor control. Neurophysiological studies have just begun but should be fruitful in the next five years. Theoretically, the concept should be extended from pure sensory motor control to cognitive domains as we have a flood of data suggesting cerebellar involvement in higher cognitive functions such as language 78, 79, 80••. This is especially important because the cerebellar

Acknowledgements

I wish to thank my collaborators in ATR and in the ERATO Dynamic Brain Project as well as those outside, especially, Daniel Wolpert, Chris Miall and Randy Flanagan. This research was partially supported by HFSP grants, special coordination funds of promotion of science.

References and recommended reading

Papers of particular interest, published within the annual period of review, have been highlighted as:

  • • of special interest

  • •• of outstanding interest

References (80)

  • M. Ito

    Neurophysiological aspects of the cerebellar motor control system

    Int J Neurol

    (1970)
  • M. Kawato et al.

    A hierarchical neural network model for the control and learning of voluntary movements

    Biol Cybern

    (1987)
  • M.I. Jordan et al.

    Forward models: supervised learning with a distal teacher

    Cogn Sci

    (1992)
  • R.C. Miall et al.

    Is the cerebellum a Smith Predictor?

    J Motor Behav

    (1993)
  • R. Shadmehr et al.

    Adaptive representation of dynamics during learning of a motor task

    J Neurosci

    (1994)
  • J.R. Lackner et al.

    Rapid adaptation to Coriolis force perturbations of arm trajectory

    J Neurophysiol

    (1994)
  • J.R. Lackner et al.

    Gravitoinertial force background level affects adaptation to Coriolis force perturbations of reaching movements

    J Neurophysiol

    (1998)
  • R.S. Johansson et al.

    Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects

    Exp Brain Res

    (1984)
  • J.R. Flanagan et al.

    The role of internal models in motion planning and control: evidence from grip force adjustments during movements of hand-held loads

    J Neurosci

    (1997)
  • T. Tamada et al.

    Activation of the cerebellum in grip force and load force coordination: an fMRI study

    Neuroimage

    (1999)
  • K. Scarchilli et al.

    The oculomanual coordination control center takes into account the mechanical properties of the arm

    Exp Brain Res

    (1999)
  • L. Snyder

    This way up: illusions and internal models in the vestibular system

    Nat Neurosci

    (1999)
  • B.J. Hess et al.

    Oculomotor control of primary eye position discriminates between translation and tilt

    J Neurophysiol

    (1999)
  • D.M. Merfeld et al.

    Humans use internal models to estimate gravity and linear acceleration

    Nature

    (1999)
  • A.G. Feldman

    Functional tuning of nervous system with control of movement or maintenance of a steady posture. 2. Controllable parameters of the muscles

    Biophysics

    (1966)
  • E. Bizzi et al.

    Posture control and trajectory formation during arm movement

    J Neurosci

    (1984)
  • T. Flash

    The control of hand equilibrium trajectories in multi-joint arm movements

    Biol Cybern

    (1987)
  • P.L. Gribble et al.

    Are complex control signals required for human arm movement?

    J Neurophysiol

    (1998)
  • M.L. Latash et al.

    Reconstruction of shifting elbow joint compliant characteristics during fast and slow movements

    Neuroscience

    (1991)
  • M. Katayama et al.

    Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models

    Biol Cybern

    (1993)
  • H. Gomi et al.

    Equilibrium-point control hypothesis examined by measured arm-stiffness during multi-joint movement

    Science

    (1996)
  • P.L. Gribble et al.

    Compensation for interaction torques during single and multi-joint limb movement

    J Neurophysiol

    (1999)
  • P.G. Morasso et al.

    Can muscle stiffness alone stabilize upright standing?

    J Neurophysiol

    (1999)
  • T.E. Milner et al.

    Compensation for mechanically unstable loading in voluntary wrist movement

    Exp Brain Res

    (1993)
  • T.D. Sanger

    Neural network learning control of robot manipulators using gradually increasing task difficulty

    IEEE Trans Robotics Automat

    (1994)
  • Katayama M, Inoue S, Kawato M: A strategy of motor learning using adjustable parameters for arm movement. Proceedings...
  • K. Doya

    What are the computations of cerebellum, basal ganglia, and cerebral cortex?

    Neural Networks

    (1999)
  • D. Marr

    A theory of cerebellar cortex

    J Physiol

    (1969)
  • J.S. Albus

    A theory of cerebellar function

    Math Biosci

    (1971)
  • M. Kawato et al.

    The cerebellum and VOR/OKR learning models

    Trends Neurosci

    (1992)
  • W.T. Thach

    A role for the cerebellum in learning movement coordination

    Neurobiol Learn Mem

    (1998)
  • K. Kawano et al.

    Visual inputs to cerebellar ventral paraflocculus during ocular following responses

    Prog Brain Res

    (1996)
  • Y. Kobayashi et al.

    Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. II. Complex spikes

    J Neurophysiol

    (1998)
  • M. Shidara et al.

    Inverse-dynamics model eye movement control by Purkinje cells in the cerebellum

    Nature

    (1993)
  • H. Gomi et al.

    Temporal firing patterns of Purkinje cells in the cerebellar ventral paraflocculus during ocular following responses in monkeys. I. Simple spikes

    J Neurophysiol

    (1998)
  • T. Kitama et al.

    Motor dynamics encoding in cat cerebellar flocculus middle zone during optokinetic eye movements

    J Neurophysiol

    (1999)
  • A. Takemura et al.

    Analysis of neuronal activities during ocular following responses in alert monkeys

    Tech Rep IEICE

    (1999)
  • K. Yamamoto et al.

    A mathematical model that reproduces vertical ocular following responses from visual stimuli

    Neurosci Res

    (1997)
  • K. Yamamoto et al.

    A computational simulation on the adaptation of vertical ocular following responses

    Tech Rep IEICE

    (1998)
  • T.J. Ebner

    A role for the cerebellum in the control of limb movement velocity

    Curr Opin Neurobiol

    (1998)
  • Cited by (1958)

    • Sensory restoration for improved motor control of prostheses

      2023, Current Opinion in Biomedical Engineering
    View all citing articles on Scopus
    View full text