Special Issue: Probabilistic models of cognition
Bayesian decision theory in sensorimotor control

https://doi.org/10.1016/j.tics.2006.05.003Get rights and content

Action selection is a fundamental decision process for us, and depends on the state of both our body and the environment. Because signals in our sensory and motor systems are corrupted by variability or noise, the nervous system needs to estimate these states. To select an optimal action these state estimates need to be combined with knowledge of the potential costs or rewards of different action outcomes. We review recent studies that have investigated the mechanisms used by the nervous system to solve such estimation and decision problems, which show that human behaviour is close to that predicted by Bayesian Decision Theory. This theory defines optimal behaviour in a world characterized by uncertainty, and provides a coherent way of describing sensorimotor processes.

Introduction

The central nervous system (CNS) constantly sends motor commands to our muscles. Determining the appropriate motor command is fundamentally a decision process. At each point in time we must select one particular motor command from the set of possible motor commands. Two components jointly define the decision problem: knowledge of the state of the world (including our own body) and knowledge of our objectives.

The sensory inputs of humans are plagued by noise 1, 2 which means that we will always have uncertainty about our hand's true location (Figure 1a). This uncertainty depends on the modality of the sensory input: when we use proprioception to locate our hand we may have more uncertainty about its position compared to when we can see it. Moreover, our muscles produce noisy outputs 3, 4 and when we quickly move to a target location (shown as a red × in Figure 1a) our final hand position will typically deviate from the intended target. Even if our sensors were perfect they would only tell us about the part of the world that we can currently sense. This uncertainty places the problem of estimating the state of the world and the control of our motor system within a statistical framework. Bayesian statistics 5, 6, 7, 8 provides a systematic way of solving problems in the presence of uncertainty (see the online article by Griffiths and Yuille associated with this issue: Supplementary material online). The approach of Bayesian statistics is characterized by assigning probabilities to any degree of belief about the state of the world (see also Conceptual Foundations editorial by Chater, Tenenbaum and Yuille).

Bayesian statistics defines how new information should be combined with prior beliefs and how information from several modalities should be integrated. Bayesian decision theory 9, 10, 11 defines how our beliefs should be combined with our objectives to make optimal decisions. Understanding the way the CNS deals with uncertainty might be key to understanding its normal mode of operation.

The cost of each movement (such as energy consumed) must be weighed against the potential rewards that can be obtained by moving. In the framework of decision theory a utility function should quantify the overall desirability of the outcome of a movement decision. We should choose a movement so that as to maximize utility. Several recent papers have addressed what functions people optimize with their movements. Understanding what human subjects try to optimize is a necessary step towards a rational theory of movement selection.

The selection of a movement can be described as the rational choice of the movement that maximizes utility according to decision theory (see Box 1). This approach thus asks why people behave the way they do. An increasing number of laboratories have addressed this question within this framework. Here we review recent studies that find human movement performance to be close to the predictions obtained from optimally combining probability estimates with movement costs and rewards. The approach has the potential to embed human behaviour into a coherent mathematical framework.

Section snippets

Estimation using Bayes rule

We need to estimate the variables that are relevant for our choice of movement. For example, when playing tennis we may want to estimate where the ball will bounce. Because vision does not provide perfect information about the ball's velocity there is uncertainty as to the bounce location. However, if we know about the noise in our sensory system then the sensory input can be used to compute the likelihood – the probability of getting the particular sensory inputs for different possible bounce

Bayesian integration in motor control

Bayes rule makes it clear that to perform optimally we must combine prior knowledge of the statistic of the task with the likelihood obtained from the sensory input. In a recent experiment [12], it was tested whether people use such a strategy. Instead of the bounce location of a tennis ball subjects had to estimate the position of a cursor relative to their hand (Figure 1b). Subjects could use two sources of information: The distribution of displacements over the course of many trials (prior),

Costs and rewards

To put movement into a rational framework it is necessary to define a function that measures how good or bad the outcome of a particular movement is. This function, often termed cost may for example be related to the energy consumed during a movement. In general people should prefer less demanding movements – movements that put less strain on the muscles or movements that can be executed using less energy. We are thus faced with the problem of selecting among the infinite set of possible

Models of optimal control: using online feedback

Understanding task statistics, the noise on our sensors and actuators and the utility function allows us to predict optimal behaviour. So far we have discussed these processes applied to discrete decisions chosen from a small number of possible decisions. However, in general we produce a continuous trajectory of movement in response to a contiguous stream of sensory input. The system will thus constantly use feedback to update its movements (Figure 3).

Future directions

The approach of formalizing human decision making as being based on partial uncertainty and utility functions formalizes the problems that are solved by the CNS. There is converging evidence from various communities that Bayesian approaches can serve as a coherent description of human decision making.

The optimal statistical approach to sensorimotor control raises many important questions (see Box 4). However, many of our movements are in the context of complicated tasks such as social

Acknowledgements

We like to thank the German Science Foundation Heisenberg Program for support (KK) as well as the Wellcome grant and the HFSP for financial support.

References (68)

  • R.T. Cox

    Probability, frequency and reasonable expectation

    Am. J. Phys.

    (1946)
  • D.A. Freedman

    Some issues in the foundation of statistics

    Found. Sci.

    (1995)
  • D.J.C. MacKay

    Information Theory, Inference, and Learning Algorithms

    (2003)
  • E.T. Jaynes

    Bayesian Methods: General Background

    (1986)
  • A. Yuille et al.

    Bayesian decision theory and psychophysics

  • D.H. Brainard et al.

    Bayesian color constancy

    J. Opt. Soc. Am. A Opt. Image Sci. Vis.

    (1997)
  • S. Russell et al.

    Principles of metareasoning

    Artif. Intell.

    (1991)
  • K.P. Körding et al.

    Bayesian integration in sensorimotor learning

    Nature

    (2004)
  • K. Singh et al.

    A motor learning strategy reflects neural circuitry for limb control

    Nat. Neurosci.

    (2003)
  • K.P. Körding

    Bayesian Integration in force estimation

    J. Neurophysiol.

    (2004)
  • M. Miyazaki

    Testing Bayesian models of human coincidence timing

    J. Neurophysiol.

    (2005)
  • Knill, D. and Richards, W., eds (1996) Perception as Bayesian Inference, Cambridge University...
  • Yuille, A. and Kersten, D. (2006) Vision as Bayesian inference: analysis by synthesis? Trends Cogn. Sci....
  • W.S. Geisler et al.

    Illusions, perception and Bayes

    Nat. Neurosci.

    (2002)
  • D. Kersten

    Object perception as Bayesian inference

    Annu. Rev. Psychol.

    (2004)
  • W.J. Adams

    Experience can change the ‘light-from-above’ prior

    Nat. Neurosci.

    (2004)
  • D. Brewster

    On the optical illusion of the conversion of cameos into intaglios and of intaglios into cameos, with an account of other analogous phenomena

    Edinburgh J. Sci.

    (1826)
  • E.H. Adelson

    Perceptual organization and the judgment of brightness

    Science

    (1993)
  • M.S. Langer et al.

    A prior for global convexity in local shape-from-shading

    Perception

    (2001)
  • Stocker, A. and Simoncelli, E. Noise characteristics and prior expectations in human visual speed perception. Nat....
  • C.Q. Howe et al.

    Range image statistics can explain the anomalous perception of length

    Proc. Natl. Acad. Sci. U. S. A.

    (2002)
  • M.O. Ernst et al.

    Humans integrate visual and haptic information in a statistically optimal fashion

    Nature

    (2002)
  • J.M. Hillis

    Slant from texture and disparity cues: optimal cue combination

    J. Vis.

    (2004)
  • R.J. van Beers

    Integration of proprioceptive and visual position-information: An experimentally supported model

    J. Neurophysiol.

    (1999)
  • Cited by (595)

    View all citing articles on Scopus
    View full text