Trends in Cognitive Sciences
Special Issue: Probabilistic models of cognitionBayesian decision theory in sensorimotor control
Introduction
The central nervous system (CNS) constantly sends motor commands to our muscles. Determining the appropriate motor command is fundamentally a decision process. At each point in time we must select one particular motor command from the set of possible motor commands. Two components jointly define the decision problem: knowledge of the state of the world (including our own body) and knowledge of our objectives.
The sensory inputs of humans are plagued by noise 1, 2 which means that we will always have uncertainty about our hand's true location (Figure 1a). This uncertainty depends on the modality of the sensory input: when we use proprioception to locate our hand we may have more uncertainty about its position compared to when we can see it. Moreover, our muscles produce noisy outputs 3, 4 and when we quickly move to a target location (shown as a red × in Figure 1a) our final hand position will typically deviate from the intended target. Even if our sensors were perfect they would only tell us about the part of the world that we can currently sense. This uncertainty places the problem of estimating the state of the world and the control of our motor system within a statistical framework. Bayesian statistics 5, 6, 7, 8 provides a systematic way of solving problems in the presence of uncertainty (see the online article by Griffiths and Yuille associated with this issue: Supplementary material online). The approach of Bayesian statistics is characterized by assigning probabilities to any degree of belief about the state of the world (see also Conceptual Foundations editorial by Chater, Tenenbaum and Yuille).
Bayesian statistics defines how new information should be combined with prior beliefs and how information from several modalities should be integrated. Bayesian decision theory 9, 10, 11 defines how our beliefs should be combined with our objectives to make optimal decisions. Understanding the way the CNS deals with uncertainty might be key to understanding its normal mode of operation.
The cost of each movement (such as energy consumed) must be weighed against the potential rewards that can be obtained by moving. In the framework of decision theory a utility function should quantify the overall desirability of the outcome of a movement decision. We should choose a movement so that as to maximize utility. Several recent papers have addressed what functions people optimize with their movements. Understanding what human subjects try to optimize is a necessary step towards a rational theory of movement selection.
The selection of a movement can be described as the rational choice of the movement that maximizes utility according to decision theory (see Box 1). This approach thus asks why people behave the way they do. An increasing number of laboratories have addressed this question within this framework. Here we review recent studies that find human movement performance to be close to the predictions obtained from optimally combining probability estimates with movement costs and rewards. The approach has the potential to embed human behaviour into a coherent mathematical framework.
Section snippets
Estimation using Bayes rule
We need to estimate the variables that are relevant for our choice of movement. For example, when playing tennis we may want to estimate where the ball will bounce. Because vision does not provide perfect information about the ball's velocity there is uncertainty as to the bounce location. However, if we know about the noise in our sensory system then the sensory input can be used to compute the likelihood – the probability of getting the particular sensory inputs for different possible bounce
Bayesian integration in motor control
Bayes rule makes it clear that to perform optimally we must combine prior knowledge of the statistic of the task with the likelihood obtained from the sensory input. In a recent experiment [12], it was tested whether people use such a strategy. Instead of the bounce location of a tennis ball subjects had to estimate the position of a cursor relative to their hand (Figure 1b). Subjects could use two sources of information: The distribution of displacements over the course of many trials (prior),
Costs and rewards
To put movement into a rational framework it is necessary to define a function that measures how good or bad the outcome of a particular movement is. This function, often termed cost may for example be related to the energy consumed during a movement. In general people should prefer less demanding movements – movements that put less strain on the muscles or movements that can be executed using less energy. We are thus faced with the problem of selecting among the infinite set of possible
Models of optimal control: using online feedback
Understanding task statistics, the noise on our sensors and actuators and the utility function allows us to predict optimal behaviour. So far we have discussed these processes applied to discrete decisions chosen from a small number of possible decisions. However, in general we produce a continuous trajectory of movement in response to a contiguous stream of sensory input. The system will thus constantly use feedback to update its movements (Figure 3).
Future directions
The approach of formalizing human decision making as being based on partial uncertainty and utility functions formalizes the problems that are solved by the CNS. There is converging evidence from various communities that Bayesian approaches can serve as a coherent description of human decision making.
The optimal statistical approach to sensorimotor control raises many important questions (see Box 4). However, many of our movements are in the context of complicated tasks such as social
Acknowledgements
We like to thank the German Science Foundation Heisenberg Program for support (KK) as well as the Wellcome grant and the HFSP for financial support.
References (68)
Statistical analysis of motor unit firing patterns in a human skeletal muscle
Biophys. J.
(1969)- et al.
Bayesian models of object perception
Curr. Opin. Neurobiol.
(2003) - et al.
The ventriloquist effect results from near-optimal bimodal integration
Curr. Biol.
(2004) Optimal integration of texture and motion cues to depth
Vision Res.
(1999)- et al.
Do humans optimally integrate stereo and texture information for judgments of surface slant?
Vision Res.
(2003) - et al.
Human altruism: economic, neural, and evolutionary perspectives
Curr. Opin. Neurobiol.
(2004) - et al.
Uncertainty, neuromodulation, and attention
Neuron
(2005) Human contrast discrimination and the threshold of cortical neurons
J. Opt. Soc. Am. A
(1987)- et al.
The computation of position sense from spindles in mono- and multiarticular muscles
J. Neurosci.
(1994) Relationship of firing intervals of human motor units to the trajectory of post-spike after-hyperpolarization and synaptic noise
J. Physiol.
(1996)
Probability, frequency and reasonable expectation
Am. J. Phys.
Some issues in the foundation of statistics
Found. Sci.
Information Theory, Inference, and Learning Algorithms
Bayesian Methods: General Background
Bayesian decision theory and psychophysics
Bayesian color constancy
J. Opt. Soc. Am. A Opt. Image Sci. Vis.
Principles of metareasoning
Artif. Intell.
Bayesian integration in sensorimotor learning
Nature
A motor learning strategy reflects neural circuitry for limb control
Nat. Neurosci.
Bayesian Integration in force estimation
J. Neurophysiol.
Testing Bayesian models of human coincidence timing
J. Neurophysiol.
Illusions, perception and Bayes
Nat. Neurosci.
Object perception as Bayesian inference
Annu. Rev. Psychol.
Experience can change the ‘light-from-above’ prior
Nat. Neurosci.
On the optical illusion of the conversion of cameos into intaglios and of intaglios into cameos, with an account of other analogous phenomena
Edinburgh J. Sci.
Perceptual organization and the judgment of brightness
Science
A prior for global convexity in local shape-from-shading
Perception
Range image statistics can explain the anomalous perception of length
Proc. Natl. Acad. Sci. U. S. A.
Humans integrate visual and haptic information in a statistically optimal fashion
Nature
Slant from texture and disparity cues: optimal cue combination
J. Vis.
Integration of proprioceptive and visual position-information: An experimentally supported model
J. Neurophysiol.
Cited by (595)
The integration of action-oriented multisensory information from target and limb within the movement planning and execution
2023, Neuroscience and Biobehavioral ReviewsA novel human learning optimization algorithm with Bayesian inference learning
2023, Knowledge-Based SystemsAnalogous cognitive strategies for tactile learning in the rodent and human brain
2023, Progress in Neurobiology