 |
|||||
|
|||||
|
|||||
|
|||||
|
|||||
The Journal of Neuroscience, April 29, 2009, 29(17):5476-5482; doi:10.1523/JNEUROSCI.0116-09.2009
Previous Article | Next Article
Behavioral/Systems/Cognitive
Sensory Weighting of Force and Position Feedback in Human Motor Control Tasks
Winfred Mugge,1 * Jasper Schuurmans,1 * Alfred C. Schouten,1,2 andFrans C. T. van der Helm1,2 1Department of BioMechanical Engineering, Delft University of Technology, 2628 CD Delft, The Netherlands, and 2Laboratory of Biomechanical Engineering, Institute for Biomedical Technology, University of Twente, 7500 AE Enschede, The Netherlands
Abstract
Top
Abstract
Introduction
In daily life humans integrate force and position feedback from mechanoreceptors, proprioception, and vision. With handling relatively soft, elastic objects, force and position are related and can be integrated to improve the accuracy of an estimate of either one. Sensory weighting between different sensory systems (e.g., vision and proprioception) has been extensively studied. This study investigated whether similar weighting can be found within the proprioceptive sensory system, more specifically between the modalities force and position. We hypothesized that sensory weighting is governed by object stiffness: position feedback is weighted heavier on soft objects (large deflections), while force feedback is weighted heavier on stiff objects (small deflections). Subjects were instructed to blindly reproduce either position or force while holding a one degree of freedom haptic manipulator that simulated a linear spring with one of four predetermined stiffnesses. In catch trials the spring was covertly replaced by a nonlinear spring. The difference in force (F) and position (
Introduction
Humans are able to effectively handle a wide range of objects with different mechanical properties, using feedback from mechanoreceptors, proprioception, and vision. The mechanical properties of an object largely determine what sensory modalities are effective. When handling stiff objects like a cup, deflections are negligible so position holds no information on the applied force. However, when handling soft objects like a sponge, deflections are large and allow position feedback to contribute to the estimated force and vice versa. Object stiffness, the physical relationship between position and force, allows translation from one modality into the other. Both force feedback (from tactile sensors in the skin and Golgi tendon organs) and position feedback (from muscle spindles and vision) are noisy and have limited accuracy (Körding and Wolpert, 2004, 2006
Many studies have shown sensory integration and weighting in the CNS (Yuille and Bülthoff, 1996
Sensory weighting between different modalities requires knowledge of the relationship between them, as described by an internal model of the physics of the world around us. The internal model enables us to make estimates of the required motor commands based on previous experiences (Shadmehr and Mussa-Ivaldi, 1994
Generally, previous research focused on sensory weighting between different sensory systems. Especially sensory weighting between vision and proprioception has been extensively studied. Sensory weighting of depth cues within the vision system has been demonstrated (Knill, 2007
Materials and Methods
Approach. To assess weighting between force and position feedback, an experimental approach is needed that differentiates between the two. Imagine a subject repeatedly interacting with an environment with certain stiffness (a spring), trying to either achieve a desired force (force task FT) (Fig. 1A) or position (position task PT) (Fig. 1B). To improve accuracy, the subject might use the acquired internal model to integrate force and position feedback. One might reason that weighting also depends on the type of task, weighting force feedback heavier during a FT and position feedback during a PT. Since sensory weighting is done within the CNS, it is impossible to discriminate between the use of force and position feedback by just measuring the exerted forces and positions. However, if the stiffness characteristic of the spring is covertly altered, the newly acquired forces and positions can be used to reveal the subject's weighting strategy. This disparity in the spring environment allows the separate weights of force and position feedback to be determined, like Ernst and Banks (2002)
View larger version (16K):
[in this window]
[in a new window]
Figure 1. Subjects were trained to blindly reproduce either a target force (A) or a target position (B) against a linear spring. During catch trials the characteristic of the spring is covertly altered to determine how the subject weights force and position feedback during task execution (C, D).
Experimental setup. Ten healthy volunteers age 18–22 years, all right-handed men, participated in the study. A subject held the handle of a one degree of freedom hydraulic haptic manipulator with his right hand (Fig. 2) (Schouten et al., 2006(2) a nonlinear spring that was designed to exert a force, which was a constant force difference
f higher at the position where the linear spring would have exerted the target force Ftarget:
A force sensor on the handle measured the force exerted by the subject. The spring model was simulated on a real-time processing board (dSPACE DS1005). The output of the spring model was the set point of the hydraulic controller which controlled the position of the manipulator. To exclude undesired visual feedback, the arm and manipulator handle were blocked from view.
View larger version (19K):
[in this window]
[in a new window]
Figure 2. Experimental setup. The subject controlled a haptic manipulator that simulated a spring. Depending on task, either force or position was displayed together with the target. During blind trials and catch trials the visual feedback on force or position was disabled. The subject operated a foot switch to indicate he believed to have acquired the target force or position.
The experimental protocol consisted of eight blocks of trials: two tasks (FT and PT) times four spring stiffnesses (50, 100, 230, 500 N/m). The protocol was fully automated with randomized trials and on-screen instructions to the subject. The target force Ftarget during FT was 10 N, and the target position during PT was such that the target position coincided with a spring force of 10 N (see Table 1). The order of the blocks was randomized, but all PT and FT remained grouped together to attain best task perception. With every new block the subject performed 15 training trials with onscreen visual feedback which enabled him to learn the task and familiarize himself with the spring stiffness. The subject was instructed to operate the foot switch when an indicator bar, representing force/position (depending on task), was aligned with the target indicators. The foot switch triggered a measurement of position and force (0.3 s at a sample rate of 500 Hz). After each measurement, the subject was instructed on-screen to return to the starting position i.e., the zero length of the spring. The next trial was automatically initiated when the subject had crossed the starting position which was indicated by the appearance of the instruction of the next trial. After training, the subject performed a series of trials composed of three different types:
View this table:
[in this window]
[in a new window]
Table 1. Target force and position for the four stiffness conditions
- (1) A reference trial was a trial where the subject was instructed to deliver the force/position that was indicated onscreen by a moving indicator bar and target indicators, exactly alike the training trials.
- (2) A blind trial was similar to a reference trial, with the exception that the onscreen force/position feedback was switched off. The subject was instructed to reproduce the trained force/position blindly and to operate the foot switch when he thought he attained the trained force/position (depending on task instruction).
- (3) A catch trial was a blind trial where the spring was covertly substituted with the nonlinear spring (Fig. 1C,D). The spring model was always substituted at the zero-length position, to prevent the subject from noticing any change in force.
Blind trials were alternated with reference trials to prevent drift from the trained force/position. On average one in three blind trials was randomly substituted with a catch trial, effectively providing one catch trial every six trials. A total of 12 catch trials were recorded per block.
Data analysis. The force and position during blind trials and catch trials were averaged over the 0.3 s measurement and the repetitions. Two repeated-measures ANOVA tests (RM-ANOVA) were performed. The first tested for effects of task, stiffness, and trial type on the measured force levels in the three trial types. The second tested for effects of task and stiffness within the difference in the force (
Maximum likelihood estimation model. We derived a maximum likelihood estimation (MLE) model to determine the optimal weights of force and position feedback during FT and PT, given the uncertainties of the force and position estimates of the subjects. The model was used to predict the differences between blind trials and catch trials.
Assuming the subject familiarizes himself with spring stiffness ki during training, he can estimate force F (Eq. 3) or position X (Eq. 4) from a weighted combination of a force estimate
and position estimate
.
MLE theory states that the optimal weight factors for force (wf) and position (wx) are inversely proportional to the variance in their respective measurements (
and
). Scaling the weights such that wf + wx = 1 results in the optimal unbiased weights, as displayed in Equations 5 and 6 for FT and Equations 7 and 8 for PT.
It is easily shown that multiplication of the numerator and denominator of Equations 5 and 6 by k
yields Equations 7 and 8. This implicates that theoretically in this experiment task instruction does not affect the weighting factors. In addition to the direct weighting of sensory information, humans are also known to weight prior information in their final estimates of sensory information (Körding and Wolpert, 2004
–8 to implement the effect of prior knowledge. We demonstrate that with sufficient training the effects of priors cancel out in the current experimental setup.
It was assumed that subjects obtained an internal model of the spring and adjusted their weights accordingly during the training sessions. The force and position during catch trials was predicted with the model by assuming that the weight factors remained the same as in the blind trials and were not influenced by the nonlinear spring of the catch trials. The exerted force in a catch trial of a FT could thus be determined by solving the set of equations in Equation 9 for F. The first equation of the set expresses the force estimated by the subject from weighted force and position feedback. The second equation is the spring characteristic of the nonlinear spring (each solution must per definition lie on the spring characteristic). The position in a catch trial of a PT could be determined likewise by solving the set of equations in Equation 10 for X.
The model has only two parameters:
Results
Experiments
Figure 3 shows the exerted forces and corresponding positions for the three trial types in one of the four stiffness conditions (50 N/m) for FT and PT. Reference trials are characterized by high accuracy and small SDs. During blind trials the subjects exerted a higher force than the target force, indicating the subjects underestimated the exerted force. During catch trials the force was slightly higher than during the blind trials; however this corresponded to a smaller position deviation due to the nonlinearity of the spring (Fig. 3, bottom panels). The results for PT show similar effects as for FT. Subjects underestimated the position, effectively moving the handle beyond its target during blind trials.
View larger version (26K):
[in this window]
[in a new window]
Figure 3. Bar diagrams of the results for the three trial types for FT and PT in a single stiffness condition (ki = 50 N/m). Error bars indicate one SD over the subjects.
Figure 4 shows that during reference trials the subjects closely approximated the target force of 10 N in all stiffness conditions. A slightly higher force is found during blind and catch trials (p < 0.001). In addition to trial type, stiffness had a significant effect on exerted force (p < 0.01). In agreement with the model prediction no significant effect of task instruction was found (p = 0.108). Of the interaction effects only the stiffness x trial type was significant (p < 0.001).
View larger version (10K):
[in this window]
[in a new window]
Figure 4. The results for the three trial types. The top windows show the measured force against spring stiffness and the bottom windows show the measured position. Left, The results for FT. Right, The results for PT. Error bars indicate one SD over the subjects.
Post hoc tests revealed significant differences between stiffness 50 N/m and all other stiffness conditions (p < 0.05 with stiffness 100 N/m, p < 0.01 with stiffness 230 N/m, and p < 0.001 with stiffness 500 N/m) and between stiffness 100 N/m and stiffness 500 N/m (p < 0.05), and between the three trial types (all p < 0.001).The graphs of the measured positions show a monotonously decreasing relation due to the linear spring relating force and position.
Model comparison
Figure 5 illustrates the predictions of the maximum likelihood model. From top to bottom it shows the force differencef/
View larger version (9K):
[in this window]
[in a new window]
Figure 5. Model predictions. Symbolic characteristics of the MLE model predictions against stiffness for the case that the difference between the linear and nonlinear spring is sufficiently small. A, Predicted force difference
In the experimental setup, target force ftarget is constant, so the corresponding position xtarget equals ftarget/k. Because of this factor 1/k, the graph of
To compare the results to the model predictions, the difference between the mean force and position during catch trials and blind trials was determined. Figure 6 shows the measured difference in force during FT (left) and the difference in position during PT (right) between the blind and the catch trials together with the model predictions. The force error during FT decreases with stiffness, while the position error shows a distinct trough at a stiffness of
100 N/m during PT. Also the repeated measures ANOVA on the force difference between the blind and the catch trials revealed a significant effect of stiffness (p < 0.01). Again the effect of task did not reach significance (p = 0.066).
View larger version (9K):
[in this window]
[in a new window]
Figure 6. Force and position difference between the catch and the blind trials for FT and PT. Left, Force difference in FT. Right, Position difference in PT. Experimental data in solid lines and model predictions in dashed lines. Error bars indicate one SD over the subjects.
Post hoc tests revealed significant differences between stiffness 50 N/m and all other stiffness conditions (p < 0.05 with stiffness 100 N/m, p < 0.05 with stiffness 230 N/m, and p < 0.01 with stiffness 500 N/m).The dashed lines in Figure 6 indicate the difference between catch and blind trials as predicted by the maximum likelihood model. The predictions were based on the variances determined in the experiment with an infinitely stiff spring (only force feedback available) and a spring with zero stiffness (only position feedback available). SD for pure force replication (
The model predicted a monotonously decreasing
Discussion
Experiment data
In agreement with the hypothesis a significant effect of stiffness on the force and position difference between the catch and the blind trials was found. Apparently, the weighting of sensory input from mechanoreceptors is variable and depends on the stiffness of the object humans are interacting with. The experiment depends on the assumption that humans do not adapt their sensory weighting in the catch trials. Care was taken to design a nonlinear spring with characteristics much alike the linear spring but which did result in sufficient force difference at the target position to overcome the variance in the blind trials. Subjects could not distinguish catch trials from regular blind trials. In fact, when asked afterward only one of the subjects reported he suspected a change in the behavior of the manipulator, but could not indicate what it was.Subjects consistently underestimated the force and position during the blind trials, effectively exerting too high forces with respect to the reference trials (Figs. 3, 4). This did not affect our results concerning the weighting because of the comparison between blind trials and catch trials, since both are performed blindly. A possible explanation may be that overestimation is a natural byproduct of neural processing as was suggested in force replication tasks before by Shergill et al. (2003)
Maximum likelihood model
Derivation of the maximum likelihood model revealed that task instruction has no effect on the optimal weighting of force and position. This means that interpreting either task instruction as "just do the same as during training" would result in equal task performance. Indeed no significant difference was found between PT and FT, indicating that although subjects were trying to perform either a FT or PT, the outcome was similar. Instructing a subject to reproduce either force or position can be interpreted as focusing the subject's attention to one of the two modalities. Recent literature has shown that attentional manipulation of a specific sensory modality does not influence the relative weighting of that modality (Helbig and Ernst, 2008The model explained the course of the data rather well, although there was a remarkable difference with the subjects' overestimation of force during a FT (Fig. 6). In terms of sensory weighting, this indicates that subjects are either biased toward using position feedback at the expense of force feedback, or that the weights are nonideal and sum up to a value other than one. Nevertheless, both model and data showed that increasing spring stiffness leads to a weight shift toward force feedback, as was hypothesized.
Conclusion
This study has shown stiffness-dependent weighting of sensory input from mechanoreceptors for force and position. The maximum likelihood model predicted a monotonously decreasing force difference between the catch and the blind trials with increasing stiffness and a distinct trough in the position difference. Both were experimentally found indicating that a maximum likelihood model not only describes sensory weighting between sensory systems as was found in previous literature, but also within one sensory system, in this case proprioception.Additionally, in reproduction tasks, subjects consistently underestimated the force/position. In the blind trials, the subjects exerted a force greater than the target force in all stiffness conditions.
Finally, it was shown that in replication tasks with known stiffness conditions, there is no difference between position and force tasks with respect to sensory weighting. The model predicted the exact same weighting factors and in accordance no significant task effect was found in the experiment.
This study successfully demonstrates sensory weighting inside the proprioceptive sensory system, i.e., between force and position feedback. The experiment shows sensory weighting that is close to optimal according to the MLE model.
Appendix A—Application of Priors in the MLE Model
In the main text force F and position X are estimated from a weighted combination of a force estimate
with weights for force (wf), position (wx), force prior (wfp), and position prior (wxp), and the expected values for force (µfp) and position (µxp). Scaling the weights such that wf + wx + wfp + wxp = 1 results in the optimal weights, as displayed in Equations 13
with
, and
the variances of respectively force, position, force prior, and position prior. Note that application of priors does not introduce an effect of task instruction since in accordance to the main text multiplication of the numerator and denominator of Equations 13
With the addition of priors, Equations 9 and 10 in the main text become:
When Equations A21 and A22 are solved with substitution of Equations A13–A20, the resulting force F and position X are both functions of ki, ftarget,
However, when the priors are assumed to be ideal, the limit case after sufficient training (µfp
ftarget and µxp
This means that the formulae in the main text apply even for the addition of priors, under the assumption that the priors are unbiased, i.e., the means of the prior distributions coincide with the target force and position during training trials.
Footnotes
Received Jan. 9, 2009; revised March 16, 2009; accepted April 2, 2009.*W.M. and J.S. contributed equally to this work.
This research was funded by Dutch Government Grant BSIK03016. We kindly thank our students Tim Kiefte, Eric-Jan den Ouden, Jeroen van Rooijen, and Wouter Wolfslag for carrying out the experiments.
Correspondence should be addressed to Winfred Mugge, Department of BioMechanical Engineering, Delft University of Technology, Mekelweg 2, 2628 CD Delft, The Netherlands. Email: w.mugge{at}tudelft.nl
Copyright © 2009 Society for Neuroscience 0270-6474/09/295476-07$15.00/0
References
Abbink DA, Van der Helm FCT, Boer ER (2004) Admittance measurements of the foot during ‘maintain position’ and ‘relax’ tasks on a gas pedal. In: 2004 IEEE international conference on systems, man and cybernetics. Vol 3:2519–2524. The Hague, Netherlands: IEEE.
Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257–262.[CrossRef][Web of Science][Medline]
Bremner AJ, Holmes NP, Spence C (2008) Infants lost in (peripersonal) space? Trends Cogn Sci 12:298–305.[CrossRef][Web of Science][Medline]
Doemges F, Rack PMH (1992a) Changes in the stretch reflex of the human first dorsal interosseous muscle during different tasks. J Physiol 447:563–573.
[Abstract/Free Full Text] Doemges F, Rack PMH (1992b) Task-dependent changes in the response of human wrist joints to mechanical disturbance. J Physiol 447:575–585.
[Abstract/Free Full Text] Ernst MO, Banks MS (2002) Humans integrate visual and haptic integration in a statistically optimal fashion. Nature 415:429–433.[CrossRef][Medline]
Ernst MO, Bülthoff HH (2004) Merging the senses into a robust percept. Trends Cogn Sci 8:162–169.[CrossRef][Web of Science][Medline]
Helbig HB, Ernst MO (2008) Visual-haptic cue weighting is independent of modality-specific attention. J Vis 8:21.1–16.[Medline]
Kalman RE, Bucy RS (1961) New results in linear filtering and prediction. J Basic Eng (ASME) 83D:95–108.
Knill DC (2007) Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. J Vis 7:5.1–24.[Medline]
Körding KP, Wolpert DM (2004) Bayesian integration in sensorimotor learning. Nature 427:244–247.[CrossRef][Medline]
Körding KP, Wolpert DM (2006) Bayesian decision theory in sensorimotor control. Trends Cogn Sci 10:319–326.[CrossRef][Web of Science][Medline]
Körding KP, Ku SP, Wolpert DM (2004) Bayesian integration in force estimation. J Neurophysiol 92:3161–3165.
[Abstract/Free Full Text] Mahboobin A, Loughlin PJ, Redfern MS, Sparto PJ (2005) Sensory re-weighting in human postural control during moving-scene perturbations. Exp Brain Res 167:260–267.[CrossRef][Web of Science][Medline]
Ostry DJ, Feldman AG (2003) A critical evaluation of the force control hypothesis in motor control. Exp Brain Res 153:275–288.[CrossRef][Web of Science][Medline]
Peterka RJ (2002) Sensorimotor integration in human postural control. J Neurophysiol 88:1097–1118.
[Abstract/Free Full Text] Peterka RJ, Loughlin PJ (2004) Dynamic regulation of sensorimotor integration in human postural control. J Neurophysiol 91:410–423.
[Abstract/Free Full Text] Säfström D, Edin BB (2004) Task requirements influence sensory integration during grasping in humans. Learn Mem 11:356–363.
[Abstract/Free Full Text] Schouten AC, de Vlugt E, van Hilten JJ, van der Helm FC (2006) Design of a torque-controlled manipulator to analyse the admittance of the wrist joint. J Neurosci Methods 154:134–141.[CrossRef][Web of Science][Medline]
Shadmehr R, Krakauer JW (2008) A computational neuroanatomy for motor control. Exp Brain Res 185:359–381.[CrossRef][Web of Science][Medline]
Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14:3208–3224.[Abstract]
Shergill SS, Bays PM, Frith CD, Wolpert DM (2003) Two eyes for an eye: the neuroscience of force escalation. Science 301:187.
[Free Full Text] van der Kooij H, Jacobs R, Koopman B, van der Helm F (2001) An adaptive model of sensory integration in a dynamic environment applied to human stance control. Biol Cybern 84:103–115.[CrossRef][Web of Science][Medline]
Vuillerme N, Chenu O, Pinsault N, Fleury A, Demongeot J, Payan Y (2008) Can a plantar pressure-based tongue-placed electrotactile biofeedback improve postural control under altered vestibular and neck proprioceptive conditions? Neuroscience 155:291–296.[CrossRef][Web of Science][Medline]
Wolpert DM, Kawato M (1998) Multiple paired forward and inverse models for motor control. Neural Netw 11:1317–1329.[CrossRef][Web of Science][Medline]
Wolpert DM, Miall RC (1996) Forward models for physiological motor control. Neural Netw 9:1265–1279.[CrossRef][Web of Science][Medline]
Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Science 269:1880–1892.
[Abstract/Free Full Text] Yuille A, Bülthoff HH (1996) Bayesian decision theory and psychophysics. In: Perception as Bayesian inference (Knill D, Richards W, eds), pp 123–161. Cambridge, UK: Cambridge UP.
Zupan LH, Merfeld DM, Darlot C (2002) Using sensory weighting to model the influence of canal, otolith and visual cues on spatial orientation and eye movements. Biol Cybern 86:209–230.[CrossRef][Web of Science][Medline]
This Article Abstract 
Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager 
Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Mugge, W. Articles by van der Helm, F. C. T. Search for Related Content PubMed PubMed Citation Articles by Mugge, W. Articles by van der Helm, F. C. T.
|
|
|
|
 |
|