Abstract
In daily life humans integrate force and position feedback from mechanoreceptors, proprioception, and vision. With handling relatively soft, elastic objects, force and position are related and can be integrated to improve the accuracy of an estimate of either one. Sensory weighting between different sensory systems (e.g., vision and proprioception) has been extensively studied. This study investigated whether similar weighting can be found within the proprioceptive sensory system, more specifically between the modalities force and position. We hypothesized that sensory weighting is governed by object stiffness: position feedback is weighted heavier on soft objects (large deflections), while force feedback is weighted heavier on stiff objects (small deflections). Subjects were instructed to blindly reproduce either position or force while holding a one degree of freedom haptic manipulator that simulated a linear spring with one of four predetermined stiffnesses. In catch trials the spring was covertly replaced by a nonlinear spring. The difference in force (ΔF) and position (ΔX) between the regular and the catch trials revealed the sensory weighting between force and position feedback. A maximum likelihood estimation model predicted that: (1) task instruction did not affect the outcome measures, and (2) force feedback is weighted heavier with increasing object stiffness as was hypothesized. Both effects were found experimentally, and the subjects' sensory weighting closely resembled the optimal model predictions. To conclude, this study successfully demonstrated sensory weighting within the proprioceptive system.
Introduction
Humans are able to effectively handle a wide range of objects with different mechanical properties, using feedback from mechanoreceptors, proprioception, and vision. The mechanical properties of an object largely determine what sensory modalities are effective. When handling stiff objects like a cup, deflections are negligible so position holds no information on the applied force. However, when handling soft objects like a sponge, deflections are large and allow position feedback to contribute to the estimated force and vice versa. Object stiffness, the physical relationship between position and force, allows translation from one modality into the other. Both force feedback (from tactile sensors in the skin and Golgi tendon organs) and position feedback (from muscle spindles and vision) are noisy and have limited accuracy (Körding and Wolpert, 2004, 2006; Körding et al., 2004). When stiffness is known, combining the sensory feedback of these two modalities (sensory integration) provides increased accuracy of the estimate of either force or position.
Many studies have shown sensory integration and weighting in the CNS (Yuille and Bülthoff, 1996; Ernst and Bülthoff, 2004). The ventriloquist effect, which is the sensation of hearing the puppet speak while it is the puppeteer who makes the sound, originates from sensory integration of vision and auditory cues. Seeing the lips of the puppet move instead of the ventriloquist's biases the sound localization toward the puppet (Alais and Burr, 2004). Other examples of sensory weighting are the sense of location of body parts in the peripersonal space (Bremner et al., 2008) and integration of visual and haptic feedback during grasping tasks (Säfström and Edin, 2004). During balance control humans weight sensory input from the vestibular system, mechanoreceptors, and vision (van der Kooij et al., 2001; Zupan et al., 2002; Mahboobin et al., 2005). Changes in environmental properties bring about changes in the relative weights between information channels: sensory reweighting (Peterka, 2002; Peterka and Loughlin, 2004). Also new information channels can be integrated in an existing control strategy: during stance even tactile feedback on the tongue could be used to replace force feedback from the soles of the feet (Vuillerme et al., 2008).
Sensory weighting between different modalities requires knowledge of the relationship between them, as described by an internal model of the physics of the world around us. The internal model enables us to make estimates of the required motor commands based on previous experiences (Shadmehr and MussaIvaldi, 1994; Wolpert and Miall, 1996; Wolpert and Kawato, 1998; Ostry and Feldman, 2003; Shadmehr and Krakauer, 2008). To effectively weight sensory feedback channels, an estimate of their accuracy is required. Bayesian inference (Körding and Wolpert, 2006) has been suggested to underlie sensory weighting. A well established example of Bayesian inference from control engineering is Kalman filtering (Kalman and Bucy, 1961; Wolpert et al., 1995).
Generally, previous research focused on sensory weighting between different sensory systems. Especially sensory weighting between vision and proprioception has been extensively studied. Sensory weighting of depth cues within the vision system has been demonstrated (Knill, 2007). Here, we wonder whether similar sensory weighting occurs between force and position within the proprioceptive system. The goal of this study was to determine how humans weight force and position feedback in an environment with known stiffness. We hypothesize that the weighting of these modalities depends largely on object stiffness. It is expected that position feedback is weighted heavier on soft objects (large deflections), while force feedback is weighted heavier on stiff objects (small deflections).
Materials and Methods
Approach.
To assess weighting between force and position feedback, an experimental approach is needed that differentiates between the two. Imagine a subject repeatedly interacting with an environment with certain stiffness (a spring), trying to either achieve a desired force (force task FT) (Fig. 1A) or position (position task PT) (Fig. 1B). To improve accuracy, the subject might use the acquired internal model to integrate force and position feedback. One might reason that weighting also depends on the type of task, weighting force feedback heavier during a FT and position feedback during a PT. Since sensory weighting is done within the CNS, it is impossible to discriminate between the use of force and position feedback by just measuring the exerted forces and positions. However, if the stiffness characteristic of the spring is covertly altered, the newly acquired forces and positions can be used to reveal the subject's weighting strategy. This disparity in the spring environment allows the separate weights of force and position feedback to be determined, like Ernst and Banks (2002) introduced disparities in a haptic environment to determine the weights of visual and haptic feedback. In one extreme, the subject could have used only force feedback during a FT. He would deliver the same force against the altered spring, but would end up at a slightly different position (Fig. 1C). In the other extreme, he could have performed the FT by reproducing the same position; the newly obtained force would then be different. If the subject would have weighted force and position feedback, the newly obtained force would be in between these two extremes. The degree in which the resulting force is shifted toward either extreme corresponds to the relative weighting between force and position feedback. The same argument holds for a PT (Fig. 1D). The dependency of weighting sensory feedback to task and stiffness is determined experimentally and with the use of a model.
Experimental setup.
Ten healthy volunteers age 18–22 years, all righthanded men, participated in the study. A subject held the handle of a one degree of freedom hydraulic haptic manipulator with his right hand (Fig. 2) (Schouten et al., 2006). During the experiment the manipulator simulated a spring and could switch between two spring models: (1) a linear spring with stiffness k_{i} (i = 1… 4): exerting a force F_{lin}, at position X, according to Equation 1: (2) a nonlinear spring that was designed to exert a force, which was a constant force difference δ_{f} higher at the position where the linear spring would have exerted the target force F_{target}: A force sensor on the handle measured the force exerted by the subject. The spring model was simulated on a realtime processing board (dSPACE DS1005). The output of the spring model was the set point of the hydraulic controller which controlled the position of the manipulator. To exclude undesired visual feedback, the arm and manipulator handle were blocked from view.
The experimental protocol consisted of eight blocks of trials: two tasks (FT and PT) times four spring stiffnesses (50, 100, 230, 500 N/m). The protocol was fully automated with randomized trials and onscreen instructions to the subject. The target force F_{target} during FT was 10 N, and the target position during PT was such that the target position coincided with a spring force of 10 N (see Table 1). The order of the blocks was randomized, but all PT and FT remained grouped together to attain best task perception. With every new block the subject performed 15 training trials with onscreen visual feedback which enabled him to learn the task and familiarize himself with the spring stiffness. The subject was instructed to operate the foot switch when an indicator bar, representing force/position (depending on task), was aligned with the target indicators. The foot switch triggered a measurement of position and force (0.3 s at a sample rate of 500 Hz). After each measurement, the subject was instructed onscreen to return to the starting position i.e., the zero length of the spring. The next trial was automatically initiated when the subject had crossed the starting position which was indicated by the appearance of the instruction of the next trial. After training, the subject performed a series of trials composed of three different types:

(1) A reference trial was a trial where the subject was instructed to deliver the force/position that was indicated onscreen by a moving indicator bar and target indicators, exactly alike the training trials.

(2) A blind trial was similar to a reference trial, with the exception that the onscreen force/position feedback was switched off. The subject was instructed to reproduce the trained force/position blindly and to operate the foot switch when he thought he attained the trained force/position (depending on task instruction).

(3) A catch trial was a blind trial where the spring was covertly substituted with the nonlinear spring (Fig. 1C,D). The spring model was always substituted at the zerolength position, to prevent the subject from noticing any change in force.
Blind trials were alternated with reference trials to prevent drift from the trained force/position. On average one in three blind trials was randomly substituted with a catch trial, effectively providing one catch trial every six trials. A total of 12 catch trials were recorded per block.
Data analysis.
The force and position during blind trials and catch trials were averaged over the 0.3 s measurement and the repetitions. Two repeatedmeasures ANOVA tests (RMANOVA) were performed. The first tested for effects of task, stiffness, and trial type on the measured force levels in the three trial types. The second tested for effects of task and stiffness within the difference in the force (ΔF) between the blind trials and the catch trials. A significance level of 0.05 was maintained for all statistical tests.
Maximum likelihood estimation model.
We derived a maximum likelihood estimation (MLE) model to determine the optimal weights of force and position feedback during FT and PT, given the uncertainties of the force and position estimates of the subjects. The model was used to predict the differences between blind trials and catch trials.
Assuming the subject familiarizes himself with spring stiffness k_{i} during training, he can estimate force F (Eq. 3) or position X (Eq. 4) from a weighted combination of a force estimate f̂ and position estimate x̂. MLE theory states that the optimal weight factors for force (w_{f}) and position (w_{x}) are inversely proportional to the variance in their respective measurements (σ_{f}^{2} and σ_{x}^{2}). Scaling the weights such that w_{f} + w_{x} = 1 results in the optimal unbiased weights, as displayed in Equations 5 and 6 for FT and Equations 7 and 8 for PT. It is easily shown that multiplication of the numerator and denominator of Equations 5 and 6 by k_{i}^{2} yields Equations 7 and 8. This implicates that theoretically in this experiment task instruction does not affect the weighting factors. In addition to the direct weighting of sensory information, humans are also known to weight prior information in their final estimates of sensory information (Körding and Wolpert, 2004). In the current setup that would mean that prior knowledge obtained during the training phase (i.e., the variance and mean of applied forces and positions) would be weighted during both blind trials and catch trials. In Appendix A priors are added to Equations 5–8 to implement the effect of prior knowledge. We demonstrate that with sufficient training the effects of priors cancel out in the current experimental setup.
It was assumed that subjects obtained an internal model of the spring and adjusted their weights accordingly during the training sessions. The force and position during catch trials was predicted with the model by assuming that the weight factors remained the same as in the blind trials and were not influenced by the nonlinear spring of the catch trials. The exerted force in a catch trial of a FT could thus be determined by solving the set of equations in Equation 9 for F. The first equation of the set expresses the force estimated by the subject from weighted force and position feedback. The second equation is the spring characteristic of the nonlinear spring (each solution must per definition lie on the spring characteristic). The position in a catch trial of a PT could be determined likewise by solving the set of equations in Equation 10 for X. The model has only two parameters: σ_{f}^{2} and σ_{x}^{2}, which express the accuracy of a force measurement and position measurement of the subject. These parameters were estimated in two additional experiments with reference trials and blind trials, but without catch trials. The subjects performed the FT protocol on an infinitely stiff spring to determine σ_{f}^{2} and the PT protocol in zero stiffness to determine σ_{x}^{2}. These conditions ensured that only a single channel of information (force or position) could be used. The variance in the blind repetitions of the instructed force and position were used as an estimator for σ_{f}^{2} and σ_{x}^{2}.
Results
Experiments
Figure 3 shows the exerted forces and corresponding positions for the three trial types in one of the four stiffness conditions (50 N/m) for FT and PT. Reference trials are characterized by high accuracy and small SDs. During blind trials the subjects exerted a higher force than the target force, indicating the subjects underestimated the exerted force. During catch trials the force was slightly higher than during the blind trials; however this corresponded to a smaller position deviation due to the nonlinearity of the spring (Fig. 3, bottom panels). The results for PT show similar effects as for FT. Subjects underestimated the position, effectively moving the handle beyond its target during blind trials.
Figure 4 shows that during reference trials the subjects closely approximated the target force of 10 N in all stiffness conditions. A slightly higher force is found during blind and catch trials (p < 0.001). In addition to trial type, stiffness had a significant effect on exerted force (p < 0.01). In agreement with the model prediction no significant effect of task instruction was found (p = 0.108). Of the interaction effects only the stiffness × trial type was significant (p < 0.001).
Post hoc tests revealed significant differences between stiffness 50 N/m and all other stiffness conditions (p < 0.05 with stiffness 100 N/m, p < 0.01 with stiffness 230 N/m, and p < 0.001 with stiffness 500 N/m) and between stiffness 100 N/m and stiffness 500 N/m (p < 0.05), and between the three trial types (all p < 0.001).
The graphs of the measured positions show a monotonously decreasing relation due to the linear spring relating force and position.
Model comparison
Figure 5 illustrates the predictions of the maximum likelihood model. From top to bottom it shows the force difference ΔF and position difference ΔX between blind trials and catch trials, and the optimal weights w_{f} and w_{x} as a function of spring stiffness k. Figure 5C shows that force is weighted heavier on stiff objects and position is weighted heavier on soft objects. In the case that the difference in force between the linear spring (blind trials) and nonlinear spring (catch trials) is sufficiently small (δ_{f} ≪ f_{target}), the stiffness where force and position feedback are equally weighted approaches k = σ_{f}/σ_{x}.
ΔF (Fig. 5A) decreases monotonously with stiffness. For zero stiffness, only position feedback is used. As a result the position in a catch trial is the same as in a blind trial and ΔF = δ_{f}. For infinite stiffness only force feedback is used and the force on the nonlinear spring will match the force on the linear spring (ΔF = 0).
In the experimental setup, target force f_{target} is constant, so the corresponding position x_{target} equals f_{target}/k. Because of this factor 1/k, the graph of ΔX (Fig. 5B) has its distinct trough. In the two extremes ΔX is zero. For zero stiffness only position is weighted and thus the position in catch trials equals the position in blind trials. For infinite stiffness the displacement of the spring is zero regardless of force. ΔX has its minimum at the intersection of the two weight curves (k = σ_{f}/σ_{x}) and its value depends only on the ratio σ_{f}/σ_{x} and on f_{target} and δ_{f} used in the experiment.
To compare the results to the model predictions, the difference between the mean force and position during catch trials and blind trials was determined. Figure 6 shows the measured difference in force during FT (left) and the difference in position during PT (right) between the blind and the catch trials together with the model predictions. The force error during FT decreases with stiffness, while the position error shows a distinct trough at a stiffness of ∼100 N/m during PT. Also the repeated measures ANOVA on the force difference between the blind and the catch trials revealed a significant effect of stiffness (p < 0.01). Again the effect of task did not reach significance (p = 0.066).
Post hoc tests revealed significant differences between stiffness 50 N/m and all other stiffness conditions (p < 0.05 with stiffness 100 N/m, p < 0.05 with stiffness 230 N/m, and p < 0.01 with stiffness 500 N/m).
The dashed lines in Figure 6 indicate the difference between catch and blind trials as predicted by the maximum likelihood model. The predictions were based on the variances determined in the experiment with an infinitely stiff spring (only force feedback available) and a spring with zero stiffness (only position feedback available). SD for pure force replication (σ_{f}) was 0.92 N, SD for position (σ_{x}) was 8.5 mm (so σ_{f}/σ_{x} = 108 N/m).
The model predicted a monotonously decreasing ΔF for the FT. The experiment data show the same trend, although the subjects showed a stronger underestimation of force in catch trials than predicted by the model. The model predicted a distinct minimum in ΔX for the PT, which was also found experimentally. The model results for the PT generally fitted the experimental data more accurately than the results for the FT.
Discussion
Experiment data
In agreement with the hypothesis a significant effect of stiffness on the force and position difference between the catch and the blind trials was found. Apparently, the weighting of sensory input from mechanoreceptors is variable and depends on the stiffness of the object humans are interacting with. The experiment depends on the assumption that humans do not adapt their sensory weighting in the catch trials. Care was taken to design a nonlinear spring with characteristics much alike the linear spring but which did result in sufficient force difference at the target position to overcome the variance in the blind trials. Subjects could not distinguish catch trials from regular blind trials. In fact, when asked afterward only one of the subjects reported he suspected a change in the behavior of the manipulator, but could not indicate what it was.
Subjects consistently underestimated the force and position during the blind trials, effectively exerting too high forces with respect to the reference trials (Figs. 3, 4). This did not affect our results concerning the weighting because of the comparison between blind trials and catch trials, since both are performed blindly. A possible explanation may be that overestimation is a natural byproduct of neural processing as was suggested in force replication tasks before by Shergill et al. (2003).
Maximum likelihood model
Derivation of the maximum likelihood model revealed that task instruction has no effect on the optimal weighting of force and position. This means that interpreting either task instruction as “just do the same as during training” would result in equal task performance. Indeed no significant difference was found between PT and FT, indicating that although subjects were trying to perform either a FT or PT, the outcome was similar. Instructing a subject to reproduce either force or position can be interpreted as focusing the subject's attention to one of the two modalities. Recent literature has shown that attentional manipulation of a specific sensory modality does not influence the relative weighting of that modality (Helbig and Ernst, 2008). Note that the lack of task effect only holds for this particular setup: the linear spring in a quasistatic experiment yields redundancy between force and position. In dynamic experiments (like perturbed posture) force and position feedback describe different dynamics, resulting in a substantial effect of task instruction (Doemges and Rack, 1992a,b; Abbink et al., 2004).
The model explained the course of the data rather well, although there was a remarkable difference with the subjects' overestimation of force during a FT (Fig. 6). In terms of sensory weighting, this indicates that subjects are either biased toward using position feedback at the expense of force feedback, or that the weights are nonideal and sum up to a value other than one. Nevertheless, both model and data showed that increasing spring stiffness leads to a weight shift toward force feedback, as was hypothesized.
Conclusion
This study has shown stiffnessdependent weighting of sensory input from mechanoreceptors for force and position. The maximum likelihood model predicted a monotonously decreasing force difference between the catch and the blind trials with increasing stiffness and a distinct trough in the position difference. Both were experimentally found indicating that a maximum likelihood model not only describes sensory weighting between sensory systems as was found in previous literature, but also within one sensory system, in this case proprioception.
Additionally, in reproduction tasks, subjects consistently underestimated the force/position. In the blind trials, the subjects exerted a force greater than the target force in all stiffness conditions.
Finally, it was shown that in replication tasks with known stiffness conditions, there is no difference between position and force tasks with respect to sensory weighting. The model predicted the exact same weighting factors and in accordance no significant task effect was found in the experiment.
This study successfully demonstrates sensory weighting inside the proprioceptive sensory system, i.e., between force and position feedback. The experiment shows sensory weighting that is close to optimal according to the MLE model.
Appendix A—Application of Priors in the MLE Model
In the main text force F and position X are estimated from a weighted combination of a force estimate f̂ and position estimate x̂. In previous research it has been shown that prior experiences can bias estimation (Körding et al., 2004). This can be modeled by adding priors: a distribution of likely forces and positions. With the application of priors for force and position, Equations 3 and 4 become the following: with weights for force (w_{f}), position (w_{x}), force prior (w_{fp}), and position prior (w_{xp}), and the expected values for force (μ_{fp}) and position (μ_{xp}). Scaling the weights such that w_{f} + w_{x} + w_{fp} + w_{xp} = 1 results in the optimal weights, as displayed in Equations 13–16 for FT and Equations 17–20 for PT. with σ_{f}^{2}, σ_{x}^{2}, σ_{fp}^{2}, and σ_{xp}^{2} the variances of respectively force, position, force prior, and position prior. Note that application of priors does not introduce an effect of task instruction since in accordance to the main text multiplication of the numerator and denominator of Equations 13–16 by k_{i}^{2} yields Equations 17–20.
With the addition of priors, Equations 9 and 10 in the main text become: When Equations A21 and A22 are solved with substitution of Equations A13–A20, the resulting force F and position X are both functions of k_{i}, f_{target}, δ_{f}, σ_{f}, σ_{x}, σ_{fp}, and σ_{xp}: However, when the priors are assumed to be ideal, the limit case after sufficient training (μ_{fp}→f_{target} and μ_{xp}→x_{target}) loses its dependency on variance of the priors σ_{fp} and σ_{xp}, yielding the following: This means that the formulae in the main text apply even for the addition of priors, under the assumption that the priors are unbiased, i.e., the means of the prior distributions coincide with the target force and position during training trials.
Footnotes
 Received January 9, 2009.
 Revision received March 16, 2009.
 Accepted April 2, 2009.

This research was funded by Dutch Government Grant BSIK03016. We kindly thank our students Tim Kiefte, EricJan den Ouden, Jeroen van Rooijen, and Wouter Wolfslag for carrying out the experiments.
 Correspondence should be addressed to Winfred Mugge, Department of BioMechanical Engineering, Delft University of Technology, Mekelweg 2, 2628 CD Delft, The Netherlands. w.mugge{at}tudelft.nl
 Copyright © 2009 Society for Neuroscience 02706474/09/29547607$15.00/0