Article Figures & Data
Figures
- Figure 1.
Simulation of multisensory adaptation. Idealized behavioral responses in a heading discrimination task based on vestibular (Vest), visual (Vis), or both (combined) cues, according to RBA (A) and VDA (B). Psychometric plots (middle 3 columns) represent the simulated ratio of rightward choices, as a function of heading direction. Data (circles) were fitted with cumulative Gaussian functions (solid lines). Baseline performance is presented for vestibular and visual cues by the blue and red curves, respectively. After introducing a heading discrepancy (Δ = 10°), vestibular and visual cues adapted to cancel out the discrepancy (cyan and magenta post-adaptation curves, respectively). For RBA, cues shifted according to the visual versus vestibular RR, i.e., for RR = 5 (top row), the vestibular cue shifted five times more than the visual cue; for RR = 1 (middle row), cues shifted equally; and for RR = ⅕ (bottom row), the visual cue shifted five times more than the vestibular cue. For VDA, the visual cue did not shift (pre-adaptation and post-adaptation curves are superimposed); only the vestibular cue shifted. This happens regardless of RR (RR = 1 was used for the example presented here). Dark to light shades of green show the combined-cue responses during adaptation, while gradually increasing Δ = 2°, 4°, 6°, 8°, 10°. The axis of the combined-cue axis (0) is defined by the heading midway between the cues (schematics on the left). In the rightmost column, the combined-cue PSE is plotted as a function of Δ. Solid black lines represent regressions through the origin. For RBA, regression slopes followed the RR: positive for higher visual weighting (top row), negative for higher vestibular weighting (third row), and 0 for equal weighting (second row). For VDA, the regression demonstrated maximal visual weighting (slope = ½; bottom row).
- Figure 2.
Example of human multisensory adaptation. Two example sessions of visual–vestibular adaptation, from one human subject, are presented. For both sessions, visual coherence = 100% (RR > 1). Plotting conventions are the same as Figure 1. However, opposite visual–vestibular heading discrepancies (Δ) were used for the two examples: in A, Δ was positive. Hence, results can be compared directly with the simulation results of Figure 1 (which also used positive Δ). In B, Δ was negative. Hence, shifts are expected in the reverse direction. R2 ≥ 0.9 for all psychometric fits. Vis, Visual; Vest, vestibular.
- Figure 3.
Example of monkey multisensory adaptation. Two sessions from one monkey of visual–vestibular adaptation are presented. For both sessions, visual coherence = 100% (RR > 1). Plotting conventions are the same as Figures 1 and 2. Opposite visual–vestibular heading discrepancies (Δ) were used for the two examples: in A, Δ was positive. Hence, results can be compared directly with the simulation results of Figure 1 (which also used positive Δ). In B, Δ was negative. Hence, shifts are expected in the reverse direction. Unlike Figures 1 and 2, there are no combined-cue data during the adaptation trials from the monkey experiment. R2 ≥ 0.9 for all psychometric fits. Vis, Visual; Vest, vestibular.
- Figure 4.
Adaptation ratio. Population data are presented for both humans (A) and monkeys (B). Data in the left, middle, and right columns are sorted by low, medium, and high RR (ratio of visual to vestibular reliability), respectively. Each data point represents the adaptation observed in one experimental session, plotted as the shift in vestibular (Ves) PSE versus the shift in visual PSE. PSE shift was calculated as the difference between the pre-adaptation and post-adaptation PSEs. Error bars on the data points represent ±1 SD of the PSE shift (calculated by bootstrapping the psychometric curve data 2000 times). Squares and circles represent sessions with positive Δ (vestibular to the right of visual) and negative Δ (vice versa) discrepancies, respectively. The brown solid lines represent type II regressions of the data points, weighting each data point by its inverse (mean visual and vestibular) SD. The regression lines were constrained to pass through the origin. The brown shaded regions represent the 95% confidence bands of the regression lines (from 1000 bootstrapped datasets). Gray shading indicates regions where vestibular and visual cues shift in the same direction (unexpected by internal consistency theory). Histograms (far right column) indicate that the sum of the absolute visual and vestibular shifts (across all RRs) were primarily <10° (vertical dashed line). n = 45, 38, 17 (100 total) for human low, medium, and high RR, respectively. n = 36, 28, 44 (108 total) for monkey low, medium, and high RR, respectively.
- Figure 5.
Correct direction PSE shift. Human (A) and monkey (B) data were normalized by the direction of cue discrepancy, such that positive values represent a PSE shift in the expected direction, and negative values represent a PSE shift in the unexpected direction (gray shaded region). In the left column, the shift in the PSE for vestibular (blue) and visual (red) cues is plotted as a function of RR (ratio of visual/vestibular reliability). Filled data points represent significant shifts. The blue and red regression lines represent the linear fit of the visual and vestibular PSE shifts, respectively. Black dashed vertical lines mark the RRs used to sort the data by low, medium, and high RR. Blue and red histograms (middle column) display the vestibular and visual PSE shift distributions, respectively. Filled bars in the histograms represent significant PSE shifts. Dashed lines represent the mean PSE shifts, presented with 95% confidence intervals. Mean PSE shifts, grouped by low, medium, and high RR, are presented in the rightmost column. Shaded regions represent ±1 SD, and vertical lines represent 1 SD. n = 100 for the human data and n = 108 for the monkey data. Vis, Visual; Ves, vestibular.
- Figure 6.
Actual versus expected proportional PSE shift. Actual versus expected proportional PSE shifts are plotted for the human (A) and monkey (B) data. Blue and red lines represent type II regressions of the vestibular and visual data (blue and red circles), respectively. The regression lines were constrained to pass through the origin. Blue and red shaded regions represent 95% confidence bands of the regressions. If PSE shifts were to follow the RBA predictions, the regression for both cues would lie on the line y = x (dashed black line). Gray shading represents regions where adaptation was in the unexpected direction. Vis, Visual; Ves, vestibular.
- Figure 7.
Adaptation data were best represented by a fixed-ratio model. When plotted on the Cartesian plane, vestibular versus visual PSE adaptation can be described by a vector. The angle of that vector represents the ratio of vestibular/visual adaptation. Here we compared the adaptation angles extracted from the data with those predicted from four different models: FRA, VDA, RBA, and VeA. The cumulative distribution of the difference between the actual and predicted angles was plotted for both the human (A) and monkey (B) data. The black diagonal line represents the average cumulative distribution for random angles. The mean of the cumulative distribution is presented on the plots for each model.
- Figure 8.
Fixed-ratio adaptation accounts for reliability-based cue combination. A, We present a simulation of the expected RBCC adaptation. RBCC was quantified by the slope of the combined-cue PSE versus Δ (as presented in Figs. 1, 2). Two models of adaptation were simulated: RBA (left plot) and FRA (right plot). For FRA, a vestibular/visual adaptation ratio of 1.75 was used (as extracted from the actual cue shifts). For both simulations, partial (65%) adaptation is presented, i.e., the cues did not reach internal consistency. Before adaptation, combined-cue PSE slopes that would result from visual or vestibular cue dominance are represented by the red and blue horizontal lines, respectively, and post-adaptation by magenta and cyan curves, respectively. RBCC is represented by the dark green and light green curves, before and after adaptation, respectively. For RBA, RBCC remained unchanged (pre-adaptation and post-adaptation curves are superimposed). For FRA, the RBCC curve amplitude changed, and its y-cut became non-0. B, The actual data of combined-cue PSE versus Δ slopes is presented (circles). Two outliers are marked by + symbols. The optimal FRA model fit is represented by the black dashed curve, and the model fit with adaptation ratio constrained at 1.75 is presented by the light green curve. C, The R2 values of the FRA model fit for all possible parameters of adaptation ratio and extent are presented. The light green and (partially hidden) black dots represent the R2 values for the ratio-constrained and optimal fits, respectively. The white vertical lines represent the adaptation-ratio extracted from the human (solid) and monkey (dashed) data. D, The average difference between the actual combined PSEs and the predictions of the models are presented. Dark and light green lines show the deviances for RBA and FRA, respectively. Error bars represent the SEM. vis, Visual; ves, vestibular.
- Figure 9.
Simulated effects of visual PSE shift variability. Each subplot presents a simulated probability density function (PDF) corresponding to the subplots in Figure 4. Left, middle, and right columns represent low, medium, and high RRs, respectively. All PDFs were generated from two bivariate Gaussians: one with mean (1°, −1°) and the other with mean (−1°,1°). Gaussian variances were set according to the actual PSE shift variances extracted from the human and monkey data (A and B, respectively). Vestibular PSE shift variance was set according to the overall variance and did not change for different RRs: 2.4° for the human data and 7.0° for the monkey data. Visual PSE shift variance was set individually for low medium and high RR: 15.0°, 2.3°, and 1.2°, respectively, for the human data; and 9.0°, 3.4°, and 0.8°, respectively, for the monkey data. A type II regression line of 2000 data points, generated from each PDF, is displayed in white. The regression lines were constrained to pass through the origin. Ves, Vestibular.
