Article Figures & Data
Figures
- Figure 1.
Bayesian decoders can misestimate uncertainty. A, Examples of posteriors for three toy Bayesian decoders: underconfident (blue), overconfident (red), and well-calibrated (black) decoders provide posterior estimates for each trial. Curves denote single-trial posteriors and lines below each posterior denote 80% the credible intervals, and credible intervals for an additional four trials. Dots denote MAP estimates. Coverage is measured by whether the HPD regions cover the true value (error = 0, in this case). B, Coverage as a function of the desired confidence level for each decoder. C, In a simulation of homogeneous neurons receiving latent input in addition to their tuning to an external variable, we find that a GLM-based decoder is increasingly overconfident as the contribution of the latent input increases (top). Modeling the latent input with a GLLVM, even though it is unknown, reduces overconfidence (bottom). For clarity, curves are averages of multiple simulations. D, Tuning curves for the simulated population (top) and median cross-validated error for the MAP estimates (bottom) for the GLM (red) and GLLVM (gray) averaged across multiple simulations. Error bars denote standard deviation across simulations.
- Figure 2.
Latent variables increase posterior uncertainty when modeled. A single neuron tuned to the reach direction may additionally be impacted by a latent variable (left) with the tuning curve scaled up or down depending on the latent state (yellow to blue curves). After fitting the encoding model, we can find the joint posterior over the value of the latent variable and the reach direction given an observed spike count (middle). Left panels show “slices” of the joint posterior evaluated at specific latent values (colors correspond to tuning curves), and the heatmaps show the full joint posterior. To decode the reach direction, we marginalize/integrate over the latent variable (right). The full model (blue) has higher uncertainty for reach direction than a model that does not take the latent variable into account (black).
- Figure 3.
Experimental decoding tasks and example posteriors. A, For M1 data, we aim to decode the target direction in single trials of a center-out reaching task. B, For V1 data, we aim to decode stimulus (full-field grating) movement direction in single trials. C, For ABI data, we aim to decode the identity of a natural image stimulus on single trials. For each case, example stimuli (top) and tuning curves for individual neurons (bottom) from the Poisson GLM fits. D–F, Example posteriors for single trials (left) as well as the average posterior aligned to the MAP estimate (right). For ABI, note that the posteriors are discrete distributions and, for clarity, only a subset of images are shown. In (F), black triangles denote the true image stimulus.
- Figure 4.
Coverage results for three experimental datasets and four decoders. Decoding the reach direction from neurons in M1 during a center-out task (A), decoding the stimulus direction from neurons in V1 during presentation of drifting gratings (B), and decoding the identity of a natural image from multiple brain regions (C), Bayesian decoders tend to be overconfident. Latent variable models (P-GLLVM and NB-GLLVM) are better calibrated than their GLM equivalents, and negative binomial models tend to be better calibrated than their Poisson equivalents. Cross-validated error/accuracy (D–F), uncertainty (G–I), and coverage (J–L) each change as a function of how many neurons are included in the model. Accuracy increases with increasing numbers of neurons and uncertainty decreases. However, calibration (the degree of overconfidence) gets worse as more neurons are included in the model. Error in D and E denotes a median error. SD in G and H is a circular standard deviation. Dashed line in I denotes maximum entropy over the natural images. Dashed lines in J–L denote a nominated 95% coverage. M1 results in D, G, and J are averaged across 200 sets of neurons; V1 results in E, H, and K are averaged across 100 sets of neurons; and ABI results are averaged across 20 sets of neurons.
- Figure 5.
Encoding models for reach direction in M1 and grating direction from V1. A, Spike counts for all neurons recorded from M1 and trials for each of the eight directions of a center-out reaching task (top). Neurons are sorted by their preferred directions, and trials are sorted first by the target direction and then by the value of the latent state. The color scale is transformed
- Figure 6.
Increasing latent dimensionality does not fully correct overconfidence. Error/accuracy (left) and coverage at 95% credibility level (right) for GLLVMs with different latent dimensionalities. GLM and GLLVM results reflect the full population of neurons for each experimental setting. For comparison, results with reduced populations of 20 neurons are included here for the GLM and one-dimensional GLLVM, selected using a greedy optimization to create the “best” and “worst” error/accuracy. Error bars denote 95% confidence intervals. Dashed lines denote nominated coverage of 95%. Light and dark colors for the best/worst greedy decoders denote results from the GLM and 1D GLLVM, respectively.
- Figure 7.
Post hoc-corrected coverage. A–C, Results for full populations in each of the three experimental settings from Figure 3A–C. For each model and experiment, there is a distinct correction parameter optimized to produce well-calibrated results. D, Underconfidence is rare, but can occur, such as when decoding from the best 20 neurons (greedy selection) from the ABI dataset using NB models. Dashed lines in C and D decode the uncorrected results, while solid lines denote the post hoc-corrected results (dashed lines in C are repeated from Fig. 4C for reference). E, The optimal correction parameter as a function of the original miscalibration. Dashed lines denote power law fits for each dataset.
- Figure 8.
Continuous decoding and coverage for position in the HC. A, The true position along the linear track (black line), along with 95% credible regions for three Bayesian decoders: (1) the traditional Poisson GLM, (2) a negative binomial GLM, and (3) the Poisson GLM after post hoc correction. Note that, in some cases, the posterior (or post hoc-corrected distribution) is multimodal, resulting in multiple HPD regions. B, Empirical coverage as a function of the desired credibility level for the four Bayesian decoders. C, Empirical coverage after post hoc correction.
- Figure 9.
Coverage for conformal prediction intervals. For M1 and V1 trial-by-trial data as well as continuous decoding of position for HC, split conformal prediction produces well-calibrated intervals for all models. Here the results show the full data. These uncertainty estimates are based on the distribution of residuals (insets) and can also be calculated for non-Bayesian decoders such as OLE (gray).
- Figure 10.
Uncertainty predicts accuracy. For reference, dots denote averages calculated in deciles. Error bars for M1 and V1 denote standard deviation. Error bars for ABI denote 95% confidence intervals. Lines for M1 and V1 denote linear, least-squares fit for single trials in the log domain. Curves for ABI denote logistic regression.
- Figure 11.
Accuracy, uncertainty, and coverage vary with stimulus contrast in V1 and with movement speed in M1. A, For static, oriented gratings, cross-validated decoding accuracy increases following stimulus onset (white triangle) but depends on stimulus contrast (top). Posterior entropy decreases, with lower entropy for higher contrast stimuli, and coverage (at 90% nominated) also varies. Dashed lines denote chance (top), maximum entropy (middle), and nominated coverage (bottom). B, At 200 ms after stimulus onset (black triangles in A), we find that the (post hoc-corrected) posterior entropy for the Poisson GLM varies with contrast. Dots denote averages in deciles, error bars denote 95% confidence intervals, and curves denote logistic regression fits. C,D, Analogous results for recordings from M1 during center-out reaching with maximum movement speed split by terciles. Cross-validated decoding accuracy increases shortly before movement onset (white triangle) but depends on the reach speed (top). Posterior entropy decreases with lower entropy for higher speeds. Results in D are for 100 ms after movement onset (black triangles in C).
