Accounting for Bias in the Estimation of r² between Two Sets of Noisy Neural Responses

Validation of estimator by simulation

To test the ability of r̂ER2 to accurately estimate true neuron-to-neuron tuning curve correlation in the face of noise, we simulated the responses of two neurons with the same trial-to-trial variability (σ²), the same number of trials (n), but potentially different SNR (Eq. 2; because noise is fixed, SNR increases with tuning curve amplitude). Both simulated neurons had sinusoidal tuning curves (as in Fig. 1A), and their relative phase offset was varied to set the true tuning correlation (see above, Materials and Methods, Eq. 3). We first consider the case where neuron X has a high SNR (SNR_X = 1.0), whereas neuron Y has a low SNR (SNR_Y = 0.5). In our simulation we set m = 371 and n = 4 to represent a typical V4 experiment with many shape stimuli and few repeats of each. This parameter regime, where m ≫ n, is typical in neuroscience because often many experimental conditions (e.g., stimuli) are needed to characterize tuning (see below, Nonvision example of correcting correlation between tuning curves for hippocampus example). In this case, the naive estimate of correlation, r̂2 (Fig. 1B, orange), is on average only one-quarter of the value of the true correlation between the simulated tuning curves. For example, r̂2=0.25 when rER2=1. On the other hand, our r̂ER2 on average lies very close to the true correlation (Fig. 1B, blue line lies on diagonal). However, although r̂ER2 is clearly the less biased estimator of rER2, it has become more variable; the 95% quantile error bars are longer than those for the naive estimator. When both neurons have a high SNR (Fig. 1C), the difference between the average of r̂ER2 and r̂2 shrinks commensurate with the increased SNR, as does the difference in the error bars. We note that r̂ER2 can exceed r² = 1 or go below r² = 0; this is necessary to obtain an unbiased estimator (e.g., if all probability mass is below 1, the expected value cannot be 1). To account for these impossible values of r², when analyzing results for single neurons, we recommend truncating the estimator to lie in [0, 1]; this will minimally add to the bias and aid interpretation.

Comparison to Spearman's correction

To demonstrate how our estimator, r̂ER2, performs relative to past attempts to correct the downward bias of correlation caused by noise, we compare it to that of Spearman as formulated by Adolph and Hardin (2007), ρ̂02, and to an extension, ρ̂0(ER)2, where we substituted unbiased estimates of dX2 and dY2 (see above, Materials and Methods, Spearman's correction for attenuation). For reference, we also compare these estimators to the naive estimate r̂2.

We compare the bias of these four estimators using simulated data in which we vary the number of stimuli m and the SNR for both neurons while holding the number of repeats constant at n = 4 and the true correlation fixed at rER2=1 (Fig. 2, top row). Our corrected estimates (ρ̂0(ER)2 and r̂ER2) can become unstable for low SNR because significant probability mass accumulates near zero in their denominators; thus, here we truncate these estimators between [–1, 2], thereby reducing the effect of extreme values. This allows our simulations to be reproducible while still reflecting the low bias and high variance of the raw corrected estimators for low SNR. In the case of a small number of stimuli (m = 20; Fig. 2A), the naive r̂2 (orange trace) is well below one and remains below for all levels of SNR. This substantial downward bias remains even as m is increased to 50 (Fig. 2B, orange) and 1000 (Fig. 2C, orange). The estimator proposed by Adolph and Hardin, ρ̂02, is less biased (red trace) but still consistently underestimates the true r² and does not improve with m. The same estimator computed using an unbiased estimate of the dynamic range, ρ̂0(ER)2 (Fig. 2, purple trace), shows an upward but, typically, smaller bias relative to the downward bias of r̂2 and ρ̂02, and it converges to the true value (r² = 1) as m increases. Finally, our r̂ER2 (blue trace) shows the least bias of the four estimators over a wide range of SNR and m, although in the case of the lowest SNR values, it becomes highly variable (Fig. 2A, large blue SD at left). For higher values of m (Fig. 2B,C), this is not an issue. For very high m, our correction to Spearman's method ρ̂0(ER)2 and our estimator r̂ER2 are essentially identical (Fig. 2C, blue and purple lines overlap).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

Comparison of r̂ER2 to alternative estimators in simulation. A–C, The mean and SD (shaded area) of four r² estimators are plotted as a function of SNR (equal in both neurons) and number of stimuli, m, in simulations of two sets of neuronal responses, each with n = 4 repeats, and true correlation is r² = 1 (see above, Materials and Methods, Simulation procedure) for 50,000 simulations. In some cases, for low SNR, the SD (shaded area) exceeds the range of the plot because the estimators become unstable so that even the 95% CI of the mean (vertical bars) remains high. In orange is the naive estimator Pearson's r², in red is the estimator of Spearman (1904) calculated according to the methods of Adolph and Hardin (2007), in purple is Spearman's estimator computed using an unbiased estimator of dynamic range (Eq. 18), and in blue is the estimator we use throughout the article, r̂ER2. A, Low number of stimuli (m). B, Intermediate number of stimuli. C, High number of stimuli. D–F, The square root of the MSE (solid line), bias² (dashed line), and variance (dotted line) are plotted for each estimator. Here, SNR is fixed (SNR = 0.5), whereas m is varied in each plot, and n is varied across plots. D, Low number of repeats. E, Intermediate number of repeats. F, High number of repeats.

Although we have found our corrected estimators can effectively reduce bias, they also typically have higher variability than the prior estimators, which can lead to overall higher mean squared error (MSE, a common measure of overall estimator accuracy). Here, we explore the trade-off between bias and variance across different numbers of repeats (n) and stimuli (m; Fig. 2, bottom row). When we set the number of repeats to be n = 4 (Fig. 2D–F), we found that for very low m, the MSE was highest for our corrected estimators and the naive estimator. Specifically, ρ̂0(ER)2 (solid purple trace) was highest, then r̂2 (orange trace), and then r̂ER2 (blue). The majority of the MSE for the corrected estimators resulted from the variance of the estimators (dotted purple and blue lines, just below the solid lines of the same colors), and only a small amount was attributable to the bias of the estimators (dashed purple and blue lines). As m increases above 10, the bias of the corrected estimators drops below those of the prior estimators, but the MSE remains higher than Spearman's ρ̂02 until m = 40. As m increases beyond 40, the MSE of the corrected estimators drops below those of the prior estimators (blue, purple below red, orange solid traces). Thus, at this point our corrected estimators are more accurate than the uncorrected estimators despite their variance remaining higher. Although the variance of all estimators decreases with m (dotted lines decreasing), the bias of the naive estimator (r̂2) remains constant, whereas the corrected estimators continue to improve in terms of both MSE and bias (blue, purple decreasing while orange remains largely flat). This is because the corrected estimator estimates of trial-to-trial variability are improving with m, thus improving the accuracy of the correction. Interestingly, the MSEs of the corrected estimators are nearly identical, but the bias of ρ̂0(ER)2 is higher than r̂ER2 throughout our simulations (blue, purple solid traces overlapped but blue dashed below purple dashed traces). When the number of repeats increases (n = 10; Fig. 2E), the performance of all estimators improves, but similarly to Fig. 2D, the MSE of the naive estimator r̂2 remains unimproved by increasing the number of stimuli. The rate at which the MSE of ρ̂02 improves with m increases (solid red line is steeper in Fig. 2E than in Fig. 2D), this is because the overestimation of dynamic range because of trial-to-trial variability (Eq. 17) is reduced by averaging across more repeats, and as m increases, the true dynamic range dominates. Finally, for a very high number of repeats (n = 50; Fig. 2F), the bias of all estimators (dashed lines) is low because the effect of trial-to-trial variability has largely been averaged out. Overall, with a high number of repeats, the difference between the estimators becomes small, but when the number of repeats is low, the corrected estimators (ρ̂0(ER)2 and r̂ER2) overcome their higher variability to outperform the prior estimators (r̂2 and ρ̂02) in terms of both MSE and bias over a wide range of m.

Split trial validation for V4 data

We have shown that our estimator, r̂ER2, works well under simulations that follow the assumptions of its derivation. Because neuronal responses are not guaranteed to be well approximated by these assumptions, it is important to test the method on real neuronal data. In most cases, we do not know rER2 and thus cannot determine whether the estimator is working. One case in which we do know rER2 is across independent trials of responses of a neuron to the same set of stimuli. Theoretically, the expected response values are the same for two tuning curves computed from alternating stimulus repeats recorded from a single neuron, thus rER2=1. Here, we correlate the mean responses from the odd trials to those from the even trials for a set of single neurons with the hope that our method correctly estimates rER2=1. Our neuronal dataset consists of single-unit recordings from 96 V4 neurons with n = 4 repeats and m = 371 stimuli (see above, Materials and Methods, Electrophysiological data). We evaluate our estimators on the first and third trial correlated to the second and fourth trial.

We first computed the naive r̂2 for all neurons (Fig. 3A, orange points) and plotted these values as a function of the estimated SNR (d̂2/mσ̂2, across all trials). Despite the theoretical value of rER2=1, the unit with the highest SNR (rightmost orange point) achieves only r̂2=0.70. Thus, for real neural data with low n, the naive r̂2 clearly underestimates rER2. In contrast, the scatter of r̂ER2 values across the population of cells is approximately centered around one (Fig. 3A, blue points), with an increasing upward bias for low SNR. The highest SNR units (rightmost blue points) have estimates that are close to one. For neurons with lower SNR, some of the r̂ER2 values become unstable, taking on values far >2 (Fig. 3A, small filled blue points, top left) or <0 (small filled blue points, bottom left) consistent with our simulations (Fig. 2A, shaded blue SD on left). The population average (r̂ER2=3.04) is made unstable by these low SNR neurons (maximum value r̂ER2=213.3). The median is a more appropriate method to aggregate results across a population that includes low SNR neurons with few repeats (median r̂ER2=1.07).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

Motivation and validation of r̂ER2 on split-half correlation for V4 neural responses from Pasupathy and Connor (2001). A, Estimated correlation between the odd and even trials of the responses of each neuron to a set of m = 371 stimulus conditions, 96 neurons total, is plotted as a function of the SNR of each neuron calculated across all trials. In blue are the corrected r̂ER2 and in orange the naive r̂2. Theoretically, rER2=1 (bold black horizontal line). Corrected estimates that went above two are set to two, or below zero are set to zero. B, Example cell with high SNR. Each point is the average of the even (x-axis) and odd (y-axis) spike rate for a single stimulus. In red is the baseline firing rate. SNR is high because of a large dynamic range in spike count. C, Example cell with smaller SNR. Despite r̂2 being lower in this case, our estimator predicts that similar to B, the neural responses on even and odd trials have the same pattern of means.

For two example neurons, plots of the raw tuning curve values on even versus odd trials (Fig. 3) provide deeper intuition into the metric. Each point in the scatter plots represents the mean response to a particular shape on the even trials versus that on the odd trials. In both plots, by the split-half construction, all the residual variance is attributable to trial-to-trial variability. The estimator r̂ER2, being ∼1.0 in both cases, is appropriately factoring out the trial-to-trial variability and is in essence predicting that with more trials, one should expect these points to settle onto a line.

Overall, our validation on simulated data demonstrates that our corrected r̂ER2 avoids the large bias of the naive estimator, r̂2, and outperforms a previously proposed metric. Furthermore, our metric appears able to achieve an approximately veridical result for a population of real neuronal data in a split-half test that had very low n (only two repeats) and thus substantial noise.

We found that low SNR neurons tended to have unstable estimates. This is because for low SNR, the distribution of denominator estimates shifts toward zero. When estimating population level correlation this can be ameliorated by calculating the median (see below, Measuring population translation invariance). When an analysis focuses on correlation at the single-neuron level, truncation of the estimate of r̂ER2 to lie in [0, 1] can reduce variability and improve interpretability, but the variability may be so high that these estimates provide little information. Use of confidence intervals is a principled approach to determining whether variability is in fact too high (see below, Measuring invariance of shape tuning to a change in a surface property). Next, we apply our method to estimate relationships in neuronal tuning where no ground truth exists.

Measuring population translation invariance

Translation invariance of neuronal selectivity is the degree to which a neuron maintains the same pattern of responses to a stimulus set regardless of where the stimulus set is presented in its RF. This can be quantified by measuring the correlation between the responses at one position to those at another. The naive r̂2 will suffer from the confounds described above. To avoid this, we use r̂ER2 to accurately assess translation invariance in V4. A population level estimate of invariance found by calculating the median across single-neuron estimates will inherit the low bias of the individual samples and thus will provide a low bias estimate of typical translation invariance in V4.

We reanalyzed data from El-Shamayleh and Pasupathy (2016), which consisted of recordings of 80 V4 neurons responding to a set of simple shapes presented at up to four positions within the RF (see above, Materials and Methods, Electrophysiological data). One potential concern is that the RFs of the neurons are diverse (Fig. 4A, gray traces); for some neurons the average response (across shapes) falls off quickly over space, whereas for others the mean remains high. Half of the neurons are at less than ∼80% of mean response for the farthest stimuli (Fig. 2A, red trace). Given the influence of SNR on r̂2 (Fig. 2A–C, top row, orange traces), it is quite possible that the naive r̂2 would display a falloff in correlation simply because the SNR was lower farther from the center of the RF. Indeed, r̂2 drops off sharply from the center of the RF (Fig. 4B), falling from a value of 1 (by definition) at the center to a median of 0.52 at the position immediately to the left of center and to 0.51 at the point immediately to the right of center. The median r̂2 value continues to drop further as the stimulus is shifted to the edges of the range tested (0.23 and 0.27 at left and rightmost points, respectively). Knowing that this naive metric is biased downward by noise and depends on SNR raises the distinct possibility that such a sharp drop would not be observed if more trials were collected. To overcome this confound, we used our corrected metric r̂ER2. The resulting single-neuron traces and their medians are shifted upward (Fig. 4C). This reveals that for the smallest shifts away from the center, tuning remains quite similar (e.g., median r̂ER2 = 0.92 left of center). Thus, the initial steep dropoff in correlation (Fig. 4B) is attributable to trial-to-trial variability. On the other hand, a substantial dropoff in tuning correlation remains for the largest offsets tested.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

RFs and translation invariance across 80 V4 cells from El-Shamayleh and Pasupathy (2016). A, Average response (across all stimuli) at each RF position tested (one gray line per neuron). Each trace is normalized to the peak average response defined to be the center of RF, and the x-axis is the position relative to the RF center, scaled by the estimated RF diameter. Red dots show the median across the population in nine nonoverlapping bins containing ∼34 samples each except at the center of the RF, points are at the center of bins. B, The r̂2 between responses at the RF center versus those at each offset position (1 trace per neuron; red shows median with same binning as A). Center correlation is by definition 1. C, Similar to B, but correlation is computed with r̂ER2. D, Comparison of example cells chosen for low and high TI (both with high SNR) and with measurements at similar positions in the RF. The cell with high TI (dark blue) maintains a high r̂2 (dash line) across the RF (dotted line), and r̂ER2 remains near 1 (solid line), reflecting near perfect invariance. For a second example cell (cyan), r̂2 drops off quickly and r̂ER2 is similar; thus, the drop is not the result of noisy responses. E, For the same two example cells, responses to each shape stimulus (1 point per shape) presented at shift –0.12 is plotted against the responses at RF center, along with SE bars. Unit 68 (dark blue) shows a strong linear relationship reflecting its high r̂2, whereas for unit 25 (cyan), one subset of stimuli evokes a higher response at the shifted position (higher points toward left) than at the center, and another subset (cyan, lower right) evokes a lower response at the shifted position; thus, tuning is clearly changing with position, consistent with the lower r̂ER2 at −0.12 for this neuron in D.

By focusing on neurons with a high SNR, we can find units that are truly invariant, and we can also identify units that have tuning that changes reliably with position in the RF. For example, consider a high-SNR neuron (neuron 68, which had high d²; Eqs. 1, 2; thus was strongly tuned for shapes) that had high translation invariance (Fig. 4D, solid blue trace is relatively flat) versus another (neuron 25) with tuning that is more sensitive to position (solid cyan trace falls off sharply). For the same fraction estimated RF shift (−0.12), one cell has near-perfect invariance, despite the RF sensitivity dropping off by ∼11% (blue dotted trace), whereas the second example cell shares only half the variance with the tuning at the center of the RF (cyan trace, RF sensitivity drops ∼20%). The important contribution of the r̂ER2 estimator is that it removes ambiguity about whether such differences are caused by changes in firing rate over position. Examining the responses at positions 0 and −0.12 across individual shapes (Fig. 4E, each point corresponds to one shape), it can be seen that there is a strong linear relationship for neuron 68 (blue). Intuitively, any deviation from a line is attributable to trial-to-trial variability because a line drawn through these points could intersect the majority of SE bars. For neuron 25 (cyan), however, there is a substantial change in tuning; many shapes with points previously in the cluster near the origin have more than doubled their responses at the shifted position, whereas for points to the right of 20 spikes/s, the average response to these shapes has been roughly halved. SE bars on these points indicate that it is unlikely their deviation from a line of best fit is solely because of trial-to-trial variability.

In summary, r̂ER2 can distinguish with confidence between units that are sensitive versus insensitive to stimulus position and can provide accurate estimates of translation invariance across a population without needing to resort to removing all but the most well-tuned neurons.

Measuring invariance of shape tuning to a change in a surface property

We now demonstrate how our metric along with its associated CIs) can be used to accurately and systematically quantify another important visual invariance, one that relates to the representation of boundary versus surface features. The neuronal data comes from a study (Popovkina et al., 2019) in which V4 neurons were tested with a set of shapes presented as either outlines (shape interior matched the background) or fills (interior was painted the same as the boundary) with the goal of determining whether responses were dictated by boundary shape alone or were dependent on interior fill. It was hypothesized that neuronal responses would have a high correlation across the transformation from fill to outline—high fill-outline invariance—if neuronal shape tuning was largely based on boundary information, but the authors found that the r values tended to be low, suggesting that most neurons did not show strong fill-outline (FO) invariance. This is an intriguing observation because many models differ strongly from V4 in this respect; for example, an H-Max model (Cadieu et al., 2007) fit to V4 shape tuning was found to have very high FO invariance (Popovkina et al., 2019). But to be able to accurately compare noisy neuronal data with noise-free models, we need a way to correct for noise.

Our r̂ER2 estimator can address this problem, but it is important to consider that the added variability in the corrected estimate can be confounding as it will tend to smear out the population distribution of invariance. To this end, we can make use of confidence intervals to select only those cells with reliable r̂ER2 values. For this subset, we can be confident that the empirical distribution of invariance is accurate and not degraded by noise. This criterion naturally removes neurons that were not significantly tuned for one or both sets of stimuli; such neurons with measurable tuning for only one condition can be considered a special case of noninvariance.

We computed fill-outline r̂ER2 values and associated confidence intervals for 42 V4 neurons and found that nearly 40% of the cells had CIs so wide they spanned the full [0, 1] range of possible values for r² (Fig. 5A, rightmost bar). These neurons correspond to the observation in the original study that many neurons failed to show tuning for one or the other of the fill or the outline stimulus sets, and this lack of tuning means that the SNR is too low to allow for accurate estimation of the correlation value.

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

Comparison of r̂2 and r̂ER2 for determining strength of correlation between responses to fill and outline shapes in V4 neurons. A, The distribution of length of 90% confidence intervals. B, Cumulative distribution of r̂2 (orange), of r̂ER2 (blue) when its associated confidence interval length was <0.5 (n = 17 of the original 42 neurons), and of r̂2 for the same 17 neurons (green). C, Example neuron with lower SNR but high fill-outline invariance. On the basis of r̂2=0.28, it might be assumed that fill-outline invariance is low, but given the high noise level and low dynamic range, this amount of correlation is quite high, and confidence intervals suggest the true invariance is upward of 0.7. D, Example cell with high SNR but middling fill-outline invariance. E, Example neuron with higher SNR but little fill-outline invariance. F, Example neuron with low SNR and confidence interval of length 1. Typical of units with long CIs, this neuron evokes significant modulation for only one set of stimuli (in this case filled shapes).

The cumulative distribution of the naive r̂2 across all neurons (Fig. 5B, orange trace) climbs quickly along the left side of the plot, suggesting that high fill-outline invariance is rare (median r̂2 = 0.02). In comparison, the distribution of r̂ER2 values for the neurons with narrow confidence intervals (Fig. 5B, blue line, CI < 0.5) is shifted substantially rightward (median r̂ER2 = 0.38) and becomes broader (slope of blue line is shallower). In this distribution, neurons that reliably change their pattern of selectivity across the fill and outline conditions are indicated by low values, whereas neurons for which selectivity for either fill or outline stimuli is absent or unmeasurable are now excluded. To allow a direct comparison between the naive and corrected correlation values for the same set of units, the green line (Fig. 5B) shows the distribution of the naive estimator for just those units with narrow CIs. From this comparison, we conclude that units with measurable tuning for both fill and outline stimuli tended to sit on the higher end of the original naive distribution (green curve lies to the right of orange curve), and that our corrected values are substantially larger than the naive values (blue lies to the right of green).

Example neurons can provide some intuition into the corrected values. Figure 5C plots outline versus fill responses for a cell with middling r̂2 but high r̂ER2 (values shown at top). The latter value indicates that the observed variation in the scatter plot is consistent with what would be expected given the trial-to-trial noise for this neuron, assuming it had identical fill and outline tuning. Cells with middling to low r̂ER2 (Fig. 5D,E) are those where some of the variance, but not all, is attributable to noise. Finally, a cell with low SNR (Fig. 5F) had a confidence interval covering [0, 1] consistent with the observation that it responded significantly only to filled shapes and not to outlines.

In summary, by focusing on r̂ER2 estimates with small confidence intervals, we were able to show that the population of neurons with significant tuning for both fill and outline stimuli displays a broad spectrum of fill-outline invariance and has a higher average invariance than would have been predicted based on a naive analysis of all neurons.

Nonvision example of correcting correlation between tuning curves

Our correction is important not only for studies of invariance in sensory areas of vision, audition, somatosensation, and olfaction (for references, see below, Discussion) but is highly relevant in any area of neuroscience where the similarity of noisy neural responses across experimental conditions or across neurons is examined. To provide a clear, concrete example from another field, we explain below how the study of hippocampal place field remapping is particularly susceptible to the confounds of noise.

Place fields are characterized by calculating firing rate tuning curves for neurons in the hippocampus with respect to the position of an animal in an enclosure. Much interest has focused on how these place fields change depending on the spatial and sensory context of the animals; the change has been termed remapping. To quantify the degree of remapping, the naive Pearson correlation coefficient has been used for more than three decades (Muller et al., 1987; Muller and Kubie, 1987; Bostock et al., 1991; Quirk et al., 1992; Skaggs and McNaughton, 1998; Anderson and Jeffery, 2003; Lee et al., 2004; Leutgeb et al., 2005, 2007; Fenton et al., 2008; Alme et al., 2014; Rueckemann et al., 2016; Fetterhoff et al., 2021), despite the fact that these experiments are especially susceptible to the confounds of trial-to-trial variability, which we have discussed above. The experimental conditions in this case are the discretized regions of the space that the animal explores, and this can easily be quite high (e.g., m ≈ 450 for an enclosure with a diameter of 76 cm Muller et al., 1987). At the same time, the number of repeats (n) is determined by the number of times an animal crosses through the same region of space, thus putting a strong limit on how much averaging can be performed. Furthermore, place fields are sparse, only a few regions in space will evoke responses; thus SNR is suppressed by the large regions over which the neurons are silent. Commensurate with these structural difficulties, studies find that correlation of rate maps in the same experimental context (same enclosure in the same location) can be very low.

Fenton et al. (2008) performed a seminal study of remapping by estimating place field maps in a large square-shaped enclosure and smaller cylinder within the square enclosure. The authors found that the correlation of firing rate maps was lower between the cylinder and square than it was between rate maps recorded in the cylinder at different times. Yet they found that this latter quantity, the correlation of a place field map to itself at a later time, was on average only r ≈ 0.34 (r² ≈ 0.11; Fenton et al. 2008, their Fig. 2B). Assuming that place fields remain the same over short time periods (60 min in this case), a common assumption (Muller et al., 1987; Latuske et al., 2017), it follows that the vast majority of variance in these tuning curves is noise, and thus the SNR is very low. Furthermore, the distribution of these rate map self-correlation values across neurons is broad, being uniformly distributed between 0 and 0.8 (Fenton et al. 2008, their Fig. 2B, inset). At the same time, the distribution of map correlations between the cylinder and square are also broad with a substantial fraction exceeding r = 0.34. The diversity in these correlation distributions could be explained solely by differences in SNR, or there may be subsets of neurons whose rate maps change rapidly over time and others that remain the same between enclosures. Reanalysis of this dataset using our metric could answer this intriguing question.

In general, we find that most studies, including place-field remapping studies, account for the confound of noise by either minimizing the effect of noise (e.g., using longer recording times, fewer stimuli, and collecting many repeats) or by restricting their conclusion to relative statements (e.g., tuning curves or maps are more similar in this condition than that condition). In the former case, an inordinate amount of repeats are collected when those trials could have been used to increase the number of experimental conditions by applying our estimator. In the latter case, results often remain ambiguous because of the possibility of changes in SNR across conditions, comparisons across studies are difficult, and there is no absolute measure of similarity between tuning curves, restricting the granularity of results.