Research Articles, Systems/Circuits

Accounting for Biases in the Estimation of Neuronal Signal Correlation

Dean A. Pospisil and Wyeth Bair
Journal of Neuroscience 30 June 2021, 41 (26) 5638-5651; https://doi.org/10.1523/JNEUROSCI.2775-20.2021

Abstract

Signal correlation (rs) is commonly defined as the correlation between the tuning curves of two neurons and is widely used as a metric of tuning similarity. It is fundamental to how populations of neurons represent stimuli and has been central to many studies of neural coding. Yet the classic estimate, Pearson's correlation coefficient, r̂s, between the average responses of two neurons to a set of stimuli suffers from confounding biases. The estimate r̂s can be downwardly biased by trial-to-trial variability and also upwardly biased by trial-to-trial correlation between neurons, and these biases can hide important aspects of neural coding. Here we provide analytic results on the source of these biases and explore them for ranges of parameters that are relevant for electrophysiological experiments. We then provide corrections for these biases that we validate in simulation. Furthermore, we apply these corrected estimators to make the following novel experimental observation in cortical area MT: pairs of nearby neurons that are strongly tuned for motion direction tend to have high signal correlation, and pairs that are weakly tuned tend to have low signal correlation. We dismiss a trivial explanation for this and find that an analogous trend holds for orientation tuning in the primary visual cortex. We also consider the potential consequences for encoding whereby the association of signal correlation and tuning strength naturally regularizes the dimensionality of downstream computations.

SIGNIFICANCE STATEMENT Fundamental to how cortical neurons encode information about the environment is their functional similarity, that is, the redundancy in what they encode and their shared noise. These properties have been extensively studied theoretically and experimentally throughout the nervous system, but here we show that a common estimator of functional similarity has confounding biases. We characterize these biases and provide estimators that do not suffer from them. Using our improved estimators, we demonstrate a novel result, that is, there is a positive relationship between tuning curve similarity and amplitude for nearby neurons in the visual cortical motion area MT. We provide a simple stochastic model explaining this relationship and discuss how it would naturally regularize the dimensionality of neural encoding.

Introduction

Signal and noise correlation are fundamental metrics used widely by neurophysiologists to measure the relationship between stimulus-evoked responses of pairs of neurons (Cohen and Kohn, 2011). Signal correlation measures how similar the tuning of one neuron is to that of another across a set of stimuli (Gawne and Richmond, 1993), whereas noise correlation measures the relationship between trial-to-trial variability across two neurons. These metrics are often related—neuron pairs with higher signal correlation often have higher noise correlation (Lee et al., 1998; Bair et al., 2001; Averbeck and Lee, 2003; Cohen and Maunsell, 2009; Ecker et al., 2014). Furthermore, the interaction between signal and noise correlation is important for determining whether correlation increases or decreases information in population codes (Oram et al., 1998; Panzeri et al., 1999; Averbeck et al., 2006; Lyamzin et al., 2015; but see Moreno-Bote et al., 2014). By itself, signal correlation is used for functional clustering (Kiani et al., 2015; Power et al., 2011), measuring invariance (Nandy et al., 2013; Popovkina et al., 2019), and as a metric of coding redundancy (Gawne and Richmond, 1993; Gawne et al., 1996; Vinje and Gallant, 2000). However, the naive estimator of signal correlation has two types of bias, described below, that have not been corrected for in the literature but can lead to artifacts or obscure important relationships.

First, signal correlation is biased toward zero by independent trial-to-trial noise. This bias, mentioned by Gawne and Richmond (1993), arises when experimental estimates of two true underlying tuning curves (Fig. 1A, solid lines) are independently deformed by noise (dashed lines). The estimated relationship between the true tuning curves, for example, a positive correlation as demonstrated in Figure 1B, will weaken as the estimated means are spread separately along the two response axes (circles indicate independent spread). This bias is large when there are too few repeats of each stimulus or when recordings are excessively noisy. Second, signal correlation is biased toward noise correlation. Noted by Rothschild et al., (2010); this occurs when correlated noise induces spurious correlation between estimated tuning curves (Fig. 1C, dashed lines). Here, the trial-to-trial noise imparts a tilted elliptical distribution (Fig. 1D, ovals), strengthening the relationship between the estimated means. This bias can inflate signal correlation and spuriously induce the frequently reported positive relationship between naive estimators for signal and noise correlation. Here, we derive an equation that unifies these two biases, determine under what conditions the biases substantially influence results, and demonstrate how to correct the biases using a method for unbiasing Pearson's r2 (Pospisil and Bair, 2021).

Using our corrected estimator, we discovered a novel characteristic of neuronal tuning while reanalyzing data from a study of interneuronal correlation in pairs of simultaneously recorded neurons in the cortical motion area MT (Bair et al., 2001). Area MT contains a preponderance of direction selective neurons (Zeki, 1974; Maunsell and van Essen, 1983; Albright, 1984; Born and Bradley, 2005), and nearby neurons tend to prefer the same direction of motion (Albright, 1984). Nevertheless, considerable unexplained diversity in signal correlation exists for neurons recorded at the same location in cortex (Bair et al., 2001). We found a significant positive relationship between signal correlation and tuning curve modulation: neuron pairs with higher signal correlation tended to have tuning curves with higher signal-to-noise ratios. This relationship would have been trivial had it been observed using the downwardly noise-biased r̂2, but our corrected estimator avoids this artifact. Thus, local diversity in MT tuning decreases among strongly modulated neurons. This fundamentally affects our understanding of the physiological makeup of nearby neurons within a population as well as our theoretical estimates of their ability to encode information, which will need to factor in a dependence of signal correlation on neuronal dynamic range and stimulus choice. We discuss consequences of this result for the dimensionality of population codes.

Materials and Methods

Stochastic model of neuronal responses

Here we describe stochastic models used in our derivations and simulations to generate spike counts for pairs of neurons responding to simple stimulus ensembles. Our derivations below and the corrected estimator of signal correlation (Pospisil and Bair, 2021) assume homoscedasticity, that is, equal variance of responses across experimental (stimulus) conditions. In the case of neuronal data, a variance stabilizing transform can be applied to make this assumption reasonable. For Poisson distributed spiking, which is a useful approximation to trial-to-trial variability in neuronal firing, the square root is a variance stabilizing transform. Other variance stabilizing transforms may be used, for example, the Box-Cox transformation (Box and Cox, 1964), thus from here on, all appropriately transformed responses are assumed to have equal variance. We modeled the variance stabilized spike counts of neurons X and Y as Xi,j and Y i,j for the presentation of the jth repeat (j=1,..,n) of the ith stimulus (i=1,..,m). Let these random variables be constructed as follows: Xi,j=ax + δxTi,x(rs) + σxVi,j,x(rn),(1) Yi,j=ay + δyTi,y(rs) + σyVi,j,y(rn),(2) where Ti,· is the component attributable to tuning curve modulation, that is, the mean response to stimulus i relative to a baseline offset, a, and Vi,j,· imparts zero-mean trial-to-trial variability (with the same variance across all stimuli because responses have been variance stabilized). The parameters δ and σ set the signal and noise SD, respectively, and we define their squared ratio, δ2/σ2, to be the neuronal signal-to-noise ratio (SNR), which will be a critical factor in Results below. The tuning curve modulation is defined as follows: Ti,x(rs)=|rs|Si + 1−|rs|Ri,x,(3) Ti,y(rs)=sgn(rs)|rs|Si + 1−|rs|Ri,y,(4) where Si is the common signal shared between the neurons, Ri,x and Ri,y are the uncorrelated signal between neurons, and all are standard normal distributions, N(0, 1). The parameter rs sets the level of signal correlation, and the sgn() function implements negative signal correlation by inverting the shared component of tuning. Similarly, the trial-to-trial variability is defined as follows: Vi,j,x(rn)=|rn|Ci,j + 1−|rn|Ni,j,x,(5) Vi,j,y(rn)=sgn(rn)|rn|Ci,j + 1−|rn|Ni,j,y,(6) where Ci,j is the common noise, Ni,j,y and Ni,j,x are the independent noise, and all are standard normal. The parameter rn sets the noise correlation.

In most cases, we will use a fixed, sinusoidal tuning curve model. In this case, the tuning curve for neuron X is no longer a function of rs, it is simply a cosine as in the following: ti,x=Ti,x=cos(θi)∑i=1mcos(θi)2,(7) where θi=(i−1)m2π,(8) and the denominator normalizes the length of the tuning vector. The tuning curve for neuron Y is simply phase shifted to achieve the desired value of rs as follows: ti,y(rs)=Ti,y(rs)=cos(θi + arccos(rs))∑i=1mcos(θi + arccos(rs))2.(9)

Sample correlation and SNR estimates

Given two sets of responses for two neurons, Xi,j and Y i,j, for m stimuli and n repeats, the typical estimators, r̂s and r̂n, for signal and noise correlation based on Pearson's r value are the following: r̂s=∑i=1m(X¯i,·−X¯·,·)(Y¯i,·−Y¯·,·)∑i=1m(X¯i,·−X¯·,·)2∑i=1m(Y¯i,·−Y¯·,·)2,(10) r̂n=1m∑i=1m∑j=1n(Xi,j−X¯·,j)(Yi,j−Y¯·,j)∑j=1n(Xi,j−X¯·,j)2∑j=1n(Yi,j−Y¯·,j)2,(11) where the overbar indicates the mean of the indicated variable computed over all values of each subscript that has been replaced by a dot. For example, X¯·,· is the grand average over all repeats of all stimuli for neuron X.

For a single neuron, the SNR can be estimated as follows: SNR=1m∑i=1m(X¯i,·−X¯·,·)2σ̂2,(12) where the estimate of trial-to-trial variability, σ̂2, is the sample variance across repeats of identical stimuli, averaged across all distinct stimuli. The SNR for a pair of neurons is defined as the geometric mean between individual SNRs.

Simulation of correlation between signal and noise correlation

To model a relationship between signal and noise correlation, we assume that zn and zs, the Fisher z transformations, z=12ln[(1+r)/(1−r)], of signal and noise correlation, respectively, are bivariate normal random variables as follows: zn∼N(μn,σn),zs∼N(μs,σs)Corr(zn,zs)=rNS.(13)

We use the inverse Fisher transformation to ensure that the support of rn and rs is in [–1, 1]. The correlation of zn and zs, rNS will give similar values to correlating rn and rs, especially if the bulk of the distribution is in [–0.7, 0.7]. The transformed correlation zn and zs, a Fisher's z is a variance stabilizing transform, will have the advantage of asymptotically known and equal variance across different levels of correlation. The estimate r̂NS is Pearson's correlation coefficient between the Fisher transformed estimates of signal and noise correlation, respectively ẑs and ẑn, across a sample of neurons.

Signal correlation estimate for random stimuli

The calculation of the asymptotic value of signal correlation (as m→∞) is straightforward in the case where stimuli are considered random. It is the correlation between X¯i,· and Y¯i,· (using the dot notation of Equation 10) in the following: r̂s→mδxδyrs + σxσyrn/n(δx2 + σx2/n)(δy2 + σy2/n).(14)

With the simplifying assumption that the dynamic ranges and noise amplitudes are the same for each neuron, that is, δx=δy=δ and σx=σy=σ, the above can be reduced to the following: r̂s→mδ2rs + σ2rn/nδ2 + σ2/n.(15)

This can be rewritten in terms of the product of n and the SNR, defined above as δ2/σ2, by defining λ=nδ2/σ2 as in the following: r̂s→mrs + rn/λ1 + 1/λ.(16)

Signal correlation estimate for fixed stimuli

The derivation of the asymptotic value of signal correlation is more complex in the case of fixed stimuli. We take the approach of separately calculating the expected value of the numerator and denominator of estimated signal correlation (Equation 10) then taking the ratio as an approximation to the expected value of estimated signal correlation. For reference, we note the following relevant moments, which follow from the definitions above: E[X¯i,·−ax]=ti,x,E[Y¯i,·−ay]=ti,y,Var[Y¯i,·]=σy2/n,Var[X¯i,·]=σx2/n,Corr[Y¯i,·,X¯i,·]=rn.(17)

Now consider the numerator of estimated signal correlation, the sample covariance between X¯i,· and Y¯i,· as follows: E[∑i=1m(X¯i,·−ax)(Y¯i,·−ay)]=∑i=1mE[(X¯i,·−ax)(Y¯i,·−ay)]=∑i=1m(Cov[(X¯i,·−ax),(Y¯i,·−ay)] + E[X¯i,·−ax]E[Y¯i,·−ay])=mσxσyrnn + ∑i=1mti,xti,y,(18) thus we see the numerator is increasing in rn similarly to the random stimuli case (Equation 16).

For the denominator of the sample signal correlation, we consider the product of the sample variances of X¯i,· and Y¯i,· as follows: E[∑i=1m(X¯i,·−ax)2∑i=1m(Y¯i,·−ay)2]=Cov[∑i=1m(X¯i,·−ax)2,∑i=1m(Y¯i,·−ay)2] + E[∑i=1m(X¯i,·−ax)2]E[∑i=1m(Y¯i,·−ay)2)].(19)

We first find E[∑i=1m(X¯i,·−ax)2] and E[∑i=1m(Y¯i,·−ay)2)] as follows: E[∑i=1m(X¯i,·−ax)2]=∑i=1mE[(X¯i,·−ax)2]=∑i=1mE[(X¯i,·−ax)]2+ ∑i=1mVar[(X¯i,·−ax)]2=∑i=1mti,x2 + mσx2n,(20) and similarly, the following: E[∑i=1m(Y¯i,·−ay)2]=∑i=1mti,y2 + mσy2n.(21)

For Cov[∑i=1m(X¯i,·−ax)2,∑i=1m(Y¯i′,·−ay)2] we note that only when i = i′ the terms in the two sums depend on each other, and thus we can move the summation out of the covariance as follows: Cov[∑i=1m(X¯i,·−ax)2,∑i=1m(Y¯i′,·−ay)2)]=∑i=1mCov[(X¯i,·−ax)2,(Y¯i,·−ay)2].(22) We then make use of the fact that for a bivariate normal we have the following: Cov[X2,Y2]=4μxμyρσxσy + 2ρ2σx2σy2.(23) Thus the sum of covariance terms in the denominator is as follows: ∑i=1mCov[(X¯i,·−ax)2,(Y¯i,·−ay)2]=4rnσxσyn∑i=1mti,xti,y + 2m(rnσxσyn)2.(24) Summing the covariance and product of expected values together, we obtain the following: E[∑i=1m(X¯i,·−ax)2∑i=1m(Y¯i,·−ay)2]=4rnσxσyn∑i=1mti,xti,y+2m(rnσxσyn)2+(∑i=1mti,x2 + mσx2n)(∑i=1mti,y2 + mσy2n).(25)

Thus, our approximation of the expected value as the ratio of the expected values of the numerator and denominator of r̂s in the case of fixed stimuli is the following: E[r̂s]≈∑i=1mti,xti,y + mσxσyrnn4rnσxσyn∑i=1mti,xti,y + 2m(rnσxσyn)2 + (∑i=1mti,x2 + mσx2n)(∑i=1mti,y2 + mσy2n),(26) where as a last approximation, we take the expected value of the square root of the denominator to be the square root of the expected value of the denominator. This approximation improves with m and provides insight into the relative contributions of the parameters of the two neurons, but it is not used as an estimator, thus its bias does not have an impact on our results.

Electrophysiological data

We reanalyzed data taken from a previous electrophysiological study in which pairs of neurons were isolated and recorded on a single electrode in two awake, fixating macaques (Zohary et al., 1994; Bair et al., 2001). Specifically, we reexamine the data for eight-point direction tuning curves in response to coherently moving dots for 81 pairs of MT neurons. For a detailed description of the visual stimuli, electrophysiological methods, and dataset, see Bair et al. (2001).

We also analyzed the publicly available data (http://dx.doi.org/10.6080/K0NC5Z4X) from V1 of anesthetized macaque monkeys. This dataset includes the spiking activity from three monkeys (n = 106, 88, and 112 single- and multiunit recordings) in response to drifting sinusoidal gratings across 12 directions (1.28 s presentation).

In addition, we analyzed simultaneously recorded responses from the awake mouse (A detailed description can be found at http://observatory.brain-map.org/visualcoding/). In the analyses here, we examined single-unit spiking activity from 75 neurons recorded in VISp of mouse visual cortex in response to a set of 118 natural images. Each natural image was presented for 0.25 s in random order with no intervening blank to achieve ∼50 repeats. For details of electrodes and spike sorting, see the Allen Brain Observatory website (http://observatory.brain-map.org/visualcoding/).

Experimental design and statistical analysis

For experimental design please see above, Electrophysiological data and the original publications (Zohary et al., 1994; Bair et al., 2001). All statistical tests (Figs. 10, 12) are Spearman's rank-order r value two-tailed t tests (Zwillinger and Kokoska, 2000) as implemented in SciPy (Virtanen et al., 2020).

Data availability

Code for calculating estimates of r̂ER2 and associated confidence intervals (CIs) are available at https://github.com/deanpospisil/er_est.

Results

Our results are organized into five sections. First, we describe analytically the origin of the biases in signal correlation, and second, we describe how to correct for these biases. Third, we examine how the correlation between signal and noise correlation can become inflated spuriously. Fourth, we demonstrate the presence of these confounds in the analysis of neuronal data. Finally, we describe a novel result obtained by using an unbiased estimator of signal correlation.

Signal correlation confounds

The de facto estimators of signal (r̂s) and noise correlation (r̂n) suffer from the following two confounds: (1) r̂s can be downwardly biased by trial-to-trial variability (Fig. 1, top row panels A and B), and (2) r̂s can be upwardly biased by rn (Fig. 1, bottom row panels C and D) and can create a spurious positive relationship between r̂s and r̂n (Equations 10, 11).

Figure 1.
Figure 1.

Simulations using sinusoidal tuning curves provide intuition into the attenuation and inflation of estimated signal, that is, tuning curve, correlation (r̂s). A, Sinusoidal tuning curves (solid lines, blue and orange) show theoretical expected value (ti,x and ti,y) for responses of two neurons to repeated stimulation. Their relative phase shift sets a fixed rs = 0.5. Simulating an experiment, single-trial responses are drawn and averaged for each neuron at each point on the tuning curves (dots and bars show expected value and variance, open circles indicate the sample mean). Signal correlation is typically estimated across such trial averages (open circles), which often have a lower correlation (here r̂s=0.32) than do the expected values (solid lines). Thus, sample signal correlation is downwardly biased compared with the correlation between the true tuning curves. B, A schematic distribution of single-trial simultaneous responses of both neurons plotted against each other. The lack of noise correlation is reflected by the circular contour lines. C, Simulation showing how signal correlation can be inflated by noise correlation. Here, deviations of the dotted lines around the means (solid lines) are correlated across the two neurons, that is, at a given x value, both dashed lines often lie above or below their means. D, Noise correlation is indicated in the joint distribution of simultaneous single-trial responses by the tilted ellipsoids around expected values. This positive tilt inflates the estimated signal correlation.

Here we describe the analytic relationship between the typical estimator, r̂s, of signal correlation and three confounding factors: noise correlation, rn; the number of repeats, n, of each stimulus; and the SNR (defined above in Material and Methods, and below). These results assume stimuli are randomly sampled from an ensemble, but the case of a fixed set of stimuli is similar (see above, Signal correlation estimate for fixed stimuli). As the number of stimuli, m, becomes large, the estimator r̂s converges to the correlation between X¯i,· and Y¯i,· (the observed mean firing rates for neurons X and Y across all stimuli, i=1,...,m), which is, r̂s→mrs + rn/λ1 + 1/λ,(27) where λ=nδ2/σ2 is the product of the SNR, δ2/σ2, and the number of repeats, n (see the derivation of Equation 16 in Materials and Methods, Signal correlation estimate for fixed stimuli). Importantly, the estimator does not converge to the true rs as m increases; however, as λ increases, for example, with greater n or higher SNR, the estimator approaches the true rs. As λ decreases, the estimator is increasingly biased toward rn, which can be an upward or downward bias depending on the relative values of rs and rn. Specifically, if rn<rs, it will be a downward bias, and if rn>rs, it will be an upward bias.

Figure 2.
Figure 2.

Analytically derived relationship between typical estimators of signal (rs) and noise correlation (rn) for electrophysiologically plausible parameters. The asymptotic value of r̂s is plotted as a function of noise correlation, rn (see above, Results, Signal correlation confounds, Equation 16). Number of stimulus repeats, n, increases from top to bottom. True signal correlation, rs, increases from left to right. Line color indicates SNR level. Trial-to-trial variability is fixed to σ2 = 0.25. Any deviation from a horizontal line (with intercept equal to rs) indicates a confounded relationship between r̂s and r̂n, and larger slopes indicate stronger biases.

The nature of this bias is depicted in Figure 2, which plots r̂s as a function of the true noise correlation, rn, for several values of rs (columns), number of repeats n (rows), and noise levels (highest noise is purple). When rs=0 (top left), the lines should be flat at 0, but instead are biased toward rn (they have a positive slope) by an amount proportional to rn. The higher the noise, the larger the bias (steeper slopes). Increasing n (going down the left column of Fig. 2) decreases the slope, as does increasing the SNR (going from purple to yellow; Equation 16). For higher values of rs (middle and right columns), the same curves from the left column are simply translated so that the point of intersection (the only nonbiased point) slides rightward up the main diagonal to rn=rs. When signal and noise correlation are equal, there is no bias at any noise level. In summary, r̂s is biased away from the true rs and toward rn, and this bias becomes substantial for low numbers of repeats and when tuning curve modulation is weak relative to trial-to-trial noise. It cannot be overcome by increasing the number of stimuli, m. We now propose and validate a solution to these biases.

Corrected estimator of signal correlation

A simple strategy to remove the bias toward rn is to compute rs based on repeats that are not recorded simultaneously. After all, rs is a measure of tuning curve similarity, and tuning curves for different neurons do not have to be acquired at the same time. Specifically, the estimated tuning curves from the odd repeats for one neuron and those from the even repeats for the other can be used to measure r̂s(1) and vice versa for r̂s(2), then the two estimates can be averaged. This estimator has the benefit that one can now seek to determine whether signal and noise correlation is in fact related across neurons in the absence of this otherwise built-in bias toward noise correlation.

The above strategy means that rn is now zero, thus leaving r̂s (or r̂split) with a bias toward zero that increases with noise as follows: r̂s→mrs1 + 2/λ,(28) where the factor of 2 in the denominator reflects that n has been cut in half. To remove this bias toward 0, we use the estimator r̂ER2 of Pospisil and Bair (2021), which estimates the correlation between the true tuning curves of the two neurons (Fig. 1A, solid blue and orange traces). More formally, it estimates the r2 between the expected values of the responses of the two neurons, ti,x and ti,y (Equations 7, 9). This quantity, rER2, is the explained fraction of variance of the expected response (ER) as follows: rER2=(∑i=1m(ti,x−t¯·,x)(ti,y−t¯·,y))2∑i=1m(ti,x−t¯·,x)2∑i=1m(ti,y−t¯·,y)2.(29)

Figure 3.
Figure 3.

Simulation-based comparison of signal correlation estimators under varying degrees of SNR, noise correlation, and true signal correlation. The estimators are the naive measure of signal correlation r̂2 (blue), the split-trial version r̂split2 (orange), the corrected version of r̂2 from Pospisil and Bair (2021): r̂ER2 (green), and the split version of this estimator r̂ERsplit2 (red). There are m = 500 stimuli and n = 8 repeats. In general, positive noise correlation inflates estimators that do not split trials (blue and green), but split estimators remove this (orange and red); whereas noise attenuates naive estimators (blue and orange), but ER estimators overcome this (green and red). As SNR increases down each column, all estimators begin to approach the true signal correlation, and variability of the estimators is reduced.

Whereas the naive r2 is heavily biased downward by trial-to-trial variability, r̂ER2 converges to the true rER2 as the number of stimuli m→∞. For finite m, r̂ER2 has little bias relative to r2 (Fig. 3 shows validation on simulated data). We make a small modification to r̂ER2 to account for noise correlation by performing the same splitting procedure proposed above and call this estimator r̂ERsplit2.

Correction for the attenuation of correlation coefficients by measurement error has received considerable attention from fields outside neuroscience (Thouless, 1939; Beaton et al., 1979; Rosner and Willett, 1988; Adolph and Hardin, 2007). The most popular correction is given in Spearman (1904). An additional correction for noise correlation is given in Saccenti et al. (2020). Rothschild et al. (2010) to our knowledge are the first to recognize the confound of noise correlation inflating r̂NS (the correlation between signal and noise correlation; Equation 13) and provide an estimate of signal correlation to account for this. The statistical properties of this potentially useful estimator are not well studied (e.g., bias, variance and asymptotic properties) and it is not clear how to extend it to account for the downward bias of trial-to-trial variability. In this article we use trial splitting to remove the effect of noise correlation for its simplicity and r̂ER2 because we have studied its statistical properties (Pospisil and Bair, 2021) and have provided validated confidence intervals, which prior methods lack. We next validate our estimators across a range of typical experimental parameters.

In Figure 3 (top left) we plot the estimated signal correlation squared as a function of the true signal correlation squared (rs2) in the case where there is no noise correlation (rn=0), and SNR is low (0.1). We evaluate the estimator at five levels of true signal correlation (rs2=[0,0.25,0.5,0.75,1]). The naive estimator (blue) shows a strong downward bias, where, for example, when rs2=1 the average r̂2 is <0.25. Thus two neurons with identical tuning would be reported as having little shared tuning. When the split-trial estimator, r̂split, is applied (orange) the downward bias is stronger because correlation is being estimated across fewer repeats. On the other hand, r̂ER2 (green trace) on average estimates the true value (green trace lies on diagonal), although it is highly variable because of low SNR (vertical bars indicate SD). For example when rs2=1, the average r̂ER2 is 1.01, although SD is 0.12. The slight overestimation is the result of the low SNR (Pospisil and Bair, 2021). Estimates could be truncated to remove impossible values of correlation (rs > 1), but this would introduce an even greater downward bias. The split-trial version of this estimator, r̂ERsplit2, shows slightly worse performance where it overestimates (average r̂ERsplit2=1.1 when rs2=1) because the estimate over fewer repeats essentially has lower SNR.

When noise correlation is introduced (Fig. 3, top middle, rn = 0.25) we observe the expected increase in r̂s2 (blue trace is higher than in left column) as it is now biased toward 0.252 rather than 0. This naive estimator gets closer to the true value but only because it is being spuriously inflated. Importantly, r̂split2 does not change across columns (orange traces remain identical), demonstrating its immunity to noise correlation. Applying r̂ER2 without splitting trials creates a large upward bias (green trace above diagonal) because its underlying assumption of independent trial-to-trial variability is violated when rn ≠ 0. Whereas, r̂ERsplit2 (red trace), like r̂split2 is not influenced by rn (red trace same across columns) because the assumption of independent noise is correct across split trials. The differences among estimators described for the top row in Figure 3 are similar but progressively diminished as SNR increases for the middle (SNR = 0.5) and lower (SNR = 1.0) rows as all curves move toward the diagonal. Overall, r̂ERsplit2 (red) shows the least bias, upward or downward, of the estimators of signal correlation considered here.

One disadvantage of r̂ERsplit2 is that it tends to be more variable than r̂s. This is one justification for using r̂s, split or not, when a relative comparison is more important than absolute accuracy. This can be the case in estimating rNS, where preserving appropriate ordering on the rs values (as opposed to achieving an unbiased mean estimate) reduces the downward bias associated with sampling variability (discussed below). Thus, in our results on estimation of rNS, we use r̂s.

Spurious correlation between signal and noise correlation

The inflation of r̂s by rn can lead to a spurious positive relationship between the estimators r̂s and r̂n. A positive relationship between signal and noise correlation has been observed across many experiments and is important to the theory of neural population coding. Thus, it is critical to ensure it is not a spurious relationship. Here we perform a series of simulations where the correlation between r̂s and r̂n is measured across a population of neurons, evaluate the severity of the confound for the classic estimators, and then validate that the split-trial estimator removes the confound.

An example of a simulation where a spurious relation between r̂s and r̂n is induced is shown in Figure 4. We begin by drawing the true signal and noise correlation between 50 pairs of neurons from a bivariate normal distribution so that rs and rn are independent (Fig. 4A). We set the SNR to be low (0.1), the condition under which rn has the greatest influence on r̂s. Indeed, when we simulate the responses from the pairs of neurons and plot the naive signal correlation estimate against the true noise correlation (Fig. 4B), there is a substantial correlation, 0.42, despite there being no relationship between the true rs and rn. This is because r̂s is biased toward rn (Equation 16); the average slope of this relationship (as calculated in Equation 26) is plotted for reference in Figure 4B (gray line). The relationship is not very tight in part because r̂s is a variable estimate. This variability is reflected in the weak relationship between the estimated r̂s and the true rs (Fig. 4C), which is further weakened by the perturbation of rn. The estimate of rn,r̂n, is not as biased as r̂s but is nevertheless variable (Fig. 4D). Finally when plotting r̂s against r̂n (Fig. 4E), there is a relationship between the estimators (r̂NS=0.38) that does not exist between the true parameters.

Figure 4.
Figure 4.

A simulation elucidating the inflating effect of noise correlation on the estimate of rNS (Equation 13). A, The true signal and noise correlation (z transformed) for neuronal pairs are randomly sampled from a bivariate normal distribution with correlation 0. The moments of the z transformed distribution of rs are σ = 0.5 and µ = 0.5. The moments of rn are σ = 0.3, µ = 0.1. The SNR of all neurons is low (SNR = 0.1). B, Estimates of signal correlation have a noisy linear relationship to rn (Equation 26, corresponding roughly to top row, middle, purple trace, Fig. 2) despite there being no relationship between the true parameters. C, Estimated signal correlation (y axis) has little relation to true signal correlation (x axis) because of low SNR and the influence of noise correlation. D, Noise correlation estimates (y axis) are themselves noisy, thus there is an imperfect relationship with true noise correlation (x axis) across the population and the estimate. E, Overall, there is a significant correlation between the simulated experimental estimates of signal and noise correlation, despite there being no true correlation between the parameters being estimated (compare to A). The correlation is lower than that of r̂s and rn (see B) because of variability in r̂n.

Figure 5.
Figure 5.

A simulation demonstrating the attenuating effect of noise on the estimate of rNS. The moments of the z transformed distribution of rs are σ = 0.5 and µ = 0.5. The moments of rn are σ = 0.3, µ = 0.1. The SNR of the neurons are higher here (SNR = 1) than in the previous figure. A, Here the true rn and rs are jointly distributed to have a strong correlation (0.8). B, Signal correlation estimates (y axis) have a strong linear relationship to rn (Equation 26, corresponding roughly to row 1, column 2, orange trace of Fig. 2), yet this is not the result of the spurious correlation in Figure 3: the gray line shows the theoretically predicted linear relationship is weak compared with the relationship in the data points. C, Signal correlation estimates have a strong relation to true signal correlation because of high SNR and corresponding lack of influence of noise correlation. D, Estimates of noise correlation are as noisy as Figure 3 as SNR does not reduce variance in the estimate r̂n. E, Overall, there is a significant correlation of the estimates, although far lower than the true correlation. Thus, in addition to inflation, there can be attenuation in estimating rNS.

Figure 6.
Figure 6.

Simulation of experiment estimating rNS (Equation 13) as a function of m, n, and SNR. Default values of simulation are: m = 100, n = 10, and SNR = 0.1. Solid lines show r̂NS using the naive estimator of signal correlation, and dashed lines show the split-trial estimator. A, Average r̂NS as a function of number of stimuli (m). We set SNR high to reduce the bias of signal correlation by noise correlation. B, Average r̂NS as function of number of stimuli (m) when SNR is low. C, Average r̂NS as function of number of repeats (n). D, Average r̂NS as function of SNR.

Alternatively, because the estimates r̂s and r̂n are themselves noisy, there can be an underestimation of rNS. This is similar to the downward bias in r̂s. We present an example of this in Figure 5 where we set the true correlation between signal and noise correlation rNS=0.8 across the population (Fig. 5A), and we set SNR to be high so there is little influence of noise correlation on signal correlation. In this case, the relationship between r̂s and rn is strong (Fig. 5B), not because of the influence of rn (small slope of gray line implies weak influence) but because there is a true underlying relationship. Yet both estimators are noisy (Fig. 5C,D estimates are scattered about the diagonal), thus the observed correlation between the estimators, r̂NS=0.54 (Fig. 5E), undershoots the true value, 0.8. These demonstrations show that r̂NS can suffer from both an upward and downward bias. Below we outline the parameter ranges where these biases remain and show how a split-trial estimator removes the upward but not downward bias.

We performed simulations like those in Figures 4 and 5 over a wide range of the key parameters n, m, and SNR and plotted r̂NS for each simulated population of pairs using the classical r̂s (Fig. 6, solid traces) and the split-trial estimator r̂split (Fig. 6, dashed traces). First, we set SNR far higher than any observed in our neural data so that the confound of noise correlation on signal correlation is weak (Fig. 6A). When rNS=0, the estimator lies close to zero (light blue trace), confirming the inability of noise correlation to substantially inflate signal correlation at a very high SNR. Yet for all other levels of correlation, rNS is underestimated inversely proportional to m (blue, purple, and pink traces increase with m to true values). This is because the variability of both r̂s and r̂n are inversely proportional to the number of stimuli m, thus with higher m there is less sampling noise corrupting the correlation between r̂s and r̂n across the population. In the case where SNR is low (Fig. 6B), r̂NS is systematically biased for all levels of correlation (solid lines), including rNS = 0. Here, the low SNR allows rn to inflate r̂s, and increasing m reduces the ability of sampling noise to attenuate this effect. Employing the split-trial estimator (Fig. 6B, dashed lines) removes the effect of noise correlation, and r̂NS converges to the true value, whereas the classic estimator (dashed lines) does not. As a function of number of repeats n (Fig. 6C), for lower rNS (0 and 0.25), the split-trial estimator converges more quickly to the true value with increasing n (compare cyan and blue solid traces to dashed). For higher rNS, inflation by noise correlation balances out attenuation by variability, thus r̂NSis closer to the true value. The effect of increasing SNR (Fig. 6D) is largely equivalent to that of increasing n because both shrink the effect of trial-to-trial variability. At all levels of correlation (separate traces), the estimated correlation between signal and noise correlation decrease as SNR increases (solid lines). Using r̂split (dashed lines) there is no longer a decrease, instead there is a rise with SNR.

In summary, split-trial estimation of signal correlation removes the upward bias of r̂NS that is caused by nonzero values of rn and is most severe for low n and low SNR. Surprisingly, increasing the number of stimuli (using higher m) can increase the spurious, upward bias by reducing sampling variability of r̂s and r̂n, which would otherwise have counteracted the bias.

Figure 7.
Figure 7.

Demonstration of attenuation of signal correlation by trial-to-trial variability and its correction by rER2 in MT data from Zohary et al. (1994). For each neuron, responses from odd trials are correlated with those from even trials, so the theoretical true signal correlation is 1. A, Naive r2 estimate for 81 area MT neurons, 8 stimulus conditions, and 10 repeats in total, thus 5 repeats for the estimate of r2 across odd-even trials. B, Same as A except using estimator r̂ER2 and 90% confidence intervals shown as gray vertical lines.

Demonstration of confounds in neural data

We have demonstrated the confounds of signal correlation both theoretically and in simulation. Here we demonstrate these confounds in neural data. We first examine the downward bias of signal correlation caused by trial-to-trial variability and then the inflation of r̂NS.

To demonstrate the effect of trial-to-trial variability in attenuating r̂2 and the efficacy of r̂ER2 to correct this bias, we estimate the correlation between tuning curves computed from the odd stimulus repeats and those computed from the even repeats for the same neuron. In this case, the signal correlation is 1 by definition because the expected value of the response to each stimulus is the same across repeats. In Figure 7A we plot r̂2 between the even-repeat and odd-repeat direction tuning curves for 81 MT neurons as a function of SNR. The positive relationship (Spearman's rank-order r = 0.91, p ≪0.001) is consistent with the theoretical curves in Figure 3 (left column, where rn = 0) where the values of the blue traces at rs2=1 increase with SNR (moving down the column). Thus, although the underlying signal correlation is 1, the classic estimator is well below 1 for the many MT neurons that have low SNR. When r̂ER2 is applied to the same responses, it on average corrects for the bias (Fig. 7B); the points now cluster around the horizontal line at 1 (Spearman's rank-order r = −0.16, p = 0.16, two-tailed t test). In summary, for units with low SNR in this dataset, the signal correlation is underestimated, and the estimator r̂ER2 corrects this bias.

Figure 8.
Figure 8.

Demonstration of inflation of r̂NS by noise correlation in neural data. Each point on a trace is an average of 1000 simulations where 2 × n trials (x axis) are randomly sampled without replacement from the original data. In the split-trial simulations (dashed), signal correlation is estimated by correlating the odd trials from the first neuron to the even trials of the second then vice versa, taking an average of the two estimates, and finally averaging across all simulations. Same trial estimates (solid trace) are similar but even and odd trials from the first neuron are respectively correlated to the even and odd trials from the second neuron. For both, noise correlation is estimated using all n randomly sampled trials. A, Simulations from MT data. B, Simulations from Allen Brain Observatory VISp data (http://observatory.brain-map.org/visualcoding/).

Next, we demonstrate the effect of noise correlation inflating r̂NS in neural data and its relation to the number of repeats, n. To reveal the influence of noise correlation on r̂NS, we measure r̂s using tuning curves computed from nonsimultaneous trials (split-trial condition; Fig. 8) and compare this to the same-trial condition where simultaneous trials are used. In the split-trial condition, noise correlation effectively has no influence because the trials are not simultaneous. Estimated noise correlation, r̂n, was computed across all trials. Figure 8A shows that same-trial estimates of r̂NS (solid trace) are consistently higher than split-trial estimates (dashed trace). The estimates converge as n increases (although the change in n is limited here), consistent with the simulations in Figure 6C. The inflation of r̂NS is modest relative to the strength of the correlation itself; nevertheless, this demonstrates that the effect occurs in neural data even when the SNR is relatively high. We next consider a lower SNR dataset.

In a previous comparison of SNRs across datasets (Pospisil and Bair, 2020), we observed that the SNR of the Allen Institute Neuropixel recordings from area VISp of mouse visual cortex was substantially lower than that of the MT data analyzed here (respective median SNRs = 0.16 and 4.0). We repeated the above analysis with this lower SNR dataset and again found that r̂NS for the same-trial condition was higher than that for the split-trial condition (Fig. 8B). Consistent with simulations in Figure 6D, the difference in the split- versus same-trial conditions was much greater in the lower SNR case. Thus, we have shown in neural data that the influence of noise correlation on r̂NS, when it is not accounted for by using nonsimultaneous trials to compute r̂s, can vary greatly depending on SNR. Comparing the two methods of estimation as demonstrated here can help experimentalists assess the magnitude of this confound and determine a range that is likely to include the true value. Some prior work has found little noise correlation among neurons (Ecker et al., 2010); in cases where noise correlation is zero, splitting trials would not be needed.

See below, Discussion, in which we cover a variety of examples where controlling for SNR could potentially make a difference in the scientific conclusions of published studies.

Novel relationship between tuning strength and signal correlation in area MT

Applying the improved estimators described above, we reanalyzed direction-tuning data from a study of signal and noise correlation in simultaneously recorded pairs of well-isolated MT neurons in awake macaques (Zohary et al., 1994; Bair et al., 2001). The dataset from these studies showed wide variation in r̂s (Bair et al., 2001, their Fig. 2A), indicating a diversity of tuning across pairs of neurons recorded from a single electrode tip. This was the case despite the expectation that nearby MT neurons should have similar direction tuning (Albright et al., 1984; Born and Bradley, 2005). We used the estimators developed in the previous section to understand what factors might have contributed to the variation in signal correlation.

Low r̂s between a pair of neurons could happen in the following two completely distinct ways: (1) trial-to-trial noise in low SNR neurons attenuates signal correlation by corrupting the ability to measure the true tuning curve (as demonstrated in Fig. 7A), or (2) there is systematic deviation between the true underlying tuning curves. With the naive estimator of signal correlation, these two hypotheses cannot be disambiguated, but with the corrected estimator r̂ER2 they can (Fig. 7B shows the ability of our estimator to remove what might have been considered an inevitable trend). Below we apply our estimator to neuron pairs with SNR high enough to accurately infer the correlation between the true tuning curves to show that the second option, systematic deviation, is common. Surprisingly, we find it is specifically neuron pairs with lower SNR, although still robustly modulated, that have strong systematic differences between their tuning curves, a finding that would be confounded had the naive estimate of signal correlation been used.

We begin by giving insight into our results on the basis of tuning curves for representative pairs of neurons. The direction-tuning curves for a pair of MT neurons are plotted in the top left panel A of Figure 9. Both neurons were strongly modulated by the stimulus (geometric mean SNR = 18.66) and have the same preferred direction (orange and blue curves both peak near 270∘) and similar classic single-cycle sinusoidal tuning profiles. The similarity in tuning is reflected unambiguously by the high r̂ER2=0.97 with a narrow confidence interval (0.9, 0.97). This pair and the other examples in Figure 9A–D demonstrate a trend whereby strongly modulated pairs of neurons (joint SNR > 15 in all cases) tended to also have a high signal correlation and sinusoidal tuning curves. On the other hand, pairs that had low joint SNR (Fig. 9F–G, geometric mean SNR <6 in all cases) tended to have low r̂ER2 values and each included at least one neuron that deviated substantially from sinusoidal tuning (blue curves indicate the more weakly tuned neuron). Importantly, the low signal correlation is not a trivial result of noisy data; r̂ER2 takes into account the low SNR, and the confidence intervals are narrow.

Figure 9.
Figure 9.

Examples of estimated direction tuning curves for pairs of MT neurons (error bars show SEM; Zohary et al., 1994). A–D, Examples of pairs of tuning curves that have high joint SNR (geometric mean), large r̂ER2 and visibly similar shapes. E–H, Example pairs of tuning curves with relatively low signal correlation, low SNR, and visibly dissimilar shapes. The tuning curve for the more weakly modulated neuron in each pair is shown in blue.

To quantitatively test whether more strongly stimulus-modulated pairs tended to have more closely matched tuning curves, we examined the relationship between SNR and rER2 across the population of MT pairs. We plotted estimates of these values against each other (Fig. 10) for pairs with narrow confidence intervals (CI < 0.25) and found a positive relationship between joint SNR and signal correlation (Spearman's rank-order r = 0.40, p = 0.005, two-tailed t test). Thus, when one or both of the neurons in a pair are not well driven by the stimuli, the tuning curves of the pair tend to be less correlated. This relationship would have been confounded had we used the naive r̂2, because noise would trivially reduce correlation, but the corrected estimator, r̂ER2, gives confidence that the relationship is intrinsic to the pattern of tuning of neurons simultaneously recorded on a single electrode. Notable exceptions to this relationship, for example, the blue point closest to top left (Fig. 10), include pairs that are poorly tuned for direction but still have very similar tuning curves. Such exceptions demonstrate that the relationship is not inevitable, that is, that low SNR neurons could have had high signal correlation. Thus, consistent with the example pairs in Figure 9, the neurons that are not well tuned for the moving dot stimuli produce low signal correlation values.

Figure 10.
Figure 10.

A positive correlation between signal correlation and SNR. Our unbiased estimate, r̂ER2, of signal correlation is plotted against the geometric mean of SNR for each pair of neurons having CI length for r̂ER2 <0.25 (n = 48). Color indicates animal ID.

We also used r̂ERsplit2, but its estimates were nearly identical to those in Figure 10 (Pearson's r = 1.0), and the increase in CI length excluded many pairs by our criterion (leaving only 29 of the original 48). The relationship between r̂ERsplit2 and SNR had Spearman's r = 0.4, but the p value, p = 0.027, was higher than that of Figure 10, p = 0.005, because of the lower number of pairs.

To demonstrate that the SNR-rs relationship is not a trivial result of trial-to-trial variability, we conducted stochastic simulations of a concrete population of pairs of nearby cortical neurons. First, consider the simple scenario in which all MT neurons have sinusoidal direction-tuning curves, and only the amplitude and phase (i.e., preferred direction) can vary. Here we do not induce noise correlation. Figure 11A (left) shows example tuning curves drawn from a uniform distribution of tuning curve amplitudes and preferred directions that are chosen independently, with preferred direction limited to a narrow range (180°±45°), consistent with cortical columnar organization (Albright et al., 1984). We simulated noisy tuning curves for 100 pairs of neurons picked randomly from the parameter distribution where the noise arises from Poisson firing statistics for n = 10 repeats (see above, Materials and Methods, Stochastic model of neuronal responses). For such a simulated population, the observed relationship between the naive r̂s2 estimate and SNR is shown in Figure 11B (left), and the distribution of Spearman's rank-order r values across 200 such populations (Fig. 11D, left, orange histogram) reveals a substantial spurious positive correlation (mean 0.31) caused by trial-to-trial variability reducing the naive r̂s when SNR is low (similar to Fig. 7A). This spurious relationship disappears as expected when we use r̂ER2, both from the example scatter (compare Fig. 11C with B, left) and from the population (Fig. 11D, left, blue histogram), because, by construction, tuning curve preferred direction differences are independent of SNR. This is similar to how our estimator removed the trend from Figure 7A to produce Figure 7B. Thus, this scenario, where tuning curves of all amplitudes are sinusoidal, is not consistent with our finding of an SNR-rs relationship, and it highlights the importance of accounting for bias in r̂s.

Figure 11.
Figure 11.

Simulation of MT neuronal data under different assumptions of tuning statistics across a population. A, Plots of example tuning curves. Parameters of tuning curves are phase, which sets preferred direction, and amplitude. The distribution of phase is uniform between 135° and 225°, amplitude is uniformly distributed from 1 to 30 spikes, and the distributions are independent. Spike counts are drawn from Poisson distributions with means set by tuning curves. In right column, tuning curves are not smooth because noise is added. B, Results of simulation of an experiment where there are 8 directions of motion, 10 repeats, and 100 pairs of neurons. Spearman's ranked correlation is measured between the geometric mean of SNR and r̂2 (naive estimator). C, Identical to B, but corrected r̂ER2 used instead of naive r̂2. D, Distribution of correlations between SNR and either r̂2 (orange) or r̂ER2 (blue) across 200 simulated experiments.

In a second scenario, we made a simple addition to reflect our observation that real neurons in area MT with lower dynamic range often have more irregular tuning curves (e.g., Fig. 9E–H, blue curves). Specifically, we added constant SD Gaussian noise to each simulated tuning curve (thus we modified the tuning curve shape), independent of the amplitude or phase of the tuning curves. An example of a pair of tuning curves with moderate SNR (Fig. 11A, right) reveals that the underlying true tuning curves are no longer perfectly sinusoidal. Crucially, the relative deviation from sinusoidal is larger at lower SNR because the average magnitude of the deforming component is constant. Under this scenario, the correlation between SNR and signal correlation is positive when measured using our corrected estimate, r̂ER2 (Fig. 11D, right, blue histogram). This simple model is sufficient to recapitulate the SNR-rs relationship observed for the MT data.

If this simple model is correct, it predicts that sinusoidal fits to tuning curves should improve with SNR. If we used the naive estimator, r̂2, of model fit, an increasing relationship could be explained in the following two ways: (1) MT neurons with low and high SNR have equally sinusoidal tuning curves, but correlation to the sinusoidal model is artifactually attenuated by trial-to-trial variability, or (2) low SNR neurons, unlike high SNR neurons, tend not to be well described by a sinusoid. We fit a sinusoidal model across all MT neurons and estimated the variance explained with r̂ER2, removing the trivial dependence of the r value on SNR, thus removing (1) as a possible explanation. We indeed found that tuning curves with higher SNR were better fit by sinusoids (Fig. 12A, red circles). In particular, for SNR > 10, nearly all neurons had r̂ER2>0.8, whereas for SNR < 10, about half of neurons had r̂ER2<0.80. As a control, we plot the results of the same analysis but performed on the stochastic responses of units drawn from the simulation in Figure 11A (left column), where tuning curves are truly sinusoidal at all SNR levels, varying only with respect to phase and amplitude. We find that these curves are consistently reported as having a nearly perfect correlation to the sinusoidal tuning curve model regardless of SNR (Fig. 12A, black circles). Thus, deviation of these MT units from the sinusoidal model is not an artifact of trial-to-trial variability but a result of systematic differences from the sinusoidal model. Furthermore, these systematic differences are more pronounced for neurons that are not as well tuned for the stimuli (i.e., have lower SNR).

Figure 12.
Figure 12.

Relationship between SNR and sinusoid model fit quality. Color keys give Spearman's rank-order r and p value of two-tailed t test. A, Fit quality (r̂ER2) of MT tuning curve to sinusoid, acos(θ)+bsin(θ)+c, where θ is the orientation of motion, as a function of SNR. For all 81 pairs of neurons (162 neurons total) we estimated the fraction of variance in the tuning curve explained by the fit of a sinusoid. As a control, we also fit the sinusoid model to the simulation of neurons with sinusoidal tuning curves from Figure 11A, left. B, Same analysis as A except for V1 orientation tuning curves in response to drifting gratings. Here, the sinusoidal model tuning curve frequency is doubled to account for the difference between orientation versus direction tuning.

Here we have uncovered a new relationship between SNR and signal correlation in area MT. If the relationship had been estimated using classic measures, it would be confounded. We proposed a simple model sufficient to explain this trend, and we demonstrated in simulation that the corrected estimator both avoids confounds and reveals the relationship between SNR and signal correlation. We provided evidence that the data were consistent with our model by showing that low SNR MT neurons were less well fit by a sinusoid. To test whether this generalizes beyond this one dataset, we conducted a similar analysis of V1 data (Kohn laboratory, (http://dx.doi.org/10.6080/K0NC5Z4X) for three monkeys, but for orientation tuning, which is more common in V1, rather than for direction tuning (see Methods). Figure 12B shows that a similar trend held for all three animals; recordings with higher SNR tended to be better fit by sinusoidal tuning curves after factoring out the influence of noise.

Discussion

We have examined two biases in the estimation of signal correlation—attenuation by trial-to-trial variability and a bias toward the noise correlation. To address these biases, we measured signal correlation across trials that were not simultaneously recorded (r̂split) and introduced r̂ERsplit2, which uses an estimate of trial-to-trial variability to correct for the correlation-attenuating effect of noise. We also examined how the often reported positive relationship between signal and noise correlation, r̂NS, could arise artifactually because of the bias of r̂s toward r̂n. We found that this positive bias was pronounced in cases of low SNR and few repeats, and we showed that the estimator r̂split prevented this bias. We demonstrated that our estimator works for neuronal and simulated data. Furthermore, to immediately demonstrate the importance of our estimator to experimental neurophysiological investigations, we conducted a reanalysis of neuronal data in which the ability of our estimator r̂ER2 to correct for the confound of trial-to-trial variability allowed us to uncover a novel positive relationship between SNR and signal correlation within cortical area MT. Our results and supporting stochastic simulations indicate that MT neurons that are not driven strongly by uniform frontoparallel motion tend to have more diverse tuning curve shapes. We showed that a similar finding holds for orientation tuning curves in V1 neurons.

Practical advantages and further applications of r̂ER-split2

Neurophysiologists can, and have, tempered the biases examined here by averaging across stimulus repeats, but this never wholly removes the biases and it restricts the number of stimuli (m) that can be presented during an experiment. A major practical benefit of using the estimator r̂ERsplit2 is that it provides more flexibility in choosing m and n. Specifically, for moderate SNR, high n is not needed to accurately estimate signal correlation, and thus the neurophysiologist can show more stimuli with fewer repeats. Additionally, whereas low SNR neurons might be excluded to address the confounds that we have described, using our corrected estimator, they can be included and reduce sampling bias. When SNR is high, the naive estimator will give similar results to r̂ERsplit2, but until r̂s is compared with r̂ERsplit2, any conclusions about signal correlation will need to be couched in the untested assumption that neither trial-to-trial variability nor noise correlation are confounding the results. Thus, our estimator is valuable even in the case where SNR is high.

As the simultaneous recording of large neuronal populations becomes more common, accounting for the effects of correlated and independent trial-to-trial variability is increasingly important for analyses that seek to understand similarities and differences in tuning across neuronal populations. Signal correlation is the basis of several multivariate data analysis methods. Examples, often used in the neurosciences, include canonical correlation analysis (for review see Zhuang et al., 2020); representational similarity analysis (for review see Kriegeskorte and Kievit, 2013); and principal component analysis. Further work should aim to quantify the effect of correlated noise on the estimation of these quantities and to determine whether applying our estimator, r̂ERsplit2, can improve inference.

Prior work

The estimator r̂ERsplit2 is built on prior work, it derives from a line of research in the neurosciences that sought to account for trial-to-trial variability in the estimation of the fit of a model to noisy data. In Pospisil and Bair (2020); we review and build on that work to provide the least biased estimator of r2, and in Pospisil and Bair (2021) we extend our estimator to the case of correlating two neuronal tuning curves with independent trial-to-trial variability. In doing so, we reviewed and followed a line of research beginning with Spearman (1904), but took an approach developed by neuroscientists, to ultimately derive the least biased estimator. Here, in the current work, we have extended the latter estimator to account for the case where neurons have correlated trial-to-trial variability by splitting trials. We note variance of the estimator can be decreased by averaging over all possible trial splits.

To our knowledge, we are the first to show a relationship between SNR and signal correlation in vivo, controlling for trial-to-trial variability. The confounds we have discussed have rarely been addressed in the neuroscience literature.

Reviewing prior experimental studies, we found the most common potential confound involves comparing signal correlation across conditions where SNR could plausibly differ, which is often the case when one stimulus set drives weaker tuning curve modulation than another. For example Vinje and Gallant (2000) report less signal correlation in macaque V1 for full field stimuli compared with stimuli restricted to the RF; however, the suppressive surround could have driven down the dynamic range of responses (Angelucci et al., 2017). Gawne et al. (1996) reported higher signal correlation in V1 for bars than for Walsh patterns; however, they do not control for differences in dynamic range for the two stimuli. Averbeck and Lee (2003); study macaque supplementary motor area, reporting signal correlation increases with spike counting bin width; however, SNR would also likely increase with bin width. Solarana et al. (2019) examined the effects of sensory deprivation in mouse primary auditory cortex, finding a decrease in signal correlation with deprivation; however, they did not determine whether this could result from differences in dynamic range. These are just a few of the relevant examples where there is great potential value in controlling for trial-to-trial variability when studying signal correlation.

The bias of signal correlation by trial-to-trial variability that we have examined tends to be downward, with an appreciable upward bias only for low SNR and atypically high noise correlation (e.g., Fig. 3, top right, blue trace). On the other hand, the bias of r̂NS is more complex. For typical ranges of values seen in neural data, we found the bias can plausibly be upward or downward depending on a variety of factors. The upward bias could either inflate or create a relationship between signal and noise correlation and remains largely unaddressed (Lee et al., 1998; Bair et al., 2001; Averbeck and Lee, 2003; Kohn and Smith, 2005; Cohen and Maunsell, 2009; Ecker et al., 2010; Martin and Schröder, 2013; Smith and Sommer, 2013; Ecker et al., 2014). In the case of Bair et al., (2001) we found a mild inflation; however, the upward confound could dominate in cases where rNS is low, m is high, and SNR is low. Future work might address the downward bias of r̂NS (caused by variability in the separate estimates of rn and rs).

SNR-signal correlation relationship

We found a positive relationship between SNR and signal correlation in area MT for the robustly encoded stimulus dimension of motion direction. This raises the possibility of a potentially ubiquitous connection between signal correlation and stimulus choice. Specifically, nearby neurons robustly encoding the same stimulus dimensions would tend to have similar tuning along those dimensions, and those that weakly encode those stimulus dimensions would not. A consequence is that signal correlation for one set of stimuli may not generalize to other sets. Although variation in noise correlation as a function of the stimulus set has been well studied (Cohen and Kohn, 2011), variation in signal correlation, unconfounded by trial-to-trial variability, has not been explored.

We modeled this relationship as the result of fixed-amplitude noise added to tuning curves of varying amplitude. We do not specify the source of this tuning curve noise, but future theoretical work could focus on its relationship to the maintenance of synaptic connectivity (Lin and Koleske, 2010) or ongoing plasticity in response to variable inputs (Rokni et al., 2007; Duffy et al., 2019).

In terms of sensory encoding, our results can be interpreted with respect to the dimensionality of the population representation of stimuli. Our MT data suggests that tuning of neurons with higher SNR (more strongly modulated) can be well captured by a low dimensional model, whereas lower SNR (poorly modulated) neurons cannot. Specifically, strongly modulated MT neurons were well fit by a sinusoidal model and thus were constrained to the two-dimensional (2D) plane defined by amplitude and phase, whereas less modulated neurons diverged from this plane (Fig. 12). This has potential consequences for downstream computation. Consider a downstream neuron building a new tuning curve as a linear function of these inputs; if inputs were only high SNR neurons in the 2D plane, then the resulting tuning of the neuron would remain in the 2D plane. However, more complex tuning would require lower SNR inputs with more diverse tuning curves, and achieving high SNR for this would require combining more of the low SNR inputs. Assuming some cost to synapse formation, the steeper the relation between SNR and signal correlation, the greater the cost to increase dimensionality. Low- versus high-dimensional representations have different advantages. In low-dimensional representations, functions can be learned with fewer samples, and estimation of underlying input drive is more resistant to noise. Whereas high-dimensional representations can be more flexible in the functions they compute. Further research is required to determine whether the form of dimensionality regularization for a given cortical region implied by a particular SNR-rs relationship has normative advantages for cortical encoding.

Footnotes

  • This work was supported by a National Science Foundation Graduate Research Fellowship DGE-1256082 (D.A.P.), National Institutes of Health Grant NEI R01-EY02999, and National Institutes of Health Grant NEI R01-EY027023 (W.B.). We thank Adam Kohn for reading and providing comments on the manuscript. We thank Ehud Zohary and William T. Newsome for sharing data and making it publicly available. We thank Greg Horwitz, Anitha Pasupathy, and Matthew Farrell for helpful discussions.

  • The authors declare no competing financial interests.

  • Correspondence should be addressed to Dean A. Pospisil at deanp3{at}uw.edu

References

  1. ↵
    2. Adolph SC,
    3. Hardin JS
    (2007) Estimating phenotypic correlations: correcting for bias due to intraindividual variability. Funct Ecology 21:178–184. doi:10.1111/j.1365-2435.2006.01209.x
    OpenUrlCrossRef
  2. ↵
    2. Albright TD,
    3. Desimone R,
    4. Gross CG
    (1984) Columnar organization of directionally selective cells in visual area MT of the macaque. J Neurophysiol 51:16–31. doi:10.1152/jn.1984.51.1.16
    OpenUrlCrossRefPubMed
  3. ↵
    2. Angelucci A,
    3. Bijanzadeh M,
    4. Nurminen L,
    5. Federer F,
    6. Merlin S,
    7. Bressloff PC
    (2017) Circuits and mechanisms for surround modulation in visual cortex. Annu Rev Neurosci 40:425–451. doi:10.1146/annurev-neuro-072116-031418
    OpenUrlCrossRefPubMed
  4. ↵
    2. Averbeck BB,
    3. Lee D
    (2003) Neural noise and movement-related codes in the macaque supplementary motor area. J Neurosci 23:7630–7641. doi:10.1523/JNEUROSCI.23-20-07630.2003
    OpenUrlAbstract/FREE Full Text
  5. ↵
    2. Averbeck BB,
    3. Latham PE,
    4. Pouget A
    (2006) Neural correlations, population coding and computation. Nat Rev Neurosci 7:358–366. doi:10.1038/nrn1888
    OpenUrlCrossRefPubMed
  6. ↵
    2. Bair W,
    3. Zohary E,
    4. Newsome WT
    (2001) Correlated firing in macaque visual area MT: time scales and relationship to behavior. J Neurosci 21:1676–1697. doi:10.1523/JNEUROSCI.21-05-01676.2001
    OpenUrlAbstract/FREE Full Text
  7. ↵
    2. Beaton GH,
    3. Milner J,
    4. Corey P,
    5. McGuire V,
    6. Cousins M,
    7. Stewart E,
    8. de Ramos M,
    9. Hewitt D,
    10. Grambsch PV,
    11. Kassim N,
    12. Little JA
    (1979) Sources of variance in 24-hour dietary recall data: implications for nutrition study design and interpretation. Am J Clin Nutr 32:2546–2559. doi:10.1093/ajcn/32.12.2546
    OpenUrlFREE Full Text
  8. ↵
    2. Born RT,
    3. Bradley DC
    (2005) Structure and function of visual area MT. Annu Rev Neurosci 28:157–189. doi:10.1146/annurev.neuro.26.041002.131052
    OpenUrlCrossRefPubMed
  9. ↵
    2. Box GEP,
    3. Cox DR
    (1964) An Analysis of Transformations. J R Stat Soc Series B Stat Methodol 26:211–252.
    OpenUrl
  10. ↵
    2. Cohen MR,
    3. Kohn A
    (2011) Measuring and interpreting neuronal correlations. Nat Neurosci 14:811–819. doi:10.1038/nn.2842
    OpenUrlCrossRefPubMed
  11. ↵
    2. Cohen MR,
    3. Maunsell JHR
    (2009) Attention improves performance primarily by reducing interneuronal correlations. Nat Neurosci 12:1594–1601. doi:10.1038/nn.2439
    OpenUrlCrossRefPubMed
  12. ↵
    2. Duffy A,
    3. Abe E,
    4. Perkel DJ,
    5. Fairhall AL
    (2019) Variation in sequence dynamics improves maintenance of stereotyped behavior in an example from bird song. Proc Natl Acad Sci U S A 116:9592–9597. doi:10.1073/pnas.1815910116
    OpenUrlAbstract/FREE Full Text
  13. ↵
    2. Ecker AS,
    3. Berens P,
    4. Keliris GA,
    5. Bethge M,
    6. Logothetis NK,
    7. Tolias AS
    (2010) Decorrelated neuronal firing in cortical microcircuits. Science 327:584–587. doi:10.1126/science.1179867
    OpenUrlAbstract/FREE Full Text
  14. ↵
    2. Ecker AS,
    3. Berens P,
    4. Cotton RJ,
    5. Subramaniyan M,
    6. Denfield GH,
    7. Cadwell CR,
    8. Smirnakis SM,
    9. Bethge M,
    10. Tolias AS
    (2014) State dependence of noise correlations in macaque primary visual cortex. Neuron 82:235–248. doi:10.1016/j.neuron.2014.02.006
    OpenUrlCrossRefPubMed
  15. ↵
    2. Gawne TJ,
    3. Richmond BJ
    (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? J Neurosci 13:2758–2771. doi:10.1523/JNEUROSCI.13-07-02758.1993
    OpenUrlAbstract/FREE Full Text
  16. ↵
    2. Gawne TJ,
    3. Kjaer TW,
    4. Hertz JA,
    5. Richmond BJ
    (1996) Adjacent Visual Cortical Complex Cells Share About 20% of Their Stimulus-Related Information. Cereb Cortex 6:482–489.
    OpenUrlCrossRefPubMed
  17. ↵
    2. Kiani R,
    3. Cueva CJ,
    4. Reppas JB,
    5. Peixoto D,
    6. Ryu SI,
    7. Newsome WT
    (2015) Natural grouping of neural responses reveals spatially segregated clusters in prearcuate cortex. Neuron 85:1359–1373.
    OpenUrl
  18. ↵
    2. Kohn A,
    3. Smith MA
    (2005) Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J Neurosci 25:3661–3673. doi:10.1523/JNEUROSCI.5106-04.2005
    OpenUrlAbstract/FREE Full Text
  19. ↵
    2. Kriegeskorte N,
    3. Kievit RA
    (2013) Representational geometry: integrating cognition, computation, and the brain. Trends Cogn Sci 17:401–412. doi:10.1016/j.tics.2013.06.007
    OpenUrlCrossRefPubMed
  20. ↵
    2. Lin YC,
    3. Koleske AJ
    (2010) Mechanisms of synapse and dendrite maintenance and their disruption in psychiatric and neurodegenerative disorders. Annu Rev Neurosci 33:349–378. doi:10.1146/annurev-neuro-060909-153204
    OpenUrlCrossRefPubMed
  21. ↵
    2. Lee D,
    3. Port NL,
    4. Kruse W,
    5. Georgopoulos AP
    (1998) Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. J Neurosci 18:1161–1170. doi:10.1523/JNEUROSCI.18-03-01161.1998
    OpenUrlAbstract/FREE Full Text
  22. ↵
    2. Lyamzin DR,
    3. Barnes SJ,
    4. Donato R,
    5. Garcia-Lazaro JA,
    6. Keck T,
    7. Lesica NA
    (2015) Nonlinear transfer of signal and noise correlations in cortical networks. J Neurosci 35:8065–8080. doi:10.1523/JNEUROSCI.4738-14.2015
    OpenUrlAbstract/FREE Full Text
  23. ↵
    2. Martin KAC,
    3. Schröder S
    (2013) Functional heterogeneity in neighboring neurons of cat primary visual cortex in response to both artificial and natural stimuli. J Neurosci 33:7325–7344. doi:10.1523/JNEUROSCI.4071-12.2013
    OpenUrlAbstract/FREE Full Text
  24. ↵
    2. Maunsell JHR,
    3. Van Essen DC
    (1983) Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. J Neurophysiol 49:1127–1147. doi:10.1152/jn.1983.49.5.1127
    OpenUrlCrossRefPubMed
  25. ↵
    2. Moreno-Bote R,
    3. Beck J,
    4. Kanitscheider I,
    5. Pitkow X,
    6. Latham P,
    7. Pouget A
    (2014) Information-limiting correlations. Nat Neurosci 17:1410–1417. doi:10.1038/nn.3807
    OpenUrlCrossRefPubMed
  26. ↵
    2. Nandy AS,
    3. Sharpee TO,
    4. Reynolds JH,
    5. Mitchell JF
    (2013) The Fine Structure of Shape Tuning in Area V4. Neuron 78:1102–1115.
    OpenUrl
  27. ↵
    2. Oram MW,
    3. Földiák P,
    4. Perrett DI,
    5. Oram MW,
    6. Sengpiel F
    (1998) The 'Ideal Homunculus': decoding neural population signals. Trends Neurosci 21:259–265. doi:10.1016/S0166-2236(97)01216-2
    OpenUrlCrossRefPubMed
  28. ↵
    2. Panzeri S,
    3. Schultz SR,
    4. Treves A,
    5. Rolls ET
    (1999) Correlations and the encoding of information in the nervous system. Proc Biol Sci 266:1001–1012. doi:10.1098/rspb.1999.0736
    OpenUrlCrossRefPubMed
  29. ↵
    2. Popovkina DV,
    3. Bair W,
    4. Pasupathy A
    (2019) Modeling diverse responses to filled and outline shapes in macaque V4. J Neurophysiol 121:1059–1077.
    OpenUrlCrossRefPubMed
  30. ↵
    2. Pospisil DA,
    3. Bair W
    (2020) The unbiased estimation of the fraction of variance explained by a model. bioRxiv. doi: 10.1101/2020.10.30.361253.
    OpenUrlCrossRef
  31. ↵
    2. Pospisil DA,
    3. Bair W
    (2021) The unbiased estimation of r2 between two sets of noisy neural responses. bioRxiv. doi: 10.1101/2021.03.29.437413. doi:10.1101/2021.03.29.437413
    OpenUrlCrossRef
  32. ↵
    2. Power JD,
    3. Cohen AL,
    4. Nelson SM,
    5. Wig GS,
    6. Barnes KA,
    7. Church JA,
    8. Vogel AC,
    9. Laumann TO,
    10. Miezin FM,
    11. Schlaggar BL,
    12. Petersen SE
    (2011) Functional network organization of the human brain. Neuron 72:665–678.
    OpenUrlCrossRefPubMed
  33. ↵
    2. Rokni U,
    3. Richardson AG,
    4. Bizzi E,
    5. Seung HS
    (2007) Motor learning with unstable neural representations. Neuron 54:653–666. doi:10.1016/j.neuron.2007.04.030
    OpenUrlCrossRefPubMed
  34. ↵
    2. Rosner B,
    3. Willett WC
    (1988) Interval estimates for correlation coefficients corrected for within-person variation: implications for study design and hypothesis testing. Am J Epidemiol 127:377–386. doi:10.1093/oxfordjournals.aje.a114811
    OpenUrlCrossRefPubMed
  35. ↵
    2. Rothschild G,
    3. Nelken I,
    4. Mizrahi A
    (2010) Functional organization and population dynamics in the mouse primary auditory cortex. Nat Neurosci 13:353–360. doi:10.1038/nn.2484
    OpenUrlCrossRefPubMed
  36. ↵
    2. Saccenti E,
    3. Hendriks MHWB,
    4. Smilde AK
    (2020) Corruption of the Pearson correlation coefficient by measurement error and its estimation, bias, and correction under different error models. Sci Rep 10:438. doi:10.1038/s41598-019-57247-4
    OpenUrlCrossRef
  37. ↵
    2. Smith MA,
    3. Sommer MA
    (2013) Spatial and temporal scales of neuronal correlation in visual area V4. J Neurosci 33:5422–5432. doi:10.1523/JNEUROSCI.4782-12.2013
    OpenUrlAbstract/FREE Full Text
  38. ↵
    2. Solarana K,
    3. Liu J,
    4. Bowen Z,
    5. Lee HK,
    6. Kanold PO
    (2019) Temporary visual deprivation causes decorrelation of spatiotemporal population responses in adult mouse auditory cortex. ENeuro 6:ENEURO.0269-19.2019. doi:10.1523/ENEURO.0269-19.2019
    OpenUrlAbstract/FREE Full Text
  39. ↵
    2. Spearman C
    (1904) The proof and measurement of association between two things. Am J Psychol 15:72–101. doi:10.2307/1412159
    OpenUrlCrossRefPubMed
  40. ↵
    2. Thouless RH
    (1939) The effects of errors of measurement on correlation coefficients. Br J Psychol 29:383–403. doi:10.1111/j.2044-8295.1939.tb00927.x
    OpenUrlCrossRef
  41. ↵
    2. Vinje WE,
    3. Gallant JL
    (2000) Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287:1273–1276. doi:10.1126/science.287.5456.1273
    OpenUrlAbstract/FREE Full Text
  42. ↵
    2. Virtanen P,
    3. Gommers R,
    4. Oliphant TE,
    5. Haberland M,
    6. Reddy T,
    7. Cournapeau D,
    8. Burovski E,
    9. Peterson P,
    10. Weckesser W,
    11. Bright J,
    12. van der Walt SJ,
    13. Brett M,
    14. Wilson J,
    15. Millman KJ,
    16. Mayorov N,
    17. Nelson ARJ,
    18. Jones E,
    19. Kern R,
    20. Larson E,
    21. Carey CJ, et al
    . (2020) SciPy 1.0: fundamental algorithms for scientific computing in Python. Nat Methods 17:261–272. doi:10.1038/s41592-019-0686-2
    OpenUrlCrossRefPubMed
  43. ↵
    2. Zeki SM
    (1974) Functional organization of a visual area in the posterior bank of the superior temporal sulcus of the rhesus monkey. J Physiol 236:549–573. doi:10.1113/jphysiol.1974.sp010452
    OpenUrlCrossRefPubMed
    2. Zhuang X,
    3. Yang Z,
    4. Cordes D
    (2020) A technical review of canonical correlation analysis for neuroscience applications. Hum Brain Mapp 41:3807–3833. doi:10.1002/hbm.25090
    OpenUrlCrossRef
  45. ↵
    2. Zohary E,
    3. Shadlen MN,
    4. Newsome WT
    (1994) Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370:140–143. doi:10.1038/370140a0
    OpenUrlCrossRefPubMed
  46. ↵
    2. Zwillinger D,
    3. Kokoska S
    (2000) CRC standard probability and statistics tables and formulae. Boca Raton, FL: Chapman & Hall.
Back to top

In this issue

Email
Print
View Full Page PDF
Citation Tools
Respond to this article
Accounting for Biases in the Estimation of Neuronal Signal Correlation
Dean A. Pospisil, Wyeth Bair
Journal of Neuroscience 30 June 2021, 41 (26) 5638-5651; DOI: 10.1523/JNEUROSCI.2775-20.2021
Twitter logo Facebook logo Mendeley logo

Jump to section

Keywords

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.