Abstract
We examined 66 complex cells in area 17 of cats that were paralyzed and anesthetized with propofol and N2O. We studied changes in ensemble responses for small (<10°) and large (>10°) differences in orientation. Examination of temporal resolution and discharge history revealed advantages in discrimination from both dependent (e.g., synchronization) and independent (e.g., bursting) interspike interval properties. For 27 pairs of neurons, we found that the average cooperation (the advantage gained from the joint activity) was 57.6% for fine discrimination of orientation but <5% for gross discrimination. Dependency (probabilistic quantification of the interaction between the cells) was measured between 29 pairs of neurons while varying orientation. On average, the dependency tuning for orientation was 35.5% narrower than the average firing rate tuning. The changes in dependency around the peak orientation (at which the firing rate remains relatively constant) lead to substantial cooperation that can improve discrimination in this region. The narrow tuning of dependency and the cooperation provide evidence to support a population-encoding scheme that is based on biologically plausible mechanisms and that could account for hyperacuities.
Introduction
The principles by which sensory information is represented in the brain are controversial. One classical viewpoint is Barlow's (1972) cardinal cell theory, in which neurons are considered independent. This would appear to be true in the visual cortex, in which information-theoretical methods have shown that the correlation between pairs of neurons is either slightly redundant (Gawne et al., 1996) or independent (Victor, 2000; Reich et al., 2001). The output of an individual neuron can represent sensory information in the form of both the average firing rate and the temporal structure of individual spike trains (Richmond and Optican, 1987; Victor and Purpura, 1996; de Ruyter van Steveninck et al., 1997). However, synchronous activity between LGN pairs enhances information by as much as 40% (Dan et al., 1998), and the representation of faces appears to be distributed across inferior temporal cortical cells (Rolls et al., 1997a,b), contradicting the convergence and specialization expected with independent neurons.
The idea that information can be represented by synchronous activity distributed across many neurons was proposed by Hebb (1949), who suggested that information could be passed between regions of the brain as spatiotemporal patterns. The difficulties of obtaining a sufficient number of simultaneous recordings and determining the relationship between a pattern and sensory input have impeded the empirical testing of Hebb's idea, but recent improvements in multiunit recording and population analysis techniques have led to supporting evidence (Maldonado and Gerstein, 1996; Nicolelis et al., 1997; Doetsch, 2000;Martignon et al., 2000; Nadasdy, 2000). These studies describe spatiotemporal activity that is related to stimuli, but do not examine directly how information is encoded in the temporal structure of individual spike trains (von der Malsburg, 1981). This issue has been addressed more directly with the analysis of bursts (clusters of spikes with <8 msec intervals) (Cattaneo et al., 1981a,b; DeBusk et al., 1997) and gamma (40–70 Hz) oscillations (Gray et al., 1989). Both are intrinsic properties of neurons (Gray and McCormick, 1996) and have been demonstrated as possible mechanisms by which spatiotemporal patterns of synchronous neural activity are selectively transmitted across brain regions (Gray et al., 1989; DeBusk et al., 1997; Snider et al., 1998).
In this study, we examine what joint aspects of the spike trains from pairs of neurons contribute to orientation discrimination. The method we use (type analysis) (Johnson et al., 2001) makes almost no assumptions about the nature of the neural code and provides a formal comparison of two stimuli with respect to the neural activity. We find that orientation discrimination is most efficient when using a temporal resolution that matches the bursting intervals (DeBusk et al., 1997) and when we consider enough discharge history to include synaptic delays. We find that the greatest cooperative advantage in discrimination is found when examining small differences in orientation (<10°), in which tuning of the dependency between the neurons is significantly narrower than their individual spike rate tuning curves. Expansion of this principle to a broader population could support phenomena (e.g., hyperacuity discrimination) that are not well explained by the integration of single-cell responses.
Materials and Methods
Preparation. Seven adult cats (2.5–4.0 kg) were prepared for electrophysiological recordings in area 17 (recordings were also made for additional experiments not described in this paper). Experimental procedures were performed under the guidelines established by the American Physiological Society and the Animal Care and Use Committee at Vanderbilt University. Each cat was initially injected intramuscularly with 0.5 ml of acepromazine maleate and 0.5 ml of atropine sulfate. Anesthesia was induced with 5% halothane in O2 and maintained with intravenous injection of 0.3 mg · kg−1 · hr−1propofol after cannulating one of the forelimb veins. A second forelimb vein and the trachea were then cannulated. Once the cat was mounted in a stereotaxic device, a small craniotomy (2 × 5 mm) was performed over the area centralis representation (Horsley-Clark coordinates P4–L2). The underlying dura was excised, and once the electrode was positioned, the hole was covered with agar mixed with mammalian Ringer's solution. Melted paraffin was poured over the agar to provide stability.
During recording, paralysis was induced with 6 mg and maintained intravenously with 0.3 mg · kg−1 · hr−1pancuronium bromide (Pavulon). The cats were artificially ventilated with a mixture of N2O:O2:CO2(75:23.5:1.5), and PCO2 was held at 3.9%. Anesthesia and health were maintained by monitoring the electrocardiogram and electroencephalograms and making bolus injections of propofol when necessary. The rectal temperature was maintained at 37.5°C with a servo-controlled heat pad. The nictitating membranes were retracted with 10% phenylephrine hydrochloride, and the pupils were dilated with 1% atropine sulfate. Contact lenses with 4 mm artificial pupils were fitted, and auxiliary lenses were added to render the retina conjugate at a viewing distance of 57 cm with direct ophthalmoscopy.
Stimuli. Initially, bars of light rear-projected onto a large tangent screen were used to characterize receptive field location and properties. Because multiple cells were recorded, the receptive field of the aggregate activity was determined and the activity center was identified. Individual receptive fields could not be distinguished because the spike sorting is performed offline. Stimuli were then generated using the Cambridge Research Systems (Rochester, UK) VSG2/4 controller board and a 21 inch Sony (Tokyo, Japan) Trinitron graphics display with a frame rate of 120 Hz and a mean luminance of 73 cd/m2. The orientation, spatial frequency, temporal frequency, and diameter of drifting sine wave gratings were varied to determine optimal stimulation characteristics for the collective response. Gratings were presented within a circular aperture with a diameter that varied from 4 to 16° (average, 9°). The stimulus size does not necessarily represent individual or even multiple or overlapping classical receptive-field sizes. The grating size was determined only by the maximum summed response of all responding cells to increase the chances of obtaining sufficiently large spike samples for type analysis.
For single electrode experiments, we collected multiunit recordings from 30 to 300 two second stimulus repetitions. We randomly repeated this for variations of orientation of 3, 7, 12, 18, 25, and 33° on both sides of the peak response (maximum combined firing rate of all neurons) or variations of spatial frequency from 0.03, 0.07, 0.12, 0.18, 0.25, and 0.33 cycles per degree on both sides of the peak. We also measured responses to spatially optimal stimuli at contrasts of 10–100% in 10% intervals.
For the multielectrode array experiment, we first used light bars to characterize receptive fields of single units. With these qualitative measurements, we determined that the population of cells could be stimulated with a single 10° sinusoid grating centered with respect to the receptive field of the aggregate activity. We then collected recordings from 84 and 104 stimulus repetitions of 2 sec each while varying orientation from 100 to 190 and 200 to 280°, respectively, at 10° increments and at spatial frequencies of 0.3, 0.5, and 0.7 cycles per degree. We used these data to measure orientation tuning with respect to firing rate and selected 20 pairs of cells with similar preferred orientations to perform type analysis calculations. The preferred orientations of these cells were grouped around two orientations. We collected recordings from 560 and 538 stimulus repetitions of 2 sec each while varying the orientation across 28° around these two orientations at 2° increments.
Data acquisition and spike classification. Recordings of multiunit activity for six cats (44 cells) were made with a single tungsten-in-glass microelectrode (Levick, 1972). The signal was amplified by 5000, band-limited between 300 and 3000 Hz, and sampled at 30 kHz by an AT&T (Allentown, PA) DSP32C digital signal processing board. The threshold for event acceptance was set at 5 SD above or below the mean noise level (Snider et al., 1998). The action potential was stored from 1 msec before to 3 msec after the trigger point (a total of 4 msec, or 120 sampled points).
Classification of cortical spikes is difficult because of amplitude shrinkage during bursting. Our classification procedure of action potentials has been described in detail previously (Snider and Bonds, 1998; Snider et al., 1998). In brief, each waveform is projected as a 120 dimension vector. Each waveform is represented as a point in space, and the waveform space is partitioned into many small clusters using the method of binary tree bisection. Although waveforms can change shape throughout recording, the method is able to combine clusters on the assumption that these changes are gradual. A score is assigned for pairs of clusters based on the individual cluster densities and the density between each cluster. If the clusters are essentially smeared together (as would be expected with the gradual nonstationary waveform), the score will be relatively low. A plot of this score versus the number of clusters can be used to determine a threshold. This plot typically yields a plateau that represents a threshold for reasonably separated clusters.
After separating the waveforms, a small number of samples remained unclassified because they resulted from noise or overlapping waveforms that could not be unambiguously separated. These waveforms typically represented only 1–3% of the data. Because of the long recording times (as long as 12 hr for a single group), the data were broken down into several files for classification. Typically, in a given recording only a pair of neurons was present with a steady response throughout all of the files and the entire recording time. Because neuron firing patterns beyond those of the strongest pair usually represented <2% of all the samples and were not consistently recorded, we limited our analysis to pairs.
Multiunit activity was recorded from an additional cat (22 cells) using the 5 × 5 Utah Intracortical Electrode Array (Bionics, Salt Lake City, UT). The array was inserted to a depth of 0.6 mm using a pneumatic implantation tool (Rousche and Normann, 1992) to minimize tissue damage. The signal on each channel was amplified by 5000 and band-limited between 250 Hz and 7.5 kHz. The threshold for each electrode was set at 3.25× the mean activity, and waveforms were sampled at 30 kHz for 1.5 msec windows. Twenty-two of the electrodes recorded single-unit activity for 30 hr; Bionics Data Acquisition spike classification software was used to remove noise and artifact.
Type analysis. We used the method of type analysis described in detail by Johnson et al. (2001), which allows examination of how neural ensemble responses differ as stimulus features (orientation, spatial frequency, and contrast) are varied. The procedure determines how two population responses vary across time in terms of the ability of an optimal classifier to discriminate them.
Each stimulus repetition is first converted into a sequence of “letters.” The letter is determined by the firing pattern that occurs within a time window (bin). We use a binary alphabet in which each neuron can have a value of 1 or 0, depending on whether a spike occurs within the bin. Each neuron represents a place in the binary representation. For example, if a population of three neurons has the first and third neuron fire within a bin, the letter would be 101 (base 2) or:
Equation 1Once this procedure is complete, each response collected is represented as a sequence of letters across 2 sec ranging from 0 to 2number of neurons − 1 or 0 to 7 for our example. The sequence length is the number of bins determined by dividing 2 sec by the bin width.
Types or probability mass functions are then formed from the repetitions of each stimulus. Essentially, a probability distribution is estimated for each bin across time for each possible letter in the alphabet. Types can then be used from two different stimuli to calculate a “distance,” which provides an estimate of the reduction in classification error when using an optimal classifier. The classification error is proportional to 2−d(t) (whered(t) is the distance at time t). An increase in the distance measure results in an exponential decrease in the classification error.
We use a modified version of the Kullback–Leibler distance described by Johnson et al. (2001) to provide an estimate of the Chernoff distance. The distance (referred to as the resistor average) is the harmonic average of the Kullback–Leibler distance from response 1 to response 2 and from response 2 to response 1. The Kullback–Leibler distance d(p∥q) for bins 1 toB and for K possible letters over Mstimulus repetitions is:
Equation 2
Equation 3The resistor average is
Equation 4The method can be extended to incorporate discharge history into the distance measure by forming conditional types on the patterns that occur in previous bins. The number of previous bins examined is the Markov order of analysis (D previous bins). Conditional types are formed from joint types, which are the probabilities of sequences of letters occurring. The joint types are formed by essentially expanding the alphabet to describe the bin of the letter (alphabet size = 2N×D). The conditional type is equal to the joint type of the sequence of letters from the current bin and all previous bins considered divided by the joint type of the sequence of letters occurring only in the previous bins.
Equation 5The Kullback–Leibler distanced(p∥q) for bins 1to B and for K possible letters over Mstimulus repetitions and a Markov order D is
Equation 6The Markov order D is limited by the available data (M stimulus repetitions) and the population size (N neurons):
Equation 7Because the estimation can depend on discharge history, it could be in error when using a Markov order that is too small. We examined the data to determine the extent of discharge history required for the distance measurement to reach a stable value (i.e., when additional bins did not change the measure). We wanted to use the smallest possible Markov order because of the data limitations of equation (7), but at the same time needed to ensure that the order was sufficiently large to describe the distance accurately.
Because many times a limited data were available, many bins ended up with probabilities of 0 for certain letters. This would result in possible infinite distances in the Kullback–Leibler calculations. To avoid this problem, the Krichevsky–Trofimov estimate (Johnson et al., 2001) was used, which initializes each probability to 0.5. The types are then normalized to compensate for the 0.5 added to the probability of each letter.
Because the Kullback–Leibler distance must always have a positive value, there will tend to be an upward bias in the estimate. The procedure we use to estimate the bias and to provide confidence limits on our measures is the bootstrap method (Efron and Tibshirani, 1993;Johnson et al., 2001). The bootstrap method creates new data sets from the original by randomly selecting samples from the Mrepetitions and allowing for repeats (i.e., the same sample can be chosen for multiple random selections). Distances are calculated for all of these new data sets (we use 200) and averaged, and the bias is obtained by subtracting the original distance measure from this average. The data sets are sorted in ascending order; depending on the confidence limits desired, certain data sets are used to produce these limits (i.e., 5th and 95th percentiles for a 90% confidence interval).
To test for the redundancy or cooperativity of neurons in a population, we form types using the ensemble alphabet (e.g., eight letters for three neurons) and form types for each individual neuron (having only two letters). The procedures for conditional types can then be repeated for all of the measures to include discharge history. The sum of the individual neuron's distances (dindependent) can then be compared with the distance computed from the ensemble alphabet (densemble):
Equation 8When these distances are equal, the neurons can be considered independent. When the ensemble measure is smaller, the neurons have negative synergy and are redundant. When the ensemble measure is larger, the synergy is positive and the neurons are cooperative. The bootstrap method (Efron and Tibshirani, 1993; Johnson et al., 2001) was used on the synergy calculation to produce confidence limits to assess the significance of these differences.
In this article, we will refer to all Kullback–Leibler resistor average distance calculations as the KL distance (Eq. 6) and all percentage synergy calculations as synergy (Eq. 8). Although the KL distance has units of bits, the calculation is not the same as an entropy or mutual information measure.
Functional dependency. The last method we explored fromJohnson et al. (2001) was the measurement of transneural correlation (throughout the article we will call this the dependency). The dependency of neurons in a population is quantified and presented as a distance measure over time. A type is formed under the assumption that the neurons are acting independently (forced-independent type). The first step is to sum the probabilities of all letters in the original type that indicate a particular neuron discharged. This is then repeated for all of the neurons, and the procedure is then repeated for when the neuron does not discharge. Multiplying the proper sequence of discharges and nondischarges then forms the independent type for each letter. For example, with two neurons:
Equation 9
Equation 10
Equation 11
Equation 12
Equation 13
Equation 14
Equation 15
Equation 16This type is compared with the original type (using a zero-order KL distance) to compare how dependent the neurons are and how this varies over time. The method can be used to determine dependency that arises from either connectivity or from shared input. By extending the measure across another dimension (i.e., orientation), we are able to examine the tuning of dependency.
We use dependency with cross-correlation (Aertsen et al., 1989; Snider et al., 1998) for pairs of neurons to assess functional dependency. The single electrode recordings of multiunit activity focus on connectivity between neurons rather than neurons paired by common input, because waveforms correlated around a zero lag time will overlap on a single electrode, and it is usually impractical to identify the individual waveforms. The multielectrode array recordings tend to focus on longer distance interactions because of the electrode spacing (400 μm). Pairs of neurons were classified as strongly synchronized when we found a large peak in the cross-correlogram at 0–4 msec and a width of 1–3 msec, moderately synchronized when we found a broader (3–10 msec) peak just above the baseline in the cross-correlogram, and uncorrelated with no noticeable correlated activity.
We emphasize again that the measures of KL distance and dependency should not be confused with other information measures. The measurements are based partly on information theory (in addition to classification theory), so the measurements are subject to many of the same obstacles that occur in entropy calculations, and we do end up with the unit of bits for both KL distance and dependency. The KL distance and dependency measures should not be confused with each other. The KL distance is a relative measure of the difference between neural responses to two different stimuli and the dependency is an absolute measure of a neural response to a single stimulus.
Results
Temporal resolution
Although type analysis is essentially unconstrained, it does depend on temporal resolution and response history. We first examined the temporal resolution used to bin the responses. Recordings were made with 30 μsec precision, but responses were represented as a letter determined by neuron activity within a longer time window. Window size was varied from 1 to 8 msec for analysis of responses for discrimination of fine and gross changes of orientation. Twenty-seven pairs of neurons were examined for orientation differences of <10 and >10° from the peak response. The number of stimulus repetitions collected for each pair of neurons ranged from 200 to 560, with a mean of 471.
Johnson et al. (2001) predicted that bin width would be essentially independent of the KL distance when discharge probabilities were relatively small. However, these predictions were made for a single-neuron, zero-Markov-order scenario in which the response difference is a difference in average firing rates. We consistently found larger KL distances when using a 2–5 msec bin width. There are two possible reasons that there is an advantage in discriminating neural responses (an increase in KL distance) in this range of temporal resolutions. First, independent interspike interval (ISI) statistics over the short term (i.e., bursting) carry information about the stimulus feature being discriminated (DeBusk et al., 1997); this information is extracted by filtering the response to emphasize this time frame. Second, dependent ISI statistics (i.e., synchronization) between the pair of neurons carry stimulus-related information that provides the best discrimination within this temporal window, which corresponds to the average delay or variation found in cross-correlation histograms. To identify the relative contribution of each of these mechanisms, we first measure the KL distance versus the bin width of the original responses and then measure the function after shuffling the stimulus repetitions for each neuron to remove spike train dependencies between the neurons.
Figure 1 shows a histogram of the number of pairs of cells versus the bin width that resulted in the largest KL distance for that particular pair of neurons. Figure1A shows the optimal bin widths for fine discrimination of orientation for the original data and the shuffled data (dependencies between the neurons removed). The average optimal resolution for 27 pairs of neurons for fine discrimination before shuffling was 3.1 ± 0.9 msec. The KL distance of these peaks provided an average increase in KL distance 26.4 ± 16.0% higher than the minimum distance seen. After the responses were shuffled, a peak remained, demonstrating that independent ISI properties of the responses (e.g., bursting) provide sizable advantages for orientation discrimination. The peak provided, on average, 40.2 ± 29.3% more KL distance than the minimum and shifted to a finer temporal resolution of 2.8 ± 0.9 msec, suggesting that the dependent advantages are slightly coarser than this resolution. Similar results were found for gross discrimination of orientation (Fig. 1B), except the increases in KL distance were proportionally smaller. The average peak for gross discrimination was at 3.6 ± 1.3 msec, with a 10.2 ± 5.0% increase in KL distance; after shuffling, the peak was at 3.1 ± 1.2 msec, with an average increase of 14.3 ± 9.6%. For both small and large orientation differences, the results suggest an optimal bin width in the range of 2–5 msec, and only 2 of 27 pairs of cells did not show any changes in KL distance with respect to bin width. We also found similar qualitative results when examining four pairs of neurons for spatial frequency discrimination.
The KL distance varies with respect to the bin width used to calculate types. A, The number of pairs of cells at the bin width that resulted in the largest KL distance (best discrimination) for fine differences in orientation of the original response and the response after shuffling out dependencies between the neurons. B, The number of pairs of cells at the optimal bin width for gross differences in orientation before and after shuffling the responses.
Discharge history
We next tested the contribution of discharge history to discrimination. Types were formed using the conditional probabilities that particular letters occur depending on the letters that occur in previous bins. When determining the characteristics of discharge history that will most improve discrimination between responses, the actual temporal duration is of most importance, not the Markov order (i.e., the number of previous bins). However, this becomes more complicated when the temporal resolution has effects on the KL distance calculation, as we have shown in the previous section. Because the conditional probabilities are calculated across more history, the KL distance measure should reach a point at which there is no more to gain. For all pairs of neurons we tested, we found that no additional distance was gained from a Markov order of >1, so we determined that the appropriate Markov order was 1. Figure2 shows a representative example of the change in KL distance as the Markov order (D) is raised from 0 to 3 (the highest D possible for our data; see Eq. 7) with a bin width of 3 msec (discharge history of 0–9 msec). There is no increase in KL distance by increasing the Markov order from 1 to 2 or 3, but a large gain from 0 to 1, so we adopted a measure that includes 1 bin of discharge history into the types. We chose the smallest possible Markov order that still accurately described the distance to minimize the data requirements of Equation 7 and to maximize the reliability of the measurement.
Determination of the relevant discharge history in a response and the Markov order used to form conditional types. The KL distances for a 5° difference in orientation using a bin width of 3 msec and a Markov order of D = 0–3. A conditional type depending on the previous bin (Markov order ofD = 1 or 3 msec of discharge history) nearly doubles the KL distance, but no improvement is found with higher orders.
We also wanted to examine how discharge history contributed to discrimination independent of the temporal resolution (i.e., what duration of previous time maximizes the KL distance and how much distance the previous time contributes). Because the temporal resolution affects the distance measure, it distorts the impact of discharge history on the KL distance. To unconfound this distortion, we calculated the percentage increase in KL distance from a zero-order calculation to a first-order calculation at several bin widths.
The responses of the same 27 pairs of neurons were tested with bin widths from 1 to 6 msec to determine the impact of increasing discharge history from 0 to 1 bin (Markov order of 0–1). As with temporal resolution, both independent and dependent spike train characteristics might lead to advantages in discrimination as a result of the discharge history information. For example, burst-length modulation (DeBusk et al., 1997) will lead to changes in the probabilities of spikes occurring in the discharge history of the individual neurons. Synchronization modulation between synaptically connected neurons will be revealed in discharge histories that include enough time to allow for synaptic delays. To separate the contributions of independent and dependent properties, we again compared the original results with the result after shuffling the stimulus repetitions for each neuron.
Figure 3 shows a histogram of the number of pairs of cells versus the percentage increase found in the KL distance from a Markov order of D = 0 toD = 1 for fine and gross differences in orientation. The histogram suggests that fine differences in orientation result in first-order distances with an average increase in KL distance of 79.1 ± 47.4% at an average bin width (discharge history) of 2.9 ± 1.2 msec. In general, the KL distances to gross differences in orientation appear to have zero-Markov-order statistics, with an average increase from D = 0 to D = 1 of only 12.1 ± 8.5%. For 12 pairs of neurons that showed a shifted peak in the cross-correlogram (described in the section on functional dependency), we also found a decrease in the percentage increase from a zero- to first-order KL distance after shuffling responses (removing dependencies between the neurons), suggesting that the discharge history for these particular pairs of neurons provided some dependent advantages in discrimination. The average percentage increase from dependencies (subtracting the shuffled percentage increase from the original percentage increase) for these 12 pairs of cells was 27.1 ± 17.8% at an average bin width of 2.7 ± 0.9 msec. The remaining 15 pairs of cells resulted in only independent (e.g., bursting) increases in KL distance. For all 27 pairs of cells, 2–5 msec of independent discharge history (examining a total of 4–10 msec) results in an average increase of 60.6 ± 33.4%.
A histogram of the percentage increase in KL distance from a Markov order of D = 0 toD = 1 (i.e., only the current bin to conditional on one previous bin) for fine and gross differences in orientation. The percentage increases are also shown for responses that have been shuffled (to remove dependencies between the neurons) for fine differences in orientation. The removal of dependencies yields a slight shift in the histogram to a smaller percentage, suggesting some dependent (e.g., synchrony) influences in the discharge history. For fine differences in orientation, the KL distance appears to be first-order; for gross differences in orientation, the KL distance generally has a Markov order of 0.
Synergy
To quantify how much the cooperation between the pair of neurons contributes to fine and gross discrimination of orientation, the ensemble KL distances were calculated for the same 27 pairs of neurons mentioned above and compared with the sum of the KL distances for each individual neuron in the pairs.
Figure 4 shows a histogram of the number of cells and the percentage increase from the independent KL distance to the ensemble KL distance (i.e., synergy) for fine and gross differences in orientation. The average amount of synergy produced across all 27 pairs of neurons for fine discrimination of orientation was 57.6 ± 31.9% using an average bin width of 4.6 ± 1.1 msec (Markov order D = 1). For the most part the neurons work independently for the gross discrimination of orientation, with an average synergy of only 2.0 ± 4.4%.
A histogram of the number of cells and the percentage synergy (KL distance not available from the KL distances for each independent cell) for fine and gross differences in orientation. A negative synergy suggests redundancy between the cells, a positive synergy suggests cooperation, and a synergy of 0 suggests that the cells discriminate independently.
The results for gross orientation discriminations agree with previous conclusions that cortical neurons act essentially independently (Gawne et al., 1996; Victor, 2000; Reich et al., 2001). However, we find that when the neurons are strongly or even moderately synchronized, they cooperate for fine discrimination of orientation. We also examined four pairs of neurons for fine spatial frequency differences (<0.1 cycles per degree) and found that to a lesser extent (average, 25%; range, 15–40%), cooperation was also present in these cases. One reason for the significant amount of cooperation across this small segment of the tuning curve for the pair of neurons is that it is in this region that synchronization is highly modulated, whereas the average firing rate (and even burst length) is nearly constant (Snider et al., 1998). We will demonstrate this idea in detail in the section on functional dependency.
Confidence in KL distance and synergy estimations
Here we have reported differences between both theD = 0 and D = 1 Markov-order KL distances and the ensemble and independent KL distances (i.e., cooperation) for fine but not coarse variations in orientation. Confirmation of the statistical reliability of these particular findings is necessary. Unless the actual probability distributions are known for the neural activity of the pairs of cells, the KL distance estimates will tend to be upwardly biased with some uncertainty. We have used the bootstrap method (Efron and Tibshirani, 1993; Johnson et al., 2001) on both KL distance and synergy calculations separately to estimate this bias and to produce confidence intervals for these estimates. For fine differences in orientation, we find a large overlap of the 90% confidence intervals between the KL distance calculated from the sum of the individual cell distances (dindependent) and the KL distance calculated with the ensemble alphabet (densemble) (Fig.5A). However, the debiased estimate for densemble itself falls outside the 90% confidence interval for thedindependent (Fig. 5A), suggesting that there is a difference between the estimates with 90% confidence. We find a similar relationship for all 27 pairs of cells, suggesting that there is cooperation with 90% confidence for fine differences in orientation. We also find in all cases that there is a significant difference between D = 0 andD = 1 KL distance estimates for small orientation differences with 90% confidence.
An examination of the confidence in the KL distance and synergy estimations. A, An example of the bootstrap debiased estimates and 90% confidence intervals of the KL distance using an ensemble alphabet (considering joint activity) and the sum of the independent KL distances. B, An example of the bootstrap estimate and 90% confidence interval for the percentage difference between the KL distances shown inA (i.e., synergy).
Because the KL distance and type analysis makes almost no assumptions about the neural code, possibilities for interactions have no constraints. Therefore, a relatively large amount of data (i.e., the number of stimulus repetitions) is required to reduce the confidence intervals of these results. We have recorded as many as 560 stimulus repetitions, which is at the limit of what can be reasonably expected with our preparation. Therefore, practical recording challenges limit to 90% our confidence in finding the differences between distances. Confidence intervals of 90% were also suggested for applications of type analysis (Johnson et al., 2001). However, we can report lower bounds on synergy with greater confidence, as described in detail below.
Because the idea of cooperative behavior can have substantial implications for information processing in area 17, merely demonstrating a difference between an independent KL distance and an ensemble KL distance does not provide a very strong case for the significance of the cooperation for discrimination, especially when the independent and ensemble KL distance calculations will have different kinds of inherent systematic biases. To ease this problem, we bootstrap on the basis of the synergy calculation rather than on the individual KL distances calculated separately to produce a confidence interval for the synergy itself.
The example for a single pair of cells of the bootstrapped estimate for synergy is shown in Figure 5B (solid line). There is still some uncertainty in the estimate when examining the 90% confidence interval (dotted lines), but the lower bound falls above 50% synergy, suggesting that there is indeed a significant amount of cooperation. When we examine this 90% confidence lower bound across all 27 pairs of cells, we find that the average amount of synergy is at least 40.7 ± 28.1% with 95% confidence. We find that the average amount of synergy is at least 32.6 ± 26.6% with 99% confidence, but this lower bound falls below 0 for four pairs of cells.
Functional dependency
We last examined the dependency (KL distance between observed type and forced-independent type; see Materials and Methods) between neurons to explore the putative substrate of the cooperative activity. The dependency was determined for pairs of neurons; the results were compared with the SD normalized cross-correlation measure (Aertsen et al., 1989; Snider et al., 1998). We tested dependency on 42 pairs of neurons while varying orientation (30 pairs), spatial frequency (10 pairs), and contrast (9 pairs). Two pairs of neurons were tested for all three stimulus parameters, two pairs for contrast and orientation, and one pair for contrast and spatial frequency. The number of stimulus repetitions ranged from 30 to 300, with a mean of 151.
We initially classified the strength of the synchronization between the pairs of neurons using cross-correlation (see Materials and Methods). Eleven of the 42 pairs of neurons were classified as strongly synchronized (15 of the 49 experiments: 10 for orientation, 1 for spatial frequency, and 4 for contrast). Twenty-two pairs of neurons (25 of the 49 experiments: 19 for orientation, 4 for spatial frequency, and 2 for contrast) showed some moderate correlation in the cross-correlogram; the last 9 pairs of neurons (1 for orientation, 5 for spatial frequency, and 3 for contrast) showed no noticeable synchronization. All 13 pairs of neurons that were at least moderately synchronized and recorded from a single electrode had a peak in the cross-correlogram centered at 2–5 msec. Five of the 20 pairs of neurons with at least moderate synchronization recorded from the microelectrode array had a shifted peak in the cross-correlogram (2–5 msec), and the peaks for the remaining 15 pairs were centered around 0 msec.
The dependency was calculated for all pairs using a temporal resolution of 1–10 msec. The resolution that resulted in the largest dependency corresponded with either the lag time or the width of the peak that was found in the cross-correlogram. The average optimal bin width of dependency of both moderately and strongly synchronized pairs of neurons was 3.4 ± 1.3 msec. The average peak in the shifted cross-correlograms was at 2.7 ± 0.9 msec, with an average width of 3.5 ± 0.9 msec; the average width was 4.8 ± 1.4 msec for the cross-correlogram peaks centered around 0 msec. The dependency was divided by the stimulus duration to produce a dependency rate (bits per second) to determine ranges of dependency for strongly synchronized, moderately synchronized, or uncorrelated pairs (determined from the cross-correlogram results). In a few cases, we found that the range of dependency rates for strongly and moderately synchronized neurons could overlap. The reason for the overlap is that the distance measure is an entropy-based measure and the absolute value of the dependency will be influenced by the strength (firing rate) of the response. Strongly synchronized neurons had dependency rates from 0.3 to 4.0 bits/sec, moderately synchronized neurons had dependency rates from 0.1 to 0.9 bit/sec, and weakly synchronized neurons had average rates of <0.1 bit/sec. We defined a cutoff of 0.1 bit/sec to classify a pair of neurons as at least moderately synchronized. This was determined by examining the 90% confidence intervals and observing a lower limit below 0 (no significant dependencies) for responses of weakly synchronized pairs of neurons and the shuffled responses (transneural dependencies removed) of moderately and strongly synchronized pairs of neurons.
The reason we represent the dependency as a rate is to present a value that is independent of the stimulus duration. A closer look at the temporal dynamics of the dependency reveals that it varies in time. Figure 6 shows the dependency of the optimal orientation and progressively nonoptimal orientations as functions of time. The slope of the response indicates variation from the predicted independent probabilities. Horizontal regions in the function indicate that the neurons are firing independently from one another. A comparison of the dependency for the peak response (288°) and 10° (or 15°) from the peak (278 or 273°) shows that from 25 to 40 msec (Fig. 6, inset), the response of the dependency is equal in each case with a very steep slope (>8.0 bits/sec). Immediately after the initial 40 msec, the slope drops to <0.5 bit/sec for 278° and <0.2 bit/sec for 273°, suggesting that a slightly delayed process, possibly inhibition, reduces the synchronous firing.
The temporal dynamics of dependency tuning for orientation. The lines represent the dependency across time at the peak orientation (288°) and two orientations away from the peak (278 and 273°). The inset zooms in at 25–50 msec to show that the dependency rates (i.e., the slope) are initially equal at all orientations. The differences in dependency between the orientations arise only after the reduction in the dependency rate. Thedotted lines represent the 90% confidence intervals.
The finer selectivity of the dependency with respect to orientation occurs only after the delayed reduction. Figure7 is an example of the dependency tuning (solid line) we find for a pair of moderately synchronized neurons as a function of orientation, with the rate tuning for the two neurons superimposed in the background (dotted lines). The dependency tuning is much narrower than the average rate tuning and can potentially support much finer discrimination between orientations around the peak. In the case of orientation, we find, in all 29 cases of highly or moderately synchronized neurons, a very sharp peak of dependency that is on average 35.5 ± 16.9% narrower than tuning for firing rate (half-height bandwidth). We also found similar results for spatial frequency tuning for five pairs of at least moderately synchronized neurons in which the dependency was on average 29.2% (range, 10.0–45.5%) narrower than tuning for the firing rate.
A comparison between the dependency tuning and average firing rate tuning for orientation. Dependency is accumulated over 2 sec to show the dependency tuning for orientation (solid line, with the dashed line representing the 90% confidence interval). The average firing rate tuning for the two neurons is superimposed in the background (fine dotted lines).
Both the refined tuning of dependency relative to average firing rate tuning and the temporal dynamics suggest that the synergy we find is a result of orientation-dependent changes in synchronization. By comparing the slope of the dependency in Figure 6 with the temporal dynamics of synergy in Figure 5B, we see that the fast rise in the synergy is likely a result of the slightly delayed orientation-selective reduction in dependency.
Discussion
Snider et al. (1998) found selective changes in correlated firing for fine changes in orientation that were independent of the firing rate. We hypothesized that these changes in synaptic efficiency would yield information that was available only from the joint firing patterns of cell pairs in cases of fine differences in orientation near the preferred orientation. We have quantitatively described this advantage, as well as others useful for discriminating responses that would not be expected, with a rate code: (1) Discrimination depends on the bin width. (2) Discharge history contributes to discrimination. (3) Cooperation enhances fine discrimination. (4) Dependency tuning is narrower than rate tuning.
ISI characteristics and synchronization
Adrian and Zotterman demonstrated in 1926 that the average spike rate depended on the intensity of sensory stimulation; since then, rate coding has been the most common measurement in neurophysiological studies. At the same time, it is not necessarily the most straightforward strategy that the brain might use for encoding (Hopfield, 1995). The spike train contains multiple distributions of activity at various temporal resolutions independently encoding different aspects of the sensory input (Cattaneo et al., 1981a,b;DeBusk et al., 1997; Victor, 2000). Synaptic properties such as facilitation and depression can take advantage of these ISI characteristics and could multiplex information on a single neuron level (Victor, 2000).
Synaptic properties and ISI characteristics work together to modulate synaptic coupling and synchronization between neurons. The low probability of neurotransmitter release along with the high threshold in the postsynaptic neuron (Creutzfeldt and Ito, 1968) makes it highly unlikely that a single spike will result in a postsynaptic spike. How do two neurons then synchronize within 3–4 msec? They manipulate transmitter release and threshold using four different properties of cortical networks: (1) bursting, (2) divergence and convergence, (3) oscillations, and (4) chaos.
Bursting
Shadlen and Newsome (1994) argued that neuronal organization on time scales of <10 msec is impractical because of synaptic unreliability. If there is an increase in synaptic reliability (attributable to changes in transmitter release probabilities or synaptic redundancy), there is a decrease in information transfer for rate coding but an increase in information transfer for temporal coding (Zador, 1998). Bursting is one way to enhance synaptic reliability (Lisman, 1997; Snider et al., 1998). Cattaneo et al. (1981a,b) suggest that rate-tuning characteristics are actually a result of burst modulation. The efficiency of short spike intervals and bursts relative to connectivity (Usrey et al., 1998) and information (Reinagel et al., 1999; Reich et al., 2000) has been demonstrated throughout the visual pathway. Reich et al. (2000) also found that bursts were disproportionately influential to the receptive field properties of neurons. In addition, bursts have more reliability from trial to trial (Guido and Sherman, 1998; Victor et al., 1998).
Divergence and convergence
Snider et al. (1998) found that the strength of synaptic coupling continued to vary when even the burst length remained constant; they proposed that coincident inputs might be another factor influenced by orientation. The anatomy of the cortex and the increased sensitivity of neurons to coincidence detection over asynchronous integration support a model of selective synchronous transmission (Abeles, 1991). Usrey and Reid (1999) have shown evidence of how synchronous activity is transmitted through the hierarchy of the visual system. Synchrony resulting from divergence could explain the long-distance (0.4–2 mm) correlated activity we found that would not likely be a result of bursting mechanisms.
Oscillations
Gray et al. (1989) demonstrated that long-distance synchronization occurred primarily between neurons that oscillated in the gamma range. Oscillation is an intrinsic property of pyramidal cells (Gray and McCormick, 1996); on the whole, theoretical studies (Ernst et al., 1995, 1998; van Vreeswijk, 1996; Karbowski and Kopell, 2000) have suggested that oscillation might serve as another mechanism for long-distance synchronization and forming synchronized assemblies.
Chaos
Theoretical models have found that disorder and chaotic behavior can lead to synchronization (Hansel, 1996; van Vreeswijk and Sompolinsky, 1996; Karbowski and Kopell, 2000). Although isolated spikes have a low probability of resulting in a postsynaptic spike (Creutzfeldt and Ito, 1968), there are thousands of connections, so they will still be passed on across cortical layers, but with a large amount of variability (Shadlen and Newsome, 1998). Isolated spikes have very broad tuning (Cattaneo et al., 1981a,b), suggesting that they are equally represented across a large population of neurons. Because these spikes activate a large portion of connections with a high amount of variability, they result in chaotic activation of both excitatory and inhibitory connections (Shadlen and Newsome, 1998). The chaos keeps the postsynaptic potential close to threshold but below saturation by carefully balancing the excitation and inhibition (Bell et al., 1995). This produces a highly temporal-sensitive state by reducing the integration time constant (Koch et al., 1996).
Synchronization and cooperation
To have cooperation, the response in multiple neurons must contain information in the form of constructive correlation that is not already represented in the individual responses of the neurons. This can occur when correlation between neurons modulates while the firing rates remain constant. In the auditory cortex, Frostig et al. (1983) found that in some cases, correlation changes were independent of presynaptic rate changes. This was again demonstrated in the frontal cortex (Vaadia et al., 1995), in which the dynamics of correlation varied between two behaviors, whereas the firing rate remained constant. We find that the dependency between two neurons continues to modulate, whereas the firing remains nearly constant (near the peak), yielding as much as a 125% increase in distance between responses to enhance orientation discrimination.
The question remains about how response mechanisms are modulated relative to the stimulus properties. The temporal dynamics of dependency offer some insight into this process. The first aspect of functional dependency that we observed was that a fast delayed reduction plays a major role. We found that the narrow tuning of dependency relative to orientation occurred only after 15 msec, which is similar to the temporal dynamics of rate tuning (Volgushev et al., 1995; Ringach et al., 1997), suggesting that the reduction is a result of feedback. The feedback might also play a role in reducing the burst length, which is influenced by GABAergic mechanisms (DeBusk et al., 1997), thereby explaining why the reduction for dependency is more dramatic and selective than for average firing rate.
Orientation discrimination
Orientation discrimination in untrained human observers is as fine as 10–20′ of arc (Westheimer, 1981). This is substantially finer than what would be expected when considering physiological properties of the most highly tuned neurons in primates (half-width at half-height of 4°); the performance is even better than expected, considering the resolution of retinal sampling in humans (hyperacuities) (Westheimer, 1981).
Psychologists have proposed population encoding to account for these performance levels. Biological substrates remain speculative, with vector summation as the most popular solution (Pouget et al., 2000). The fast synaptic modulations of synchronization can provide sizable contributions to orientation discriminations. In the present study, we provide clues into some of the temporal characteristics of this framework. With only two cells, we cannot reasonably predict the advantage of a cooperative code beyond the percentages we report (i.e., the cooperation), but our results do suggest that cooperation can provide the same level of discrimination as independent coding with fewer cells or in less time (or even both). This efficiency can in turn be used to provide the finer discriminations that are found in perceptual tests.
In addition to interval information being easily modulated by cortical mechanisms, the representation of visual information as synchronous activity is advantageous (von der Malsburg, 1981; Abeles, 1991; Singer and Gray, 1995) over an integrated rate code (Shadlen and Newsome, 1998) for several reasons. The rate information is ambiguous in terms of the feature encoded and is less flexible in participating in multiple and new representations (Singer and Gray, 1995). Although the rate is simply pooled across a population, synchronous patterns can participate at different times in the representation of different patterns across different subpopulations. We do not argue that spatial integration is not crucial in transmitting visual information, but suggest that the finer salient information is found in the synchronous activity. With only two neurons, the synchronization becomes much more selective for orientation. The synchronous pattern directly affects which neurons participate in the next assembly (Abeles, 1991), making it a reasonable code for fast hyperacuity representations.
Footnotes
This work was supported by National Eye Institute Grant RO1EY-03778-19. We thank Don Johnson for his assistance and helpful discussion on the type analysis methods. We also thank Jonathan Victor for helpful discussion and the anonymous reviewers for their suggestions.
This work was presented in part at the 2002 meeting of the Visual Sciences Society.
Correspondence should be addressed to A. B. Bonds, Department of Electrical Engineering, Vanderbilt University, 255 Featheringill Hall, 400 24th Avenue South, Nashville, TN 37212. E-mail:ab{at}vuse.vanderbilt.edu.
