Abstract
Many synapses in the CNS transmit only a fraction of the action potentials that reach them. Although unreliable, such synapses do not transmit completely randomly, because the probability of transmission depends on the recent history of synaptic activity. We examine how a variety of spike trains, including examples recorded from area V1 of monkeys freely viewing natural scenes, are transmitted through a realistic model synapse with activity-dependent depression arising from vesicle depletion or postrelease refractoriness. The resulting sequences of transmitted spikes are significantly less correlated, and hence less redundant, than the presynaptic spike trains that generate them. The spike trains we analyze, which are typical of those recorded in a variety of brain regions, have positive autocorrelations because of the occurrence of variable length periods of sustained firing at approximately constant rates. Sustained firing may, at first, seem inconsistent with input from depressing synapses. We show, however, that such a pattern of activity can arise if the postsynaptic neuron is driven by a fixed population of direct, “feedforward” inputs accompanied by a variable number of delayed, “reverberatory” inputs. This leads to a prediction for the number and latency distribution of the inputs that typically drive a cortical neuron.
Information transfer across many central synapses seems remarkably ineffective because of frequent synaptic transmission failures (Stevens and Wang, 1995; Murthy et al., 1997). However, if presynaptic spike trains encode information redundantly, unreliable transmission can save resources by allowing a synapse to filter out redundant aspects of its inputs. Spike sequence-dependent filtering can arise because the probability of release for most synapses is modulated by activity-dependent processes (Zucker, 1989). As a result, the sequence of transmissions can, in principal, be less redundant than the presynaptic spike train.
Many recorded spike trains display temporal autocorrelations over time scales of hundreds of milliseconds, indicative of redundant coding (Dan et al., 1996; Baddeley et al., 1997). This redundancy can limit the amount of information carried by a neural system (Barlow, 1961; Atick, 1992). Spike train autocorrelations may reflect intrinsic properties of the spiking neuron, such as bursting, or features of its inputs. In sensory areas, temporal autocorrelations may be generated by mechanisms of sensory acquisition, such as saccadic eye movements, whisking, and sniffing, that sample at frequencies of a few to several hertz.
In the first part of this article, we study the transmission of four types of spike trains through a model synapse constructed to match data on synaptic depression in slice preparations from various brain regions (Abbott et al., 1997; Hjelmstad et al., 1997; Thomson and Deuchars, 1997; Tsodyks and Markram, 1997; Varela et al., 1997; Dobrunz and Stevens, 1999; Dittman et al., 2000). The four types are: (1) a Poisson spike train, (2) a spike train based on a simple model of visually evoked activity during free viewing (which we call the saccade model), (3) a model spike train based on experimentally recorded bursting neurons, and (4) spike trains recorded in area V1 of awake monkeys freely viewing natural scenes. Our results indicate that activity-dependent, stochastic synapses can reduce spike train redundancy by removing autocorrelations within these trains.
In the second part of the article, we study how sustained firing can be generated in neurons receiving input through depressing synapses. The correlations in the spike trains we study arise from periods of sustained firing typically lasting for hundreds of milliseconds. Depressing synapses that receive such input tend to produce transient rather than sustained synaptic currents. Despite this, we show that a reasonable model of the total input to a neuron that includes both immediate and delayed components can generated sustained periods of firing.
The overall claim of this work is that activity-dependent, stochastic synapses remove autocorrelations from presynaptic spike trains, but that cross-correlations among multiple inputs reintroduce autocorrelations into postsynaptic spike trains. Thus, it might appear that the benefits of stochastic filtering at single synapses are undone by the multiple inputs to a neuron. However, as we analyze further in the Discussion, this is not the case. Even when multiple inputs reintroduce autocorrelations into postsynaptic spike trains, the ability of depressing synapses to enhance important temporal features of the input is not lost. Furthermore, noisy input signals that arrive synchronously across many inputs can be averaged instantaneously, something that could only be done at the single-input level by lengthy temporal averaging.
MATERIALS AND METHODS
Experimental procedures
Subjects and surgical procedures. Two adult female rhesus monkeys (Macaca mulatta) weighing 7–10 kg served as subjects for this study. A pair of scleral search coils were implanted for monitoring eye position (Judge et al., 1980). During the same surgery, a recording chamber was mounted over the opercular surface of the striate cortex. A small craniotomy (5 × 10 mm) was made in the bone overlying one cortical hemisphere. All surgical procedures were performed at the California Regional Primate Research Center and were supervised by the attending veterinary staff. Surgical and experimental techniques were in accordance with institutional and National Institutes of Health guidelines.
Behavioral training. For receptive field characterization, the monkeys were trained to maintain their gaze within 1.0° of a fixation spot, in the presence of moving or stationary visual stimuli, for up to 3 sec. Successful trials were rewarded with a drop of diluted apple juice. Access to water was restricted during behavioral training and recording. After receptive field characterization was completed, natural stimuli were presented for 3 sec, and the monkeys were allowed to freely view the natural scene.
Recording techniques. Electrophysiological signals were recorded with custom-fabricated nichrome tetrodes separated by either 250 μm or 3–4 mm. The signals were amplified, bandpass-filtered (0.6–6 kHz), and digitized (30 kHz/channel) using custom software. After recording, individual units were resolved from the multiunit activity through principal components analysis of waveforms or by clustering spikes on the basis of peak-to-peak amplitude, peak time, and spike width, as described by Gray et al. (1995). The extracted spike trains were stored at 1 msec resolution. The eye coil signals were digitized (1 kHz/channel) and stored using custom software.
Stimuli. Visual stimuli consisted of natural scenes, composed primarily of landscapes, presented with a slide projector and projected from behind the animals onto a screen located 1 m in front of them. Receptive field location and properties were measured using single or multiple light bars or square wave gratings (0.5–3.0 cycles/°) presented on a computer monitor with a dark background and ranging in mean luminance from 0.003 to 12 cd/m2.
Modeling Procedures
Synaptic depression. Several models of synaptic depression were tested, and all gave qualitatively similar results. Here, we present results from a simple stochastic model of a single active zone (Vere-Jones, 1966; Matveev and Wang, 2000). Although we formally model a single active zone, our results extend to single synapses with multiple independent active zones or multiple synapses receiving synchronous input, provided that the active zones have the same characteristics and that their outputs add without saturation.
The active zone is assumed to have N vesicles available for release, where N cannot exceed a maximum numberNmax. On arrival of a presynaptic action potential, each of the N available vesicles has a probability p of being released, with the restriction that not more than one vesicle can be released per action potential (Korn and Faber, 1991; Stevens and Wang, 1995). This gives the synaptic transmission probability, P = 1 − (1 −p)N. When a vesicle is released, N is decreased by 1, causing synaptic depression attributable to vesicle depletion. Each released vesicle has an independent, constant probability per unit time, 1/τD, of being replenished. On average, this yields exponential recovery from depletion with time constant τD. For the Poisson and saccade model spike trains (Figs. 1, 2),Nmax = 3. In the studies involving the model bursting neuron and real V1 neuron spike trains (Figs. 3, 5),Nmax = 1.Nmax in this model governs the strength of depression immediately after vesicle release (strongest depression for Nmax = 1) and represents an “effective” number of vesicles that takes into account both the possibility of having a population of vesicles with heterogeneous probabilities of release and the postrelease refractoriness thought to occur across an active zone (Dittman et al., 2000). For purposes of comparison, we also consider a model in which synaptic transmission occurs with constant probability.
To study the effects of multiple inputs, we use a deterministic rate model of depression (Abbott et al., 1997; Tsodyks and Markram, 1997). In this model, the steady-state probability of release for a presynaptic spike train generated by a Poisson process at rater is:
Equation 1If the presynaptic rate changes suddenly fromr0 tor1 at time 0, the transmission probability decays exponentially fromPss(r0) toPss(r1):
Equation 2Spike trains. Spike trains generated by the saccade model are produced by a piecewise, constant-rate Poisson process that simulates the response of a visual neuron during a sequence of saccades to different targets. After each simulated saccade, the model generates a Poisson spike train at a constant rate until the time of the next saccade. A new Poisson rate is chosen independently at each saccade from an exponential distribution with a mean of 15 Hz (Levy and Baxter, 1996; Baddeley et al., 1997). The saccade durations were chosen to fit an observed distribution of intersaccade intervals,tsacc (Viviani, 1990), using the function (exp[(4.55 Hz) tsacc] + exp[8.82 − (54.28 Hz)tsacc])−1. This distribution peaks at 192 msec and has a mean intersaccade interval of 365 msec.
The model burst train is adopted from a fit to bursting cells recorded in area MT of monkeys performing a visual discrimination task, as described by Bair et al. (1994). Interburst intervals were chosen from an exponential distribution with a mean of 31 msec, with intervals shorter than a minimum duration discarded and chosen again. The minimum interburst interval was chosen randomly from a Gaussian distribution with a mean of 16 msec and an SD of 7 msec, clipped at 0. The durations of individual bursts were chosen from a Gaussian distribution with a mean of 5.2 msec and an SD of 1.1 msec. Within the bursts, spikes were generated with Gaussian-distributed interspike intervals with a mean of 1.8 msec and an SD of 0.5 msec. The burst duration and interspike interval distributions were clipped at 0.
For the real V1 spike trains, twenty 5 sec responses recorded in each cell were concatenated to make a 100 sec train. The transmission autocorrelations for these trains were computed from enough repetitions to gather 100,000 presynaptic spikes. Model results were also based on 100,000 presynaptic spikes.
Integrate-and-fire neuron. We used a leaky integrate-and-fire model neuron to generate the postsynaptic responses used in Figure 6. Synaptic transmissions produced an excitatory postsynaptic conductance with an instantaneous rise and exponential decay with a time constant of 2 msec. Neuronal and synaptic parameters were chosen so that the postsynaptic neuron had an approximate balance of excitatory input and effectively inhibitory decay to rest (Shadlen and Newsome, 1994; Troyer and Miller, 1997) and had matching input and output average firing rates. For all simulations, we used a membrane time constant τm = 30 msec and spike thresholdVth = −55 mV. For simulations with 1000 synaptic inputs, the resting potentialVrest = −68 mV, and the reset voltageVreset = −58 mV. Each synaptic transmission increased the synaptic conductance by Δgsyn = 2.5% of the resting membrane conductance. For simulations with 30 synaptic inputs,Vrest = −65 mV,Vreset = −60 mV, and Δgsyn = 55% for runs that used dynamic synapses, and Vrest = −63 mV,Vreset = −59 mV, and Δgsyn = 39% for runs that used constant-probability synapses.
Autocorrelations. Autocorrelations, representing a normalized probability density of two spikes being separated by a time τ, are defined by:
Equation 3and satisfy −1 ≤ A(τ) < 〈 〉. 〈 and 〉 indicate time averaging. In practice, the autocorrelation is computed by dividing time into discrete bins, withs(t) equal to the number of presynaptic or transmitted spikes within the bin containing the time tdivided by the bin width. For reference, A(τ) = 1 indicates that, each time a spike occurs in a train, it is twice as likely as average that another spike will arrive a time τ later (to within a bin width). A(τ) = −1 indicates that a spike is never followed by a spike occurring a time τ later (to within a bin width).
RESULTS
Transmission autocorrelations
We are interested in how efficiently the sequence of action potentials transmitted by an activity-dependent, stochastic synapse encodes the information carried by the full presynaptic spike train. Unfortunately, a full information calculation is beyond the range of existing methods because of the long correlation times of the spike trains we consider. Instead, we use decorrelation as an approximate indication of the maximization of transmitted information at fixed average transmission probability. This approximation is accurate for high signal-to-noise ratios in the synaptic transmissions, meaning that, for multiple presentations of the same stimulus, the sequence of spikes that gets transmitted is approximately the same for each presentation. Specifically, we compare the autocorrelation of spike sequences arriving at a synapse with the autocorrelation of the sequence of spikes transmitted through the synapse and use the level of autocorrelation as an approximate measure of redundancy (Barlow, 1961;Atick, 1992). Within this approximation, redundancy reduction is equivalent to reducing the magnitude of autocorrelations.
Synaptic failures add negative autocorrelations to Poisson spike trains
We begin by examining how the model synapse transmits spike trains generated by a homogeneous Poisson process, which are uncorrelated (Fig. 1A). Figure 1,B and C, shows the results of sending Poisson spike trains through a stochastic synapse with constant transmission probability (Fig. 1B) or with vesicle depletion (Fig.1C). Each synapse transmits 23% of the incoming spikes. Transmission with constant probability thins the presynaptic spike train but does not change its autocorrelations; therefore, it produces an uncorrelated sequence of transmissions (Fig. 1B). In contrast, the transmissions through the depleting synapse have negative autocorrelations, reflecting the dynamics of the activity-dependent transmission process (Fig. 1C). The correlation time for the negative autocorrelations is a decreasing function of rate, and it is shorter than the refilling time constant τD. This is because, in the presence of presynaptic input, the rate of recovery to the partially depleted state seen during activity is controlled in parallel by the vesicle refilling process and the rate-dependent vesicle release process. This leads to a recovery rate that is an approximately additive combination of the refilling and vesicle depletion rates.
The activity-dependent dynamics of vesicle depletion add negative autocorrelations to sequences of synaptic transmissions. A, Autocorrelation of a Poisson spike train of average rate 15 Hz. B, Autocorrelation of the transmissions of this spike train through a model synapse with a constant probability of transmission. C, Autocorrelation of the transmissions of the Poisson train through a model synapse withp = 0.2 and τD = 500 msec. In both B and C,Nmax = 3, and 23% of presynaptic spikes were transmitted.
Synaptic failures remove positive autocorrelations from saccade model spike trains
We next consider more realistic spike trains with positive autocorrelations, using a model that simulates the responses of a visual neuron during a sequence of saccades to different targets (see Materials and Methods). Spike trains produced by the saccade model have positive autocorrelations that are nearly exponential, with a decay time constant τcorr = 235 msec (Fig.2A, black line). Positive spike train autocorrelations indicate that, each time a spike arrives at the synapse, there is a greater than average probability that additional spikes will arrive over a subsequent time of order τcorr. Synapses with vesicle depletion remove positive autocorrelations, because their dynamics have the complementary property that each time a spike is transmitted, future spikes are less likely to be transmitted over a time determined by the vesicle recovery dynamics, τD. The decorrelation is optimal when τD ∼ τcorr (Fig. 2A, black bars), but nearly complete decorrelation occurs over a hundreds of milliseconds range of τD.
Decorrelation of a saccade model spike train by a stochastic synapse with vesicle depletion. A, Autocorrelation of the presynaptic saccade model train of average rate 15 Hz (black line) and of the transmissions through a model synapse with p = 0.5 and τD = 350 msec (black bars). Thirty-three percent of presynaptic spikes were transmitted.B, Autocorrelation of the transmissions through a model synapse with p = 0.5 and τD = 70 msec (black line) or 1750 msec (gray line). Sixty-five and 10% of presynaptic spikes were transmitted, respectively. C, Autocorrelation of the transmissions through a model synapse with p = 0.5 and τD = 350 msec when the average presynaptic rate is 3 Hz (black line) or 75 Hz (gray line). Sixty-nine and 10% of presynaptic spikes were transmitted, respectively. D, Autocorrelation of the transmissions through a model synapse with τD = 350 msec and p = 0.1 (black line) or 1.0 (gray line). Seventeen and 38% of presynaptic spikes were transmitted, respectively. In A,B, and D, the average presynaptic rate is 15 Hz. In A–D, Nmax = 3.
A synapse with much shorter vesicle refilling time constant τD approaches the nondecorrelating behavior of a synapse with constant transmission probability (Fig.2B, black line). A synapse with much longer τD not only produces small negative autocorrelations but also greatly degrades the spike train by transmitting only a small fraction (10%) of presynaptic spikes (Fig.2B, gray line). With the synaptic parameters set to decorrelate the 15 Hz presynaptic train, as in Figure2A, black bars, significant lowering of the presynaptic firing rate reduces the decorrelating effect because of the greater recovery of the synapse between presynaptic spikes (Fig.2C, black line). However, increases in the presynaptic firing rate have little effect on the statistics of the transmitted train, even for large increases (Fig. 2C,gray line). At higher rates, the decorrelation effect appears to be dominated by the time constant of depression, rather than by the faster, rate-dependent time constant that governs the negative autocorrelations observed for Poisson trains. Changing the single-vesicle release probability p is similar to changing the presynaptic firing rate except for a reduced sensitivity to reductions in p that results from the nonlinear dependence of the transmission rate on p. As a result, the effect of both positive and negative changes in p on autocorrelations is weak (Fig. 2D). This indicates that the activity-dependent vesicle depletion and replenishment processes, not the vesicle release probability, govern the observed decorrelation.
Synaptic failures remove positive autocorrelations from burst trains
A model burst train was adopted from the work of Bair et al. (1994), who modeled the burst firing of cells recorded in area MT of awake monkeys (see Materials and Methods). The autocorrelations for these spike trains are positive at short times because of the relatively regular spiking during the bursts and negative for longer intervals because of interburst intervals (Fig.3A).
Removal of positive autocorrelations from a model burst train by a stochastic synapse with activity-dependent transmission failures. A, The presynaptic burst train has strong positive autocorrelations followed by near-maximal negative autocorrelations. B, Autocorrelation of the transmissions through a model synapse with τD = 15 msec, p = 0.5, andNmax = 1 (left) or 3 (right). Twenty-six and 67% of presynaptic spikes were transmitted, respectively.
The three-vesicle model (Nmax = 3) used for Figures 1 and 2 does not produce enough depression to remove the large positive autocorrelations characteristic of the burst trains (Fig. 3B, right). To increase the amount of depression, we reduced Nmax to 1. We also decreased the recovery time constant τD to a value just less than the mean of the minimum interburst interval to match the shorter autocorrelation time of the burst trains. This removes the burst-associated positive autocorrelations in the spike train without adding excessive negative autocorrelations at longer times (Fig. 3B, left). A strong and rapidly recovering form of depression such as this has been hypothesized to be caused by refractoriness of the entire active zone rather than vesicle depletion (Hjelmstad et al., 1997; Thomson and Deuchars, 1997; Dittman et al., 2000). The single-vesicle model, by having zero transmission probability immediately after a spike, simulates this process within a simple exponential recovery model.
Synaptic failures remove positive autocorrelations from V1 spike trains recorded from monkeys freely viewing natural scenes
Spike trains were recorded from area V1 of awake monkeys freely viewing natural scenes. Various response types were observed (Fig.4). Some neurons fired in a pattern that was qualitatively similar to that of the model saccade trains, maintaining a stimulus-dependent firing rate during fixations (Fig. 4, cells 1, 5). Others appeared to have more bursty behavior or a mixture of bursting and irregular firing behavior, with bursts often appearing at the beginning or end of fixations (Fig. 4, cell 14). All cells studied exhibited strong positive autocorrelations that decayed over various time scales. A few cells had strong positive autocorrelations for small τ and small negative autocorrelations for intermediate τ. Generally, the large positive autocorrelations in all trains tended to decay more rapidly than those of the model saccade trains (Fig. 2).
Spike trains recorded from area V1 of a monkey freely viewing natural scenes. Traces 1–14 show the responses of 14 different neurons, and traces V andH indicate the horizontal and vertical eye positions.
We analyze three spike trains that approximately sample the types of autocorrelations observed: one with strong positive short-time autocorrelations and a long tail of weaker positive autocorrelations (Figs. 4, cell 13,5, right, black line), one with positive autocorrelations of an intermediate duration (Figs. 4, cell 1, 5, middle,black line), and one with strong but short positive autocorrelations (Figs. 4, cell 14, 5, left,black line). To test the robustness of the decorrelating effect, we examined transmission for each of these spike trains using a model synapse with a fixed set of parameters (Nmax = 1; τD= 150 msec; p = 1.0). These parameters are consistent with a combination of the refractory and vesicle depletion forms of depression found in slice preparations (Abbott et al., 1997; Hjelmstad et al., 1997; Thomson and Deuchars, 1997; Tsodyks and Markram, 1997;Varela et al., 1997; Dittman et al., 2000). The sequence of transmissions from the model synapses is significantly less correlated than the input spike sequence for all three trains (Fig. 5, black bars), although the long-tail autocorrelations (Fig. 5,left, black bars) are not entirely removed, and a negative autocorrelation is introduced into the train with low firing rate and short-time autocorrelations (Fig. 5, right,black bars). Considering the range of different spike train statistics exhibited by these trains, decorrelation appears to be a robust effect.
Decorrelation of three typical spike trains recorded from freely viewing monkeys. Black lines, Spike trains from three recorded neurons showing strong positive autocorrelations of differing magnitudes and time scales. Average firing rates were left, 15.2 Hz; middle, 12.5 Hz; and right, 3.2 Hz. Black bars, Corresponding autocorrelation of the transmissions through a model synapse with Nmax=1, p =1, and τD = 150 msec. Percentages of presynaptic spikes transmitted were left, 19%; middle, 27%; and right, 52%. The left,middle, and right panels correspond to neurons 14, 1, and 13, respectively, in Figure 4.
Sustained firing with depressing synaptic input
Although depressing synapses decorrelate their inputs, neuronal firing is autocorrelated, as we have discussed. If synapses decorrelate their inputs, how can this be consistent with autocorrelated neuronal firing? Here we provide an illustrative example of how delayed recurrent input can resolve this apparent inconsistency.
Correlations arise in the saccade model and V1 spike trains, as well as in other spike trains from animals viewing natural scenes (Baddeley et al., 1997), from sustained periods of approximately constant firing at a variety of rates over variable-length periods, with sudden steps between these rates. We will show how autocorrelations may reappear in the spike train of a postsynaptic neuron by showing how such periods of sustained firing can occur.
The basic idea for maintaining sustaining firing is to recruit additional inputs over time to compensate for the drive lost because of synaptic depression. This is done by postulating that a group ofNff direct inputs change rates rapidly and approximately synchronously (compared with τD) in response to a stimulus. This response is then followed by a set of Nrev delayed inputs that change their rates later with various latencies. In this arrangement (shown schematically in Fig.6C), the coincident inputs produce the initial step in the postsynaptic rate, and the delayed inputs compensate for the depression of the coincident group of inputs, maintaining a sustained firing rate. In the context of a visually responsive neuron, this has a natural interpretation in terms of rapid, feedforward input immediately after a saccade, followed by a set of reverberatory inputs that arrive with variable latencies. We therefore refer to the immediate, synchronous inputs as feedforward and call the delayed inputs reverberatory.
Matching of input and output firing rates and autocorrelations. A, Comparison of the postsynaptic firing rate (solid line) of an integrate-and-fire neuron receiving 30 inputs (as described in Results and inC) with the firing rate of one of its feedforward inputs (dashed line). The slight discrepancy between the input and postsynaptic firing rates for the initial lower-rate segment arises because the integrate-and-fire neuron fires more rapidly at low rates than the linear model used in the calculation of the matching conditions. B, Autocorrelation of one of 333 presynaptic spike trains generated from the saccade model (black line) and of the corresponding transmissions through one of the model synapses (Nmax = 3;p = 0.25; and τD = 350 msec;black bars). The other 667 presynaptic inputs were generated by a 15 Hz constant-rate Poisson process. C, Schematic of the pattern of arrivals of an identical change in rate for the 333 saccade model afferents to the integrate-and-fire model neuron. For the simulations, we have defined a fixed order in which neurons switch rates for each simulated saccade. D, Autocorrelation of the spike train generated by the postsynaptic neuron (average rate, 15 Hz). The smaller autocorrelations in the first two time bins reflect the postspike dynamics of the integrate-and-fire neuron, an effect not included in the input trains.
In the , we show that a postsynaptic neuron will duplicate a step in the firing rates of its inputs from an initial rater0 to a final rater1 provided that the numbers of feedforward and reverberatory inputs are given by:
Equation 4and provided that the latencies of the reverberatory inputs are distributed exponentially with mean latency equal to τD. Here, k is a constant of proportionality relating the firing rate of the postsynaptic neuron to the rate at which it receives synaptic transmissions. Equation 4implies that we require a fixed number of synchronously rate-switching feedforward inputs and a number of delayed, reverberatory inputs that increase linearly with the firing rate. This seems reasonable because a stronger feedforward response is likely to recruit a proportionally larger population of recurrently excited neurons. Substituting values for the parameters into these equations gives an estimate for the number of responsive inputs to a neuron. Assuming that 1/k = 20 EPSPs per spike, as would be the case if the average EPSP caused a 0.5 mV depolarization and the difference between the reset and firing threshold potentials of the postsynaptic neuron was 10 mV (Shadlen and Newsome, 1994), andr1 = 15 Hz, τD= 380 msec, and p = 0.6 (Varela et al., 1997), we findNff ∼ 35 feedforward inputs andNrev ∼ 115 reverberatory inputs. This gives a total of Nff +Nrev ∼ 150 responsive inputs, a number similar to what has been estimated on the basis of a different argument using a consistent feedforward cortical model (Shadlen and Newsome, 1998).
Figure 6A shows the results of applying this input scheme to a leaky integrate-and-fire neuron (see Materials and Methods) with 30 inputs that jump from 10 to 30 Hz. The postsynaptic firing rate (obtained by averaging over 100 repeats) reproduces the step in input firing rates quite well and is sustained after the initial upward step.
Figure 6B–D demonstrates that this arrangement of inputs also leads to a matching of the autocorrelations of spike train of the postsynaptic neuron (Fig. 6D) to that of its presynaptic inputs (one of which is shown in Fig. 6B,black line). The autocorrelated postsynaptic firing occurs despite the fact that the individual synaptic transmissions are uncorrelated (Fig. 6B, black bars) and reflects sustained periods of constant-rate firing that occur when fluctuations produced by feedforward inputs are extended over time by the reverberatory afferents. Such consistency of input and output spiking statistics is necessary because of the strongly recurrent nature of cortical circuits, in which a large fraction of the inputs to a neuron come from neurons in the same local region with similar firing statistics (Shadlen and Newsome, 1994).
The results shown in Figure 6B–D come from a 1000-input simulation, with one-third of the inputs generated by the saccade model at an average rate of 15 Hz and the other two-thirds of the inputs, modeling background activity, generated from a constant-rate Poisson process at 15 Hz. We fixed the number of saccade model inputs at 333 and, at each saccade, assigned a fractionNrev/Nff= pr1τD of these inputs to be reverberatory (i.e., to have delayed transitions in firing rate). We did this because varying the total number of inputs at each saccade in accordance with Equation 4 would require a model of how reverberatory inputs are recruited from the background. We verified that this approximation had only a small effect on the ability of the model to reproduce input steps as in Figure 6A. Similar results were obtained for the integrate-and-fire neuron with 30 saccade model inputs used for Figure 6A. The background input in the 1000-input model primarily adds a baseline level of depolarization, and its summed contribution is primarily uncorrelated, although each individual input produces negatively autocorrelated transmissions (Fig. 1), because the arrival times of the inputs are uncorrelated.
Finally, we are not claiming that the particular model we have presented is a unique solution to the problem of generating sustained firing and autocorrelations. Rather, it provides an existence proof that these patterns of firing are not inconsistent with the idea of decorrelation by synaptic depression at individual synapses.
DISCUSSION
Autocorrelations in recorded spike trains suggest redundant representation of information (Barlow, 1961; Atick, 1992). We have shown that activity-dependent, stochastic synaptic transmission can reduce this redundancy by removing autocorrelations (Figs. 2, 3, 5). The tuning of synaptic time constants does not need to be precise to achieve a significant decorrelating effect (Figs. 2, 5). Although decorrelation is not strictly equivalent to information maximization, it is an effective method of compressing the information in a redundant signal. Furthermore, activity-dependent failures reduce total transmissions, allowing for a more efficient use of synaptic resources. In studying the optimal decoding of bursty spike trains recorded from area MT during motion discrimination, Bair et al. (1994) found that the best prediction of a monkey's responses was obtained when individual spikes and individual bursts were weighted equally. This finding could reflect transmission through synapses with a strong refractory form of depression that allows transmission of an isolated spike or the first spike of a burst with equal fidelity but that prevents subsequent, redundant spikes in the burst from being transmitted.
We have focused on the role of synaptic depression in spike train decorrelation, but other short-term processes such as spike rate adaptation (Liu and Wang, 2001), synaptic facilitation (Lisman, 1997), and postsynaptic receptor dynamics could allow more general filtering of spike trains (Maass and Zador, 1999). In general, the forms and time scales of short-term dynamics in different brain regions may be tuned to the statistics of the spike trains in those regions.
Although synaptic depression removes autocorrelations at individual synapses, neuronal spike trains exhibit autocorrelations that we have shown may arise from the timing of and cross-correlations between different inputs (Fig. 6). Thus, it might seem that the benefits of decorrelation at single synapses are canceled by the effect of cross-correlations across synapses. However, decorrelating individual synaptic inputs and then generating sustained firing from the arrangement of inputs we have discussed may have several computational advantages. Depression in the feedforward connections and synchrony across these inputs could allow a relatively small number of feedforward inputs to have a relatively large influence on the firing of the postsynaptic neuron during postsaccadic changes in firing rate. Synchrony across the feedforward group of inputs may also remove noise inherent in stochastic synaptic transmissions. Such spatial averaging of synaptic noise, unlike temporal averaging, is instantaneous, permitting fast and reliable responses to new inputs. Cross-correlations among the reverberatory inputs may reflect positive feedback loops in the highly recurrent cortical circuitry. Depression in recurrent cortical connections may exist in part to dampen runaway excitation in such circuitry (Thomson and Deuchars, 1994).
More generally, depressing synapses may underlie a mode of neuronal computation in which individual postsynaptic spikes emphasize specific temporal features of individual inputs, rather than responding equally to all presynaptic spikes arriving along a particular input (Dobrunz and Stevens, 1999). Whereas simpler models of neurons consider them as nonselective integrators of their inputs, a postsynaptic neuron with decorrelating synapses preferentially reflects interesting (i.e., nonredundant) features of its individual inputs. For the example of a neuron receiving saccade model input as in Figure 6, the postsynaptic spikes produced by a neuron with depressing synapses preferentially reflect the jumps in firing rate that are the most prominent feature of this input (Goldman, 2000). In this manner, decorrelating synaptic inputs can provide a neuron with a matched filter that is tuned to the statistics of its individual inputs (Abbott et al., 1997; Maass and Zador, 1999).
The results in this article depend on an approximate coincidence of time scales for several processes. The decay time constant for the autocorrelations in the saccade spike trains, τcorr, is directly related to the characteristic time scale of the intersaccade intervals, τsacc. These autocorrelations can be removed by stochastic depressing synapses with a recovery time, τD ∼ τcorr, in the same range as has been seen experimentally (Abbott et al., 1997;Tsodyks and Markram, 1997; Varela et al., 1997). The consistency of presynaptic and postsynaptic firing statistics results from matching the characteristic spread in the latencies of reverberatory inputs, τdelay ∼ τD, which implies a corresponding time scale for network activity. Together, this suggests that τD ∼ τcorr ∼ τdelay sets a basic time scale for cortical processing.
Appendix
Number of synchronous and delayed inputs and distribution of latencies for matched input-output responses
Here we show how sustained firing after a steplike rise in input rates can be generated when synapses depress.
We postulate a group of Nff direct feedforward inputs and a set of Nrevdelayed reverberatory inputs that all increase their firing rates from an initial rate r0 to a final rater1. The feedforward inputs jump to the rate r1 immediately, whereas the reverberatory inputs remain at rate r0for variable delay or latency periods and then jump to rater1. Immediately after the feedforward inputs have jumped, the probability of release for all synapses isPss(r0) because of the preceding period of sustained firing at rater0 (see Materials and Methods). At this time, the feedforward synapses transmit collectively at a ratePss(r0)r1Nffbecause their presynaptic inputs fire at rater1. The reverberatory inputs transmit at a total ratePss(r0)r0Nrevbecause their input rates have not yet changed fromr0.
We assume that the firing rate of the postsynaptic neuron is proportional to the rate at which it receives synaptic transmissions, with a proportionality constant k. Then the presynaptic firing rate immediately after the jump in the rates of the feedforward inputs is, adding together the contributions of both feedforward and reverberatory inputs:
Equation 5Now consider the firing rate of the postsynaptic neuron long after the initial jump in rates, when the feedforward and reverberatory inputs are all firing at rate r1. At this time, the probability of release at all Nff+ Nrev synapses isPss(r1) because of the sustained firing at rater1. Thus, the transmission rate at each synapse isPss(r1)r1, and the postsynaptic firing rate is:
Equation 6We now determine the conditions under which the firing rates at the beginning (Eq. 5) and end (Eq. 6) of the period in question match the input firing rate, that is, rpost= r1. Using Equation 1 forPss, we find:
Equation 7which can be verified by substitution into Equations 5 and 6.
We next extend this result to times intermediate between the beginning and end of the period. Suppose that theNff feedforward synapses transition from rate r0 tor1 at time t = 0. At later times, these synapses transmit at a total rate (using Eq.2):
The transmission rate for a reverberatory synapse that changes its rate at time tstep, rather than at time 0, isr0Pss(r0) for t < tstep and:
for t ≥ tstep.
We now show that sustained firing at a constant rate is maintained by a set of delayed, reverberatory inputs that have a latency distribution exp(−tstep/τdelay)/τdelay, with τdelay a constant to be determined. Heretstep indicates the time when the rate of a delayed input jumps from r0 tor1. Adding up the contributions from the feedforward and reverberatory inputs, we can write the postsynaptic firing rate at time t > 0 as:
Although the algebra is a bit tedious, doing the integrals and using the results of Equation 7, we find thatrpost(t) =r1 if τdelay = τD. Note that the time constant governing the decay of arrivals of steps matches the depression time constant τD, not the effective time constant τDPss(r1)/pthat appears in the exponentials throughout the calculation and governs the decay of the feedforward group of inputs.
Footnotes
This work was supported by National Institutes of Health Grant MH58754, National Science Foundation Grant IBN-9817194, the Sloan Center for Theoretical Neurobiology at Brandeis University, and the W. M. Keck Foundation. Experiments were performed in the laboratory of Charlie Gray, and we thank him for his assistance. We thank Sacha Nelson, Geoff Hinton, Ken Sugino, and John Birmingham for helpful discussions.
Correspondence should be addressed to Dr. Mark Goldman, Brain and Cognitive Sciences, Massachusetts Institute of Technology, E25-210, 45 Carleton Street, Cambridge, MA 02139. E-mail:mark_g{at}mit.edu.
