Abstract
The ability of synchronous population activity in layered networks to transmit a rate code is a focus of recent debate. We investigate these issues using a patterned unidimensional hippocampal culture. The network exhibits population bursts that travel its full length, with the advantage that signals propagate along a clearly defined path. The amplitudes of activity are measured using calcium imaging, a good approximate of population rate code, and the distortion of the signal as it travels is analyzed. We demonstrate that propagation along the line is precisely described by information theory as a chain of Gaussian communication channels. The balance of excitatory and inhibitory synapses is crucial for this transmission. However, amplitude information carried along this layered neuronal structure fails within 3 mm, ∼10 mean axon lengths, and is limited by noise in the synaptic transmission. We conclude that rate codes cannot be reliably transmitted through long layered networks.
- layered network
- hippocampal culture
- rate coding
- Shannon information
- Gaussian chain
- inhibition
- one-dimensional network
Introduction
Layered networks have been the focus of much attention and constitute a major framework for discussion of information transfer (Wilson and Cowan, 1973; Miles et al., 1988; Abeles, 1991; Ermentrout and McLeod, 1993; Idiart and Abbott, 1993; Traub et al., 1993; Shadlen and Newsome, 1994, 1998; Diesmann et al., 1999; van Rossum et al., 2002; Litvak et al., 2003; Reyes, 2003). Despite their simple structure, these systems display a variety of complex phenomena and are primarily of interest because they give a reasonable model of cortical organization (Mountcastle, 1957; Wilson and Cowan, 1973; Christian and Dudek, 1988; Miles et al., 1988; Abeles, 1991; Traub et al., 1993; Mountcastle, 1997). Because it is impractical to follow information transfer directly in the cortex, computer simulation and theoretical modeling were a natural substitute.
Using layered networks for analysis of coding was suggested by Abeles (1991), and shortly thereafter such a system was directly applied for an in depth discussion of coding strategies (Shadlen and Newsome, 1994, 1998). They investigated whether rate (Adrian and Zotterman, 1926; Stein, 1967; Barlow, 1972) or temporal (Abeles and Gerstein, 1988; Abeles et al., 1993; Bair and Koch, 1996; van Steveninck et al., 1997; Hatsopoulos et al., 1998) codes are the preferred mode for reliable information transfer. They showed that, in the presence of an excitatory/inhibitory balance, the accumulating transmission errors could be significantly reduced by averaging over the activity of large neuronal populations (Mountcastle, 1957, 1997) and concluded that rate coding is the preferred cortical coding strategy.
Diesmann et al. (1999) realized that, in such systems, activity tends to synchronize and is characterized by large-amplitude population bursts termed “synfire” (Abeles, 1991). Although this leads to large-scale correlations that destroy the advantages of rate-code averages, it may still allow exact temporal patterns to reliably pass through the layers (Diesmann et al., 1999; Litvak et al., 2003). Their conclusion is that rate code cannot be used in layered cortical structures. although temporal schemes cannot be ruled out.
The first attempt to build a fully accessible layered network that incorporates experimental data were performed by Reyes (2003). He incorporated a single recording neuron from a hippocampal slice into a simulation of a layered network, using simulated data to control current injected to the recording neuron. The output response was used in an original bootstrap scheme to calculate the input for the next layer. Using this system, he showed that, despite the evolution of synchrony, information on spike rate was sustained and could be propagated onto as far as 10 layers.
In this paper, we study coding efficiency using a fully experimental, in vitro system in which dissociated hippocampal neurons are patterned to form a layered structure (Feinerman et al., 2005). This culture may be regarded as a simplified yet biological model of layered structures that yields a direct measurement of neural activity and of information transfer. We identify fluorescence intensities with population spiking rates to allow the extension of the experimental results to the stability of rate-coded information progressing along the culture.
Materials and Methods
Primary cultures of hippocampal neurons and coverslip patterning were prepared as described previously (Papa et al., 1995; Feinerman et al., 2005). Experiments were performed on patterned one-dimensional networks cultured on lines 170 μm wide and up to 8.05(!)cm long (Feinerman et al., 2005) (Fig. 1). Neuronal density varied, with an average value of 0.31 ± 0.02 neurons/μm (all means are given as mean ± SEM) and an SD of 0.08 neurons/μm. At this density, axons travel an average of 308 ± 16 μm along the line, and only 3% are >800 μm long. The layered structure is created because individual axons extend far less than the full length of the line. The network is layered in the sense that its connectivity is local, and its structure forces the activity to propagate sequentially through successive groups of neurons. Axons propagate in both directions of the line, with no apparent preference.
An 8.05-cm-long and 170-μm-wide culture containing ∼2.5 × 104 neurons. The line was patterned on a Φ = 13 mm coverslip (Menzel-Gläser, Braunschweig, Germany) whose edges are visible in the image. Dissociated hippocampal neurons were plated on it to form a single continuous culture, in which population bursts excited at one end travel the full length of the line and reach the other edge ∼800 ms later. The image was taken through a Zeiss (Oberkochen, Germany) Stemi SV 11 stereoscope.
Cultures exhibit spontaneous population bursts of activity that are assumed to be synchronous [synfire chains (Abeles, 1991)] and emanate from localized sources or hotspots (Murphy et al., 1992; Maeda et al., 1995; Streit et al., 2001; Tscherter et al., 2001; Darbon et al., 2002). We directly verified the existence of hotspots in the linear culture (Feinerman et al., 2005) (our unpublished data). From these hotspots, the population bursts are relayed from one area to the next and propagate throughout the culture within, typically, hundreds of milliseconds. A period of quiescence that is on the order of 10 s separates these spontaneous bursts. Similar bursts were stimulated by local application of either 20 or 200 μm of glutamate using concentric pipettes (Feinerman and Moses, 2003; Feinerman et al., 2005).
We track the time evolution of these bursts over areas of the culture separated by distances of up to 8 cm, testing the reliability of their transmission. Neural activity at different parts of the linear culture is measured simultaneously via intracellular calcium concentrations as monitored by fluorescence levels of the calcium-sensitive dye fluo4 (Teflabs, Austin, TX) (Feinerman et al., 2005). These measurements are averaged at video rate over groups of ∼100 neurons [regions of interest (ROIs)]. A crucial fact for our subsequent analysis is that calcium dye fluorescence levels are practically linear with spike number for the low spike counts (Canepari and Mammano, 1999; Lips and Keller, 1999; Smetters et al., 1999; Froemke et al., 2002; Murayama et al., 2005). The bursts we monitor are characterized by up to approximately five spikes per neuron (data not shown). It is also well known that subthreshold responses fail to cause any calcium transients (Smetters et al., 1999; Froemke et al., 2002). We can conclude that, in this regime, the fluorescence amplitudes sum the spikes over all neurons in a group and can thus be identified with a population rate code.
Hippocampal cultures are primarily composed of pyramidal neurons and typically include ∼15–20% inhibitory neurons (Goslin et al., 1998). The excitatory/inhibitory balance in the network can be broken by bath application of 40 μm of the GABA antagonist bicuculline (Sigma-Aldrich, Rehovot, Israel). This allows us to study the predicted importance of this balance for transmission reliability (Shadlen and Newsome, 1994, 1998; Beggs and Plenz, 2003).
Amplitude lists, their entropy, and mutual information.
The fluorescence amplitude of a population burst on a specific area of the line reflects the integrated spiking activity of a group of neurons and varies from burst to burst. Similar fluctuations are described by Murphy et al. (1992), Maeda et al. (1998), Bacci et al. (1999), Tscherter et al. (2001), Tabak and Latham (2003), and van Pelt et al. (2005). By binning the amplitudes, we can assign each event with a discrete symbol (e.g., “a” for the highest amplitude events, “b” for the next highest, etc.). A succession of such events would thus form a “word” (e.g., “abgdefdgid” for 10 bursts). Activity propagates along the line as neurons synaptically excite their neighbors (Feinerman et al., 2005) so that the amplitude of an event varies as it progresses along the line. Measuring the fluorescence for a sequence of bursts gives a slightly different word at every location along the line. The distortion of these words reflects the reliability of transmission and may be quantified using information theory.
The line is divided into nonoverlapping rectangular ROIs that span the line in width (170 μm) and whose length is of the same order (200–300 μm). The particular choice of ROIs is not critical and can even be arbitrary to some degree. The typical distance between the centers of two neighboring ROIs was 390 μm. Although discrete ROIs do not reflect the continuous architecture of the culture, they do provide a way of quantifying propagating activity along the linear network. We assign to each event (population burst) at every region its amplitude, defined as the difference between maximal and minimal fluorescence during the event, integrated over the region, and divided by the minimum (the baseline). This reduces the continuous signal taken from each area into a list of numbers. Because the slow decay of fluorescence after a burst may influence the amplitude of a subsequent burst that is too close in time, only events that followed a quiescent period were taken (usually >4 s of no activity). We also limited our analysis to the rising of the signal (typically <200 ms), ignoring any spikes that appear in follow-up activity (Wu et al., 1999) after the initial burst. The slope of this initial rise was measured by taking the point with maximal first derivative of the rising fluorescence signal.
Each ROI contains on the order of 100 neurons, which is a reasonable number to average a population or rate code over (Shadlen and Newsome, 1994). The amplitude lists of all ROIs, which is analogous to the words mentioned above, are of equal length, with the nth entry in all lists referring to the nth event that occurred on the line. Because experiments included at least 10 min of fluorescence measurements, bleaching was evident. This effect was compensated for by normalizing each list by an exponential decay fit (Smetters et al., 1999; Kerr et al., 2000; Galizia and Kimmerle, 2004).
The differential entropy (self-information) of each list as well as the mutual information between pairs of lists were estimated numerically (Kraskov et al., 2004). The algorithm used provides unbiased estimates of entropy and mutual information with small systematical errors even on small sets of data. The (absolute) entropies were calculated by quantizing the random variable by the experimental noise (Cover and Thomas, 1991), defined as the SD of the baseline signal. This quantization is analogous to discretizing the amplitudes, which are a continuous variable, into letters as explained above.
In experiments in which activity was stimulated, the ROI that is directly excited may be regarded as the origin, or input, of an information channel. In such cases, the mutual information was calculated between the origin and all other ROIs, which may be regarded as channel outputs at various distances. Spontaneous activity is similar to the stimulated case because it too originates in discrete hotspots along the line. However, in experiments in which spontaneous activity was monitored, we disregard its source. We sample activity at different locations along the progression of the activity and estimate mutual information for all possible pairs of ROIs. Because these two methods yielded the same results, we could be confident that localization of the source, although possible, was not important for the determination of information propagation.
Gaussian chains.
Transmission of information through a line is common to a wide variety of communication networks, e.g., the telephone or Internet, and has received wide attention from the engineering community. Adequate theoretical tools for describing the efficiency and capacity of the transmission in such communication lines have been developed and give precise quantitative answers within the framework of information theory. Although technology has advanced significantly, so that for example telephone lines are nowadays relatively noise free, the analog case in neuronal transmission is highly error prone and noisy.
It is particularly useful to consider the concatenation of channels as a model for the transmission of information between ROIs and use the information theoretical model of Gaussian chains. The model describes a prevalent communication line in which signals accumulate random noise as they propagate (for an introductory text, see Cover and Thomas, 1991).
A Gaussian communication channel (Cover and Thomas, 1991) is a time-discrete noisy transmission line. The output of a unity gain channel is the same as its input, but with a random additive noise chosen from a Gaussian probability distribution function (pdf). The distributions of input and noise terms are considered independent. The ratio between the variance of the signal and the variance of the noise is defined as the signal-to-noise ratio (SNR) of the channel. The capacity of such a channel is defined as the maximal mutual information between a list of inputs and the corresponding list of outputs and is given by the following (Cover and Thomas, 1991): C = 0.5 × log2(1 + SNR) bits.
This is an upper bound that can be attained if the inputs are also taken from a Gaussian distribution.
Gaussian channels may be “daisy-chained” in series to form a Gaussian chain in which the output of one channel serves as the input of the next. If the noise terms of the channels in the chain are independent of each other, then their sum will also be distributed according to a Gaussian, with a variance that is the sum of their individual variances. This makes the Gaussian chain itself a Gaussian channel. The SNR of a Gaussian chain that is composed of N identical channels connected in series will be the SNR of a single channel, divided by N.
Results
Signal transmission
Bursts propagate stably at an average velocity of 55 ± 4.5 mm/s (Feinerman et al., 2005) and can travel distances of 8 cm. We measured the transmission of these bursts in both the stimulated and spontaneous cases. Almost all population bursts traveled the full length of the line to excite activity on its distant edge. The arrival of a signal without regard to its amplitude is a yes/no event that carries information only about its timing. For this reason, we monitor the amplitudes of activity in each burst.
More specifically, in stimulated cultures, 88% (157 of 178) of bursts propagated at least 8.5 mm from the point of stimulation, as measured on five cultures. A total of 84% (96 of 114) of locally stimulated population bursts propagated the full 17 mm of the line as measured on four cultures.
To identify propagation characteristics in spontaneous activity, three distant areas on a 17 mm line were visualized: its two ends and its central part (Feinerman et al., 2005). A total of 94% (299 of 319 bursts) of bursts propagated to reach two of these three areas (the central part and one of the ends). In this case, bursts had propagated a total of at least 8.5 mm. In 88% of bursts (144 of 164), activity was evident in all three areas. In this case, activity had initiated in a hotspot situated along the line (which we did not attempt to localize), from which it propagated to excite the full 17 mm of the culture.
The gain in amplitude when the signal traverses from one ROI to its neighbor must, on average, be close to 1. Any deviation from unity gain at the transfer between ROIs would be exponentiated by 
∼50 mm (the line length over the mean axonal length in millimeters), leading to either a blow up or decrease in amplitude to 0, neither of which were observed.
The fact that the amplitude does not saturate at a maximal value as it progresses along the line can also be seen by the removal of inhibition. Shutting down inhibition by bath application of bicuculline leads to a global rise in amplitude up to approximately a factor of 4 and to an increase of propagation velocities to an average value of 97 ± 10 mm/s (Feinerman et al., 2005). Addition of bicuculline furthermore increases the stability of propagation and allowed 96% of spontaneously induced bursts to reach 8.5 mm (268 of 279), 93% to reach 17 mm (153 of 164), and 83% to reach 45 mm (30 of 36) of the culture.
Reliability of amplitude propagation
The population bursts commence locally, stimulated either spontaneously or by glutamate application (Feinerman et al., 2005), and then spread to the whole culture. In analogy, we may associate the local group of neurons in which bursting commenced to sensory neurons, whose activity carries information about an “external stimulus.” As the population burst propagates, this information is conveyed to more distal parts of the culture. This process involves a deterioration of the signal, so that distant groups of neurons receive an inaccurate account of the “sensory” activity. A quantitative comparison between the firing levels in distal groups of neurons was performed by use of information theory.
To check whether a coding scheme can reliably transmit information through the experimental system, we need first to identify the specific critical parameters that carry information along the channel (such as millisecond interspike intervals for temporal coding). We can then ask whether these succeed to progress through the system without an accumulation of errors. A few possible choices for such parameters were considered for our experimental measurements and analysis. One such parameter is the interburst interval (IBI), the time interval between two consecutive population bursts, but this is typically on the order of ten of seconds (Feinerman et al., 2005). Such long timescales do not seem to be relevant for the coding of any realistic information. Another option is the internal temporal structure of the burst, which may contain several spikes. However, the shorter timescales between the spikes in a single burst cannot be measured in our experiment because of the long decay times of the calcium-sensitive dye as well as the 20 ms time resolution of video imaging. What is practically accessible are the event amplitudes as measured by the concurrent increase in fluorescence, as well as the initial slope for each event. These are a critical variable whose progression can be tracked using our experimental system, and we therefore chose to concentrate on this measurement. As mentioned above, it is linearly related to a rate code.
In principle, the slope of each burst could hold different information than the amplitude, for example, on the timing between initial spikes. However, a close examination showed that the relationship of initial slopes of the fluorescence signal of bursts to their amplitude, as measured in 18 different cultures and >180 ROIs, is linear. On average, the amplitude of a signal is just 0.2 ± 0.004 s times its initial slope. This strict relationship actually identifies spike number (amplitude) with spike rate (slope), connecting these two related modes of coding in the activity of the one-dimensional cultures. We therefore concentrate our discussion on amplitude measurements.
We define the area in which activity commenced as the input of an information channel and another, distal area as the output. The reliability of the channel is obtained by measuring the mutual information between a given activity parameter in these two areas. To get ideally good estimates on channel performance, no limitations should be imposed by the stimuli chosen; they should be randomly chosen over a wide dynamical range (Werner and Mountcastle, 1965; Rieke et al., 1997).
Our experimental design supplies us with a reasonably wide range of amplitudes, as much as a factor of 2 between the maximum and the minimum of a random Gaussian signal. These fluctuations in event amplitude are inherent to the system and are common to a variety of neuronal culture preparations (Murphy et al., 1992; Maeda et al., 1998; Bacci et al., 1999; Tscherter et al., 2001; Tabak and Latham, 2003; van Pelt et al., 2005). Interestingly, such variations were not observed by Pinto et al. (2005), perhaps because of the fact that, in this slice preparation, inhibition was blocked to achieve propagating waves.
Propagating waves in slice preparations usually require special treatment such as the removal of inhibition (Golomb and Amitai, 1997; Wu et al., 1999; Stoop and Pralong, 2000; Harris and Stewart, 2001; Pinto et al., 2005). As shown by Houweling et al. (2005), this may be attributed to the fact that neurons in cortical slice preparations develop in vivo and experience constant synaptic bombardment that may tune them to low responsiveness. Culture preparations allow for neurons to be studied in the same environment in which they have developed. As stated above, no special treatment is required for propagating waves to be generated.
The amplitude variations obtained while stimulating the culture at constant time intervals are similar to those in the spontaneous case (see below). It is possible that the variations in response to a constant stimulus are because the stimulation causes large-scale population response only indirectly. The initial response is local and may be specific to the stimulation, but it gets amplified by recurrent activity before developing into the large-scale event (Feinerman et al., 2005; Pinto et al., 2005). Event amplitudes depend only weakly on the preceding quiescent period.
Because the average fluorescence amplitude of events in a specific area depends on local parameters, such as local neuron density, it carries no information (Litvak et al., 2003), and we must normalize each list by its mean. We end up with a list of normalized deviations from the average that do carry information and typically range between 0.7 and 1.3 times the mean. This normalized data are proportional to the spike count, averaged over the neurons in a given ROI, regardless of their total number. We quantify the efficiency of the rate-code communication by estimating the mutual information of normalized amplitude lists recorded in different ROIs.
As an example, Figure 2shows nine regions of interest chosen from a 3.6-mm-long and 170-μm-thick line (Fig. 2A). Their fluorescence was monitored at 50 Hz during ∼7 min of spontaneous activity. Time plots of the average fluorescence level in areas 1, 2, and 7 are given in Figure 2B, and their normalized lists of amplitudes, obtained from 69 bursting events, are in Figure 2C. The amplitudes range between 80 and 120% of the average amplitude of each area and fluctuate in a seemingly random manner around their mean. Fitting the fluctuations to a Gaussian pdf in separate experiments gave R2 = 0.9 ± 0.01 (n = 10 cultures), whereas averaging over all of these experiments by shifting maxima to 0 and normalizing variances to 1 yielded an excellent fit with R2 = 0.99 (Fig. 2F).
Relationship between fluorescent amplitudes in different areas of the line during population bursts. A, Nine ROIs were chosen along a 3.6-mm-long, 170-μm-wide line. B, The fluorescent signal was gathered at 50 Hz resolution and averaged over each ROI. The time traces of regions 1, 2, and 7 are shown here. C, Normalized amplitudes for 69 consecutive events in the same three areas. D, The amplitudes of an event in area 1 plotted against the amplitude of the same event in area 2 (full green dots). Red dots are the same for areas 1 and 7. A linear relationship (slope of 1.04) between the amplitudes is evident. C ensures that this relationship is not the result of a global drift or bleaching of the fluorescent signal. The linear relationship is not exact, and more noise is apparent between areas 1 and 7, whose distance from each other is 2.8 mm, than areas 1 and 2, separated by 0.35 mm. E, The mutual information (in bits per event) between the amplitude lists of areas 1 (blue) and 9 (red) with the amplitude lists of all ROIs. The mutual information is a decreasing function of the separation between ROIs attributable to the fact that the noise increases with distance. A sharper decrease is evident through region 6 because of the lower density there (see also A), which induces lower connectivity. The effect of such fluctuations on information measurements is elaborated in Results. F, Histogram of the fluctuations around the mean of fluorescence amplitudes, averaged over n = 10 experiments. The fit is to a Gaussian (R2 = 0.99).
The nonrandom structure is revealed once the simultaneous amplitudes of two areas are plotted one against the other (Fig. 2D). A noisy yet clearly linear relationship between the amplitudes is evident, with a gain of 1.04 in this instance.
Once the amplitude lists are independently normalized for each area, the relationship between lists of neighboring ROIs was always linear and very close to 1. Measuring this slope over 93 pairs of neighboring ROIs (∼300 μm apart) gave a mean deviation of only 0.013 ± 0.0015 from 1. Taking into account that these are normalized data, the conclusion is that the deviations from averages, measured in percentages, are practically equal in neighboring ROIs regardless of what that average is. An existing model (van Rossum et al., 2002) gives an example of how, in a feedforward network, such linearity is possible in near-neighbor transmission and demonstrates that it is a necessary condition for rate coding to be transmitted.
Figure 2D shows how bursts generated somewhere along the line maintain their relative amplitude on the other parts of the line: big waves stay big. This is not an exact relationship attributable to the noise, and distortion becomes more significant as the ROIs are farther apart (Fig. 2D). These observations lead (below) to a model of a relay of identical communication channels of unity gain, each adding to the noise.
The use of information theory allows a variety of comparisons to be made, as well as the application of theoretical models for transmission, which cannot be done with other tools. However, reliability of transmission can be evaluated to some extent also by correlation analysis. The average correlation between pairs of ROIs decreases monotonically as the distance between them increases. For nearest neighbor ROIs, the correlation is 0.89 ± 0.03 (averaged over n = 8 pairs), and it drops to 0.76 ± 0.05 for areas separated by three ROIs (n = 6) and to 0.61 ± 0.03 for those separated by six ROIs (n = 3). The correlations between all possible ROI pairs in this example are all statistically significant (p < 0.016).
As the channel connecting two ROIs gets longer, it also becomes noisier, so that knowing the amplitude list in one area yields less information about the other area. Thus, the mutual information between two areas should be a decreasing function of the distance between them. The mutual information of two ROIs (1 and 9) with all other areas was estimated (see Materials and Methods) and gave the expected decay with distance (Fig. 2E). The mutual information of an area with itself, its entropy, serves as an upper bound for its mutual information with any other area, which is achieved only in a perfect noiseless channel.
Figure 3gives the main result of this paper. Blue circles depict the efficiency of information transfer by plotting the dependence of mutual information on the distance between the ROIs. Mutual information of the amplitude lists was measured and averaged over n = 17 experiments with >700 spontaneous events. The mutual information per event was calculated for all possible pairs of ROIs. To obtain the average self-information (or entropy, i.e., distance = 0) of the amplitude lists for each individual ROI, we quantized the relative entropy by the baseline noise. This is the fluorescence background with no activity, and it is important to note that its SD was significantly smaller than the dynamic range of the signal itself, by a factor of 7.0 ± 0.5. Averaging over 82 ROIs on 10 lines yields a self-information of 3.14 ± 0.09 bits per event.
Mutual information between amplitudes of areas at different distances along a 170-μm-thick linear culture during spontaneous activity. Each blue point is an average over at least four experiments, and many pairs of areas (from 40 pairs at short distances down to a minimum of 16 pairs at the longest distance). The mutual information was calculated numerically and is presented in binary units (bits). This is the amount of information transferred per event. The green curve is the theoretical information capacity of a relay of Gaussian channels, determined by the measured average signal-to-noise ratio of adjacent areas. This and the average distance between ROIs are the only parameters specifying this capacity, and the theoretical curve is not a fit. Red points denote similar measurements in the presence of 40 μm bicuculline and show that inhibition is important for the transmission of information. Inset, Stimulated activity propagates the same as spontaneous activity. Plotted is the average mutual information between the stimulated region of the culture and ROIs at different distances away from it. The theoretical curve is the same as in the main figure.
The salient feature of Figure 3 is that, as distance increases, the information decays rapidly. From a value of 3.14 bits at 0, it falls to ∼1.4 at the nearest neighbor. Within 9 mm, the information decreases by a factor of 4.5 and plateaus out as the distance grows to 17 mm at a value that is less by an additional 30%. There is practically no mutual information left between areas this far apart.
We verified that the correlation between the amplitude lists of two ROIs also decreases as their separation grows. Correlations drop from 0.91 ± 0.02 at a distance of 0.31 mm, to 0.73 ± 0.04 at 2.2 mm, and 0.44 ± 0.03 at 8.8 mm. This remains the asymptotic correlation at 17 mm. Correlations between all pairs are statistically significant, with p < 0.06 even for the most distal pairs.
Similar measurements were repeated for activity stimulated at the center of the culture (8.5 mm from both ends). The inset in Figure 3 plots the mutual information between the stimulated area in which activity commences and areas at different distances along the line. The results were averaged over n = 13 experiments with 477 stimulated population bursts. The average self-information is 3.32 ± 0.38 bits, similar to the spontaneous case. Error bars here are bigger than in the main figure because averages were taken over a small number of ROI pairs (only those that include the stimulated area) compared with all possible pairs. These results, however, are still quite well described by the theoretical curve of a Gaussian chain of information channels (see below, The Gaussian chain model). This demonstrates that the mutual information between two areas is independent of where the activity had started.
The transmission between two areas in not necessarily symmetric; it may be the case that one group of neurons (e.g., ROI A) makes more connections onto a second group (ROI B) than B does onto A. We checked for asymmetry in 18 nearest neighbor ROIs in n = 3 cultures. The direction of activity propagation was first determined by monitoring distal parts of the culture, in which the video resolution was sufficient to tell us which side fired first (Feinerman et al., 2005).
Mutual information between neighboring ROIs was then measured for the two directionalities: activity propagates either from ROI A toward ROI B or from B to A. The averaged absolute value of the difference between information propagation in each direction was small, 0.19 ± 0.04 bits (n = 18). This number should be compared with other sources of noise in the system, for example, spatial fluctuations in transmission arising from fluctuations of neuronal density. To evaluate the contribution of density on spatial fluctuations, we considered different pairs of neighboring ROIs in the stimulated experiments, in which directionality is externally determined. These differences in transmission turn out to be larger and were measured on average to be 0.34 ± 0.03 bits (n = 78 couples of nearest neighbor pairs). The left/right asymmetry on a specific location along the line thus turns out to be somewhat smaller than the spatial asymmetry along the line. We conclude that the major contribution to fluctuations in information transfer arise from density fluctuations along the line rather than from any local asymmetry. Both the asymmetry and density effects were averaged over in Figure 3, which takes into account all events regardless of the directionality of transmission.
Red circles in Figure 3 depict mutual information between spontaneous bursts with inhibition blocked by bicuculline (336 events in 9 experiments). Interestingly, after this treatment, the burst amplitudes become higher (Feinerman et al., 2005), but their variance stays the same as in the previous case. The overall result is that there is little effect on the self-entropy (2.9 ± 0.1 bits for n = 44 ROIs). The situation is very different when one looks at propagation of information. A random relationship is evident even between amplitudes of neighboring ROIs rather than the linear one displayed in the absence of bicuculline (Fig. 2D). This is quantitatively manifested in the fact that mutual information is reduced drastically compared with when inhibition is active. Because the value of the self-entropy is similar to the inhibited case, it is clear that the reduction in information is attributable to bad transmission rather than to the sending of low-information messages.
Although lines that are <50 μm in thickness cannot support cultures for extended periods of time (Feinerman et al., 2005), short sections of approximately this thickness can be integrated within thicker lines. If the width of the line is decreased, then the mutual information between areas also decreases. Information transport was measured between two long, 170-μm-thick lines that were connected by a thin (80 μm) and short (500 μm) bridge. The mutual information between neighboring areas drops to 0.4 ± 0.21 bits (n = 9 pairs), a factor of approximately 3 smaller than what was measured for the full 170 μm line.
As a control, we checked the effect of the length of a period of quiescence on the burst that follows it. In principle, the time required for recovery of the neurons could create correlations between the activities of areas that have just been active together. The mutual information was calculated between the lists of burst amplitudes, and the corresponding lists of the IBIs preceding these bursts. The mutual information was only 0.17 ± 0.023 bits (n = 132 ROIs in 16 experiments) and 0.066 ± 0.026 bits (n = 62 ROIs in 7 experiments) when bicuculline was added. These values are much lower than the measured mutual information between different areas.
The Gaussian chain model
Activity that passes along the line may be regarded as traveling through a chain of ROIs, the output of one serving as the input of the next. The properties of such a chain may be inferred from those of its components: pairs of adjacent ROIs. Data gathered from neighboring ROIs may be regarded as samples from both the input and the output of a unity gain Gaussian channel. As explained in Materials and Methods, the capacity of such a chain is specified by two parameters, the SNR of a single channel and the number of channels in the chain (n). These two parameters are directly obtained from the experimental data.
The SNR of a neighboring pair of ROIs is the ratio of two terms, formally the variance of the signal divided by the variance of the noise. The signal variance is just the variance of the amplitude list, and we average this over both ROIs in the pair. This should be divided by the variance of the channel noise. The noise for our unity gain channels is the arithmetical difference between input and output. Because we measure both the input and the output lists of the pair, the random noise terms are experimentally available. Averaging over 91 nearest neighbor ROIs gives an SNR of 9.38 ± 0.89.
The mean distance between the centers of two neighboring ROIs (averaged over the same 91 pairs) is 0.39 ± 0.01 mm. This determines n, the mean number of channels in x mm of culture as n = x/0.39.
The theoretical capacity of the chain is therefore 
with x the distance between any two areas (in millimeters) and C an upper bound, in bits, on the mutual information between them. The calculated curve (which has no free parameters) is plotted in green in Figure 3 and exhibits a striking fit to the data. The mutual information saturates the capacity bound because, as we showed above, the distribution of input signals is Gaussian. At large distances, the mutual information drops like 1/x, reflecting the sharp decrease in information as the distance grows.
The inherent symmetry of mutual information and of Gaussian chains guarantees that mutual information depends primarily on the distance between input and output. The location of the node at which signals initiated makes no difference as long as we measure differences between nodes. It is thus appropriate to average mutual information over all possible pairs of ROIs with a given distance, even if they happen to be on different sides of the initiation point.
The SNR between all neighboring ROIs is not precisely constant; it may depend on variation in neuronal density and connectivity and fluctuates around a value of 9.38 ± 0.89. Despite this, in the model, we concatenated identical units with a constant SNR of 9.38 to form a Gaussian chain. The fact that this model provides a good description of the experimental measurements shows that the effect of spatial fluctuations in connectivity (and SNR) on the average mutual information is small.
Discussion
The linear neural network is very efficient in passing population waves of activity, as demonstrated by the very high percentage that travel to excite the full 17 mm of the culture. Such stability was suggested previously in a theoretical network with similar converging/diverging connections (Abeles, 1991). There is no rounding off of the signal: it keeps its characteristic form and propagates along the line. This is similar to in vivo measurements in cortex (Bair and Koch, 1996) as well as measurement in slice preparations (Wu et al., 1999).
The one-dimensional structure and anatomy (Feinerman et al., 2005) of the culture induces causality in the propagation of the signal as each area excites its neighbor. The physical distance along the line is easily measured and can be identified with the temporal evolution of the activity, which is much harder to quantify. This is beautifully manifested by the way in which the measured values of mutual information fall unto a curve once plotted versus distance in Figure 3. One dimensionality also allows the accurate comparison of this curve with one obtained from a simple theoretical model of linear information transport. This comparison suggests that the main obstacle for information propagation, at the natural dynamic range exhibited by both spontaneous and evoked activity, is synaptic transmission noise (Koch, 1999). This noise is even greater for transmission through thin parts of the culture because of the reduced synaptic connectivity (Feinerman et al., 2005). We can conclude that the integration of postsynaptic potentials from a large group of presynaptic neurons serves to decrease the noise of the channel. The linearity of the noise term in the Gaussian chain ensures that the parameters for the theoretical curve do not rely on our choice of ROIs.
Our experimental system comprises neurons that are uniformly distributed along the line, and their connection probability falls with distance (Feinerman et al., 2005). This is similar to previous experimental and theoretical frameworks (Miles et al., 1988; Ermentrout and McLeod, 1993; Idiart and Abbott, 1993; Traub et al., 1993; Golomb and Amitai, 1997) concerning, for example, propagation within cortical layers, but differs from models of discrete-node feedforward networks (Diesmann et al., 1999; Litvak et al., 2003; Reyes, 2003). The connectivity is bidirectional rather than the theoretically preferred feedforward-only structure, and some feedback connections must exist. However, a basic theorem in the information theory approach that we use ensures that the existence of feedback will have no effect on the channel capacity as long as there is no memory of the noise in the system (Cover and Thomas, 1991).
Using the close relationship between fluorescence levels and rate codes shows that this linear layered network cannot transmit rate codes over long distances. It is perhaps suggestive that, when rate code is used to convey sensory information in the lateral spinothalamic pathway, only two synapses are used between the first-order sensory neuron and the third-order neuron in the thalamus (Nauta and Feirtag, 1990; Kandel et al., 1991). Because our experimental setup provided us with little temporal information regarding individual spikes, we cannot rule out reliable propagation of temporally coded information along the one-dimensional cultures.
Because the dynamic range of stimuli in this experimental setup is limited to the variability of the spontaneously generated or externally stimulated signals, the absolute value of the self-information, or entropy, was not high. However, the important issue is our ability to monitor and measure quantitatively the propagation of that information from the initial site to the rest of the line. An increase in the creation of information would not affect the transport of information, nor the manner in which it decays over distance attributable to noise. The importance of these results is not in finding precise upper bounds on the amount of rate-coded information that may be conveyed through the linear culture but rather in understanding the mechanisms that prevent information from reliably reaching distant areas.
Blocking inhibition by addition of bicuculline increases the propagation velocity of population bursts (Feinerman et al., 2005) but leads to a sharp drop in information transmission rates. The limited information that does propagate through the culture relies on the existence of a self-regulated excitatory/inhibitory balance that is disrupted during addition of bicuculline. Without inhibition, the input/output relationship between parts of the culture may induce runaway activity and loss of correlation (Beggs and Plenz, 2003). It was argued previously (Douglas and Martin, 1991; Berman et al., 1992) that the absence of inhibition would lead to saturation of the signal and the loss of gain control. This seems to be consistent with our findings.
One must worry whether burst amplitudes or durations may be linked to the amount of neurotransmitters depleted in a previous burst (Murphy et al., 1992; Staley et al., 1998). The uptake of these neurotransmitters happens over long timescales and may control the amplitude of a subsequent burst. If this mechanism were to determine burst amplitudes, information could be regarded as stored in each area (in the form of available neurotransmitters) rather than transmitted across the culture. This possibility was disqualified in large part by the fact that the mutual information between the period of quiescence and the amplitude of the following burst is small compared with that between neighboring areas. Furthermore, such a mechanism would not explain the fact that mutual information is a decreasing (rather than constant) function of the distance. The mutual information encoded in the history is not 0, however, which could explain why mutual information, as well as correlations, does not decay to 0 at large distances (Fig. 3).
In relation to previous theoretical studies, this system may be viewed in two ways. First, this model for a layered network possesses real, biological components and connections (synapses), unaffected by computational considerations, and thus complements the existing theoretical models. Second, this model is a bridge between the in vivo system and its theoretical models in the sense that the activity suggested by different theoretical models can be more readily compared with our simply structured experimental system than to the cortex.
Comparisons can be made to the theoretical works mentioned in Introduction. The excitatory/inhibitory balance exhibited by this experimental system is not sufficient to allow a stable propagation of rate codes (Shadlen and Newsome, 1994, 1998). Overall, the information loss over distance is better described by the predictions of Litvak et al. (2003), but the specific mechanism by which information is lost may be different.
Although the measurement protocol for spontaneous and stimulated activity was very different, the dependence of information on distance was identical. Information decay is only a function of distance so, for example, the relative decay from the initiation point to a distance L will be the same as the decay from a point at L to a distance 2L. We can conclude that some information does pass between areas even if they are far from the burst initiation site. Litvak et al. (2003) use the fact that activity evolves into a fixed stationary rate (Diesmann et al., 1999; Litvak et al., 2003) to disqualify the use of rate coding. In the case of a fixed point, the decay in distance would be practically immediate, followed by a region with little or no information, which is different from what we observe.
Our experimental results show that information contained in the deviations from this stationary rate may still pass through a layered network. Reyes (2003) concludes that rate codes may be passed through layered networks despite synchronization. His conclusions are similar to ours, because we also see that synchronous activity (Feinerman et al., 2005) can carry a rate code. Reyes (2003) investigates the effect of noise in the creation of synchrony and not on propagation itself. However, our additional observation is that transmission noise interferes with this transmission.
The architecture of our experimental system does not involve discrete nodes or a strictly feedforward connectivity. Comparisons to the models of Diesmann et al. (1999) and Litvak et al. (2003) are strictly valid only to the extent that they address directly the issue of information transfer through linear neuronal structures. One might expect that, in locally recurrent networks, self-consistency relationships should push firing rates into a fixed point as activity propagates (Abbott, 1992; Idiart and Abbott, 1993). However, because such a fixed point is also predicted by the discrete-node feedforward models, this leaves our conclusions intact: the linear culture can transmit rate-coded information through fluctuations around this fixed point, but noise in the synaptic transmission impedes its propagation.
Footnotes
-
This work was supported in part by Israel Science Foundation Grant 298/01 and the Minerva Foundation (Munich, Germany). We thank Menahem Segal for support and insight and Michail Tsodyks, Moshe Abeles, and Haim Sompolinsky for useful discussions. We also thank Varda Greenberger and Ruth Tal for biological aid.
- Correspondence should be addressed to Ofer Feinerman, Department Physics of Complex Systems, Weizmann Institute of Science, Herzel Street, Rehovot 76100, Israel. Email: ofer.feinerman{at}weizmann.ac.il
