Abstract
Learning and long-term memory rely on plasticity of neural circuits. In adult cerebral cortex, plasticity can result from potentiation and depression of synaptic strengths and structural reorganization of circuits through growth and retraction of dendritic spines. By analyzing 166 distributions of spine head volumes and spine lengths from mouse, rat, monkey, and human brains, we determine the “generalized cost” of dendritic spines. This cost universally depends on spine shape, i.e., the dependence is the same in all the analyzed systems. We show that, in adult, synaptic strength and structural synaptic plasticity mechanisms are in statistical equilibrium, the numbers of dendritic spines in different cortical areas are nearly optimally chosen for memory storage, and the distributions of spine lengths and head volumes are governed by a single parameter—the effective temperature. We suggest that the effective temperature may be viewed as a measure of circuit stability or longevity of stored memories.
Introduction
Learning and long-term memory formation in the brain are attributed to changes in synaptic connectivity (Tanzi, 1893; Hebb, 1949). Several substrates for such changes, or plasticity, had been identified experimentally. Some plasticity mechanisms act by modulating the properties of existing synaptic connections (Bliss and Lomo, 1973; Markram et al., 1997; Turrigiano et al., 1998), whereas others are responsible for structural modifications of circuits in the neural network (Chklovskii et al., 2004; Holtmaat and Svoboda, 2009). In this study, we examine two major plasticity mechanisms capable of altering synaptic connectivity in adult cerebral cortex with a single synapse resolution: potentiation and depression of excitatory synapses on dendritic spines (Yuste and Bonhoeffer, 2001) and structural synaptic plasticity by means of growth and retraction of spines accompanied with the formation of new synapses and elimination of the existing ones (Grutzendler et al., 2002; Trachtenberg et al., 2002). Because the majority of synaptic connections in the cerebral cortex are excitatory and the majority of excitatory synapses are made on dendritic spines (Braitenberg and Schüz, 1998), these plasticity mechanisms are expected to provide the major avenues for altering neural circuits in adult brain. Furthermore, because of their experience-dependent nature, these mechanisms have been suggested to play essential parts in learning and long-term memory formation (Chklovskii et al., 2004; Holtmaat and Svoboda, 2009; Xu et al., 2009; Yang et al., 2009).
To quantify the potentials of synaptic strength and structural synaptic plasticity mechanisms for learning and memory, we developed a statistical theory that encompasses both contributions. Our theory is based on the premise that the strength of an excitatory synaptic connection on a spine is correlated with the volume of the spine head (Harris and Stevens, 1988; Schikorski and Stevens, 1999; Matsuzaki et al., 2004; Zhou et al., 2004; Kopec et al., 2006; Arellano et al., 2007; Harvey and Svoboda, 2007; Yang et al., 2008; Zito et al., 2009). Thus, by analyzing the distribution of spine head volumes, one may infer the potential of the neuropil for synaptic strength plasticity. Similarly, the number of different connectivity diagrams that can be built with structural synaptic plasticity is dependent on the distribution of dendritic spine lengths (Escobar et al., 2008). Hence, the shapes of dendritic spines (spine head volumes and spine lengths) (see Fig. 1) in different cortical systems may contain clues to the roles these systems play in learning and long-term memory storage.
Materials and Methods
Distributions of dendritic spine head volumes and spine lengths.
The dataset used in this study was compiled from a number of published articles reporting the distributions of dendritic spine shape measurements, such as spine length, spine neck length, spine head diameter, spine head area, and spine head volume. Initially, 44 such articles were identified through an extensive literature search. Next, because our analyses rely on accurate and detailed distributions, histograms based on <110 measurements or containing less than five bins were excluded. This resulted in 30 publications containing 166 distributions of spine shapes measured in animals of either sex. Table 1 contains a partial list of these experiments. A custom-made MATLAB (MathWorks) algorithm was used to extract accurate quantitative information from high-resolution digital images of distributions contained in these publications.
Our theory uses the distributions of spine head volumes and spine lengths. However, in some studies, these measurements were not reported directly. In these instances, we generated the necessary distributions from the reported data with Monte Carlo sampling algorithms. For example, if an article reported a distribution of spine head diameters, this distribution had to be converted into the distribution of spine head volumes. To this end, we sampled 10,000 spine head diameters and, for each diameter, calculated the corresponding spine head volume, assuming spherical geometry. We note that, because this procedure greatly amplifies systematic errors, only the relative values of spine head volumes can be considered to be reliable. Such data were only used in Figure 2, A and E, to illustrate the universality of the volume-related component of the generalized spine cost and in Figure 3, A and C, to illustrate the increase of the effective temperature, TV, with age. No quantitative conclusions were made in these cases.
In cases in which neck length and head diameter distributions were reported, these distributions were combined to generate the spine length (neck length + head diameter) distribution. Here we assumed that there is no correlation between the two measurements (Benavides-Piccione et al., 2002; Arellano et al., 2007), sampled 10,000 spine neck lengths and head diameters from their corresponding distributions, and randomly paired them with each other to generate the spine length distribution.
Because of the limited resolution of optical microscopy, dendritic volume measurements of <0.075 μm3 (diameter of ∼0.5 μm) were deemed unreliable and were left out from the analyses. Similarly, spine length measurements of <1 μm were thought to be unreliable. These assumptions are based on the limited resolution of light microscopy, as well as the fact that short spines are more likely to be shadowed by dendritic branches (∼1 μm in diameter), resulting in an underestimate of the number of short spines. Fractions of unreliably measured spine head volumes and spine lengths were typically low (see Fig. 5, gray bars), and every considered system contained a relatively large measurement range and fraction of spines suited for the analyses.
Dendritic length density and other anatomical parameters of cortical neuropil.
Dendritic length density, ρd, which is the combined length of dendrites in a unit volume of neuropil, is a key parameter of the presented theory. Previously, we estimated the value of ρd in several cortical systems (Escobar et al., 2008): 0.48 μm−2 in mouse occipital cortex, 0.59 μm−2 in rat CA1, 0.47 μm−2 in monkey primary visual cortex area V1, and 0.42 μm−2 in human temporal cortex. These estimates were based on the ratio of the volume density of asymmetric synapse, ns, and the spine frequency on dendrites, fd.
In this study, we analyzed several other systems. We found that ρd = 0.51 μm−2 in mouse barrel cortex. This result was calculated using the reported values of ns = 0.92 μm−3 (layers L2/3, postnatal day 16) (De Felipe et al., 1997) and fd = 1.8 μm−1 (L2/3, postnatal day 15) (Ultanir et al., 2007). In mouse visual cortex, we estimated that ρd = 0.66 μm−2, calculated based on ns = 0.66 μm−3 (all layers, adult) (Cragg, 1967) and fd = 1 μm−1 (L3, postnatal day 60) (Benavides-Piccione et al., 2002). In rat motor cortex, we found that ρd = 0.3 μm−2, based on ns = 0.47 μm−3 (L5, 4–6 months old) (Jones, 1999) and fd = 1.55 μm−1 (L5/6, 3 months old) (Wallace and Bear, 2004). Direct ultrastructural measurements in layer 1 of young monkey visual and prefrontal cortexes resulted in ρd = 0.35 μm−2 (Peters et al., 2001) and 0.5 μm−2 (Peters et al., 1998). Because the exact values of ρd in different systems are unknown and because the conclusions of this study are not very sensitive to the values of ρd in a biologically plausible range (0.1–1.0 μm−2) (see below and Fig. 6B), we used the value of ρd = 0.5 μm−2 for all the analyzed systems.
Parameter δ, which is the sum of the average radii of dendritic branches and axonal boutons, was estimated to be in the range of 0.5–0.7 μm in mouse and rat and 0.95–1.35 μm in monkey and human cerebral cortexes (Escobar et al., 2008). Again, because exact values of this parameter are not known in all the analyzed systems and because the conclusions of our theory do not depend strongly on the values of δ in the 0.5–1.5 μm range (see Fig. 6C), we used δ = 0.7 μm throughout this study.
Parameter ΔV is the change in the spine head volume corresponding to the unitary change in synaptic strength. We assumed that this parameter fundamentally depends on the molecular composition of synaptic machinery, which is expected to be the same in different species or cortical areas. Hence, in all calculations, ΔV was treated as a fundamental constant. At present, we are unable to provide an accurate estimate of this constant. However, conclusions of our theory are not sensitive to the values of ΔV in the biologically plausible range of this parameter (0.001–0.11 μm3) (see below and Fig. 6C), and we used ΔV = 0.05 μm3 for all the analyzed systems.
Model of a dendritic spine.
We modeled dendritic spines as protrusions extending along straight lines perpendicularly to the dendritic shafts (see Fig. 1A). This is a reasonable simplification, motivated by inspection of light and electron microscopy images of dendritic spines. In reality, the entropy associated with different conformations of dendritic spine necks and deviations of dendritic spines from the normal direction, leading to new connectivity patterns, may not be negligible. This entropy, however, amounts primarily to a constant contribution (Escobar et al., 2008) and does not affect the main conclusions of this study (see Discussion).
In our model, dendritic spines have straight necks and spherical heads with a synapse located at the furthest tip of the spine. This model is more appropriate for long spines and may not describe well short stubby spines, which, as mentioned above, were already disregarded because of their short lengths. The joint probability density for spines of head volume V and length s, or spine shape probability density, is denoted with p(V, s). The probability densities for V and s alone are given by the following expressions:
The spine shape, spine head volume, and spine length probability density functions are normalized to one:
In the following, we calculate synaptic entropy, which is the natural logarithm of the number of different connectivity configurations attainable with synaptic strength and structural synaptic plasticity mechanisms combined, making a simplifying assumption that V and s are independent variables. This assumption is based on the idea that, in a crowded neuropil environment, a dendritic spine is confined to the space between the presynaptic bouton and the postsynaptic dendrite. In such a setting, potentiation/depression of the synapse is likely to result in the spine head diameter changes at the expense of the spine neck length, leaving s unchanged. Spine length, s, is defined by the relative positions of the presynaptic and postsynaptic branches and, therefore, is independent of V. To substantiate this assumption, we performed numerical simulations based on the data from the studies by Benavides-Piccione et al. (2002) and Arellano et al. (2007) in which spine head areas and spine neck lengths were measured in individual spines. We calculated s for individual spines, by added neck lengths and head diameters, and confirmed that there is no significant correlation between s and V.
Because of the assumption of independence of V and s, synaptic entropy can be calculated as the sum of the synaptic strength and structural synaptic contributions: I = Istrength + Istructure. These contributions are derived in the next two subsections.
Entropic contribution attributable to potentiation and depression of dendritic spines.
Here we calculate the number of different connectivity diagrams that can be built by altering the strengths (but not the pattern) of existing synaptic connections (for similar calculations, see Balasubramanian et al., 2001; Varshney et al., 2006). Consider a large volume of neuropil containing Ns excitatory synapses on dendritic spines. Each synapse can be in one of n discrete states (i) of synaptic strength. In a stationary state, the strengths of individual synapses can change as a result of synapse potentiation or depression, but the distribution of synapses (Ni) across different states remains the same. The entropy of synaptic strengths, which is the natural logarithm of the number of different circuits that can be built from the population of Ns = N1 + · · · + Nn synapses, is as follows:
where Pi = Ni/Ns is the probability that a given synapse is found in the synaptic state of strength i. To arrive at the final expression in Equation 3, we used the fact that numbers Ni are typically very large in any biologically relevant unit of the cerebral cortex (cortical area, column, layer) and used the Stirling's approximation for the factorials.
Because strength of a synaptic connection is proportional to the dendritic spine head volume (Harris and Stevens, 1988; Schikorski and Stevens, 1999; Matsuzaki et al., 2004; Zhou et al., 2004; Kopec et al., 2006; Arellano et al., 2007; Harvey and Svoboda, 2007; Yang et al., 2008; Zito et al., 2009), probabilities Pi in Equation 3 can be determined from experimental measurements of spine head volumes. In particular, if ΔV is the change in the spine head volume associated with the unitary change in synaptic strength, and pV(Vi) is the spine head volume probability density, Pi = pV(Vi)ΔV.
As discussed above, ΔV is assumed to be a fundamental constant, independent of spine origin. Consequently, the upper bound of ΔV can be determined from the fact that the probabilities Pi = pV(Vi)ΔV have to be <1 for all states of synaptic strengths, i. Hence, ΔV < 1/max(pV(V)). The max[pV(V)] for the systems in which spine head volumes were measured reliably (Table 1) was 9.4 μm−3, leading to ΔV < 0.11 μm3. This upper bound on ΔV was used in Figure 6C. Without the knowledge of the precise value of ΔV, the entropic contribution, Istrength, is ill-defined in absolute terms yet the relative (difference) values of Istrength are informative, because they are independent of ΔV.
For reasonably small ΔV, summation in Equation 3 can be replaced with integration:
In the following, this continuous approximation of Istrength is used to provide a unified description of the entropic contributions attributable to changes in synaptic strengths and patterns. The results of this study are not dependent on this approximation, and Equation 3 could be used instead, leading to the same conclusions.
Entropic contribution attributable to structural synaptic plasticity.
The gaps between axonal and dendritic branches within potential synaptic appositions differ in size (see Fig. 1A). Hence, they must be bridged by spines of various lengths, s, to form synaptic connections. The probability of having an actual synaptic connection at a potential synaptic site is described by the connectivity fraction (Escobar et al., 2008):
The lower bound for ρd can be deduced from the fact that f(s) has to remain below 1 for all values of s. Hence, ρd > max[ps(s)/(s + δ)]/2π, which is in the [0.04–0.21] μm−2 range for the systems from Table 1.
In a large neuropil volume, containing a large number of dendritic spines, Ns, the number of structurally different connectivity patterns that can be attained by spine remodeling is characterized by the following entropic contribution (Escobar et al., 2008):
Discussion of the synaptic entropy maximization hypothesis.
According to the first hypothesis in Results, dendritic spine shapes in a given cortical region of adult brain are in statistical equilibrium. Two conditions must be met in such an equilibrium state: (1) the distribution of spine shapes must be stationary, and (2) for a fixed total “generalized cost” and number of dendritic spines, the spine shape distribution must maximize the total synaptic entropy, I. There are two motivations behind the entropy maximization proposal. Presently, it is not clear which explanation is biologically more plausible. Nonetheless, both explanations lead to mathematically identical formulations and do not affect the results of this study. First, it is possible that synaptic entropy has been maximized in the course of evolution of the cerebral cortex. Large synaptic entropy is beneficial for learning and memory storage because it can give rise to a large number of connectivity diagrams that can be attained with strength and structural synaptic plasticity mechanisms.
Second, because of very large numbers of dendritic spines in any macroscopic cortical unit (area, column, layer), it is reasonable to suppose that spines are experimentally observed in states for which the entropy of their shape configurations is maximal. This supposition requires the assumption of ergodicity or randomness of spine shape configurations (Pathria, 1996) and may appear contrary to the idea of specificity of synaptic connectivity. We note that the assumptions of ergodicity and specificity are not necessarily mutually exclusive. This is because the majority of experimentally measured spines in a small cortical region receive synapses from very distant presynaptic cells (Stepanyants et al., 2009). Only a very small fraction of spines on a dendritic branch of a neuron share presynaptic partners (Fares and Stepanyants, 2009) and may be correlated in size. Therefore, the properties of nearby synapses and, consequently, the experimentally measured spine shapes are expected to be independent. Because transient states of dendritic spines (such as those encountered during spine retraction and outgrowth) are short-lived, spines spend most of their lifetime in synaptic states, and the entropy of spine configurations is expected to equal synaptic entropy. This leads to the synaptic entropy maximization hypothesis.
Determining the generalized cost of dendritic spines from the entropy maximization principle.
We denote the generalized cost of a dendritic spine of head volume V and length s with ε(V, s). The total cost of Ns dendritic spines to the organism can be obtained by integrating ε(V, s) with the spine shape distribution function:
According to the first hypothesis in Results, the spine shape distribution function, p(V, s), in adult brain maximizes synaptic entropy subject to two constraints: p(V, s) has to be normalized according to Equation 2, and the total cost of dendritic spines to the organism, E, in Equation 7, must remain fixed. Such constrained optimization problem is typically solved by maximizing the objective (or Lagrange) function, which combines synaptic entropy with the constraints weighted by Lagrange multipliers (Pathria, 1996):
Because of the analogy between this optimization problem and the treatment of non-equilibrium Fermi and Boltzmann gases in statistical physics (Pathria, 1996), we had chosen (without loss of generality) to denote the Lagrange multipliers in Equation 8 with −1/T and μ/T. Parameter T, or the effective temperature, controls the relative contributions of the dendritic spine cost and the synaptic entropy to the objective function, L. Because the generalized cost of dendritic spines is always detrimental to the organism, the effective temperature must be non-negative, T ≥ 0. Parameter μ is referred to as the effective potential. This parameter describes the contribution of a single spine to the objective function and can be positive or negative. It follows from Equation 8 that the effective temperature, the effective potential, and the generalized spine cost must have the same units. These units are referred to as arbitrary cost units or [cu].
Because the experimentally measured spine shape distribution function, p(V, s), is precisely the one that maximizes the objective function, the functional derivative of the objective function with respect to p(V, s) must be zero. Taking the functional derivative of Equation 8, we arrived at the following:
The generalized cost of dendritic spines was found from this equation (for similar methodology, see Balasubramanian et al., 2001; Varshney et al., 2006):
Because of the simplifying assumption of independence of V and s, which led to I = Istrength + Istructure, the generalized cost of dendritic spines in Equation 10 splits into spine-head-volume-related and spine-length-related components. These components are captured by the dimensionless functions ε̃V(V) and ε̃s(s), which are entirely dependent on experimentally measurable parameters (see Fig. 2A,B). The effective temperature and potential determine the scale and the offset for the generalized cost, ε(V, s).
In Results, we use a slightly different formulation of the problem, in which two different temperatures, TV and Ts, were assigned to the volume and length dimensions of dendritic spines. Subsequently, instead of maximizing the Lagrange function of Equation 8, we minimize the grand potential of synaptic connectivity (Pathria, 1996):
Such two-temperature formulation is used to describe quasi-equilibria in some statistical physics problems (Tool, 1946; Nieuwenhuizen, 1998), in which different components of the systems are not in equilibrium with each other but are in quasi-equilibria on their own. Results shown in Figures 2, E and F, and 3A suggest that this may be the case in the developing cerebral cortex. Quasi-equilibrium state may result from synaptic strength and structural synaptic plasticity mechanisms having separate energy budgets or because the equilibration timescales of the two plasticity mechanisms are much shorter than the overall equilibration time (i.e., the time it takes to reach adulthood). Needless to say, the optimization problems of Equations 8 and 11 become identical in statistical equilibrium where TV = Ts = T.
The idea of quasi-equilibrium of spine shapes in the developing brain is beyond the scope of this study, and no quantitative conclusions are drawn from developing systems. The two-temperature formulation is only introduced as a convenient way of testing the statistical equilibrium hypothesis in adult. Because in statistical equilibrium TV and Ts must equal, in Results we calculate these effective temperatures based on the experimental distributions of spine head volumes and spine lengths and infer statistical equilibrium from the comparison of the two temperatures.
The cost of dendritic spines resulting from minimization of Equation 11 has the following form:
This expression can be broken down into two independent contributions to the cost: one related to volume of spine head and another related to spine length:
Here μV + μs = μ of Equation 12.
Universality of the generalized spine cost and the reference system.
According to the second hypothesis in Results, the generalized spine costs, ε(V, s), and therefore their components εV(V) and εs(s), are the same in different cortical systems. As a result, appropriate choice of parameters TV,s and μV,s should make the εV(V) and εs(s) curves calculated for different systems collapse on top of each other (see Fig. 2E,F). It is evident that the choice of such parameters TV,s and μV,s is not unique. This is because all the effective temperatures can be rescaled by two arbitrary positive multiplicative factors, one for TV and one for Ts. By fixing these factors, we define the absolute scales for the two effective temperatures. Similarly, the effective potentials, μV,s, are defined up to two arbitrary additive constants, the sum of which specifies the absolute energy scale.
In choosing the absolute effective temperature and energy scales, we used mouse dentate gyrus (Knafo et al., 2009) as a reference system (Table 1, system 8). The choice of this system as the reference was motivated by several considerations. First, this system contains the largest number of measured dendritic spines (n = 10,677). Second, measurements were performed in adult brain (12–14 months), and both spine head areas and spine neck length were measured, making accurate estimates of spine head volume and spine length probability densities possible. Finally, because of continued neurogenesis, connectivity in dentate gyrus remains very plastic even in adult. Hence, this system may have a better chance of reaching statistical equilibrium. With no loss of generality, we set TV = Ts = 1 cu for the reference system. Parameters μV and μs were chosen in such a way that the generalized costs εV(V) and εs(s) would have straight line asymptotes passing through the (0,0) points. This choice has no effect on the results of this study, with the exception of Figure 6A, in which the effect is typically small (see Fig. 6B). Because dendritic spine costs appear to be linear in the limit of large V and s, the reference costs εV(V) and εs(s) were linearly extrapolated onto the larger V and s values. The resulting reference costs are shown in Figure 2, C and D.
The values of the parameters TV,s and μV,s for all the systems from Table 1 were determined according to the last two expressions in Equation 13. The reference costs were used in the left sides of these expressions, and the right sides contained ε̃V(V) and ε̃s(s) of individual systems, making this formulation equivalent to a linear regression problem. Best-fit values of TV,s and μV,s, and the corresponding 95% confidence intervals were obtained from the fitting procedures. As described above, points corresponding to spine head volumes of <0.075 μm3 and spine lengths of <1.0 μm were ignored during fitting. Next, by plugging the best-fit values of TV,s and μV,s into Equation 13, we calculated the generalized spine costs, εV(V) and εs(s), for every system (see Fig. 2E,F).
Dependence of the synaptic entropy on the number of spines.
As a result of the above described universality and statistical equilibrium, the functional forms of the probability densities of dendritic spine head volumes and spine lengths, as well as the connectivity fraction in any given cortical system are governed by only four parameters (see Eqs. 5 and 10): ΔV, ρd, δ, and T:
The effective potentials, μV,s, in Equation 14 are determined from the normalization conditions of Equation 2.
Other quantities describing the state of cortical neuropil can be expressed in terms of the four parameters by using Equations 4–7:
The choice of the four parameters describing the state of cortical neuropil is not unique. For example, in the following, it is more convenient to use the total energy per spine E/Ns = (EV + Es)/Ns instead of T. This way, the total synaptic entropy per spine, I/Ns = (Istrength + Istructure)/Ns, can be expressed from Equation 15 in terms of E/Ns and ΔV, ρd, δ:
The functional form of g in Equation 16 cannot be deduced analytically as a result of the empirical nature of the universal spine costs, εVu(V) and εsu(s), but the shape of this function can be obtained numerically. Here, this was done by using the Newton's method for solving nonlinear systems of equations. An alternative form of Equation 16 is used in Results.
Results
Hypotheses
Our theory is based on two hypotheses that are rooted in statistical theory and are supported by the experimental data. Formal descriptions and motivations for these hypotheses are given in Materials and Methods.
Statistical equilibrium hypothesis
The distribution of spine shapes in a given cortical region changes during the development of the brain but reaches a statistical equilibrium in adult. Two conditions must be met in statistical equilibrium. First, in this dynamic equilibrium, individual spines may change their shapes, new spines may appear, and the existing ones may be retracted, but the overall distribution of spine shapes must remain stationary. Second, by drawing analogy with the treatment of equilibrium ensembles in statistical physics (Pathria, 1996), we theorized that, for a fixed total generalized cost and number of dendritic spines in a given cortical region, the experimentally observed spine shape distribution must maximize the entropy associated with the number of different circuits that can be built with synaptic strength and structural synaptic plasticity mechanisms combined.
Universality of the generalized spine cost
The generalized cost of a dendritic spine to the organism is a universal function of the spine shape, i.e., it is not dependent on the origin of the spine (different species, age, cortical areas, layers, or dendritic domains).
In the next two subsections, we analyze 166 experimental distributions of spine head sizes and spine lengths (Table 1), measured in culture or slice preparations, in different cortical areas of mouse, rat, monkey, and human, and show that these distributions are consistent with the statistical equilibrium and the universality hypotheses. In the subsequent subsections, we illustrate the predictive power of the theory resulting from the hypotheses and show that the cerebral cortex is nearly optimally designed to store information in strengths and patterns of synaptic connections.
Evidence supporting the spine cost universality hypothesis
Consider a small volume of cortical neuropil (Fig. 1A). The majority of excitatory synaptic connections in this volume are formed on dendritic spines that bridge the gaps [potential synapses (Stepanyants et al., 2002; Stepanyants and Chklovskii, 2005)] between axonal and dendritic branches. The strengths of these synapses are correlated with the volumes of the spine heads (Harris and Stevens, 1988; Schikorski and Stevens, 1999; Matsuzaki et al., 2004; Zhou et al., 2004; Kopec et al., 2006; Arellano et al., 2007; Harvey and Svoboda, 2007; Yang et al., 2008; Zito et al., 2009), and, thus, the number of synaptic connectivity patterns attainable with synaptic strength plasticity can be deduced from the distribution of spine head volumes (Fig. 1B,D,F,H). Because the gaps between axonal and dendritic branches within potential synaptic appositions differ in sizes, they must be bridged by spines of various lengths to form excitatory synapses. Therefore, the number of synaptic connectivity patterns attainable with structural synaptic plasticity is dependent on the distribution of spine lengths (Fig. 1C,E,G,I). It is not entirely clear whether a spine outgrowth precedes the formation of a presynaptic bouton or whether new dendritic spines preferentially contact existing boutons in the process of synaptogenesis (Knott et al., 2006). Theoretical frameworks for these two models of synapse formation have been described by Escobar et al. (2008). Because the two models are quantitatively very similar, in the present study, we only considered the latter model, in which a potential synapse is an apposition between a bouton and a dendrite that can be bridged by a dendritic spine.
Synaptic strength and structural synaptic plasticity of neural circuits. A, Left, Schematic illustration of a spherical neuropil volume of 3 μm in diameter. Densities of axons (blue segments), dendrites (red segments), axonal boutons (blue spheres), and dendritic spines (red protrusions) are based on mouse occipital cortex. Boutons and spines that are in contact form excitatory synaptic connections. Any axonal and dendritic segment pair is potentially connected if it can be bridged by a dendritic spine. Right, In this model, spines are assumed to extend perpendicularly to dendritic shafts and have straight necks and spherical heads. B, D, F, H, Spine head volume probability densities (n = 70) for dissociated culture and cortical slices from different areas of mouse, rat, monkey, and human brains. C, E, G, I, Spine length probability densities (n = 96).
Any change in the configuration of a neural network, such as a change in the size of a single spine head or a change in the location of a single spine, may alter the functional connectivity of the network. To quantify the number of possible connectivity configurations, we introduced the notion of synaptic entropy, I. This entropy is the natural logarithm of the number of different connectivity configurations attainable with synaptic strength and structural synaptic plasticity mechanisms combined. Thus, I consists of synaptic strength and structural entropic contributions, I = Istrength + Istructure (for details, see Materials and Methods).
Longer spines can make larger contributions to I as they can contact larger numbers of potential boutons. Similarly, wider distributions of spine head volumes, containing larger spine heads, can give rise to higher numbers of possible connectivity configurations. However, something prevents dendritic spines from growing beyond a few micrometers in length (s) or from having very large spine head volumes (V). We call this impeding factor the generalized cost of dendritic spines, ε(V, s). It is not known what contributes to the generalized spine cost. One part of the cost must be related to maintenance of dendritic spines and may be metabolic in nature (Attwell and Laughlin, 2001) or functional, e.g., associated with molecular transport or signal attenuation in the spine neck. Another part of the cost must be related to synaptic plasticity, associated with new spine construction and potentiation, as well as depression and degradation of the existing spines. Although the exact composition of the generalized spine cost is not known, it is expected that both maintenance- and plasticity-related components are costlier for longer spines with larger spine heads and, thus, ε(V, s) must be an increasing function of V and s.
Making no assumptions about the nature of the generalized spine cost, we were able to deduce its functional form from the statistical equilibrium hypothesis (for details, see Materials and Methods):
Equation 17 show that the generalized spine cost, ε(V, s), is the sum of spine head volume and spine length dependent contributions, εV(V) and εs(s). The functional forms of εV(V) and εs(s) are captured by their dimensionless components, ε̃V(V) and ε̃s(s), which are completely determined by experimentally measurable neuropil characteristics. Specifically, these components depend on spine head volume and spine length probability densities, pV(V) and ps(s) (Fig. 1B–I), dendritic length density, ρd, which is the total length of dendrites in a unit neuropil volume, sum of the average bouton and dendritic branch radii, δ, and the unitary change in spine head volume, ΔV, corresponding to the unitary change in synaptic strength (for the numerical values of these parameters, see Materials and Methods).
The shapes of ε̃V(V) and ε̃s(s) for all the analyzed systems are shown in Figure 2, A and B. Parameters of the theory, TV and Ts, which determine the steepness of the corresponding dendritic spine cost contributions, are referred to as the effective spine head volume and spine length temperatures. Parameters μV and μs specify the offsets of the dendritic spine cost components and are termed the effective potentials. The effective temperatures and potentials should not be confused with the physiological temperature of the brain and the chemical potential, regardless of the similarities in the theoretical treatment. Instead, because the effective temperatures uniquely determine the distributions of spine shapes according to Equation 14, one can think of these parameters as the measures of the average dendritic spine length and head volume. Higher-temperature cortical areas contain longer spines with larger spine heads and vice versa. The effective potential, according to Equation 11, is the cost of creation of a single spine. Below we describe how the effective temperatures and potentials of a cortical region can be measured.
Universality of the dendritic spine cost. A, B, Functions ε̃V(V) and ε̃s(s) calculated based on the distributions from Figure 1. C, D, Volume- and length-related components of the reference cost of dendritic spines were created by linear extrapolation (dashed lines) of ε̃V(V) and ε̃s(s) for adult mouse dentate gyrus (Table 1, system 8; blue lines). E, F, The generalized costs associated with dendritic spine volume, εV(V), and spine length, εs(s), are universal.
According to the second hypothesis, ε(V, s) has a universal shape, i.e., this function is the same in different systems. The implication of this hypothesis is that it should be possible to adjust parameters TV,s and μV,s in Equation 17 in such a way that the ε(V, s) curves for different systems would collapse on top of each other. To this end, we selected rat dentate gyrus as the reference system (Fig. 2C,D) [Table 1, system 8 (Knafo et al., 2009)] and, with no loss of generality, chose TV = Ts = 1 cu for this system (see Materials and Methods). Here, [cu] denotes arbitrary cost units used in describing T, μ, and ε. Consistent with the second hypothesis, the dendritic spine cost curves collapse on top of each other (Fig. 2E,F). The goodness of this collapse can be described by the adjusted R2 coefficient, which was 0.94 ± 0.05 (mean ± SD, n = 70) for εV(V) and 0.95 ± 0.06 (n = 96) for εs(s).
In agreement with the idea of volume being a major resource, limiting the number of possible synaptic connectivity configurations (Konur et al., 2003; Chklovskii, 2004; Varshney et al., 2006), the generalized spine cost ε(V, s) in different systems increases linearly with the volume of spine head. This result is also consistent with the exponential spine volume distribution measured with electron microscopy in rat CA1 (Mishchenko et al., 2010). In addition, ε(V, s) appears to be linear in s for spines longer than 1 μm. For shorter spines, the dependence on s is nonlinear. This nonlinearity could result from crowding of the space in the vicinity of dendritic shafts attributable to inhibitory shaft synapses and glial processes. It is also likely that systematic undercounting of short spines attributable to the limited optical resolution and the effect of shadowing of short spines by dendritic branches contributed to the nonlinearity.
Effective temperature and evidence of statistical equilibrium in adult cerebral cortex
Values of the effective temperatures, TV and Ts, which lead to the collapse of the generalized spine cost curves, are shown in Figures 3 and 4A (also see Table 1). In younger animals, dendritic spines are longer and have smaller heads, and Ts progressively decreases and TV increases in development (Fig. 3A). In statistical equilibrium, TV and Ts must equal (see Materials and Methods), and, thus, there is no statistical equilibrium in the developing brain. Both effective temperatures in rodents seem to stabilize by the end of the first postnatal month.
Statistical equilibrium in adult cerebral cortex. A, Effective temperature Ts decreases in the course of development, whereas TV increases. These effective temperatures in rodents appear to stabilize approximately at the end of the first postnatal month. B, In wild-type (wt) adult cortical areas, effective temperatures TV and Ts are nearly equal, suggesting that synaptic strength and structural synaptic plasticity mechanisms are in equilibrium with each other. This equilibrium is perturbed in a transgenic (APP/PS1) model of Alzheimer's disease mice (red dots with error bars). C, Effective temperatures TV and Ts are not statistically different in different layers of a given cortical area or on apical and basal dendritic branches. With the exception of very distal apical branches (*p < 0.01, two-tailed Student's t test for samples with unequal variances), TV on CA1 neurons remains constant with spine distance to the soma. In A and C, only the relative measurements (ratios) of TV within individual groups are reliable. Numerals in A–C correspond to the system numbering in Table 1. Error bars indicate SEM values. BC, Barrel cortex; V, visual; M, motor; L, layer.
Consistent with the first hypothesis, synaptic strength and structural synaptic plasticity mechanisms in wild-type adult cerebral cortex appear to be in statistical equilibrium with each other. This equilibrium is implied by the fact that the effective temperatures TV and Ts within several cortical systems are nearly equal (Fig. 3B). In contrast, the statistical equilibrium is perturbed in transgenic (APP/PS1) model Alzheimer's disease mice (Knafo et al., 2009) (Fig. 3B, red points with error bars).
In addition to statistical equilibrium of synaptic strength and structural synaptic plasticity mechanisms (TV = Ts) in individual adult cortical systems, there appears to be equilibrium among different synaptic domains within the systems as well. Figure 3C shows that TV and Ts are not statistically different in layers 2/3 and 5/6 of adult rat motor cortex. The average effective temperature TV is the same on apical and basal dendrites in mouse V1, as well as mouse CA1. In the latter system, however, TV increases with distance from the cell body along apical dendrites and is significantly higher on very distal apical (but not basal) branches.
Implications of spine shape differences across cortical areas
Although synaptic strength and structural synaptic plasticity mechanisms appear to be in statistical equilibrium within individual cortical areas, different cortical areas can have vastly different effective temperatures. Results in Figure 4A suggest that the effective temperature decreases from primary sensory and motor areas to association areas in the neocortex to hippocampal areas (CA1 and dentate gyrus). For example, the effective temperature of monkey V1 is more than twofold higher than those of areas V2, V4, 7a, and 46, and the average effective temperature in rodent hippocampus is approximately threefold lower than that in the neocortex. This finding is consistent with the fact that dendritic spines in CA1 are typically much smaller than their counterparts in other cortical areas (Schikorski and Stevens, 1997; Genoud et al., 2006; Mishchenko et al., 2010).
Spine shape differences across cortical areas. A, Effective temperatures TV (green) and Ts (blue) in hippocampal areas are much lower than those in the neocortical areas. B, C, Likewise, both volume- and length-related components of the generalized cost and synaptic entropy per dendritic spine in hippocampus are lower than the corresponding components in the neocortex. Because of uncertainty in the absolute value of Istrength (see Materials and Methods), only the differences in this quantity are meaningful. TE, Temporal; V, visual; M, motor; BC, barrel cortex; DG, dentate gyrus; A, auditory; FS, forelimb somatosensory.
Knowledge of the effective temperatures in different systems makes it possible to calculate the average volume- and length-related components of the generalized costs per dendritic spine. Figure 4B shows these costs measured from the corresponding minimal cost levels, ΔEV,s/Ns = EV,s/Ns − εV,smin (Fig. 2C,D). Because of the universality of the dendritic spine cost, ε(V, s), the average cost per spine is well correlated with the effective temperature. This cost in rodent hippocampus is several-fold lower than those in neocortical areas. In comparison, synaptic entropy per dendritic spine (in particular, Istructure/Ns) in rodent hippocampus is less affected by the differences in the effective temperatures (Fig. 4C).
Effective temperature in adult cerebral cortex determines the shape of dendritic spines
Because of the universality of the dendritic spine cost and the statistical equilibrium in adult cortical areas, the functional forms of probability densities of dendritic spine volumes and spine lengths are completely determined by only four parameters: ΔV, ρd, δ, and T (see Materials and Methods):
u in εu was added to emphasize the fact that dendritic spine cost is universal. Its functional form is shown in Figure 2, C and D. Because of statistical equilibrium between the spine head volume and the spine length degrees of freedom in adult cortical areas, TV = Ts were replaced with T. The effective potentials, μV,s, in Equation 18 are determined from the normalization conditions for pV,s(s) (see Eq. 2).
Parameters ΔV, ρd, and δ do not change dramatically from one cortical region to another and only have a limited effect on the shapes of pV,s(s) (see Materials and Methods), leaving the effective temperature, T, as the main determinant of the spine shape. To show how well the effective temperature predicts the distribution of dendritic spine shapes, we performed simultaneous single-parameter (T) fits of the experimental spine head volume and spine length distributions with Equation 18. Figure 5 illustrates goodness of the fits for four systems (Benavides-Piccione et al., 2002; Ballesteros-Yáñez et al., 2006) in which both spine head volume and spine length measurements were made reliably (Table 1, systems 1–4). In addition, separate single-parameter fits of all the analyzed pV(V) and ps(s) distributions yielded adjusted R2 of 0.93 ± 0.10 (mean ± SD, n = 70) for pV(V) and 0.94 ± 0.10 (n = 96) for ps(s).
Distributions of spine head volumes and spine lengths (for large dendritic spines) in adult cortical areas are completely determined by a single parameter: the effective temperature of the area. A1, A2, Experimental distributions of spine head volumes and spine lengths (bars) from human temporal cortical area TE (Table 1, system 1) were fitted simultaneously (red lines) with Equation 18 containing a single free parameter, T. Spine head volumes of <0.075 μm3 and spine lengths of <1.0 μm were ignored during fitting (gray bars). The areas under the best-fit lines are normalized to 1. The experimental distributions are scaled along the y-axes to match the areas under the best-fit lines in the regions in the blue bars. The best-fit effective temperature and the adjusted R2 coefficient are shown. Similar results are obtained for mouse cortical areas V1 (B1, B2), auditory/somatosensory cortex areas A1/S2 (C1, C2), and motor cortex area M2 (D1, D2).
Are cortical circuits optimally designed to store information in the strengths and patterns of synaptic connections?
Consider a cortical system with a fixed amount of total generalized cost, E, available to Ns dendritic spines. Assume that the number of spines may vary: the question is, what value of Ns maximizes synaptic entropy? To answer this question, we expressed synaptic entropy per unit cost, I/E, as a function of four parameters: the number of dendritic spines per unit cost, Ns/E, ΔV, ρd, and δ (see Materials and Methods):
Function h in this expression is completely defined by the shape of the universal dendritic spine cost, ε(V, s). Equation 19 clearly shows that, for a constant value of the total cost of dendritic spines, E, and the experimentally observed values of ΔV, ρd, and δ, the total synaptic entropy depends only on Ns.
The shape of the curve in Equation 19 is shown in Figure 6A (solid line). For small Ns, spines must be large and long on average to accommodate the fixed total dendritic spine cost constraint. In this limit, the effective temperature is high, and the total synaptic entropy decays because it is proportional to the number of spines. In the limit of large numbers of dendritic spines, increase in Ns must be accompanied by a decrease in the generalized cost of individual spines. Hence, spines assume shapes for which the generalized cost is minimal. This is the low effective temperature limit. At certain value of Ns, the distribution of spine lengths becomes so narrow that spines occupy all the available potential synaptic sites in that range of spine lengths. At this point, there is only one possible connectivity pattern, and Istructure = 0. A similar argument shows that Istrength = 0 in this limit as well. This abrupt transition occurs at a value of Ns/E = 1/εmin ≈ 0.15 cu−1, where εmin denotes the minimal dendritic spine cost of the reference system (εmin = εVmin + εsmin ≈ 6.7 cu) (Fig. 2C,D).
Cortical neuropil is nearly optimally designed for storing information in strengths and patterns of synaptic connections. A, For a fixed total cost of dendritic spines, E, the dependence of synaptic entropy on the spine number is shown with the black line. The values of I/E versus Ns/E in different cortical systems from Table 1 are shown with red and blue points. B, C, Relative deviations of I/E (for the reference system) from the maximum of the synaptic entropy curve from A. The deviations are small for all biologically plausible values of parameters εmin, ρd, ΔV, and δ. Asterisks show the default parameter values used throughout this study.
Figure 6A also shows the values of I/E and Ns/E calculated for individual cortical systems (red and blue points). The analyzed systems operate in the 0.86–5.0 cu effective temperature range, and their neuropil states are located near the maximum of the synaptic entropy curve. Because large synaptic entropy is advantageous for learning and memory storage, the results of Figure 6A suggest that the cerebral cortex is optimally designed to perform these functions. It is important to note that the maximum of the synaptic entropy curve is quite broad, and neural networks can operate within a large range of spine numbers with little detriment to synaptic entropy. This finding is consistent with close to twofold (maximum/minimum) interbrain variability in volume density of asymmetric synapses observed in many systems (Bourgeois and Rakic, 1993).
Results of Figure 6A do not depend strongly on the values of the parameters εmin, ρd, ΔV, and δ. Figure 6, B and C, shows that the position of the reference system (system 8) with respect to the maximum of the I/E curve does not change substantially if the values of the above parameters are varied in the biologically plausible range (see Materials and Methods). With the exception of very low εmin (<1 cu), the deviation from the optimum remains within 15%.
Discussion
We developed a statistical theory of synaptic connectivity in which the shape of dendritic spines is governed by the generalized cost and two effective temperatures describing spine head volumes and spine lengths. We found that the generalized cost universally depends on spine head volume and spine length, and this dependence is approximately linear for large spines. The effective temperatures in a given cortical area change in development but reach the same equilibrium point in adult, suggesting that synaptic strength and structural synaptic plasticity mechanisms in adult are in statistical equilibrium with each other. We show that anatomical diversity of dendritic spine shapes (lengths and head volumes) across different cortical areas of adult cerebral cortex is primarily governed by a single parameter: the effective temperature.
We demonstrated that the numbers of dendritic spines in human, monkey, mouse, and rat cortical areas are chosen in a way that nearly maximizes their synaptic entropy. Large synaptic entropy, which can give rise to a large number of circuits that can be built with synaptic strength and structural synaptic plasticity mechanisms, is expected to be beneficial for learning and memory storage. Hence, it is possible that maximization of synaptic entropy was one of the driving forces in the evolution of the cerebral cortex. The maximum of the I/E curve in Figure 6A is so broad that, for a relatively large effective temperature range (0.86–5.0 cu) encountered in different cortical systems, the numbers of dendritic spines in these systems are nearly optimally chosen for storage of memories in strengths and patterns of synaptic connections. As it is often the case in biological systems, getting to the exact optimum solution may be too costly and unnecessary. Multiple “good enough” solutions to biological problems usually suffice, giving rise to the diversity of organization observed in individual animals (Prinz et al., 2004).
Our results are based on a diverse experimental dataset (different experimental methodologies, species, brain regions, and ages). Measurements in some of these experiments may not be perfect, but the fact that our conclusions are drawn from 166 distributions surmounts concerns related to the quality of data in individual systems. Conclusions about different cortical areas in this study were often based on spine measurements from single cortical layers. Supporting such generalizations, Figure 3C shows limited evidence that the effective temperatures are equal on apical and basal dendrites of neurons, as well as on neurons belonging to different cortical layers within an area. It is clear that more high-quality experimental measurements of dendritic spine shapes (preferably done with electron microscopy) are needed to further validate the idea of statistical equilibrium of circuits in the cerebral cortex.
The underlying assumption of the presented model is that spine head volume, V, can be used to infer the strength of a synaptic connection. Because spine head volume is proportional to the area of the postsynaptic density (Harris and Stevens, 1989; Harris et al., 1992), which, in turn, is proportional to the number of AMPA receptors (Nusser et al., 1998; Kharazia and Weinberg, 1999; Takumi et al., 1999), we assumed that V changes in finite-size steps, ΔV. Spine head volume quantization fits naturally in the presented framework because the entropy associated with this quantity would be ill-defined otherwise. Unitary change in spine volume, ΔV, was treated as a fundamental biological constant, the value of which is presently unknown. It is not possible to predict ΔV based on the presented model because ΔV enters the generalized energy (Eq. 10), as an additive constant and, thus, does not impact any experimentally measurable parameters. It is also likely that ΔV is too small to be observable in direct light microscopy experiments. However, an indirect estimate of ΔV can be made by correlating spine head volume measurements with the numbers of synaptic receptors obtained with calibrated immunogold localization techniques.
Dendritic spines in this study were modeled as straight processes extending perpendicularly to dendritic shafts. As a result, the entropy associated with different dendritic spine conformations leading to new connectivity patterns was not accounted for by the model. In other words, we are underestimating the number of potential synapses accessible to a spine of length, s. It is not easy to quantify the amount of missing entropy because, on the one hand, spines are too short to be treated in the framework of statistical physics of polymers, and, on the other hand, the amount of available experimental data is not sufficient to deduce the probabilities of different spine conformations. A rough estimate of the missing entropy was provided previously (Escobar et al., 2008), leading to a correction term in the generalized spine cost of Equation 10: −ln[2π(s + δ)Δs/A(s)]. The numerator in the argument of the logarithm in this expression is the cross-sectional area available to a straight spines of length s, and A(s) denotes a larger cross-sectional area available to other spine conformations. It is not clear how exactly A depends on s, yet, for reasonable (non-exponential) dependences, correction to the generalized spine cost amounts primarily to a constant contribution. This is attributable to the slow increase of the above logarithmic correction term compared with the linear increase of the generalized spine cost (Fig. 2F). Hence, we expect that inclusion of the conformational entropy of dendritic spines will not affect the major conclusions of this study.
Could there be an alternative explanation accounting for all the results of the theory? For example, because the distributions of spine head volumes and spine lengths have nearly exponential tails, they lead to linear generalized cost components (see Eq. 17) (Fig. 2). Linear curves can always be collapsed on top of each other, giving rise to a false impression of universality of the generalized spine cost. We do not believe that this explanation is valid because it neither accounts for the TV = Ts condition of statistical equilibrium (Fig. 3B) nor does it explain the maximization of synaptic entropy with respect to the number of spines (Fig. 6A).
The generalized cost of dendritic spines may be a factor limiting the rate of synaptic plasticity. Structural remodeling of large dendritic spines is costly for the organism, and, hence, it is observed less frequently (Kasai et al., 2003; Holtmaat et al., 2005; Zuo et al., 2005; Bourne and Harris, 2007). Because the average generalized cost of dendritic spines is directly related to the effective temperature (Fig. 4), the latter can be viewed as a measure of circuit stability. Low effective temperature areas, such as CA1, are expected to be more plastic than areas in which the effective temperatures are high, e.g., primary neocortical areas. This view is supported by the fact that, in adult rats performing associative memory tasks, the density of dendritic spine in CA1 can increase by as much as 39% over 24 h after training (Leuner et al., 2003). In comparison, motor skill learning and novel sensory experience have much smaller effects on structural synaptic plasticity in the forelimb and primary somatosensory areas, in which only ∼5–7% of additional new spines are formed over a 2 d period (Yang et al., 2009). An experimentally falsifiable prediction of this line of thinking is that the baseline in vivo motility rate in rodent hippocampus is much higher than those in primary neocortical areas.
It is believed that longevity of stored memories is related to stability of dendritic spines (Yang et al., 2009). Thus, higher effective temperature areas, containing larger and presumably more stable dendritic spines, are expected to store longer-lasting memories. Such “hardwired” systems should include primary areas (e.g., sensory and motor), in which there is no clear necessity to change synaptic connectivity in normal adulthood. Consistent with this argument, the highest effective temperatures are found in monkey and rat V1, as well as in rat and mouse motor and barrel cortical areas. In contrast, the effective temperatures of mouse and rat hippocampus are the lowest, which is in agreement with temporary storage and relatively fast turnover of hippocampal (e.g., declarative) memories (Marr, 1970; Frankland and Bontempi, 2005; Fuster, 2009).
Notes
Supplemental material for this article is available at http://www.northeastern.edu/neurogeometry/wp-content/uploads/SM_JN_2011.pdf. The supplemental table shows experimental studies reporting distributions of dendritic spine shapes. This material has not been peer reviewed.
Footnotes
-
This work was supported by NIH Grants NS047138 and NS063494. We thank Dmitri Chklovskii, Vincent Hakim, and Quan Wen for useful discussions.
- Correspondence should be addressed to Armen Stepanyants, Department of Physics, Northeastern University, 110 Forsyth Street, Boston, MA 02115. a.stepanyants{at}neu.edu
