Abstract
We extend previous models for separate development of ocular dominance and orientation selectivity in cortical layer 4 by exploring conditions permitting combined organization of both properties. These conditions are expressed in terms of functions describing the degree of correlation in the firing of two inputs from the lateral geniculate nucleus (LGN), as a function of their retinotopic separation and their “type” (ON center or OFF center and left eye or right eye).
The development of ocular dominance requires that the correlations of an input with other inputs of the same eye be stronger than or equal to its correlations with inputs of the opposite eye and strictly stronger at small retinotopic separations. This must be true after summing correlations with inputs of both center types. The development of orientationselective simple cells requires that (1) an input’s correlations with other inputs of the same center type be stronger than its correlations with inputs of the opposite center type at small retinotopic separation; and (2) this relationship reverse at larger retinotopic separations within an arbor radius (the radius over which LGN cells can project to a common cortical point). This must be true after summing correlations with inputs serving both eyes.
For orientations to become matched in the two eyes, correlated activity within the receptive fields must be maximized by specific betweeneye alignments of ON and OFF subregions. Thus the correlations between the eyes must differ depending on center type, and this difference must vary with retinotopic separation within an arbor radius.
These principles are satisfied by a wide class of correlation functions. Combined development of ocularly matched orientation maps and ocular dominance maps can be achieved either simultaneously or sequentially. In the latter case, the model can produce a correlation between the locations of orientation map singularities and local ocular dominance peaks similar to that observed physiologically.
The model’s main prediction is that the above correlations should exist among inputs to cortical layer 4 simple cells before vision. In addition, mature simple cells are predicted to have certain relationships between the locations of the ON and OFF subregions of the left and right eyes’ receptive fields.
To what extent do cortical receptive fields (RFs) and maps result from simple, activitydependent rules for synaptic development? A primary model system for addressing this is the development of monocular and binocular, orientationselective simple cells and their arrangement into orientation (ORI) and ocular dominance (OD) maps in the primary visual cortex (Hubel, 1982; Wiesel, 1982).
In previous work, we studied the conditions required to develop either OD or ORI preference systems alone, under a general class of synaptic modification rules (Miller et al., 1989; Miller, 1994). This class, “correlationbased” rules (Miller, 1990a, 1997), includes mechanisms that have in common the dependence of the development of a structure on the correlations among the activities of its inputs. These mechanisms include simple versions of Hebbian synaptic modification, activitydependent release, and uptake of diffusible modifying factors, or synaptic sprouting and retraction with activitydependent stabilization. We now examine whether the coexistence of multiple features in single RFs and maps is compatible with correlationbased mechanisms by studying the conditions necessary for the combined development of OD and ORI.
A significant problem is raised by this combined development. ORI maps are continuous across OD column borders, regardless of whether the OD map contains many or few binocular cells, and binocular cells show nearly identical preferred ORIs and spatial frequencies in each eye’s RF (Skottun and Freeman, 1984). How can correlationbased development yield binocularly matched ORI maps, which presumably requires correlation between the activities of the two eyes, and also yield OD segregation, which presumably requires a sufficient lack of interocular correlation?
We address this question through study of the development of simple cells. These are cells with RFs composed of one or more spatially segregated, elongated subregions, each giving exclusively ON (response to light onset/dark offset) or exclusively OFF (response to light offset/dark onset) excitatory input (Hubel and Wiesel, 1962). These comprise most or all cells in layer 4 of cat primary visual cortex (Hubel and Wiesel, 1962; Gilbert, 1977; Bullier and Henry, 1979). In that layer, the segregation of a simple cell’s lateral geniculate nucleus (LGN) inputs appears to underlie its RF properties. Segregation of left and righteye inputs underlies OD (LeVay et al., 1978), whereas segregation of ONcenter and OFFcenter inputs underlies spatial RF structure (Tanaka, 1983; Ferster, 1988; Reid and Alonso, 1995) as proposed by Hubel and Wiesel (1962). This spatial RF structure in turn appears to underlie ORI selectivity (Movshon et al., 1978;Jones and Palmer, 1987; Ferster et al., 1996). Thus, we will study the development of the LGN inputs to cat layer 4 simple cells. We ignore for simplicity the simultaneous development of intracortical connections (Callaway and Katz, 1991; Löwel and Singer, 1992;Durack and Katz, 1996; Ruthazer and Stryker, 1996).
Our previous work studied segregation of only two LGN input types. We determined how OD maps can emerge from competition among inputs serving the left and right eyes (Miller et al., 1989) and how simple cells and ORI maps can emerge through competition among ON and OFFcenter inputs (Miller, 1994). We now study the outcome of correlationbased competition among four input types—ON and OFFcenter cells serving left and right eyes (see Fig. 1a) —to address combined development of ORI and OD.
The primary outcome of our model is a specification of the spatial patterns of correlated activity among these inputs that are sufficient to yield both binocularly matched ORI maps and OD segregation. The activity patterns are robust, not requiring fine tuning. They are simple enough to plausibly exist in spontaneous neural activity before the onset of vision, although they are also plausible attributes of visually induced activity. These results are shown to generalize the results of the earlier separate OD and ORI models: the earlier results remain applicable, but new results emerge, including the conditions for developing binocularly matched orientation maps and a novel prediction for the interocular relationships of ON and OFF RF subregions. These results should in turn remain applicable if further subcategorizations of LGN cell types are considered (see discussion in Appendix ).
It is possible that ORI and OD normally develop at separate times in response to differing activity patterns. We consider such sequential development and find that it increases the range of correlation structures compatible with combined ORI and OD development. It also induces relationships between the ORI and OD maps such as those observed experimentally (Crair et al., 1997b). Simple explanations of the origins of these relationships are proposed.
Although we focus on the development of connections from LGN to layer 4 of cat primary visual cortex, simple cells might also develop through the segregation of monocular, ON and OFFcenter cell inputs in other systems. Possible examples include the avian visual Wulst (Pettigrew, 1979) and the projections from visual cortical layer 4 to upper layers in species in which layer 4 cells are primarily not ORIselective [these include monkey (Blasdel and Fitzpatrick, 1984), ferret (Chapman and Stryker, 1993), and tree shrew (Fitzpatrick, 1996)]. If so, our analysis could also apply to such systems.
Preliminary reports of this work have appeared (Erwin and Miller 1995,1996a,b).
MATERIALS AND METHODS
Model system
We use a model system based on those previously used to study activitydependent formation of OD columns (Miller et al., 1989) and of simplecell RFs and ORI columns (Miller, 1994; for review, see Miller, 1996a). Except for the change from two to four LGN input classes, and other very minor changes discussed below, the model is as in Miller (1994).
Model cortical cells are arranged in a 32 × 32 grid with positions denoted by Roman letters, e.g., x⃗ ory⃗ (Fig. 1a). Model LGN cells are arranged in a corresponding 32 × 32 grid, with positions denoted by Greek letters, e.g., α⃗ or β⃗. We use common, e.g., retinotopic, coordinates so that locations in the two grids can be equated. Each LGN grid location represents a “projection column” containing four classes of cells: LN, LF, RN, and RF. Here the first letter stands for the left (L) or right (R) eye layer, and the second letter stands for the ON (N) or OFF (F) center type of the LGN receptive field. We will use the variables E and C to represent eye and center type respectively: E ∈ {L, R}, C ∈ {N, F}. We do not assume that ON and OFF cells are segregated into separate LGN layers as in the illustration but only that cells of each center type, receiving input from each eye, are present for all retinotopic positions.
For simplicity, we ignore distinctions among LGN cells beyond these four types. Ignoring competition among X, Y, and Wcell inputs may be justified by considering the model to represent the dominant input to a cortical region: Xcells for area 17 and Ycells for area 18 (Humphrey et al., 1985; Ferster, 1990a,b; Ferster and Jagadeesh, 1991; Boyd and Matsubara, 1996). Distinguishing between cells of differing temporal envelopes (e.g., lagged and nonlagged; Mastronarde 1987a,b) would likely only extend, not invalidate, the present results, as explained in Appendix .
Connectivity from LGN to cortex is modeled by a static arbor function, A(x⃗, α⃗), and an evolving synaptic weight function, S(x⃗, α⃗). The arbor function, A(x⃗, α⃗), models activityindependent biases in connectivity, e.g., the retinotopically allowed range of cortex over which an LGN afferent may arborize or sprout. It can be imagined roughly as being 1 for (x⃗, α⃗) such that there can be a connection from α⃗ to x⃗, and 0 otherwise. More generally, we consider a function that tapers from 1 to 0 with increasing retinotopic separation of α⃗ andx⃗ (Fig. 1b). The arbor function can be interpreted as a measure of the retinotopic affinity between α⃗and x⃗ as determined by activityindependent cues (Miller et al., 1989; Miller, 1997), or alternatively as the relative number of synapses initially made by an LGN cell at α⃗ to a cortical cell at x⃗ (Miller et al., 1989; Miller, 1990b). The synaptic weight variables, S^{EC}(x⃗, α⃗), represent the total efficacy or strength of the synaptic connections from LGN cells at position α⃗ in eye layer E and of center type C to cells at cortical position x⃗ (Fig. 1a). This efficacy is necessarily zero where A(x⃗, α⃗) is zero. The spatial RF of the cortical cell at x⃗ is determined by its pattern of geniculate input, represented by S^{EC}(x⃗,α⃗).
Model dynamics
Overview. We use a simple, linear learning rule, which we will refer to for simplicity as “Hebbian.” However, our equations can be derived from simple models of various underlying mechanisms, as mentioned in the introductory remarks (Miller et al., 1989; Miller, 1990b, 1997). Each component of the model may thus have multiple interpretations in terms of biological processes. Some of these are discussed here. For more details, see Miller (1994).
The dynamic variables in the model are the synaptic weights, S^{EC}, which define the cortical spatial RFs. These weights develop over time according to a linear “Hebbian” learning rule, described below, subject to several constraints. We have shown previously that under this rule, correlations in firing activity of LGN cells, (Fig. 1a, C) are the primary determinants of RF structure, whereas interactions between cortical cells (Fig. 1a, I) are the primary determinants of map structure.
We first present the Hebbian learning rule without any constraints. This learning rule involves the functions A, I, and C, so we next describe the forms of these functions we will use. Then we describe the saturation constraints, which keep individual synaptic strengths positive and bounded, and the competitive constraints, which cause lesscorrelated inputs onto a cortical cell to weaken in response to competition from morecorrelated inputs onto the same cell. The constraint is intended to model the effects, but not the mechanism, of unknown physiological processes that achieve a competitive outcome. Finally we describe in detail the iterative numerical procedure used to integrate the differential equations of the model and to allow joint enforcement of the saturation and competitive constraints.
Unconstrained Hebbian learning. To study the pattern of synaptic weights that initially develops from a nearly uniform projection, it is not necessary to explicitly specify sequences of input activity patterns. Rather, we only need to specify “correlation functions,” C^{EC,E′C′}(α⃗,β⃗), each representing the time averaged pattern of correlation between pairs of LGN cells, one of type EC at locationα⃗ and another of type E′C′ at location β⃗(Fig. 1a) (for further details of the averaging procedure, see Miller, 1990b, and references therein). The time over which this averaging occurs is purposely kept vague: it corresponds to the time separation of two input activities that yields cooperative interactions under whatever plasticity rule is acting biologically. More generally, it would involve some weighted average over time separation, weighted by the degree of cooperative interaction. The assumptions being made are that (1) one can summarize the temporal details of activity of any input pair by a number describing their degree of cooperation under the plasticity rule; and (2) these numbers can on average be predicted simply from the separation of two inputs and their input types (as supported by, e.g., Mastronarde, 1983a,b; Meister et al., 1995; Wong and Oakley, 1996).
Unconstrained development under a Hebbian or other correlationbased synaptic modification rule then takes the form:
Although this equation may look daunting, it expresses a simple rule. It states that the development of one synapse (S^{EC}(x⃗, α⃗)) is determined by a sum of the average influences exerted on it by all other synapses (e.g., S^{E′C′}(y⃗, β⃗)). The influence on one synapse by another is a product of three factors: (1) the synaptic strength of the “influencing” synapse, S^{E′C′}(y⃗, β⃗); (2) the strength of correlation in the firing of the two synapses, C^{EC,E′C′}(α⃗, β⃗); and (3) the average influence I(x⃗, y⃗) exerted across the cortex when the two synapses fire together. The influence between two synapses is attributable to their cooperation (via activity correlation, C) in producing a “reward” signal. This signal may be the firing of each synapse’s postsynaptic cell in a Hebbian model, or the activitydependent production and uptake of a diffusible modification factor in other models. The propagation of this influence between cortical cells is summarized by I, which could represent the effects of intracortical connectivity in a Hebbian model, and/or of diffusion in a model involving diffusible factors. Multiplication by the arbor function, A(x⃗,α⃗), ensures that influence is modulated by the intrinsic retinotopic affinity between α⃗ and x⃗ and in particular ensures that no synapses can develop where they are retinotopically disallowed (i.e., where A(x⃗, α⃗) = 0).
The advantage of this formulation is that it allows the central determinants of development to be isolated, independent of underlying mechanism. That is, for any proposed mechanism, it may be possible to summarize the elements driving development as (1) a pattern of input correlations, (2) propagation of influence of correlated input activity between cortical cells, and (3) retinotopic affinity or retinotopic limitations on connectivity. Our formulation isolates and describes the influence of each of these three factors on the developmental outcomes and thus simultaneously describes the determinants of development under multiple mechanisms. This formulation is further described by, e.g.,Miller (1990a,b, 1997).
The use of a linear equation can be justified by considering early development of the differences between initially approximately equivalent input projections, e.g., left eye and right eye or ON and OFF (Miller, 1990a,b). To the extent to which initial differences are small, early development of these differences will be determined by linear equations. This early linear development determines many features of the final mature pattern (in particular, those features that are shared by the fastest growing patterns in the linear regime typically persist in the final pattern). However, nonlinearities can also play important roles in developmental outcome (Feidler et al., 1997). Furthermore the different input projections are not really equivalent, because initial projections show contralateraleye and OFFcenter dominance (Albus and Wolf, 1984; Braastad and Heggelund, 1985; Crair et al., 1997a). Thus, to some extent this simple approach is justified simply by its success in yielding insight into the biology. It is of value to understand the simplest models before adding complexity to them, the moreso when that complexity is not well constrained by experiment. Many insights gained from the simplest models will persist as more complexities are added; we comment at several points on which features of the outcome are likely to be most susceptible to alteration by nonlinearities.
The functions A, I, and C. We will show analytically that the arbor function, A, and the intracortical interaction function, I, play developmental roles in the present model similar to their previously studied roles in the twoinputtype forms of the model (Miller et al., 1989; Miller, 1994). Thus in the present numerical simulations we hold constant the forms of A and I while we study the role of the correlation functions, C^{EC,E′C′}(α⃗, β⃗), within and between the four input types.
The correlation between two inputs of given types is assumed to depend only on their types and on the distance separating them: C^{EC,E′C′}(α⃗, β⃗) = C^{EC,E′C′}(‖α⃗ − β⃗‖). Such dependence is shown in experimental measurements of correlations (Mastronarde, 1983a,b; Meister et al., 1995; Wong and Oakley, 1996) and is sufficient to produce RFs with alternating ON/OFF subregions, such as simple cells.
Two basic functional forms will be used throughout the paper. The first form has purely positive correlations that taper with distance ‖α⃗ − β⃗‖, as given by a Gaussian function determined by a parameter γ:
For simplicity, we take the arbor function to be identical for each eye, E, and center type, C, and constant in time. We let A be radially symmetric, largest nearx⃗ = α⃗ and falling to zero as a function only of lateral distance, A(x⃗, α⃗) = A(‖x⃗ − α⃗‖). We use the default arbor function of Miller (1994), which is zero beyond a radius of 6.5 grid intervals, as illustrated in Figure 1d. Taking grid intervals to be on the order of 100 μm, arbor diameters are a little >1 mm (for more detailed discussion, see Miller, 1994, p 433). This arbor function imparts an initially smooth retinotopy on a coarse scale. Because the receptive fields that evolve are typically significantly smaller than allowed by the arbor function, fine scale variations in retinotopy can emerge from activitydependent synaptic modification.
Intracortical interactions are modeled by a function, I(x⃗,y⃗). For simplicity, so that we can focus on the effects of varying input correlations and also to ease computation, we use a twodimensional cortical architecture and take I to be a function of only the distance between two cortical column positions, I(‖x⃗ − y⃗‖). This function is not meant to be a realistic model of interactions attributable to cortical connectivity, which are threedimensional and cellspecific and develop along with the geniculocortical weights. This grossly simplified implementation of cortical interactions means that our model focuses more on development of RFs than on details of map structure because, as discussed above, correlations are the primary determinants of RF structure, whereas intracortical interactions are the primary determinants of map structure (for extensive discussion, see Miller, 1994.)
Setting I to be a radially symmetric, Mexicanhat function of lateral distance, with the form illustrated in Figure 1e, is sufficient to cause the emergence of continuous, approximately periodic maps of RF properties. The form illustrated is used throughout this paper; it is identical to the function M defined in Equation 4, except with the factor 0.25 rather than 0.24 in Equation 3defining the underlying Gaussian functions. (The value 0.25 was chosen simply as a round number that gave reasonable orientation maps inMiller (1994); the similarity of the Mexicanhat functions used for I and C has no significance for the results presented here.) Because there is little evidence for such Mexicanhatshaped intracortical connections, it should be noted that very similar model behavior is achieved if purely excitatory intracortical interactions are combined with an additional competitive “arbor” constraint ensuring that each presynaptic afferent with a given mean activity maintains approximately constant total synaptic projection strength (Miller et al., 1989; Miller, 1994; Miller and MacKay, 1994). Thus one can think of the Mexicanhat I as a computationally convenient standin for this combination of excitatory interactions and arbor constraints.
Saturation constraints. Synaptic weights must be constrained to remain positive and bounded. We refer to these as saturation constraints. We take these to be:
Very little is known about the mechanism by which such competition is enforced physiologically (but see Davis and Goodman, 1998; Turrigiano et al., 1998). We therefore do not try to model a specific mechanism but instead simply enforce competition by demanding that the total synaptic efficacy projecting onto each cortical cell be held constant:
When the number of plastic synapses P(x⃗, t) at cortical position x⃗ is not changing, the value of ε(x⃗, t) that will satisfy the constraint (7) is:
Initialization. Synapses are initialized as S^{EC}(x⃗, α⃗) = A(x⃗,α⃗)[1 + ξ(x⃗, α⃗)] where each ξ is drawn randomly from a distribution uniform between −0.2 and 0.2. Note that this embodies our assumption that initial differences between the projections of the different input types are small (see discussion above on use of a linear equation).
For ease of comparing the effects of changing parameters, a single random initial condition (same random seed) is used for all illustrated simulations. We have run simulations for many initial conditions; results are qualitatively invariant.
Numerical algorithm. This section describes in detail how the differential equation governing synaptic growth is numerically integrated while the saturation and competitive constraints are also enforced. These details are necessary for a complete description of the model sufficient to allow reconstruction of the results. This section may be safely skipped by most readers.
We integrate the differential equation, Equation 8, with a threestep difference method (Birkhoff and Rota, 1978). The initial condition is taken as time t = 0. Application of the equations proceeds as follows for each cell x⃗ at each time step t:
 (1)
 Calculate H^{EC}[S](x⃗,α⃗, t) from Equation 2.
 (2)
 Determine the sets of plastic synapses P(x⃗, t) from Equation 6.
 (3)
 Initially assume ζ(x⃗, t) = 0.
 (4)
 For a given value of ζ(x⃗, t):
 (a)
 Calculate ε(x⃗, t) from Equation 10.
 (b)
 For each of the four weights S^{EC}, calculate (d/dt)S^{EC}(t) using Equation8.
 (c)
 Calculate ΔS^{EC}(t) = Δt(f_{0}(d/dt)S^{EC}(t) + f_{1}(d/dt)S^{EC}(t − Δt) + f_{2}(d/dt)S^{EC}(t − 2Δt)). The factors are f_{0} = 1, f_{1} = f_{2} = 0 for the first time step; f_{0} = 2, f_{1} = −1, f_{2} = 0 for the second time step; and f_{0} = 23/12, f_{1} = −16/12, and f_{2} = 5/12 for subsequent steps.
 (d)
 For each of the four weights S^{EC}, calculate S^{EC}(t + Δt) = S^{EC}(t) + ΔS^{EC}(t), followed by setting any weight S^{EC}(x⃗, α⃗, t + Δt) < 0 to 0, and any weight S^{EC}(x⃗, α⃗, t + Δt) > 8A(x⃗, α⃗) to 8A(x⃗,α⃗).
 (e)
 Calculate D(x⃗) = Σ_{E,C,α⃗}[S^{EC}(x⃗,α⃗, t + Δt) − S^{EC}(x⃗,α⃗, t)] to determine whether the input to cellx⃗ was held constant for this value of ζ(x⃗, t).
 (f)
 If D(x⃗) is zero (or smaller than our criterion 0.00001), then accept the trial values of all the S^{EC}(t + Δt) forx⃗ and continue from step 2 with the nextx⃗. Otherwise, discard these S^{EC}(t + Δt) and repeat from step 4a using a new value of ζ(x⃗) selected by the method of bisection (Press et al., 1992). This method is guaranteed to converge [to find, in a finite number of steps, a ζ(x⃗) that yields an arbitrarily small D(x⃗)], because D(x⃗) decreases monotonically with ζ(x⃗).
For the first four time steps, we take Δt = 1, but we double the time step to Δt = 2 afterward. Thus, after t = 4 no synaptic values are computed at oddnumbered time steps. [Similar results can be obtained using Euler onestep integration with a smaller time step. The threestep method allows the use of a larger Δt or η but requires the storage of the intermediate results (d/dt)S^{EC}(x⃗, α⃗, t − Δt) and (d/dt)S^{EC}(x⃗,α⃗, t − 2Δt).] In twostage simulations, the correlation functions are changed at the beginning of the second stage. In this case we again use Δt = 1 for the first four time steps of the second stage and Δt = 2 thereafter.
As described by Miller and MacKay (1994), with the subtractive competitive constraint (Eq. 10), the development will cease only when all of the synapses have reached either the upper or lower cutoff specified in Equation 5. For efficiency, we terminate a simulation when 90% of the synapses have reached the upper or lower cutoff. We choose the learning rate η small enough such that each complete simulation will last at least 40 iterations. (Note: because Δt = 2 after the first four time steps, 40 iterations corresponds to t = 76.)
RF and map analysis
It is not possible to show images of the RFs and maps from each of the many simulations conducted. Thus, we define numerical measures through which we may study the effects of the model parameters. It is most convenient to describe map properties in terms of the composite weight vectors defined in Table 1. (These composite weight vectors are introduced and extensively discussed in Results.)
The degree of OD segregation of a cell, m(x⃗), is defined as the difference in connection strength between the right and left eyes, divided by the sum of those connection strengths: m(x⃗) = Σ_{α⃗}S^{OD}(x⃗,α⃗)/Σ_{α⃗}S^{SUM}(x⃗,α⃗). Complete right or lefteye dominance is indicated by m(x⃗) = +1 or −1. The overall degree of OD segregation 〈m〉 can be assessed as the root mean square of m(x⃗):
We measure the degree to which ON and OFF inputs are segregated by an index Z:
We define preferred orientation of the R or L eye, φ_{x⃗} ^{R} or φ_{x⃗} ^{L}, respectively, as the orientation of the flashed sinusoidal grating stimulus that maximizes the direct LGN input to a cortical cell for stimulation of that eye. This orientation is computed as that corresponding to the peak of the Fourier transform of ON minus OFF input for that eye (Miller, 1994). Note that we do not directly calculate cortical activity, and we neglect input from other cortical cells for two reasons: (1) we are interested in the degree of orientation tuning arising from segregation of the LGN input; and (2) our model does not incorporate development of the intracortical connections, and if we interpret the static I( ) function as the strength of synaptic input between cortical cells, then including these inputs causes only subtle changes in cell responses and the orientation map (Miller, 1994).
Orientation selectivity, q_{x⃗}
^{R} or q_{x⃗}
^{L}, measures the degree to which a cell’s input to sinusoidal gratings is peaked at its preferred orientation. We use a measure q_{x⃗}
^{E}, defined by Miller (1994). With the simple response model used here, in practice 0 ≤ q_{x⃗}
^{E} ≤ 0.27. Mean cortical orientation selectivity, Q, is computed as a weighted average of monocular orientation selectivities q^{R} and q^{L}across cortex:
In a few cases in which the preferred ORI is nearly the same in each eye of all binocular cells, we show a “binocular” ORI map, rather than monocular maps, to save space. Such a map is constructed by averaging, at each cortical position x⃗, the two monocular preferred orientations, φ_{x⃗}
^{E}, E ∈ {L, R}, weighted by their ORI selectivities, q_{x⃗}
^{E}, and by the corresponding eye’s proportion of synaptic strength at x⃗, (1 ± m_{x⃗})/2. The “binocular” preferred orientation, φ_{x⃗}, and selectivity, q_{x⃗}, are found from
Positive and negative singularities in the ORI maps are defined as locations centered between four grid positions around which the sum of the changes in preferred orientation rotates through 180° in the same or opposite direction as the movement around the singularity.
To determine the similarity of any two ORI maps, A and B, we use a method similar to that which has been used in optical imaging experiments (Gödecke and Bonhoeffer, 1996). We first compute singleorientation response maps at 18 discrete orientations θ at 10° intervals. We define the responses, A(x⃗, θ) and B(x⃗, θ), as the maximum LGN input to cells at x⃗ attributable to any grating (of any spatial phase or frequency) with preferred orientation within ±5° of θ. Correlation coefficients (Pearson’s r; Press et al., 1992) are computed between the two singleorientation maps, A(x⃗, θ) and B(x⃗, θ), giving a measure r(θ), which is then averaged over θ to give a single measure of similarity between the maps.
RESULTS
The dynamical variables in our model are the four synaptic weights, S^{EC}, defined in Figure 1a. These represent the projection to layer 4 from LGN cells of center type C ∈ {N, F} (ON and OFF), and of eye E ∈ {R, L} (right and left) (for simplicity, we use “of eye E ” to mean “driven by eye E ”). These develop under Equation 8, as determined by the correlation functions C, which describe the patterns of correlation in the firing of pairs of LGN cells, and by the intracortical interaction function I (Fig. 1e), which describes interactions across cortical positions attributable either to synaptic connectivity or diffusible factors. This development occurs subject to constraints, including the limits on retinotopically appropriate connections specified by the arbor function A (Fig. 1d), competition for limited synaptic resources on each postsynaptic cell imposed by Equation 10, and minimum and maximum weight values (saturation limits) imposed by Equation 5.
The functions I and A and the saturation limits all play roles in the present model that are formally identical to their roles in earlier models (Miller et al., 1989; Miller, 1994), which described development of OD or ORI maps alone. Because the effects of changes in these functions were fully described there, we hold them constant here and focus on the role of the correlation functions.
We make the simplifying assumptions that the activity patterns in LGN of the right and left eyes are statistically indistinguishable, as are the activity patterns of the ON and OFF populations. In reality, there are differences between ON and OFF spontaneous activity patterns (in retina; Mastronarde, 1983a,b; Wong and Oakley, 1996). As previously noted, we also make the simplifying assumption that the arbor function is identical for all four projection types (in reality, the contralateral eye has a somewhat stronger anatomical projection than the ipsilateral eye; LeVay et al., 1978; Shatz and Stryker, 1978). We will refer to these, along with the assumption that the four input types begin in approximately equal, unstructured conditions (see Materials and Methods), as symmetry assumptions between the two eyes and between the two center types.
Correlations among input activities are in general defined by a set of 16 correlation functions, C^{EC,E′C′}(α⃗− β⃗), describing the correlation in activity between an input of eye E and center type C at positionα⃗, and one of eye E′ and center type C′ at position β⃗. Because of the symmetry assumptions, the functions C^{EC,E′C′} depend only on whether E and E′ are the same or opposite eyes (S_{E} or O_{E}) and on whether C and C′ are the same or opposite center types (S_{C} or O_{C}). Thus, there are only four distinct correlation functions, which we describe as C^{SE} ^{SC}, C^{SE} ^{OC}, C^{OE} ^{SC}, and C^{OE} ^{OC} (Fig. 2a).
Composite weight modes and correlation functions
The four synaptic weight variables and correlation functions correspond to physical quantities that are, in principle, measurable. However, the behavior of the model is more easily described in terms of four composite synaptic weight variables, ormodes, called S^{μ}, where μ ∈ {SUM, OD, ORI+, ORI−}, and four corresponding correlation functions, C^{μ}. These are defined, as shown in Figure 2d and Table 1, as different linear combinations of the respective physical weight variables (Fig. 2b) and correlation functions (Fig. 2c). The modes can be understood as follows: (1) S^{SUM}(x⃗, α⃗) is the summed synaptic connection strength from all cell types atα⃗ to x⃗; (2) S^{OD}(x⃗, α⃗) is the total righteye minus total lefteye synaptic strength from α⃗to x⃗; it describes monocularity and OD maps; (3) S^{ORI+}(x⃗, α⃗) is the total ONcenter minus total OFFcenter synaptic strength fromα⃗ to x⃗; it describes one type of ORI selective RFs and their ORI maps; and (4) S^{ORI−}(x⃗, α⃗) is the righteye ON/OFF difference minus the lefteye ON/OFF difference from α⃗ to x⃗; it describes a second type of ORIselective RFs and their ORI maps.
Near the beginning of a simulation, when most synapses are far from their maximum or minimum values, each mode S^{μ} develops independently of the other three, under the influence of its corresponding composite correlation function, C^{μ}. This can be seen in equations, by substituting the definitions of the composite weight vectors and correlation functions into Equation 8. Ignoring the effects of synaptic saturation (Eq. 5), this yields the following fourindependent dynamical equations:
When the symmetry assumptions apply, the outcome of development is most conveniently described in terms of the four modes, because they initially (before weights saturate) develop independently of one another. The final RF and map properties are largely determined by this early phase of development. In contrast, the development of each of the four original weight vectors, S^{EC} (Fig. 2b), depends on the other three original weight vectors. Thus, the modes S^{μ} and their corresponding composite correlation functions C^{μ} provide the natural language with which to characterize model behavior.
Because Equations 1414 neglect the weight saturation constraints, Equation 5, they do not completely describe the model dynamics. It is awkward to express the saturation constraints in terms of the composite weight vectors. Thus, all simulations are actually carried out using the numerical algorithm described in Materials and Methods, which does not make reference to the modes or the composite correlation functions.
Independently growing synaptic patterns and their growth rates
Within each mode, S^{μ}, μ = {SUM, OD, ORI+, ORI−}, there are multiple, independently growing weight patterns, which we label S_{i} ^{μ}. For example, one pattern of the OD mode, S_{1} ^{OD}, might involve monocular cells with eye preference alternating in OD stripes having a period of four grid intervals, whereas another pattern, S_{2} ^{OD}, might be similar but with a period of five grid intervals. Similarly, S_{1} ^{ORI+} might involve cells developing ON/OFF segregation with three horizontal subregions per RF, whereas another pattern, S_{2} ^{ORI+}, might involve two vertical subregions per RF.
Each synaptic pattern, S_{i} ^{μ}, has its own corresponding growth rate, λ_{i} ^{μ}, which we number from largest to smallest: λ_{0} ^{μ} ≥ λ_{1} ^{μ} ≥ λ_{2} ^{μ} ≥ … . Thus, the fastestgrowing pattern of mode S^{μ}is S_{0} ^{μ}, with corresponding growth rate λ_{0} ^{μ}. For fixed A and I, the shapes of the patterns of S_{i} ^{μ} are determined by the shape of C^{μ}, whereas their growth rates λ_{i} ^{μ} increase with the magnitude of C^{μ} and also depend on the shape of C^{μ}. [The growth rates are defined by the independent exponential growth of each pattern: letting S_{i} ^{μ}(t) be the amplitude of S_{i} ^{μ} at time t; then for small times t, i.e., before weights saturate, S_{i} ^{μ}(t) = S_{i} ^{μ}(0)e^{λ iμt}. Technically, the S_{i} ^{μ} and λ_{i} ^{μ} are the eigenvectors andeigenvalues of Eqs. 1414.]
The various synaptic patterns, S_{i} ^{μ}, grow independently until synapses begin to saturate at the upper or lower cutoffs in Equation 5. The fastest growing patterns will dominate the overall weight pattern by the time saturation begins. Thus, the final RFs and map typically are determined by a mixing of only the fastestgrowing weight patterns, so that central features of the outcome of development can be characterized by determining the structure of these patterns (this approach is discussed in more detail by Miller et al., 1989; Miller, 1990a, 1994). The fastest growing patterns may all belong to a single mode, such as OD or ORI, or may include members of several modes.
Overview
We will begin by discussing how each mode, S^{μ}, develops independently. This determines the conditions necessary for development of OD or of ORI selectivity alone. We then turn to the simultaneous development of several modes, for which we determine the conditions for binocular matching of preferred orientations and for combined development of OD and ORI. Finally, we consider the effects of sequential development, in which initially ORI develops alone, and OD development begins subsequently because of a change in input correlations. We find that sequential development does not qualitatively change the previously determined conditions but does quantitatively relax some of them. In addition, it leads to development of significant correlations between OD and ORI maps similar to those observed experimentally.
Our results are presented in terms of the form that the correlation functions C^{μ} must take to achieve a given developmental outcome. The results are very simple in this form. However, the C^{μ} functions are not directly accessible experimentally. As shown in Table 1 and Figure 2d, the C^{μ} functions are linear combinations of the correlation functions, C^{SE} ^{SC}, C^{SE} ^{OC}, C^{OE} ^{SC}, and C^{OE} ^{OC}, which correspond to more intuitive, experimentally measurable quantities. To allow our results to be understood in terms of these more intuitive quantities, we show in Figure3 the forms that C^{SE} ^{SC}, C^{SE} ^{OC}, C^{OE} ^{SC}, and C^{OE} ^{OC} take in our simulations. Figure 3 should be referred to along with each simulation. We have limited ourselves in the simulations to cases in which each of the C^{μ} functions can be expressed as a Gaussian (Eq. 3, Fig. 1b) or Mexicanhat (Eq.4, Fig. 1c) function.
Separate development of individual modes
Each mode develops independently in the early part of development, before synaptic saturation begins to break the independence of Equations 1414. In this section, we describe the determinants of the independent development of each mode. These results are summarized schematically in Figure 4. These results derive directly from the results of our previous twoinputtype models, as explained in Appendix , so discussion here will be brief.
The basic intuition behind these results is that correlationbased development leads to RFs consisting of a maximally correlated subset of inputs (for the mathematical expression of this, see Miller and MacKay, 1994; Miller, 1997). Correlations are maximized when the spatial RF pattern of each S^{μ} closely follows the spatial structure of the corresponding C^{μ}: if a given C^{μ} is all of one sign, then so is the RF pattern that results for the corresponding S^{μ}; if a given C^{μ} oscillates in sign, so does the RF pattern of the corresponding S^{μ}, with a similar spatial period. In particular, the fastest growing patterns of each S^{μ} follow the form of the corresponding C^{μ} in this manner, and these patterns in turn determine the structure that develops in S^{μ}.
Development of ocular dominance
Ocular dominance is a net difference in total lefteye versus righteye strength, summed over all input types of either eye. OD is represented only by the S^{OD} mode (Table1). Monocularity develops if cortical cells become dominated by a single eye throughout their RF, which means that S^{OD} has the same sign throughout the RF (positive for righteye domination, negative for lefteye domination). The condition for this to occur is as follows (Fig. 4a):
Condition 1. For monocularity to develop, C^{OD} should be positive at least at small distances, and it should not significantly oscillate in sign within an arbor radius. (More precisely: the peak of the Fourier transform of C^{OD} must be at a wavelength long compared to the arbor diameter.) By arbor radius, we mean the radius over which LGN cells can project to a common cortical point.
Condition 1 was found by Miller et al. (1989). The condition there was expressed in terms of the net difference in total lefteye versus righteye strength, which here is represented by S^{OD}; and in terms of the difference between withineye and betweeneye correlations, which here is represented by C^{OD}. Note that both S^{OD} and C^{OD}include a sum over ON and OFF center types (Table 1), whereas distinct center types were not considered by Miller et al. (1989). The applicability of those results to the present case is shown in Appendix.
The independent development of an OD map in the present fourinputtype model is shown in Figure 5a, for a case in which only the S^{OD} mode develops (because all the C^{μ} values except C^{OD} are set to zero), and C^{OD} satisfies Condition 1. Here, C^{OD} was a broad Gaussian. Most cortical cells become monocular, as shown by the RFs of an 8 × 8 subset (Fig. 5c,d). ON and OFFcenter inputs develop identically. The form and spatial period of the OD map are jointly determined by the functions I and A precisely as in Miller et al. (1989).
Development of orientation selectivity:S^{ORI+} and S^{ORI−}
A monocular RF develops ORI selectivity if it develops simplecell structure: elongated, spatially segregated ON and OFF subregions. This means that the monocular RF must spatially oscillate, along one direction, between regions of ON domination and OFF domination; that is, the difference between ONcenter and OFFcenter strength must spatially oscillate in sign. For a binocular cell, each monocular RF should have such a structure, with matching preferred orientations.
S^{ORI+} is the sum of the righteye and lefteye ON/OFF differences. ORI selectivity develops if S^{ORI+} spatially oscillates in sign within an RF. Oscillations in S^{ORI+} correspond to ON/OFF segregation that is in phase in the two eyes: ON subregions in the right eye and ON subregions in the left eye grow in corresponding locations; likewise, OFF subregions in the two eyes are matched (Fig. 4b).
S^{ORI−} is the difference of the righteye and lefteye ON/OFF differences. Just as for S^{ORI+}, ORI selectivity also develops if S^{ORI−} spatially oscillates in sign within an RF. Oscillations in S^{ORI−}correspond to ON/OFF segregation that is antiphase in the two eyes (Fig. 4c): ON subregions in the right eye and OFF subregions in the left eye grow in corresponding locations; likewise, OFF subregions in the right eye are matched to ON subregions in the left eye.
Only these two modes involve differences between ONcenter and OFFcenter innervation, and thus only these modes can yield simple cells. This yields the following condition (Fig. 4b,c):
Condition 2. For ORI selectivity to develop, the fastest growing patterns of either the ORI+ or ORI− mode must have RFs with simplecell structure: elongated, spatially segregated ON and OFF subregions. This in turn means that the corresponding ORI correlation function, C^{ORI+} or C^{ORI−}, should have a Mexicanhat or similar form that oscillates in sign within an arbor radius. (More precisely, it must have the peak of its Fourier transform at a wavelength corresponding to one or more cycles within an arbor diameter.)
Note that development of either S^{ORI+} or S^{ORI−} alone yields RFs in which orientation is binocularly matched. Later, we will consider the case in which both modes develop and determine the more general conditions required for binocular matching of preferred orientation.
Condition 2 was found by Miller (1994), which considered two center types but ignored the existence of two eyes. The condition there was expressed in terms of the net difference in total ONcenter versus OFFcenter strength and the net difference between withincentertype and betweencentertype correlations. The applicability of these results both to the binocular sum of these differences (S^{ORI+} and C^{ORI+}, respectively) and to the binocular difference of these differences (S^{ORI−}and C^{ORI−}, respectively) is shown in Appendix .
The independent development of an ORI map in the present fourinputtype model is shown in Figure 6a for a case in which only the S^{ORI+} mode develops: C^{ORI+} has a Mexicanhat shape, whereas C^{SUM}, C^{OD}, and C^{ORI−} are all set to zero. All cortical cells develop segregated ON and OFF RF subregions, as shown by the RFs of an 8 × 8 subset of cells (Fig. 6b), and this yields a continuous map of ORI preference (Fig. 6a). All cortical cells are binocular, with the two monocular RFs in phase: ON and OFF subregions and ORI maps are matched in the two eyes. If instead only the S^{ORI−} mode develops ( C^{ORI−} has a Mexicanhat shape, and C^{SUM}, C^{OD}, and C^{ORI+} are all zero) an ocularly matched ORI map again develops. However, in this case the two monocular RFs develop in antiphase (Fig. 6c). In either case, the periodicity of the ORI map depends on C^{ORI±}, I, and A in the same complex way as did the maps of Miller (1994).
RFs corresponding to S^{ORI−} may seem functionally implausible: it may seem that they would not respond to binocular input, because excitation in the ON or OFFtype region of one eye would be canceled by inhibition from the oppositetype region at the same location in the other eye. However, independent movements of the right and left eyes, as well as presentation of stimuli at various depths, complicate this simple picture. In fact, RF patterns such as S^{ORI+} and S^{ORI−} have been proposed to underlie socalled “tunedexcitatory” and “tunedinhibitory” cells, respectively (Freeman and Ohzawa, 1990; Nomura et al., 1990; DeAngelis et al., 1995). There is, however, some evidence that “tunedinhibitory” cells are rare or absent in the cat (LeVay and Voigt, 1988).
Development of the summed synaptic strength
The S^{SUM} mode is the only mode affected by the competitive constraint, represented by the second, subtracted term in Equation 14a. This term ensures that the total synaptic strength to each cortical cell remains constant. This constrains the patterns of S^{SUM} that may grow: the strength of S^{SUM} to a cortical cell may increase from some LGN locations only if it correspondingly decreases from other LGN locations.
The development of S^{SUM} can be understood from the development of the summed synaptic strength in our previous twoinput models (Appendix ), as discussed for example by Miller (1990a). A typical effect of nonzero C^{SUM} is refinement of the RFs, meaning that the RFs grow at their centers and decay at their peripheries without changing their total synaptic weight. Because our constraint is only a poor standin for the unknown mechanisms of biological competition, the details of growth of the S^{SUM} mode in our model are unlikely to be of biological relevance (also see Discussion). Growth of S^{SUM} is of relevance to us only insofar as it interacts with, and thus alters the growth of, the other modes. This interaction is discussed below.
Joint development of orientation and ocular dominance modes
We now consider the additional conditions on the correlation functions that must be met for both ORI and OD maps to codevelop, with binocularly matched ORI preferences. This requires study of the codevelopment of multiple modes.
The different modes grow independently until synapses begin to saturate at their upper or lower cutoffs (Eq. 5). As multiple synapses saturate, the overall synaptic pattern becomes locked in. Thus, the main effect of the different modes on one another involves their relative synaptic strengths at the time that saturation begins to halt development. This in turn depends on their relative growth rates. In general, for several modes to codevelop, their fastest growing patterns must have comparable growth rates, whereas modes with fastest growing patterns that have significantly smaller growth rates will have little influence.
Interaction of the two ORI modes
We have seen that each ORI mode alone yields binocularly matched ORI preferences. It turns out that the requirement for binocular matching of ORI preferences is that one of these modes should remain predominant; the mixing of the two modes destroys binocular matching. The general condition is as follows:
Condition 3. For ocular matching of ORI preferences to develop, the growth rates of one of S^{ORI+}and S^{ORI−} must sufficiently dominate those of the other. This in turn requires that betweeneye correlations must be centertypespecific (C^{OE} ^{SC} and C^{OE} ^{OC} must differ significantly from one another).
To demonstrate this, we consider the case in which each is driven by correlations of identical Mexicanhat form but with independent magnitudes: C^{ORI+} = r^{+} M; C^{ORI−} = r^{−} M (Fig. 3a). We have seen above that, if only r^{+} is positive (C^{ORI+} dominant), left and right RFs will be in phase, whereas if only r^{−} is positive (C^{ORI−} dominant), left and right RFs will be in antiphase (Figs. 4b,c, 6a–c). One might imagine that, if r^{+} and r^{−} are both positive, the resulting codevelopment of the two ORI modes could lead to the development of ocularly matched ORI maps with other spatial phase relationships between the two eyes. However, this is not the case.
Instead, if S^{ORI+} and S^{ORI−} codevelop with approximately equal growth rates, the two eyes develop independent, uncorrelated ORI maps and RFs (Fig. 6d,e). To understand this, consider first the case in which r^{+} = r^{−}, so that C^{ORI+} = C^{ORI−}. A glance at Table 1 shows that C^{ORI+} = C^{ORI−} implies C^{OE} ^{SC} = C^{OE} ^{OC}. This means that an ON (or OFF) input of one eye cannot “tell the difference” (in terms of activity correlations) between an ON input and an OFF input of the other eye. In that case, although firing correlations within each eye have the form needed to generate simplecell RFs, there is no mechanism to relate the structure of ON or OFF subregions in one eye to the ON or OFF subregions in the other eye. ON or OFF subregion structure, and thus preferred orientations, develop independently in each eye. When r^{+} and r^{−} are not exactly equal but have similar magnitude, the argument is the same: the intereye correlations differentiate the two center types only weakly, too weakly to cause the ON/OFF subregion structures that develop for each eye to become related. [These arguments can also be understood mathematically as follows: when the fastest growing patterns of S^{ORI+} and S^{ORI−} have identical growth rates, then any two orthogonal linear combinations of these patterns also grow independently. But one such orthogonal pair is (S^{ORI+} + S^{ORI−})/2 = S^{RN} − S^{RF}, and (S^{ORI+} − S^{ORI−})/2 = S^{LN} − S^{LF}. Thus, the two eyes develop independent ON/OFF segregation patterns and ORI maps.]
In Figure 6f, we plot the degree of similarity between the two eyes’ maps as a function of λ_{0} ^{ORI−}/λ_{0} ^{ORI+} = r^{−}/r^{+}, the ratio of the fastest growth rates of S^{ORI+} and S^{ORI−}. When λ_{0} ^{ORI−}≈ λ_{0} ^{ORI+}, the lefteye and righteye ORIs become independent. For the choices of functions explored here, the condition for ocular matching of ORI maps is λ_{0} ^{ORI−} < (2/3) λ_{0} ^{ORI+} or λ_{0} ^{ORI+} < (2/3) λ_{0} ^{ORI−} (Fig. 6f). The quantitative factor 2/3 may change for other choices of shapes for C^{ORI+} and/or C^{ORI−} or for the functions A and I (although in limited exploration, we have found that it does not seem to vary much).
How much difference must exist between C^{OE} ^{SC} and C^{OE} ^{OC} for ORI preferences to become binocularly matched? Note that C^{ORI±} = C^{SE} ^{ΔC} ± C^{OE} ^{ΔC}, where C^{SE} ^{ΔC} = C^{SE} ^{SC} − C^{SE} ^{OC} and C^{OE} ^{ΔC} = C^{OE} ^{SC} − C^{OE} ^{OC}. Thus, the requirement is that C^{OE} ^{ΔC}, the difference between C^{OE} ^{SC} and C^{OE} ^{OC}, must be of significant size relative to C^{SE} ^{ΔC}. Furthermore, the difference between C^{OE} ^{SC} and C^{OE} ^{OC} must vary significantly within an arbor radius (and therefore, it cannot simply be attributable to differing mean rates of ON and OFFcenter activity). This can be seen in two ways. Intuitively, for the ORI maps of the two eyes to match, C^{OE} ^{ΔC} must be such that correct alignment of the two eyes’ RFs will significantly increase the betweeneye correlations relative to other alignments. If C^{OE} ^{SC} and C^{OE} ^{OC} do not vary significantly within an arbor radius, then the betweeneye correlations would not vary if one eye’s RF is rotated relative to the other eye’s RF, and so correct alignment would not be favored. Mathematically, C^{OE} ^{ΔC} must couple to the leading pattern of C^{ORI+} or C^{ORI−}, whereas for orientation to develop, that fastest growing pattern should involve an oscillation across the RF. If C^{OE} ^{ΔC} were flat over an arbor radius, it would not have any influence on such a mode.
Small local variations in left and righteye dominance are visible in Figure 6e, even though C^{OD} = 0. This occurs primarily because the RFs of the left and right eyes begin with different random initial conditions, so S^{OD} begins with small random variations. Although S^{OD} does not change during linear development (because C^{OD} = 0), saturation nonlinearities can induce weak additional changes in S^{OD}. This does not yield strong or spatially structured OD segregation like that observed biologically; that can only emerge in the model as a pattern of S^{OD} driven by C^{OD}.
Interaction of the ORI and OD modes
For OD and ocularly matched ORI maps to jointly develop from a single, timeinvariant set of correlation functions, the following condition must also be met:
Condition 4. The fastest growing patterns of OD and of ORI must have comparable growth rates.
We demonstrate this by varying the relative magnitudes of OD and ORI correlation functions while ensuring that the previous conditions are met.
To meet Condition 1, we set C^{OD} to the Gaussian function used to develop OD columns in Figure 5, scaled by a parameter d: C^{OD} = dG_{3}. To meet Conditions 2 and 3, we set one of the ORI correlation functions, which we arbitrarily choose to be C^{ORI−}, to zero and the remaining one, C^{ORI+}, to the Mexicanhat function M used to develop ORI columns in Figure 6, a and b. We continue to set C^{SUM} = 0; we will consider C^{SUM} more carefully in the next section. Changes in the correlations with d are illustrated in Figure 3b.
Figure 7 illustrates the effect of varying the ratio of OD and ORI growth rates by varying d. Because only the ORI+ and OD modes have nonzero growth rates, the outcome is determined by the relative magnitudes of their fastest growing patterns. In this case, it turns out numerically that d is approximately equal to the ratio of growth rates, λ_{0} ^{OD}/λ_{0} ^{ORI+} = 0.97 d, so that d can be used as a convenient standin for this ratio.
When this ratio is small, as for d = 0.5 (Fig. 7c), no pattern of OD grows fast enough to compete with the rapidly growing ORI patterns. Thus the final map contains a map of ORI preference matched in the two eyes but no OD map. No cells are monocular, and most cells have well segregated ON and OFF subregions generating ORI preferences. At d = 1, patterns of OD and ORI grow at similar rates, generating concurrent representation of an ORI and an OD map (Fig. 7a,d). The map includes both binocular and monocular RFs. Very similar maps develop for d = 1.6, except that few cells remain binocular (Fig. 7b,e). As d is raised further, ORI patterns contribute less and less to the final outcome. Thus the final RFs for d = 4 (Fig. 7f) contain an OD map but show little separation into ON and OFF subregions and thus little ORI selectivity or map.
How similar must the growth rates of ORI and OD be to support codevelopment of both kinds of maps? To explore this, we now allow the shape as well as the amplitude of C^{OD} to vary: C^{OD} = dG_{γ}(see Eq. 3). The parameter γ controls the width of the Gaussian function forming C^{OD}: γ = 3 gives the function just studied in Figure 7, whereas larger γ yields a proportionately wider Gaussian. (Note from Eq. 3 that G_{γ} is normalized so that its magnitude, i.e., its twodimensional integral, is independent of γ. Thus, γ controls width, whereas d controls magnitude.) For the functions used, we can numerically solve Equation 14b to find their independently growing patterns and their growth rates. The fastest growing pattern, S_{0} ^{OD}, is always monocular for C^{OD} ∝ G_{γ}; its dominance in growth rate over binocular patterns increases with increasing γ (Miller, 1990a). We restrict attention to γ ≥ 2.5, a large enough value to ensure robust development of OD segregation. The growth rate, λ_{0} ^{OD}, of S_{0} ^{OD}decreases with increasing γ and is proportional to the amplitude d.
The range of growth rate ratios, λ_{0} ^{OD}/λ_{0} ^{ORI+}, that allow codevelopment of ORI and OD is quantified in Figure8. As the growth rate ratio λ_{0} ^{OD}/λ_{0} ^{ORI+} increases, the degree of OD segregation, 〈m〉 (Eq. 11), rises monotonically from near 0.0 to near 1.0 (Fig. 8a, solid curves), whereas the degree of ON/OFF segregation, Z (Eq. 12), falls monotonically (Fig. 8a, dotted curves). The ORI selectivity, Q (Eq. 13), also falls with increasing λ_{0} ^{OD}/λ_{0} ^{ORI+} when this ratio is sufficiently large (Fig. 8b), in parallel with the decrease in ON/OFF segregation. Perhaps surprisingly, ORI selectivity increases with increasing λ_{0} ^{OD}/λ_{0} ^{ORI+} for smaller values of this ratio. A likely explanation for this rise in ORI selectivity is that the patterns of ORI that are most impeded through competition with OD are those that have the lowest growth rates, and these generally have poorer ORI selectivity than the fastergrowing ORI patterns. Modestly high OD growth rates, by outcompeting the slowergrowing ORI patterns, can thus increase ORI selectivity.
Different curves in Figure 8 correspond to different values of γ. Wider γ allows both ORI selectivity Q and ON/OFF segregation Z to develop for larger values of λ_{0} ^{OD}/λ_{0} ^{ORI+}, for the following reason. As γ widens, the growth rates of nonmonocular patterns of S^{OD} become smaller relative to those of the leading, monocular patterns. Therefore, for a fixed λ_{0} ^{OD}/λ_{0} ^{ORI+}, wider γ means that fewer OD patterns are competing against growth of S^{ORI+}, allowing fuller development of S^{ORI+}. The net effect of widening γ is that the range of relative growth rates that yields combined maps becomes less restricted, requiring less “fine tuning” of λ_{0} ^{OD}/λ_{0} ^{ORI+}. These ranges are quantified in Figure 8c.
ORI selectivity emerges more rapidly than OD, even though joint map development occurs only when ORI and OD modes grow at similar rates (Fig. 9). The similar rate of growth is reflected in the similar time courses of ON/OFF segregation Z and left/right segregation 〈m〉. However, ORI selectivity Q develops more rapidly than ON/OFF segregation: ORI selectivity can be strong as a result of only weak segregation of ON and OFF inputs. (The time courses of development of Z and Q can differ because the former is a linear function of the weights, whereas the latter is a nonlinear function.) The result is that the ORI map approaches its final state before OD segregation significantly develops. This is shown by the time course of the correlation, r, between the ORI map computed at any time step t and the final ORI map (Fig. 9a; compare time course of degree of OD segregation 〈m〉).
ORI selectivity develops before OD segregation over the entire range of λ_{0} ^{OD}/λ_{0} ^{ORI+} in which OD and ORI maps jointly develop. This provides one possible explanation for the experimental fact that ORI selectivity reaches a reasonably high level before OD bands are observable (LeVay et al., 1978; Fregnac and Imbert, 1984; Crair et al., 1997a). However, this could also be explained if the correlations driving ORI development exist during a first developmental stage, before the onset of another correlation structure driving OD development; this “twostage” developmental scenario is explored later in Results.
OD and ORI columns have been shown to preferentially intersect at sharp angles in primary visual cortex of both monkey (Obermayer and Blasdel, 1993) and cat (Crair et al., 1997b). This effect also occurs in many abstract feature map models (for review, see Erwin et al., 1995,Swindale 1996). Although the initial developments of OD and ORI map modes under Equation 14 are independent, relationships between them could develop after synapses have begun to saturate via Equation 5. In the simulations discussed so far, these saturation effects only occur during a short period near the end of a simulation, and we have found no statistically significant relationship between the OD and ORI map patterns. (Such relationships might weakly exist but might be lost in the noise because of the current low resolution of simulated ORI maps; testing this will require higherresolution simulations.) We show later that a clearly demonstrable relationship between the maps does develop in twostage development.
Influence of the SUM mode on ORI and OD development
Just as rapid growth in one of the OD or ORI modes can prevent significant growth in the other of these modes, rapid growth of patterns in the SUM mode can push some synaptic strengths to their limiting saturation values (Eq. 6) before OD and ORI map patterns have had time to develop. Thus we may state the final condition for joint development of ORI and OD maps:
Condition 5. Patterns of the SUM mode must not grow fast enough to prevent development of the OD or ORI modes.
The growth of the SUM mode is distinguished from that of the other modes in two ways. First, the competitive constraint in Equation 10requires that growth of S^{SUM} in some regions of any RF must be exactly canceled by decay of S^{SUM} in other regions of the same RF, whereas the other modes are unaffected by this constraint. Second, the initial condition of the SUM mode is close in shape to A(x⃗,α⃗), whereas the other modes are initially close to 0. This means that the SUM mode begins with some patterns [those similar to A(x⃗, α⃗)] having large size, whereas the other modes start with all patterns having approximately equal, small sizes. The initially large SUM patterns can dominate development through exponential growth, even with relatively modest growth rates. Both of these conditions make it difficult to predict when the SUM mode growth will interfere with growth of the other modes.
Because both the competitive constraint and the method of initializing weights are not likely to be correct in their details, we have restricted our analysis above to the case in which C^{SUM} = 0 and patterns of the SUM mode do not interfere. A more biologically accurate description of limits on synaptic growth is clearly needed before the effects of such limits on codevelopment of OD and ORI can be properly understood.
For the sake of completeness, however, we have made a limited study of the effect of variation in C^{SUM} on development. The results of this study are available as a supplement to the present paper (Erwin and Miller, 1998).
Timevarying correlation functions: sequential development of ORI and OD modes
We showed above that OD and ORI maps can jointly develop under a timeinvariant set of correlation functions. Experimental evidence suggests that development of a binocular ORI map precedes development of an OD map (Crair et al., 1997a). Although this arises naturally in the above scenarios (Fig. 9), it also suggests an alternative developmental scenario in which the correlation functions change in time. In this scenario, a period of ORI but not OD development is followed by a period in which both ORI and OD are developing. This change could occur gradually, but it is easier to study a twostage process with a sudden change. One might imagine, for example, that a sudden change in correlations occurs at the time of eye opening or when the vascular tunic of the eye’s lens begins to fall away [the latter approximately coincides with (Thorn et al., 1976) or somewhat precedes (Freeman and Lai, 1978) the onset of the critical period and the major onset of ocular dominance development (LeVay et al., 1978)].
Here we study the effect of such a twostage scenario on our conclusions. We find that Conditions 1–5 for development of OD and ocularly matched ORI maps all remain true, but their application becomes more flexible. Conditions 1–3 must still each be met during at least one stage of development, but they need not all be met at the same time. Condition 4 remains qualitatively unchanged, as applied to the period in which both ORI and OD are developing, but the relative growth rates of OD and ORI modes can now cover a wider range.
We also find that twostage development offers an additional benefit. It leads to stronger coupling of the OD and ORI maps, with relationships between their patterns similar to those that are experimentally observed. Although such coupling might also occur in the later time steps of a singlestage development, it is clearly demonstrable in maps from a twostage process.
The twostage simulations proceed as follows. The first stage, beginning from random initial conditions, involves development up to a specified time, t_{1}, with C^{ORI+} the only nonzero correlation function, as in Figure 6a. Thus, only the S^{ORI+} mode develops. The second stage continues the simulation with the same C^{ORI+}, but in addition now C^{OD} is nonzero: it is set to a broad Gaussian, with magnitude proportional to the parameter d, as in Figures 3b, 8, and 9.
An example of such a twostage simulation is shown in Fig.10. Here, the first stage lasts only a short time (t_{1} = 26). This short first stage yields only poorly organized, similar but nonidentical left and righteye ORI maps (Fig. 10a,b). Most synaptic connections are still weak, and there is only weak segregation of ON and OFF inputs (Fig. 10e).
The second stage continues with C^{OD}nonzero; in this case, d = 2. When the second stage terminates, the ORI map has fully developed (Fig. 10c), and an OD map has formed (Fig. 10d). Most synaptic connections have saturated at their upper or lower bounds (Fig. 10f).
The ORI map at the end of the second stage is shown as a binocular ORI map, because the ORI maps in the two eyes have become well matched. This can be seen by noting that, although most cells have connections to only one eye, the cells that are binocular have similar RFs in the two eyes (Fig. 10f), and the binocular ORI map is continuous across OD column borders (Fig. 10c,d).
Twostage development allows greater flexibility than a onestage process in the relative magnitudes of correlation functions that can be used to develop combined ORI and OD maps. We judge this range in the same way as in Figure 8. As the first stage is carried to longer times, t_{1}, joint OD and ORI maps develop in a progressively larger range of secondstage growth rate ratios, λ_{0} ^{OD}/λ_{0} ^{ORI+}, Fig.11. Onestage development allowed a range of 0.96 ≤ λ_{0} ^{OD}/λ_{0} ^{ORI+}≤ 2.0, that is, variation by a factor of 2.1. For twostage development with t_{1} = 26, this range expands slightly to a factor of 2.2; as t_{1} increases to 66, the range expands to allow variation by a factor of 6.5 (Fig. 11e).
The two eyes’ ORI maps can develop independently in the second stage and still become matched: the degree of correlation between the two eyes’ maps shown in Figure 10, a and b, induced in the first stage, is sufficient to cause the two eyes to independently arrive at nearly identical ORI maps. The outcome of such development is shown in Figure 11, right column. Here we have made an additional change in the secondstage correlation functions: in addition to making C^{OD} nonzero, we also set C^{ORI+} = C^{ORI−}. As shown previously, this equality implies that the left and right eyes’ orientation maps are developing independently, except for saturation effects. The range of variation of growth rate ratios, λ_{0} ^{OD}/λ_{0} ^{ORI+}, allowing joint development of OD and ORI is greater still under this scenario, growing to a factor of 8.8 for t_{1} = 66 (Fig. 11f).
We have visually verified that such independent secondstage development leads to ocularly matched maps. However, it is difficult to characterize the correlation of the ORI maps in the two eyes when OD also develops, because most cells come to receive input from only a single eye. Therefore, to quantitatively demonstrate that independent development of the two eyes’ maps in the second stage leads to matched maps, we show the outcome of development of ORI alone (Fig.12). Here, the first stage is as before (only C^{ORI+} is nonzero), whereas in the second stage, C^{ORI−} = C^{ORI+}, and C^{OD} remains zero. Thus, in the first stage, the two eyes’ ORI maps develop in a correlated fashion, yielding results such as in Figure 10, a and b; in the second stage, the two eyes’ ORI maps continue development completely independently of one another, and there is no OD development. As shown in Figure 12, the correlation between the maps of the two eyes continues to increase in the second stage. For t_{1} ≥ 26, the two maps come to be essentially perfectly correlated. Once an ORI map has developed sufficiently far along a path, its outcome becomes well determined; the two eyes’ maps, having developed in a correlated way in early development, independently arrive at a common outcome.
We can summarize by saying that the twostage process removes the requirement for centertypespecific betweeneye correlations in the second stage of development (recall that such correlations are required to correlate the development of the two eyes’ ORI maps; and that the condition C^{ORI+} = C^{ORI−} implies that such correlations do not exist). It is sufficient that such correlations exist only in the first stage of development, e.g., up to the state illustrated in Figure10, a, b, and e.
A further benefit of the twostage process is that it leads to a stronger coupling of the ORI and OD maps, in a direction that is observed biologically. The S^{OD} and S^{ORI+} weights develop independently, under Equation 14, until the saturation constraints of Equation 5 begin to take effect, linking their development in complex ways. In onestage development, the OD and ORI maps develop almost independently until they are nearly complete; therefore, coupling of the maps is very weak. In twostage development, saturation may begin to couple OD and ORI development early in the second stage, i.e., early in OD map development. This coupling grows stronger for increasing t_{1}. The OD map that develops therefore can have a somewhat different form than that which develops in a onestage process (compare Figs. 7d, 13a, developed with t_{1} = 66).
The relationship induced between the OD and ORI maps by the saturation constraints is similar to the relationship observed experimentally in cats (Crair et al., 1997b) in at least one respect: the most monocular regions of the OD map tend to lie closer to singularities in the ORI map than they would by chance (Fig. 13a–c). This coupling in the model is in the same direction as that observed biologically, but is weaker. Because the effect grows with increasing t_{1}, we illustrate the effect by showing the case t_{1} = 66 (for t_{1} = 26, the effect is barely visible).
The reason for this coupling of ORI and OD can be intuitively understood as follows. OD can grow most strongly where it has the least competition from ORI. Near pinwheels, ORI patterns on different nearby cells are least able to contribute to one another’s growth. OD patterns are oblivious to ON/OFF segregation patterns and hence grow equally well near pinwheels or near linear ORI regions. Hence, the OD patterns are able to develop most fully near pinwheels, where they face the least competition, or equivalently where synapses are least saturated when OD development occurs. It is likely that this tendency of pinwheels to form in the centers of OD bands is sufficient to explain the experimentally observed tendency toward perpendicular crossings of isoORI lines and OD borders. However, as in the onestage simulations, we can find no statistically significant tendency toward perpendicular crossings in our twostage maps; this may be attributable to the weakness of these effects in the model as well as to the noisiness in (low resolution of) the ORI maps.
Experimental predictions and tests
From these results we can extract two major sets of experimental predictions. One set relates to correlation structures, the other to the relation between the left and righteye RFs of mature simple cells.
Correlation structures
The correlation structures we have described should be observed in cat LGN at the time that ORI selectivity and OD are developing in layer 4 of cat V1. These structures would be measured by simultaneously recording pairs of LGN cells. The degree of correlations in each pair’s activity would be plotted against the retinotopic separation of the pair’s RFs. Separate such plots would be made for pairs of same or opposite eyes and same or opposite center types, yielding the four correlation functions C^{SE} ^{SC}, C^{SE} ^{OC}, C^{OE} ^{SC}, and C^{OE} ^{OC} (Fig. 2a).
The overall prediction is that Conditions 1–5 should be met. In more directly biological terms, the essential predictions are: (1) when ocularlymatched simple cell RF structure develops: (a) within eyes, the difference between samecentertype and oppositecentertype LGN correlations, C^{SE} ^{SC} − C^{SE} ^{OC}, should have Mexicanhat or oscillating structure; (b) intereye LGN correlations should be centertypespecific: C^{OE} ^{SC} and C^{OE} ^{OC} should be nonzero and differ from one another, in a manner that varies over an arbor radius; and (2) when OD develops, total withineye LGN correlations, C^{SE} ^{SC} + C^{SE} ^{OC}, should be larger than total betweeneye LGN correlations, C^{OE} ^{SC} + C^{OE} ^{OC}.
When and under what conditions should these correlations be measured? Optical measurements in cat show that ORI maps are ocularly well matched from their first appearance, at postnatal day 12 (P12), regardless of whether the animal has had visual experience (Crair et al., 1997a). Because this development is independent of vision, the relevant correlations would be in LGN spontaneous activity, in darkness (“dark activity”) or with eyelids closed. It is not precisely clear when ocularly matched ORI maps develop. P12 is only 1 d after synaptic connections from layer 4 to upper layers first cross into layer 3 (Callaway and Katz, 1992); optical measurements, which measure upperlayer activity, probably could not see an earlier map. With electrode studies, ORIselective cells are found in deep layers as early as recording is possible (P6, before normal eye opening) (Albus and Wolf, 1984; Braastad and Heggelund, 1985) (earlier work reviewed byMovshon and Van Sluyters, 1981; Fregnac and Imbert, 1984). The optical results make it seem likely that the initial development of ORI preference in the deep layers is also ocularly correlated. Thus, the predicted correlation structure should probably be looked for in the period P6–P12, although it is possible that ocularly matched ORI maps develop even earlier.
Ocular dominance appears to develop somewhat later (LeVay et al., 1978;Crair et al., 1997a). Physiologically, some OD is first visible optically at P14 (Crair et al., 1997a). Until approximately P18–P20, visual experience appears to have no effect on ORI or OD development (Fregnac and Imbert, 1984; Crair et al., 1997a). The major development of OD, including equalization of the contralateral and ipsilateral projections and anatomical segregation, develops immediately after this time and depends strongly on visual experience (LeVay et al., 1978;Crair et al., 1997a). Our studies have shown that OD may appear later than ORI selectivity even if the relevant correlation structures are present at the same time. Thus, the ODrelevant correlations might be present in spontaneous activity as early as the ORIrelevant correlations and certainly should be present by P14. Such correlations should be present in visually induced activity after P20.
Binocular spatial phase relationships
Our model predicts “binocular subregion correspondence”: the ON and OFF subregions in the two eyes’ RFs for a given cell should be arranged with either a 0° (S^{ORI+}dominant) or 180° (S^{ORI−}dominant) spatial phase shift, measured in absolute LGN coordinates. This result is compatible with observed interocular phase shifts (Freeman and Ohzawa, 1990), because those phases are defined in relativecoordinates that move with the centers of each eye’s RF. Our model allows any value of such a relative phase shift, as long as it is accompanied by an appropriate position shift between the spatial envelopes of the two eyes’ RFs (Fig.14). Because of our choice of a narrow arbor function, only small position shifts between RFs of the two eyes develop in our simulations. Use of a wider arbor function relative to final RF size would allow development of larger position shifts (Berns et al., 1993) and thus a larger range of relative phase shifts.
Some support for subregion correspondence exists in the finding that most tuned excitatory cells in both cat (Ferster, 1981; LeVay and Voigt, 1988) and macaque V1 (Poggio and Fischer, 1977) have preferred disparity at 0°. This arises naturally from subregion correspondence, whereas it is not predicted if relative phase shifts and position shifts are uncorrelated (although nonzero preferred disparities can also arise from subregion correspondence; this will be discussed elsewhere). The same authors report that tuned excitatory cells are binocular, whereas near and far cells tend to be monocular (disparity tuning arising from inhibition from the nondominant eye). This is also consistent with the idea that when both eyes have significant input to a cell, their RFs are in subregion correspondence.
More realistic models of intracortical circuitry might lead to more complex possibilities. For example, multiple forms of coexistence between left and righteye RFs on individual cells might arise through chains or loops of excitatory and inhibitory network interactions. It seems inescapable, though, that the set of absolute spatial phases of left versus righteye RFs in individual layer 4 cells should not be consistent with a random distribution: there should be correlations between the absolute phase found in one eye’s RF and that found in the other eye’s for the preferred ORIs of the two eyes to become matched. This general prediction is robust and can be tested in adult binocular simple cells in cat layer 4.
We are aware of only one experiment (Anzai et al., 1997) that attempted to measure both position and phase shifts in the same set of cells. Unfortunately, we have found that the referencecellpair method used there is not sufficient to distinguish between the coordinated shifts of position and relative phase that we predict, and an uncorrelated distribution of these shifts (Erwin and Miller, 1997). Measuring many cells’ RFs simultaneously yields a stronger test. We will describe these results, and other consequences of binocular subregion correspondence that can be more easily tested, in detail elsewhere.
DISCUSSION
We have studied correlationbased competition among four input types (ON and OFFcenter cells from left and right eyes), and determined the conditions necessary for development of both (1) OD segregation and (2) ORIselective simple cells with ocularly matched ORI preferences. These conditions, approximately summarized, are as follows:
(1) For OD to develop, total withineye activity correlations (summed between a pair of samecentertype and a pair of oppositecentertype LGN cells at a given separation) must be greater than or equal to total betweeneye correlations for LGN input separations within the retinotopic equivalent of a geniculocortical arbor radius.
(2) For ORI to develop, the activity correlations within each eye should have a “Mexicanhat” or oscillatory structure (correlations between samecentertype cell pairs greater than those between oppositecentertype cell pairs at small retinotopic separations but smaller at larger retinotopic separations within an arbor radius).
(3) For preferred ORIs to be matched in the two eyes, betweeneye activity correlations should be centertypespecific, in a manner that varies with input separation over an arbor radius. This ensures that maximal correlations arise when centerspecific subregions in the two eyes are appropriately aligned; as a byproduct, this yields matching of preferred orientations.
(4) For OD and ocularly matched ORI to develop together, the growth rates of OD and of ORI should be approximately comparable.
Conditions 1–3 are fairly general, met by a wide range of correlation structures. They may be met at different times, e.g., Condition 3 may be met only in an early stage in which binocularly matched ORI selectivity develops, whereas Condition 1 may be met only in a later stage in which OD develops. Condition 4 also does not require tight tuning: if ORI and OD develop under a single correlation structure, the ORI/OD growth rate ratio can vary by factors of ≥2–8; if ORI and OD develop in separate stages, this range can be increased as much as three times.
The maxima of OD in the model tend to arise near ORI singularities, as observed experimentally (Crair et al., 1997b), at least when ORI and OD develop in separate stages. A qualitative explanation is proposed: pinwheels slow the growth of ORI but not OD and hence are sites where OD development faces less competition.
Robustness of the results
Model results fall into two categories. Some are general to a large class of correlationbased learning rules; others depend on fairly arbitrary choices in modeling competition between modes.
The first category includes results describing the correlation structure needed to develop a given RF or map structure, e.g., Conditions 1–3 above and the qualitative explanation of the OD maxima–ORI pinwheel relationship. These results should be reasonably robust: they follow from the idea that development leads to RFs and maps that, in some sense, maximize input activity correlations.
Results in the second category involve competition between different modes, e.g., between OD and ORI, where each mode captures a different aspect of the input correlations. Examples include Condition 4 above, the fact that OD maxima–ORI pinwheel relationships are only demonstrable in twostage and not onestage simulations, or Condition 5 (stated only in Results), which involves suppression of OD or ORI development by growth of the SUM mode. These results depend on our simpleminded pictures of saturation and competition, in which different modes grow independently until they are coupled and then stabilized by saturation of synaptic strengths. Biology likely contains more subtle mechanisms of competition (Miller, 1996b; Davis and Goodman, 1998; Turrigiano et al., 1998). These might allow multiple aspects of input activity to be captured: strong development of one feature might allow subsequent development of another feature, rather than freezing development. Such behavior is seen in models in which cortical cells compete nonlinearly for activation (Ritter et al., 1992;Sirosh and Miikulainen, 1997; . Before model predictions in the second category can be given much weight, better understandings of competitive and nonlinear mechanisms are needed, both theoretically and biologically. This might considerably broaden the conditions allowing codevelopment of OD and ORI.
Piepenbrock and Obermayer, Piepenbrock et al. (1996, 1997), studying the present model, independently noted that nonlinearities determine the precise conditions allowing OD and ORI codevelopment (also see Erwin and Miller, 1996b). They concluded from this that codevelopment could not occur robustly. We have shown the opposite and believe the correct conclusion is simply that the quantitative details of conditions allowing codevelopment are not to be trusted.
Some nonlinearities might also modify results in the first category. For example, Feidler et al. (1997) report that certain nonlinearities can yield RFs that would not result from a linear rule. They studied development of direction tuning, but the results translate directly to the present model, with direction tuning replaced by disparity tuning (Wimbauer et al., 1997a,b). After translation, these results show that, in a onedimensional environment in which the distribution of stimulus disparity has nonzero peaks symmetric about zero, certain nonlinearities allow intereye absolute phase differences other than 0 or 180° to develop. It will be interesting to explore these ideas in a 2D environment with alternative intereye stimulus distributions.
Evaluation of assumptions
Our results depend on several simplifying assumptions. First, we have assumed symmetries between the two eyes and between the two center types. As discussed previously, this ignores actual asymmetries in projection strengths and activity patterns.
The symmetry between ON and OFF is not crucial. The “Mexicanhat” correlation structure is sufficient to robustly give simplecell structure within each eye’s RFs. This yields an approximate balancing of total ON and OFF innervation, independent of asymmetries in overall activities of ON versus OFF inputs.
The symmetry between left and right is more important but can be regarded as a simple standin for the unknown mechanisms of competition, that is, for whatever forces lead to equalization of the projections of the two eyes. As discussed elsewhere (Miller and MacKay, 1994; Miller, 1996a), at least two mechanisms can achieve equalization: a Mexicanhat pattern of intracortical interactions, i.e., shortdistance excitation and longerdistance inhibition; or a competitive process (e.g., competition for neurotrophins) that ensures that all afferents with approximately equal average activity achieve approximately equal projection strength. (Because we used a Mexicanhat cortical interaction here, we did not need to assume that the two eyes begin with equal strength; however, this assumption simplified analysis considerably). Given either symmetry assumptions or competitive mechanisms to render the eyes equal, the impact of correlations on development should be as described here.
The facts that there is initial contralateral dominance and that ocular equalization occurs only in the presence of vision have been suggested to require nonHebbian mechanisms of development (Crair et al., 1997a). Instead, we believe they suggest that vision is required to induce adequate competition, e.g., adequate development of inhibition or release of neurotrophins.
We also assume that the LGN inputs from each eye can be divided into two homogeneous groups, ON and OFF (this could be extended to more groups; Appendix ). These groups are heterogeneous in their visual response properties; for example, the temporal phase of response to a drifting sinusoid is widely distributed among neurons of a single type (Saul and Humphrey, 1990). The key requirement of our model is an overall tendency to a Mexicanhat correlation structure between ON and OFF in darkness. A fair amount of diversity in the details, which amounts to addition of noise to the correlations assumed here, can be tolerated; the amount tolerable is a quantitative issue not explored here. Measurements have shown clear separations of ON from OFF retinal cells in their dark correlation properties (Mastronarde, 1983b; Meister et al., 1995; Wong and Oakley, 1996).
Possible sources of the postulated activity correlations
Because the development of ocularly matched ORI maps is independent of vision (Crair et al., 1997a), we presume it depends on LGN spontaneous activity. Are the correlation structures we predict plausible attributes of this activity?
For the ORI maps of the two eyes to become matched, we predict that the LGN must possess centertypespecific intereye correlations. In adults, both excitatory and inhibitory LGN interocular interactions exist (Xue et al., 1987; Guido et al., 1989; Schroeder et al., 1990; Moore et al., 1992; Tong et al., 1992). There are at least three potential sources of intereye LGN correlations at the developmentally appropriate times: the perigeniculate nucleus, the corticogeniculate feedback, and the inhibitory intrinsic connections within the LGN. The perigeniculate nucleus induces spindle waves that, in vitro, propagate across the retinotopic map of the LGN in register between the layers of the two eyes (Kim et al., 1995). These waves arise in ferret (McCormick et al., 1995) shortly before the major development of cortical ORI selectivity (Chapman and Stryker, 1993). Corticogeniculate feedback is established even earlier in development (Shatz and Rakic, 1981; Weber and Kalil, 1987; Johnson and Casagrande, 1993). Cortical simple cells, which retain centertypespecific information, project to LGN only in binocular regions (Tsumoto and Suda, 1980), suggesting such information may be important for LGN binocular interactions.
The existence of strong intereye correlations, apparently generated by corticogeniculate feedback, has recently been demonstrated in developing (P24–P27) ferret LGN, 0.5–1 week before the major development of ORI selectivity (Weliky and Katz, 1998). These correlations appear to have the proper form to drive the major features described thus far in ferret layer 4 (Zahs and Stryker, 1988): OD segregation (because the correlations are stronger within than between eyes) and ON/OFF segregation into cortical patches (because correlations within a single eye are stronger between cells of the same rather than opposite center type; note that differing ON and OFF mean activity levels could explain ON/OFF patch continuity across OD borders). It is not clear whether these correlations have the proper form to account for simple cells or their ocular matching, because retinotopic positions of the cells were not measured. However, it is also not clear whether such correlations should exist in ferret LGN: only 40% of the cells in ferret layer 4 are ORIselective (Chapman and Stryker, 1993), and there is as yet no evidence as to whether these are simple cells.
For development of simplecell RF structure, withineye correlations should have Mexicanhat structure. As extensively discussed by Miller (1994), such correlations are expected from (1) the connectivity yielding centersurround receptive fields, combined with (2) the assumption that correlations in spontaneous activity are attributable to common input from photoreceptors (Mastronarde, 1983b). A possible flaw in this argument was also discussed: it relies on centersurround RF structure but discusses correlations in spontaneous activity, which may involve darkadapted cells. In adult cats, darkadapted cells tend to lose their surrounds in retina (EnrothCugell and Lennie, 1975;Barlow and Levick, 1976) and perhaps also LGN (Virsu et al., 1977;Kaplan et al., 1979). Recent studies in salamander retina demonstrate a Mexicanhat correlation structure between ON and OFF ganglion cells in spontaneous activity both in light and darkness (Meister et al., 1995). These cells also lose their surrounds in darkness (M. Meister, personal communication). Thus, at least in salamander, the circuitry that produces the RF surround in light may equally influence correlations in light and darkness.
These arguments focus on adultlike correlations based on spontaneous quantal events in photoreceptors (Mastronarde, 1983b, 1989). An alternative correlation structure exists in early developing retina, attributable to spatially traveling waves of activity (Wong et al., 1993; Wong and Oakley, 1996). It was argued by Miller (1994) that these waves probably are not involved in development of simple cells and ORI selectivity. The arguments were threefold: the waves are far too wide to produce simple cell subregions; they disappear about or slightly before the time that ORI selectivity first arises; and no ON/OFF difference had been observed in the waves. An ON/OFF difference has since been observed in ferrets (this has not yet been studied in cats) beginning at P14, when ON/OFF segregation begins in LGN (Wong and Oakley, 1996), but ∼2.5 weeks before significant development of ORI selectivity (Chapman and Stryker, 1993; Chapman et al., 1996). Although this difference is appropriate to drive ON/OFF segregation in ferret LGN (Lee and Wong, 1996; Miller, 1996b), the timing and size of the waves continue to render them unlikely candidates for a role in simple cell development.
Other models
Only one other model has addressed ocular matching of ORI preferences in the absence of vision (Wolf et al., 1996). That work concluded that boundary conditions such as those of cat area 18, but not area 17, could cause well correlated ORI maps to develop from activitydependent mechanisms. We predict that interocular correlations in spontaneous activity induce matching of the two eyes’ maps in both areas 17 and 18; boundary conditions may also contribute in area 18 but are not needed given such correlations.
Previous models of joint OD and ORI map development are of two types. Some used reduced description of inputs and RFs in terms of a few lowdimensional “features,” such as OD, ORI, and retinotopic position (Obermayer et al., 1992; Swindale, 1992) (Durbin and Mitchison, 1990; as extended in Erwin et al., 1995). These models assume a single ORI map and thus do not address ocular matching of the maps. More generally, they do not explain how RF features are generated from the inputs of many separate synapses. Synapsebased models have been studied by several authors. Tanaka (1996) and Olson and Grossberg (1998) assumed betweeneye correlations were zero and so obtained only ocularly uncorrelated orientation maps. Shouval et al. (1996) studied development of a single cortical cell’s RF in response to natural images (well correlated between the two eyes); the problem of ocular matching without vision was not addressed, and the approach was not extended to multiple cells and maps. The more general relationship of our separate ORI and OD models to these and other modeling approaches has been extensively discussed (Miller, 1990a, 1994, 1996a; also seeErwin et al., 1995; Swindale, 1996).
Conclusion
Our model is very simple and has many inadequacies. Nonetheless, the richness and diversity of the experimental results accounted for, starting from extremely simple assumptions of a correlationbased synaptic competition, suggest that the basic explanations proposed here may underlie the more complex results of biological development.
The model reveals that the two most salient RF properties of primary visual cortical cells, OD and ORI selectivity, arise naturally from competition among inputs representing two eyes and two center types. Given the symmetry assumptions, the natural variables describing development in this system are ocular difference (S^{OD}), centertype difference (S^{ORI+}), and their combination (S^{ORI−}). The neural organizations of OD and ORI arise as a result of simple properties of the input correlations driving these variables (Fig. 4). It seems possible that many RF properties may arise from such competitions between differing input types [e.g., lagged vs nonlagged, which may yield development of direction selectivity (Feidler et al., 1997; Wimbauer et al., 1997a,b) and colorselective input subtypes in the monkey parvo system; also see Appendix ]. As discussed by Miller (1994), this raises the possibility that such RF properties may arise, not through selection for their computational role in visual processing (although such selection shapes internally generated input correlation structures), but rather as byproducts of deeper and more general, modalityindependent principles of cortical computation.
In conclusion, cortical selforganization via correlationbased synaptic competition can explain the development of linked ORI maps in the two eyes in the absence of vision, along with development of OD segregation, and can naturally account for the observed correlation between OD maxima and ORI singularities. The role of selforganization has two key tests: appropriate LGN correlations as described here must exist at the times that ORI selectivity and OD first develop; and binocular simple cells in adult layer 4 should have nonrandom absolute phase relationships between their right and lefteye receptive fields.
FURTHER DETAILS OF MODE DECOMPOSITION
The decomposition of Equation 14 (which amounts to diagonalization of the matrix of correlation functions; e.g., see Erwin and Miller, 1996b) can be easily found as follows. We assume a symmetry of our equations under exchange of righteye and lefteye labels and under exchange of ONcenter and OFFcenter labels. If our equations (e.g., Eq. 2) are invariant under such exchange, then the eigenvectors of those equations can be expressed as eigenvectors of the “exchange operators.”
Eigenvectors under exchange are easily found as the sum (Σ) or difference (Δ) of the weights designated by the two exchanged labels. For example, for the ocular label E ∈ {L, R}, the sum S^{ΣE} = S^{R}+ S^{L} is equal to itself under L/R exchange and hence has eigenvalue +1 under such exchange, whereas the difference S^{ΔE} = S^{R} − S^{L} is equal to its negative under L/R exchange and hence has eigenvalue −1 under such exchange. The same procedure applies to the centertype label C ∈ {N, F}. This provides a basis for finding the decomposition, by proceeding one label at a time. We let λ_{E}, λ_{C} be the eigenvalue under L/R or N/F exchange, respectively. Then the patterns are found as follows:
Begin with two labels, L and R. Eigenvectors are:
We now identify:
This method can be applied to arbitrary dualistic symmetries of input types. For example, if there were eight input types corresponding to three symmetries (e.g., left/right, ON/OFF, and lagged/nonlagged), then there would be eight independently growing composite weights, corresponding to the four above with eigenvalue +1 and −1 under the new symmetry (i.e., each of the four composite S^{x}s above would split into two composite weights, [(S^{x})^{lagged} ± (S^{x})^{nonlagged}], where ± gives eigenvalue ±1 under lagged/nonlagged exchange; and the corresponding correlation functions would be (C^{x})^{ΣT} and (C^{x})^{ΔT}, where Σ_{T} and Δ_{T} stand for sum and difference, respectively, of sametemporaltype and oppositetemporaltype correlation functions).
More generally, no matter how many types of cells there may be and whether they form dualistic symmetries, as long as the symmetries hold between left and right and between ON and OFF, then, if we characterize inputs only by L/R/N/F (ignoring all other distinctions), then the development of L/R/N/F will still be described in the linear regime by the above four modes and composite correlation functions. However, the underlying correlation functions C^{EC}, E ∈ {S_{E}, O_{E}}, C ∈ {S_{C}, O_{C}} may be more difficult to determine, because they will involve various linear combinations over correlation functions involving the various other subtypes. Nonetheless, assuming we have calculated the C^{EC} correctly, our analysis will then correctly describe the development of L/R/N/F (ignoring all other distinctions) in terms of these correlation functions.
RELATIONSHIP OF TWOINPUTTYPE AND FOURINPUTTYPE MODELS
We previously considered twoinputtype models of OD (Miller et al., 1989) and ORI (Miller, 1994) development. We label the two input types S^{I}, I ∈ {1, 2}. In the OD model, S^{1} and S^{2} were righteye and lefteye inputs, S^{R} and S^{L}, whereas in the ORI model, S^{1} and S^{2} were ONcenter and OFFcenter inputs, S^{N} and S^{F}. These models used equations identical to those of the present model (Eqs. 1, 2, 59), except for the use of only two rather than four input types (replacing the combination E, C by I and E′, C′ by I′, the equations become identical).
In both twoinputtype models, we assumed a symmetry of the equations under exchange of S^{1} and S^{2}, which implies that C^{11} = C^{22} ≡ C^{SAME} and C^{12} = C^{21} ≡ C^{OPP}. Development before saturation limits took effect was then reexpressed in terms of composite variables: S^{S} = S^{1} + S^{2}, S^{D} = S^{1} − S^{2}, C^{S} = C^{SAME} + C^{OPP}, and C^{D} = C^{SAME}− C^{OPP}. Again, the procedures of the present paper are identical except for the change in the number of input types.
The resulting equation for development of S^{D} in terms of C^{D} was identical to each of Equations1414 for development of S^{μ} in terms of C^{μ}, μ ∈ {OD, ORI+, ORI−}. We previously showed how the independently growing patterns of S^{D} and their growth rates are determined by C^{D}. Because the equations are identical, these results apply directly to tell how the independently growing patterns of each S^{μ}, μ ∈ {OD, ORI+, ORI−}, and their growth rates are determined by the corresponding C^{μ}.
The resulting equation for development of S^{S} in terms of C^{S} was identical to Equation 14a for development of S^{SUM} in terms of C^{SUM}. [There is a superficial difference: a factor of 2 rather than 4 multiplying ε(x⃗), which reflects the number of input types. The equations become identical if Eq. 9 for ε is rewritten in terms of S^{SUM} (or S^{S}) rather than S^{EC} (or S^{I}).] Thus, our previous studies of the development of S^{S} can be applied directly to understand development of S^{SUM} under Equation 14a.
Beyond formal equivalence of the equations, the twoinputtype models can also be seen as special cases of the present model. If we define S^{R} = S^{RN} + S^{RF}, S^{L} = S^{LN} + S^{LF}, C^{SAME} = C^{SE} ^{SC} + C^{SE} ^{OC}, and C^{OPP} = C^{OE} ^{SC} + C^{OE} ^{OC}, then S^{D} and C^{D}in the OD model are identical to S^{OD} and C^{OD} in the present model. Thus, the twoinputtype OD model can be understood as a special case of the present model in which ON/OFF differences are zero, so that S^{ORI+} = S^{ORI−} = 0. Similarly, if we define S^{N} = S^{RN} + S^{LN}, S^{F} = S^{RF} + S^{LF}, C^{SAME} = C^{SESC} + C^{OE} ^{SC}, and C^{OPP} = C^{SE} ^{OC} + C^{OE} ^{OC}, then S^{D} and C^{D}in the ON/OFF model are identical to S^{ORI+} and C^{ORI+} in the present model. So the twoinputtype ON/OFF model can be understood as a special case of the present model in which left–right differences are zero, so that S^{OD} = S^{ORI−} = 0. These equivalences can also be seen from Appendix : setting all terms involving Δ_{E} to zero, we arrive at the ON/OFF model, whereas setting all terms involving Δ_{C} to zero, we arrive at the left/right model.
Footnotes

This work was supported by National Institutes of Health Grants NS07067 and EY11001 and by grants from the Searle Scholars’ Program and the Lucille P. Markey Charitable Trust. We gratefully acknowledge advice on multiple versions of this manuscript from Steve Lisberger, as well as helpful comments from Mark Kvale, Michael Crair, Ed Ruthazer, and Michael Stryker. We thank the latter three and Deda Gillespie for sharing their experimental results with us at preliminary stages.
Correspondence should be addressed to Kenneth D. Miller, Department of Physiology, University of California, San Francisco, CA 941430444.