Neurons in the primary visual cortex of higher mammals respond selectively to light/dark borders of a particular orientation. The receptive fields of simple cells, a type of orientation-selective cell, consist of adjacent, oriented regions alternately receiving ON-center and OFF-center excitatory input. I show that this segregation of inputs within receptive fields can occur through an activity-dependent competition between ON-center and OFF-center inputs, just as segregation of inputs between different postsynaptic cells into ocular dominance columns appears to occur through activity-dependent competition between left-eye and right-eye inputs. These different outcomes are proposed to result, not from different mechanisms, but from different spatial structures of the correlations in neural activity among the competing inputs in each case. Simple cells result if ON-center inputs are best correlated with other ON-center inputs, and OFF with OFF, at small retinotopic separations, but ON-center inputs are best correlated with OFF-center inputs at larger separations. This hypothesis leads robustly to development of simple cell receptive fields selective for orientation and spatial frequency, and to the continuous and periodic arrangement of preferred orientation across the cortex. Input correlations determine the mean preferred spatial frequency and degree of orientation selectivity. Estimates of these correlations based on measurements in adult cat retina (Mastronarde, 1983a,b) produce quantitative predictions for the mean preferred spatial frequencies of cat simple cells across eccentricities that agree with experiments (Movshon et al., 1978b). Intracortical interactions are the primary determinant of cortical organization. Simple cell spatial phases can play a key role in this organization, so arrangements of spatial phases and preferred orientations may need to be studied together to understand either alone. Possible origins for other cortical features including spatial frequency clusters, afferent ON/OFF segregation, blobs, pinwheels, and opponent inhibition within simple cell receptive fields are suggested. A number of strong experimental tests of the hypothesis are proposed.