Visual cortex shows smooth retinotopic organization on the macroscopic scale, but it is unknown how receptive fields are organized at the level of neighboring neurons. This information is crucial for discriminating among models of visual cortex. We used in vivo two-photon calcium imaging to independently map ON and OFF receptive field subregions of local populations of layer 2/3 neurons in mouse visual cortex. Receptive field subregions were often precisely shared among neighboring neurons. Furthermore, large subregions seem to be assembled from multiple smaller, non-overlapping subregions of other neurons in the same local population. These experiments provide, to our knowledge, the first characterization of the diversity of receptive fields in a dense local network of visual cortex and reveal elementary units of receptive field organization. Our results suggest that a limited pool of afferent receptive fields is available to a local population of neurons and reveal new organizational principles for the neural circuitry of the mouse visual cortex.