From the analyses of our own and others' anatomical and physiological data for the macaque visual system, we arrive at a conclusion that three pathways can provide the V1 neuron with access to information from the visual field and affect its response. First, direct thalamic input can determine the size of the initial activating RF at high contrast. Second, lateral connections can enlarge the RF at low contrast by pooling information from larger regions of cortex that are otherwise ineffective when high contrast thalamic input is driving the cortical neuron. Thirdly, feedback from extrastriate cortex (possibly together with overlap or interdigitation of coactive lateral connectional fields within V1) can provide a large and stimulus specific surround modulatory field. The stimulus specificity of the interactions between the center and surround fields, may be due to the orderly, matching structure and different scales of intra-areal and feedback projection excitatory pathways. The observed activity changes of single recorded excitatory neurons could be a result of the relative weight of excitation on the excitatory neurons themselves and on local inhibitory interneurons that synapse on them. Inhibitory basket neurons, driven by the local excitatory neurons, could govern local interactions between cortical patches of different tuning properties, resulting in more distant changes in excitatory input in the laterally connected intra-areal neuronal pools.