Single-cell responses in visual cortex to a target falling within their receptive field can be modified by collinear flanking stimuli concurrently presented outside the receptive field. Here, we report the presence of four types of contrast-dependent lateral effects: (1) facilitation at low target contrasts and suppression at high contrasts, (2) facilitation that increases with contrast, (3) suppression that increases with contrast, and (4) suppression at low contrasts with facilitation at high contrasts. We propose a sensitivity modulation model that accounts for all the four types of lateral effects by changes in two parameters. In this model, activation of neighboring neurons changes the sensitivities of the target neuron to both the direct feedforward input and inhibitory, divisive feedback from neighboring neurons.