Sensory maps are reshaped by experience. It is unknown how map plasticity occurs in vivo in functionally diverse neuronal populations because activity of the same cells has not been tracked over long time periods. Here we used repeated two-photon imaging of a genetic calcium indicator to measure whisker-evoked responsiveness of the same layer 2/3 neurons in adult mouse barrel cortex over weeks, first with whiskers intact, then during continued trimming of all but one whisker. Across the baseline period, neurons displayed heterogeneous yet stable responsiveness. During sensory deprivation, responses to trimmed whisker stimulation globally decreased, whereas responses to spared whisker stimulation increased for the least active neurons and decreased for the most active neurons. These findings suggest that recruitment of inactive, 'silent' neurons is part of a convergent redistribution of population activity underlying sensory map plasticity. Sensory-driven responsiveness is a key property controlling experience-dependent activity changes in individual neurons.