Cortical inhibition plays an important role in shaping neuronal processing. The underlying synaptic mechanisms remain controversial. Here, in vivo whole-cell recordings from neurons in the rat primary auditory cortex revealed that the frequency tuning curve of inhibitory input was broader than that of excitatory input. This results in relatively stronger inhibition in frequency domains flanking the preferred frequencies of the cell and a significant sharpening of the frequency tuning of membrane responses. The less selective inhibition can be attributed to a broader bandwidth and lower threshold of spike tonal receptive field of fast-spike inhibitory neurons than nearby excitatory neurons, although both types of neurons receive similar ranges of excitatory input and are organized into the same tonotopic map. Thus, the balance between excitation and inhibition is only approximate, and intracortical inhibition with high sensitivity and low selectivity can laterally sharpen the frequency tuning of neurons, ensuring their highly selective representation.