The direction of frequency-modulated (FM) sweeps is an important temporal cue in animal and human communication. FM direction-selective neurons are found in the primary auditory cortex (A1), but their topography and the mechanisms underlying their selectivity remain largely unknown. Here we report that in the rat A1, direction selectivity is topographically ordered in parallel with characteristic frequency (CF): low CF neurons preferred upward sweeps, whereas high CF neurons preferred downward sweeps. The asymmetry of 'inhibitory sidebands', suppressive regions flanking the tonal receptive field (TRF) of the spike response, also co-varied with CF. In vivo whole-cell recordings showed that the direction selectivity already present in the synaptic inputs was enhanced by cortical synaptic inhibition, which suppressed the synaptic excitation of the non-preferred direction more than that of the preferred. The excitatory and inhibitory synaptic TRFs had identical spectral tuning, but with inhibition delayed relative to excitation. The spectral asymmetry of the synaptic TRFs co-varied with CF, as had direction selectivity and sideband asymmetry, and thus suggested a synaptic mechanism for the shaping of FM direction selectivity and its topographic ordering.