Physiological rhythmic activity in cortical circuits relies on GABAergic inhibition to balance excitation and control spike timing. With a focus on recent experimental progress in the hippocampus, here we review the mechanisms by which synaptic inhibition can control the precise timing of spike generation, by way of effects of GABAergic events on membrane conductance ('shunting' inhibition) and membrane potential ('hyperpolarizing' inhibition). Synaptic inhibition itself can be synchronized by way of interactions within networks of GABAergic neurons, and by excitatory neurons. The importance of GABAergic mechanisms for generation of cortical rhythms is now well established. What remains to be resolved is how such inhibitory control of spike timing can be harnessed for long-range fast synchronization, and the relevance of these mechanisms to network function. This review is part of the INMED/TINS special issue Physiogenic and pathogenic oscillations: the beauty and the beast, based on presentations at the annual INMED/TINS symposium (http://inmednet.com).