Rapid memory formation relies, at least in part, on long-term potentiation (LTP) of excitatory synapses. Inhibitory interneurons of the hippocampus, which are essential for information processing, have recently been found to exhibit not one, but two forms of LTP. One form resembles LTP that occurs in pyramidal neurons, which depends on N-methyl-D-aspartate receptors and is triggered by coincident pre- and postsynaptic activity. The other depends on Ca2+ influx through glutamate receptors that preferentially open when the postsynaptic neuron is at rest. Here we review these contrasting forms of LTP and describe how they are mirrored by two forms of long-term depression. We further discuss how the remarkable plasticity of glutamatergic synapses on interneurons greatly enhances the computational capacity of the cortical microcircuit.