The level of synchronization in distributed systems is often controlled by the strength of the interactions between individual elements. In brain circuits the connection strengths between neurons are modified under the influence of spike-timing-dependent plasticity (STDP) rules. Here we show that when recurrent networks with conduction delays exhibit population bursts, STDP rules exert a strong decoupling force that desynchronizes activity. Conversely, when activity in the network is random, the same rules can have a coupling and synchronizing influence. The presence of these opposing forces promotes the self-organization of spontaneously active neuronal networks to a state at the border between randomness and synchrony. The decoupling force of STDP may be engaged by the synchronous bursts occurring in the hippocampus during slow-wave sleep, leading to the selective erasure of information from hippocampal circuits as memories are established in neocortical areas.