Anatomical studies demonstrate that excitatory connections in cortex are not uniformly distributed across a network but instead exhibit clustering into groups of highly connected neurons. The implications of clustering for cortical activity are unclear. We studied the effect of clustered excitatory connections on the dynamics of neuronal networks that exhibited high spike time variability owing to a balance between excitation and inhibition. Even modest clustering substantially changed the behavior of these networks, introducing slow dynamics during which clusters of neurons transiently increased or decreased their firing rate. Consequently, neurons exhibited both fast spiking variability and slow firing rate fluctuations. A simplified model shows how stimuli bias networks toward particular activity states, thereby reducing firing rate variability as observed experimentally in many cortical areas. Our model thus relates cortical architecture to the reported variability in spontaneous and evoked spiking activity.