The prevalence of coherent oscillations in various frequency ranges in the central nervous system raises the question of the mechanisms that synchronize large populations of neurons. We study synchronization in models of large networks of spiking neurons with random sparse connectivity. Synchrony occurs only when the average number of synapses, M, that a cell receives is larger than a critical value, Mc. Below Mc, the system is in an asynchronous state. In the limit of weak coupling, assuming identical neurons, we reduce the model to a system of phase oscillators that are coupled via an effective interaction, gamma. In this framework, we develop an approximate theory for sparse networks of identical neurons to estimate Mc analytically from the Fourier coefficients of gamma. Our approach relies on the assumption that the dynamics of a neuron depend mainly on the number of cells that are presynaptic to it. We apply this theory to compute Mc for a model of inhibitory networks of integrate-and-fire (I&F) neurons as a function of the intrinsic neuronal properties (e.g., the refractory period Tr), the synaptic time constants, and the strength of the external stimulus, Iext. The number Mc is found to be nonmonotonous with the strength of Iext. For Tr = 0, we estimate the minimum value of Mc over all the parameters of the model to be 363.8. Above Mc, the neurons tend to fire in smeared one-cluster states at high firing rates and smeared two-or-more-cluster states at low firing rates. Refractoriness decreases Mc at intermediate and high firing rates. These results are compared to numerical simulations. We show numerically that systems with different sizes, N, behave in the same way provided the connectivity, M, is such that 1/Meff = 1/M - 1/N remains constant when N varies. This allows extrapolating the large N behavior of a network from numerical simulations of networks of relatively small sizes (N = 800 in our case). We find that our theory predicts with remarkable accuracy the value of Mc and the patterns of synchrony above Mc, provided the synaptic coupling is not too large. We also study the strong coupling regime of inhibitory sparse networks. All of our simulations demonstrate that increasing the coupling strength reduces the level of synchrony of the neuronal activity. Above a critical coupling strength, the network activity is asynchronous. We point out a fundamental limitation for the mechanisms of synchrony relying on inhibition alone, if heterogeneities in the intrinsic properties of the neurons and spatial fluctuations in the external input are also taken into account.