Sex differences in brain and behavior are ubiquitous in sexually reproducing species. Developmental differences in circulating concentrations of gonadal steroids underlie many sexual dimorphisms. During the late embryonic and early perinatal periods, the testes produce androgens, thus, male brains are exposed to testosterone, and in situ testosterone is aromatized to estradiol. In contrast, females are not exposed to high concentrations of testosterone or estradiol until puberty. In many species, neural sex differences and sexually dimorphic behaviors in adults are initiated primarily by estradiol exposure during early development. In brain, estradiol activates two independent processes: masculinization of neural circuits and networks that are essential for expression of male-typical adult behaviors, and defeminization, the loss of the ability to display adult female-typical behaviors. Here, data for the roles of each of the known estrogen receptors (estrogen receptor alpha and estrogen receptor beta) in these two processes are reviewed. Based on work done primarily in knockout mouse models, separate roles for the two estrogen receptors are suggested. Estrogen receptor alpha is primarily involved in masculinization, while estrogen receptor beta has a major role in defeminization of sexual behaviors. In sum, estradiol can have selective effects on distinct behavioral processes via selective interactions with its two receptors, estrogen receptor alpha and estrogen receptor beta.