The human brain is organized into a collection of interacting networks with specialized functions to support various cognitive functions. Recent research has reached a consensus that the brain manifests small-world topology, which implicates both global and local efficiency at minimal wiring costs, and also modular organization, which indicates functional segregation and specialization. However, the important questions of how and when the small-world topology and modular organization come into existence remain largely unanswered. Taking a graph theoretic approach, we attempt to shed light on this matter by an in vivo study, using diffusion tensor imaging based fiber tractography, on 39 healthy pediatric subjects with longitudinal data collected at average ages of 2 weeks, 1 year, and 2 years. Our results indicate that the small-world architecture exists at birth with efficiency that increases in later stages of development. In addition, we found that the networks are broad scale in nature, signifying the existence of pivotal connection hubs and resilience of the brain network to random and targeted attacks. We also observed, with development, that the brain network seems to evolve progressively from a local, predominantly proximity based, connectivity pattern to a more distributed, predominantly functional based, connectivity pattern. These observations suggest that the brain in the early years of life has relatively efficient systems that may solve similar information processing problems, but in divergent ways.