Diffusion imaging can be used to estimate the routes taken by fiber pathways connecting different regions of the living brain. This approach has already supplied novel insights into in vivo human brain anatomy. For example, by detecting where connection patterns change, one can define anatomical borders between cortical regions or subcortical nuclei in the living human brain for the first time. Because diffusion tractography is a relatively new technique, however, it is important to assess its validity critically. We discuss the degree to which diffusion tractography meets the requirements of a technique to assess structural connectivity and how its results compare to those from the gold-standard tract tracing methods in nonhuman animals. We conclude that although tractography offers novel opportunities it also raises significant challenges to be addressed by further validation studies to define precisely the limitations and scope of this exciting new technique.