Humans show a preference for using the right hand over the left for tasks and activities of everyday life. While experimental work in non-human primates has identified the neural systems responsible for reaching and grasping, the neural basis of lateralized motor behavior in humans remains elusive. The advent of diffusion imaging tractography for studying connectional anatomy in the living human brain provides the possibility of understanding the relationship between hemispheric asymmetry, hand preference, and manual specialization. In this study, diffusion tractography was used to demonstrate an interaction between hand preference and the asymmetry of frontoparietal tracts, specifically the dorsal branch of the superior longitudinal fasciculus, responsible for visuospatial integration and motor planning. This is in contrast to the corticospinal tract and the superior cerebellar peduncle, for which asymmetry was not related to hand preference. Asymmetry of the dorsal frontoparietal tract was also highly correlated with the degree of lateralization in tasks requiring visuospatial integration and fine motor control. These results suggest a common anatomical substrate for hand preference and lateralized manual specialization in frontoparietal tracts important for visuomotor processing.