Studies have converged in their findings of relatively less gray matter volume (GMV) in developmental dyslexia in bilateral temporoparietal and left occipitotemporal cortical regions. However, the interpretation of these results has been difficult. The reported neuroanatomical differences in dyslexia may be causal to the reading problems, following from, for example, neural migration errors that occurred during early human development and before learning to read. Alternatively, less GMV may represent the consequence of an impoverished reading experience, akin to the experience-dependent GMV differences attributed to illiterate compared with literate adults. Most likely, a combination of these factors is driving these observations. Here we attempt to disambiguate these influences by using a reading level-matched design, where dyslexic children were contrasted not only with age-matched controls, but also with younger controls who read at the same level as the dyslexics. Consistent with previous reports, dyslexics showed less GMV in multiple left and right hemisphere regions, including left superior temporal sulcus when compared with age-matched controls. However, not all of these differences emerged when dyslexics were compared with controls matched on reading abilities, with only right precentral gyrus GMV surviving this second analysis. When similar analyses were performed for white matter volume, no regions emerged from both comparisons. These results indicate that the GMV differences in dyslexia reported here and in prior studies are in large part the outcome of experience (e.g., disordered reading experience) compared with controls, with only a fraction of the differences being driven by dyslexia per se.