This study investigated effects of age and sex on regional brain structure in humans, focusing on the frontal and temporal lobes. Hemispheric volumes were obtained from magnetic resonance images (MRIs) of 96 young (53 men, 43 women; aged 18–40 years) and 34 older (17 men, 17 women; aged 41–80) healthy volunteers. Images (5 mm axial spin-echo, repetition time of 3000 msec and echo times of 30 and 80 msec) were resliced along the anterior commissure-posterior commissure (AC-PC) axis to standardize for difference in head tilt, and imported into a computer program where borders of the frontal and temporal lobes were delineated. The program calculated regional brain volumes based on slice data from which CSF was segmented out. An age x sex x hemisphere x region interaction indicated that age-related reductions in brain volume were sexually dimorphic, lateralized, and region specific. Greater decrements in brain volume occurred with age in the frontal lobe than in the temporal lobe. Age-related reductions in both regions were greater in men than in women, demonstrating that sexual dimorphisms in human neuroanatomy are not fixed, but continue to change throughout adulthood. The possibility that gonadal hormones play a role in the promotion and/or prevention of neural atrophy with aging is discussed.