Tip-of-the-tongue (TOT) experiences are frustrating word-finding failures where people are temporarily unable to produce a word they are certain they know. TOT frequency increases with normal aging during adulthood, and behavioral evidence suggests that the underlying deficit is in retrieving the complete phonology of the target word during production. The present study investigated the neural correlates of this phonological retrieval deficit. We obtained 3-D T1-weighted structural magnetic resonance images (MRI) for healthy participants between 19 and 88 years old and used voxel-based morphometry to measure gray matter density throughout the brain. In a separate session, participants named celebrities cued by pictures and descriptions, indicating when they had a TOT, and also completed Raven's Progressive Matrices (RPM), a task that does not involve phonological production. The number of TOTs increased with age and also with gray matter atrophy in the left insula, an area implicated in phonological production. The relation between TOTs and left insula atrophy cannot be attributed to the correlation of each variable with age because TOTs were related to insula atrophy even with age effects removed. Moreover, errors on the RPM increased with age, but performance did not correlate with gray matter density in the insula. These results provide, for the first time, an association between a region in the neural language system and the rise in age-related word-finding failures and suggest that age-related atrophy in neural regions important for phonological production may contribute to age-related word production failures.