A number of theories have emerged to explain the well-studied changes in memory that occur with age. Many of these theories invoke mechanisms that have the potential to affect multiple cognitive domains, in addition to memory. Such mechanisms include alterations in attentional or inhibitory function, or dysfunction of specific brain areas, such as the frontal lobes. To gain insight into these mechanisms, we used functional magnetic resonance imaging to examine brain activity during encoding and recognition tasks in young, middle-aged, and older adults to identify correlations between age and brain activity across the various tasks. The goal was to see whether these correlations were task-specific or common across tasks, and to determine whether age differences emerged in a linear fashion over the adult years. Across all memory tasks, at both encoding and recognition, linear increases of activity with age were found in areas normally decreased during task performance (e.g., medial frontal and parietal regions), whereas activity in regions with task-related activation (e.g., dorsolateral prefrontal cortex) decreased with age. These results suggest that there is a gradual, age-related reduction in the ability to suspend non-task-related or "default-mode" activity and engage areas for carrying out memory tasks. Such an alteration in the balance between default-mode and task-related activity could account for increased vulnerability to distraction from irrelevant information, and thereby affect multiple cognitive domains.