Functional magnetic resonance imaging (fMRI) is a non-invasive technique that has come into common use to examine neural network function in normal and impaired cognitive states. Using this promising type of analysis, researchers have identified the presence of anatomically distributed regions operating as large-scale neural networks, which are observed both during the performance of associative memory tasks and in the resting state. The assembly of these anatomically distinct regions into functional ensembles and their choreographed activation and deactivation sets the stage for complex behaviors such as the formation and retrieval of associative memories. We review progress in the use of task-related and task-free MRI to elucidate the changes in neural activity in normal older individuals, patients with mild cognitive impairment, and those with Alzheimer's disease, focusing on the altered activity of the default mode network and medial temporal lobe. We place task-free fMRI studies into the larger context of more traditional, task-based fMRI studies of human memory, which have firmly established the critical role of the medial temporal lobe in associative encoding. Lastly, we discuss the data from our group and others that suggests task-free MRI and task-based fMRI may prove useful as non-invasive biomarkers in studying the progression of memory failure over the course of Alzheimer's disease.