Memory for past events can be based on recollection or on assessments of familiarity. These two forms of human memory have been studied extensively by philosophers and psychologists, but their neuroanatomical substrates are largely unknown. Here we examined the brain regions that are involved in these two forms of memory by studying patients with damage to different temporal lobe regions. Our results come from (i) structural covariance modeling of recall and recognition, (ii) introspective reports during recognition and (iii) analysis of receiver operating characteristics. In sum, we found that the regions disrupted in mild hypoxia, such as the hippocampus, are centrally involved in conscious recollection, whereas the surrounding temporal lobe supports familiarity-based memory discrimination.