Previous fMRI results suggest that extrastriate visual areas have a predominant role in perceptual processing while the prefrontal cortex (PFC) has a predominant role in working memory. In contrast, single-unit recording studies in monkeys have demonstrated a relationship between extrastriate visual areas and visual working memory tasks. In this study we tested whether activity in both the PFC and fusiform face area (FFA) changed with increasing demands of an n-back task for gray-scale faces. Since stimulus presentation was identical across conditions, the n-back task allowed us to parametrically vary working memory demands across conditions while holding perceptual and motor demands constant. This study replicated the result of PFC areas of activation that increased directly with load n of the task. The novel finding in all subjects was FFA activation that also increased directly with load n of the task. Since perceptual demands were equivalent across the three task conditions, these findings suggest that activity in both the PFC and the FFA vary with face working memory demands.