Adaptation to faces leads to face aftereffects and currently this topic attracts a lot of attention because it clearly shows that adaptation occurs even at the higher stages of visual cortical processing. Recently it has been found that long-term exposure to a face stimulus results in adaptation of a position-specific population of face sensitive neurons in addition to a position-invariant neural population, the later being also adapted in the case of short-term adaptation. Here we used the fMRI adaptation technique to investigate the neural locus of position-specific and position-invariant face adaptation. We show that in the right fusiform face area adaptation effects are position invariant and can be evoked by short (500 ms) as well as long (4500 ms) adaptation durations. On the other hand adaptation effects in the right occipital face area are position-specific and require long-term adaptation to develop. These findings imply that the behaviourally observed face aftereffects reflect time-dependent adaptation processes of both position-specific and invariant face sensitive neurons at different stages of visual processing.