Visual images of our own and others' body parts can be highly similar, but the types of information we wish to extract from them are highly distinct. From our own body we wish to combine visual information with, at least, somatosensory, proprioceptive and motor information in order to guide our interpretation of sensory events and our actions upon the world. For others' bodies we only have visual information available, but from that we can derive much useful social information including their age, health, gender, emotional state and intentions. Consequently, a challenge for the brain is to sort visual images of our own bodies, to be integrated with processing from other sensory modalities, from highly similar images of others' bodies for social cognition. We explored the possibility that the extrastriate body area (EBA) may help to accomplish this sorting. Previous work had suggested that the EBA is responsive to images of both our own and others' body parts but does not distinguish between them. Here, using fMRI adaptation, we provide evidence that the right EBA contains separate neural sub-populations that are selectively sensitive to images of our own or others' body parts. Thus, we argue that the right EBA may perform an important sorting of body part images by identity (including self-recognition) and may interact both with brain areas involved in sensory processing and social cognition having identified our own and others' body part images respectively.