One of the most provocative and exciting issues in cognitive science is how neural specificity for semantic categories of common objects arises in the functional architecture of the brain. More than two decades of research on the neuropsychological phenomenon of category-specific semantic deficits has generated detailed claims about the organization and representation of conceptual knowledge. More recently, researchers have sought to test hypotheses developed on the basis of neuropsychological evidence with functional imaging. From those two fields, the empirical generalization emerges that object domain and sensory modality jointly constrain the organization of knowledge in the brain. At the same time, research within the embodied cognition framework has highlighted the need to articulate how information is communicated between the sensory and motor systems, and processes that represent and generalize abstract information. Those developments point toward a new approach for understanding category specificity in terms of the coordinated influences of diverse regions and cognitive systems.