Understanding complex movements and abstract action goals is an important skill for our social interactions. Successful social interactions entail understanding of actions at different levels of action description, ranging from detailed movement trajectories that support learning of complex motor skills through imitation to distinct features of actions that allow us to discriminate between action goals and different action styles. Previous studies have implicated premotor, parietal, and superior temporal areas in action understanding. However, the role of these different cortical areas in action understanding at different levels of action description remains largely unknown. We addressed this question using advanced animation and stimulus generation techniques in combination with sensitive functional magnetic resonance imaging adaptation or repetition suppression methods. We tested the neural sensitivity of fronto-parietal and visual areas to differences in the kinematics and goals of actions using kinematic morphs of arm movements. Our findings provide novel evidence for differential involvement of ventral premotor, parietal, and temporal regions in action understanding. We show that the ventral premotor cortex encodes the physical similarity between movement trajectories and action goals that are important for exact copying of actions and the acquisition of complex motor skills. In contrast, whereas parietal regions and the superior temporal sulcus process the perceptual similarity between movements and may support the perception and imitation of abstract action goals and movement styles. Thus, our findings propose that fronto-parietal and visual areas involved in action understanding mediate a cascade of visual-motor processes at different levels of action description from exact movement copies to abstract action goals achieved with different movement styles.