Legal decision-making in criminal contexts includes two essential functions performed by impartial "third parties:" assessing responsibility and determining an appropriate punishment. To explore the neural underpinnings of these processes, we scanned subjects with fMRI while they determined the appropriate punishment for crimes that varied in perpetrator responsibility and crime severity. Activity within regions linked to affective processing (amygdala, medial prefrontal and posterior cingulate cortex) predicted punishment magnitude for a range of criminal scenarios. By contrast, activity in right dorsolateral prefrontal cortex distinguished between scenarios on the basis of criminal responsibility, suggesting that it plays a key role in third-party punishment. The same prefrontal region has previously been shown to be involved in punishing unfair economic behavior in two-party interactions, raising the possibility that the cognitive processes supporting third-party legal decision-making and second-party economic norm enforcement may be supported by a common neural mechanism in human prefrontal cortex.