During speech comprehension the auditory association cortex in the superior temporal cortex is involved in perceptual analysis of the speech signal, whereas the basal language area in the inferior temporal cortex mediates access to word meaning. Disruption of the interaction between the superior and inferior temporal cortices is one factor that may determine recovery from aphasic stroke. We used positron emission tomography to investigate semantic processing within inferior temporal cortex in control subjects and after infarction involving the superior temporal cortex. In the control group, semantic decision making on clear speech activated both anterior fusiform gyri. Chronic aphasic patients were impaired at the task and demonstrated reduced activation within the left anterior fusiform gyrus. A similar pattern of impaired performance and reduced left anterior fusiform gyrus activation was observed when control subjects heard perceptually degraded speech. Performance in both groups predicted activity in the right anterior fusiform gyrus and the temporal poles, where accuracy linearly correlated with activity. These results demonstrate that the function of the basal language area is sensitive to changes in the quality of perceptual input. In addition, different profiles of response observed in each hemisphere suggest distinct contributions of both left and right inferior temporal cortices to the semantic processing of speech.