Event-related potential (ERP) studies of the human brain have shown that object categories can be reliably distinguished as early as 130-170 ms on the surface of occipito-temporal cortex, peaking at the level of the N170 component. Consistent with this finding, neuropsychological and neuroimaging studies suggest major functional distinctions within the human object recognition system, particularly in hemispheric advantage, between the processing of words (left), faces (right), and objects (bilateral). Given these observations, our aim was to (1) characterize the differential response properties of the N170 to pictures of faces, objects, and words across hemispheres; and (2) test whether an effect of inversion for highly familiar and monooriented nonface stimuli such as printed words can be observed at the level of the N170. Scalp EEG (53 channels) was recorded in 15 subjects performing an orientation decision task with pictures of faces, words, and cars presented upright or inverted. All three categories elicited at the same latency a robust N170 component associated with a positive counterpart at centro-frontal sites (vertex-positive potential, VPP). While there were minor amplitude differences at the level of the occipital medial P1 between linguistic and nonlinguistic categories, scalp topographies and source analyses indicated strong hemispheric and orientation effects starting at the level of the N170, which was right lateralized for faces, smaller and bilateral for cars, and as large for printed words in the left hemisphere as for faces. The entire N170/VPP complex was accounted for by two dipolar sources located in the lateral inferior occipital cortex/posterior fusiform gyrus. These two locations were roughly equivalent across conditions but differed in strength and lateralization. Inversion delayed the N170 (and VPP) response for all categories, with an increasing delay for cars, words, and faces, respectively, as suggested by source modeling analysis. Such results show that early processes in object recognition respond to category-specific visual information, and are associated with strong lateralization and orientation bias.