Early electrophysiological responses to multiple face orientations correlate with individual discrimination performance in humans
Introduction
Presenting face stimuli upside-down (either flipped vertically or rotated by 180°) dramatically affects their recognition (e.g. Hochberg and Galper, 1967). This observation has become particularly important in the face processing literature, for several reasons. First, the effect of inversion for faces is much larger than for other object categories, a phenomenon known as the ‘face inversion effect’ (Yin, 1969). This effect is one of the first cited evidence in favor of specific brain mechanisms for faces (i.e. those that are particularly affected by inversion, Yin, 1969), together with the observation of recognition impairments specific for faces following brain damage (‘prosopagnosia’, Bodamer, 1947) and of inferior–temporal cortex neurons responding selectively to faces in the monkey brain (Gross et al., 1972). Second, the effect is extremely robust, and has been observed in a variety of conditions: for familiar and unfamiliar faces, in old/new recognition tasks or matching tasks, with or without delay between stimuli to match (for a recent review, see Rossion and Gauthier, 2002). Finally, a large number of behavioral studies support the view that inversion of a face affects mainly, but not exclusively, the processing of facial configuration. For instance, when faces are inverted the recognition of facial features is no longer (or less) affected by the presence of other features, or by the whole face stimulus (e.g. Sergent, 1984, Tanaka and Farah, 1993, Young et al., 1987). Moreover, most studies show that the perception of metric distances between features (i.e. mouth–nose distance, eyes height within the face stimulus…) is more affected by inversion than the perception of local cues (shape of the mouth, color of the eyes…) (e.g. Barton et al., 2001, Collishaw and Hole, 2000, Freire et al., 2000, Goffaux and Rossion, in press, Le Grand et al., 2001, Leder and Bruce, 2000, Rhodes et al., 1993). These observations are particularly important for experimental studies of face perception because a simple stimulus transformation such as inversion can be used to disrupt configural processes (e.g. Collishaw and Hole, 2000, Goffaux and Rossion, 2006, Young et al., 1987).
Inversion appears to affect perceptual encoding of faces, since large decreases of performance with inversion are observed during simultaneous presentation of unfamiliar faces, i.e. without any memory component involved (e.g. Farah et al., 1998, Moscovitch et al., 1997, Searcy and Bartlett, 1996). Moreover, the inversion effect is equally large whether the faces are presented simultaneously or with various delays (1, 5 or 10 s) during individual discrimination tasks (Freire et al., 2000). Recent neuroimaging studies support a perceptual encoding account of the face inversion effect since the processing of upright and inverted face stimuli differ mainly in visual areas responding preferentially to faces, such as the middle fusiform gyrus or ‘fusiform face area’ (‘FFA’, e.g. Haxby et al., 1999, Mazard et al., 2006, Yovel and Kanwisher, 2005). However, establishing the perceptual nature of the face inversion effect requires high temporal resolution methods such as event-related potentials (ERPs) recorded on the human scalp, which are able to track the time course of upright and inverted face processing at the millisecond (ms) range. Early ERP work by Jeffreys (1993), who focused on the so-called vertex-positive potential (VPP) peaking between 140 and 180 ms following face stimulation, showed that face inversion delays this electrophysiological component by about 10 ms. More recent ERP studies have reported the same delay on the N170, the negative counterpart of the VPP (Joyce and Rossion, 2005) at occipito-temporal sites (e.g. Bentin et al., 1996, de Haan et al., 2002, Eimer, 2000, Itier et al., 2006b, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998, Milivojevic et al., 2003, Rebai et al., 2001, Rossion et al., 1999, Rossion et al., 2000, Rossion et al., 2003, Rousselet et al., 2004, Sagiv and Bentin, 2001; for magnetoencephalography (MEG) evidence, see e.g. Itier et al., 2006a, Linkenkaer-Hansen et al., 1998). Paradoxically, most but not all of these studies have also found a significant increase of voltage amplitude of the N170 to inverted relative to upright faces (e.g. Itier et al., 2006b, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998, Rossion et al., 1999, Rossion et al., 2000, Rousselet et al., 2004, Sagiv and Bentin, 2001). A few studies have also disclosed latency and amplitude increases as starting earlier, at the level of the P1, a visual ERP component peaking at about 100 ms following stimulus onset and originating from striate and extrastriate visual areas (Itier et al., 2006a, Itier and Taylor, 2002, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998).
Despite the numerous studies that have described the effects of face inversion on early visual ERPs, the relationship between these early electrophysiological modulations and the behavioral effect of inversion is unclear. In particular, how does the P1/N170 latency and amplitude increases relate to the difficulty of encoding faces presented upside-down? The main goal of the present ERP study was to address this question. That is, to determine the exact time-course of the behavioral face inversion effect: when does the processing of upright and inverted faces start to differ reliably in the human brain, leading ultimately to an increase in error rates and response times during individual discrimination/recognition tasks? Clarifying the relationships between these early electrophysiological effects and behavior may be a critical step towards our understanding of the nature of the face inversion effect. If there were no relationship between early electrophysiological parameters and behavioral effects of inversion, this would seriously cast doubt on the perceptual encoding hypothesis. However, if there were systematic relationships between the electrophysiological parameters of early visual components and behavior, this would clearly demonstrate the perceptual encoding basis of the face inversion effect, and would strengthen the interest of these measures to understand the nature of this effect (i.e. which perceptual processes are affected by inversion).
To clarify the time course of the relationships between behavioral effects of face inversion and early face processes identified in human ERP studies, the present study differs from previous work in several ways. First, while most ERP studies on this topic have examined the effect of face inversion on the P1 and N170 components during tasks that are not associated with a behavioral inversion effect (i.e. orientation judgments, unrelated target detection or passive viewing) (e.g. Bentin et al., 1996, de Haan et al., 2002, Eimer, 2000, Linkenkaer-Hansen et al., 1998, Milivojevic et al., 2003, Rebai et al., 2001, Rossion et al., 2000, Rossion et al., 2003, Sagiv and Bentin, 2001), we measured ERP responses to faces during a delayed individual face matching (same/different) task. Second, and most importantly, we recorded ERPs in response to photographs of faces presented at 12 different orientations from 0° to 360° in 30° steps, rather than contrasting only upright and inverted faces as in previous studies. Presenting face stimuli at multiple orientations allowed us to characterize and correlate both the patterns of modulation of the P1 and N170 parameters and the pattern of behavioral responses, as a function of angles of face rotation. Two previous ERP studies have used multiple face orientations to characterize the time-course of face processes, but did not address the questions raised here. Jeffreys (1993) observed a linear increase of the latency of the VPP as a function of face orientation from 0° to 120°, and then slightly decreased again between 120° and 180°. There was no amplitude modulation of the VPP with the angle of face rotation due to face orientation reported in that study. Most importantly, there were no quantitative data analysis, and no behavioral data to compare with the ERP latency modulations. More recently, Milivojevic et al. (2003) recorded ERPs during a sex classification task of ‘Thatcherized faces’1 presented at six angular departures from the upright but only analyzed their data in terms of an interaction between “thatcherization” and orientation, and did not report main effects of face orientation. Third, and finally, besides the analysis of specific ERP components (P1, N170), we took advantage of the variability in the ERP and behavioral responses introduced by our parametric manipulation of face orientation to relate the two measures, and search for spatio-temporal regions where the pattern of ERP modulations paralleled behavioral performance. More precisely, to determine the time-course of the face inversion effect at a global level, we correlated behavioral data with ERP signal at each time point and at each scalp electrode. Similar approaches of correlating behavioral and neurophysiological responses to characterize the relationships between face perceptual processes and decision making have been recently applied to single neuron recordings in the monkey inferior–temporal cortex (Keysers et al., 2001) or single-trial EEG analysis in humans (Philiastides and Sajda, 2006).
According to the perceptual encoding view, the effect of face orientation measured behaviorally should be directly related to early ERP differences between face orientations. We hypothesized that systematic relationships between behavior and ERPs would emerge during the N170 time window, following the P1 component. This is because the N170 is the first component that shows a reliable larger response to faces than objects (e.g. Bentin et al., 1996, Rossion et al., 2000, Rousselet et al., 2004) and is thought to reflect a structural encoding stage (Bentin et al., 1996, Eimer, 2000). That is, a stage at which faces are not only discriminated from other object categories, but also at which individual representations of faces are activated (Jacques and Rossion, 2006), and which should thus be particularly sensitive to inversion (Rossion and Gauthier, 2002).
Section snippets
Subjects
Ten paid volunteers (10 males, 2 left-handed, mean age 21.8 ± 1.8 years) participated in this study. All subjects had normal or corrected-to-normal vision.
Stimuli
Thirty front view face pictures (15 males) without glasses, facial hair and make-up, and with neutral expression were used in this study. All face photographs were edited to remove backgrounds, hair, and everything below the chin, and were equated for mean luminance within the same face–gender. All stimuli subtended approximately 2.8° × 3.7° of
Behavioral results
The relationship between face orientation and behavioral responses is plotted in Fig. 2. The points plotted at 360° are duplicates of the 0° point and are included for clarity. Analyses revealed a significant effect of orientation for response times (F(3.8,34) = 13.8, p < 0.001), error rates (F(3.9,34.9) = 15.57, p < 0.001) and inverse efficiency (F(3.5,31.4) = 24.14, p < 0.001). The pattern of modulation with orientation was an increase in response times, error rates and inverse efficiency as the face was
Discussion
To summarize our findings, we observed significant effects of face orientation on both the P1 and N170, in line with previous observations. However, the patterns of modulation of behavioral responses with orientation were clearly different from the patterns observed for the P1 parameters (amplitude and latency), while they were remarkably similar to the patterns observed for the N170 parameters. Point-by-point correlation analyses performed over the entire scalp showed that the effect of face
Acknowledgments
The authors are supported by the Belgian National Fund for Scientific Research (FNRS). We are grateful to Quentin Duvivier for his help during data collection, and to three anonymous reviewers for their helpful comments on a previous version of the manuscript.
References (83)
- et al.
Relations between covert orienting and filtering in the development of visual attention
J. Exp. Child Psychol.
(1989) - et al.
The Thatcher illusion seen by the brain: An event-related brain potentials study
Cogn. Brain Res.
(2005) Effects of face inversion on the structural encoding and recognition of faces – evidence from event-related brain potentials
Cogn. Brain Res.
(2000)- et al.
Thresholding of statistical maps in functional neuroimaging using the false discovery rate
NeuroImage
(2002) - et al.
The effect of face inversion on activity in human neural systems for face and object perception
Neuron
(1999) - et al.
Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs
NeuroImage
(2002) - et al.
Effects of repetition learning on upright, inverted and contrast-reversed face processing using ERPs
NeuroImage
(2004) - et al.
Inversion and contrast – reversal effects on face processing assessed by MEG
Brain Res.
(2006) - et al.
Face, eye and object early processing: what is the face specificity?
NeuroImage
(2006) - et al.
The face-sensitive N170 and VPP components manifest the same brain processes: the effect of reference electrode site
Clin. Neurophysiol.
(2005)
Face-selective processing in human extrastriate cortex around 120 ms after stimulus onset revealed by magneto- and electroencephalography
Neurosci. Lett.
Shape representation in the inferior temporal cortex of monkeys
Curr. Biol.
Recovery from adaptation to facial identity is larger for upright than inverted faces in the human occipito-temporal cortex
Neuropsychologia
Specialized face processing and hemispheric asymmetry in man and monkey: evidence from single unit and reaction time studies
Behav. Brain Res.
Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations
Cognition
What’s lost in inverted faces?
Cognition
Spatio-temporal localization of the face inversion effect: an event-related potentials study
Biol. Psychol.
Early lateralization and orientation tuning for face, word, and object processing in the visual cortex
NeuroImage
The spatiotemporal dynamics of the face inversion effect: a magneto- and electro-encephalographic study
Neuroscience
The neural basis of the behavioral face-inversion effect
Curr. Biol.
Electrophysiological studies of human face perception: I. Potentials generated in occipitotemporal cortex by face and non-face stimuli
Cereb. Cortex
Discrimination of spatial relations and features in faces: effects of inversion and viewing duration
Br. J. Psychol.
Controlling the false discovery rate – a practical and powerful approach to multiple testing
J. R. Stat. Soc., B Met.
Electrophysiological studies of face perception in humans
J. Cogn. Neurosci.
Die-Prosop-agnosie
Arch. Psychiatr. Nervenkr.
Effect of disorientation on visual analysis, familiarity decision and semantic decision on faces
Br. J. Psychol.
Identification of early visual evoked potential generators by retinotopic and topographic analyses
Hum. Brain Mapp.
Featural and configurational processes in the recognition of faces of different familiarity
Perception
Is there a linear or a nonlinear relationship between rotation and configural processing of faces?
Perception
Specialization of neural mechanisms underlying face recognition in human infants
J. Cogn. Neurosci.
Cortical sources of the early components of the visual evoked potential
Hum. Brain Mapp.
What is “special” about face perception?
Psychol. Rev.
The face-inversion effect as a deficit in the encoding of configural information: direct evidence
Perception
Faces are “spatial” – holistic face perception is supported by low spatial frequencies
J. Exp. Psychol. Hum.
Sources or attention-sensitive visual event-related potentials
Brain Topogr.
Visual properties of neurons in inferotemporal cortex of the macaque
J. Neurophysiol.
Electrophysiological and haemodynamic correlates of face perception, recognition and priming
Cereb. Cortex
Source localization of early stages of face processing
Brain Topogr.
Recognition of faces: an explanatory study
Psychol. Sci.
The speed of individual face categorization
Psychol. Sci.
Cited by (134)
Face motion form at learning influences the time course of face spatial frequency processing during test
2023, Biological PsychologySocial and non-social working memory in neurodegeneration
2023, Neurobiology of Disease