Elsevier

NeuroImage

Volume 36, Issue 3, 1 July 2007, Pages 863-876
NeuroImage

Early electrophysiological responses to multiple face orientations correlate with individual discrimination performance in humans

https://doi.org/10.1016/j.neuroimage.2007.04.016Get rights and content

Abstract

Picture-plane inversion dramatically impairs face recognition. Behavioral and event-related potential (ERP) studies suggest that this effect takes place during perceptual encoding of the face stimulus. However, the relationship between early electrophysiological responses to upright and inverted faces and the behavioral face inversion effect remains unclear. To address this question, we recorded ERPs while presenting 10 subjects with face photographs at 12 different orientations around the clock (30° steps) during an individual face matching task. Using the variability in the electrophysiological responses introduced by the multiple orientations of the target face, we found a correlation between the ERP signal at 130–170 ms on occipito-temporal channels, and the behavioral performance measured on the probe stimulus. Correlations between ERP signal and behavioral performance started about 10 ms earlier in the right hemisphere. Significant effects of orientation were observed already at the level of the visual P1 (peaking at 100 ms), but the ERP signal was not correlated with behavior until the face-sensitive N170 time window. Overall, these observations indicate that plane-inversion affects the perceptual encoding of faces as early as 130 ms in occipito-temporal regions, leading directly to an increase in error rates and RTs during individual face recognition.

Introduction

Presenting face stimuli upside-down (either flipped vertically or rotated by 180°) dramatically affects their recognition (e.g. Hochberg and Galper, 1967). This observation has become particularly important in the face processing literature, for several reasons. First, the effect of inversion for faces is much larger than for other object categories, a phenomenon known as the ‘face inversion effect’ (Yin, 1969). This effect is one of the first cited evidence in favor of specific brain mechanisms for faces (i.e. those that are particularly affected by inversion, Yin, 1969), together with the observation of recognition impairments specific for faces following brain damage (‘prosopagnosia’, Bodamer, 1947) and of inferior–temporal cortex neurons responding selectively to faces in the monkey brain (Gross et al., 1972). Second, the effect is extremely robust, and has been observed in a variety of conditions: for familiar and unfamiliar faces, in old/new recognition tasks or matching tasks, with or without delay between stimuli to match (for a recent review, see Rossion and Gauthier, 2002). Finally, a large number of behavioral studies support the view that inversion of a face affects mainly, but not exclusively, the processing of facial configuration. For instance, when faces are inverted the recognition of facial features is no longer (or less) affected by the presence of other features, or by the whole face stimulus (e.g. Sergent, 1984, Tanaka and Farah, 1993, Young et al., 1987). Moreover, most studies show that the perception of metric distances between features (i.e. mouth–nose distance, eyes height within the face stimulus…) is more affected by inversion than the perception of local cues (shape of the mouth, color of the eyes…) (e.g. Barton et al., 2001, Collishaw and Hole, 2000, Freire et al., 2000, Goffaux and Rossion, in press, Le Grand et al., 2001, Leder and Bruce, 2000, Rhodes et al., 1993). These observations are particularly important for experimental studies of face perception because a simple stimulus transformation such as inversion can be used to disrupt configural processes (e.g. Collishaw and Hole, 2000, Goffaux and Rossion, 2006, Young et al., 1987).

Inversion appears to affect perceptual encoding of faces, since large decreases of performance with inversion are observed during simultaneous presentation of unfamiliar faces, i.e. without any memory component involved (e.g. Farah et al., 1998, Moscovitch et al., 1997, Searcy and Bartlett, 1996). Moreover, the inversion effect is equally large whether the faces are presented simultaneously or with various delays (1, 5 or 10 s) during individual discrimination tasks (Freire et al., 2000). Recent neuroimaging studies support a perceptual encoding account of the face inversion effect since the processing of upright and inverted face stimuli differ mainly in visual areas responding preferentially to faces, such as the middle fusiform gyrus or ‘fusiform face area’ (‘FFA’, e.g. Haxby et al., 1999, Mazard et al., 2006, Yovel and Kanwisher, 2005). However, establishing the perceptual nature of the face inversion effect requires high temporal resolution methods such as event-related potentials (ERPs) recorded on the human scalp, which are able to track the time course of upright and inverted face processing at the millisecond (ms) range. Early ERP work by Jeffreys (1993), who focused on the so-called vertex-positive potential (VPP) peaking between 140 and 180 ms following face stimulation, showed that face inversion delays this electrophysiological component by about 10 ms. More recent ERP studies have reported the same delay on the N170, the negative counterpart of the VPP (Joyce and Rossion, 2005) at occipito-temporal sites (e.g. Bentin et al., 1996, de Haan et al., 2002, Eimer, 2000, Itier et al., 2006b, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998, Milivojevic et al., 2003, Rebai et al., 2001, Rossion et al., 1999, Rossion et al., 2000, Rossion et al., 2003, Rousselet et al., 2004, Sagiv and Bentin, 2001; for magnetoencephalography (MEG) evidence, see e.g. Itier et al., 2006a, Linkenkaer-Hansen et al., 1998). Paradoxically, most but not all of these studies have also found a significant increase of voltage amplitude of the N170 to inverted relative to upright faces (e.g. Itier et al., 2006b, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998, Rossion et al., 1999, Rossion et al., 2000, Rousselet et al., 2004, Sagiv and Bentin, 2001). A few studies have also disclosed latency and amplitude increases as starting earlier, at the level of the P1, a visual ERP component peaking at about 100 ms following stimulus onset and originating from striate and extrastriate visual areas (Itier et al., 2006a, Itier and Taylor, 2002, Itier and Taylor, 2004, Linkenkaer-Hansen et al., 1998).

Despite the numerous studies that have described the effects of face inversion on early visual ERPs, the relationship between these early electrophysiological modulations and the behavioral effect of inversion is unclear. In particular, how does the P1/N170 latency and amplitude increases relate to the difficulty of encoding faces presented upside-down? The main goal of the present ERP study was to address this question. That is, to determine the exact time-course of the behavioral face inversion effect: when does the processing of upright and inverted faces start to differ reliably in the human brain, leading ultimately to an increase in error rates and response times during individual discrimination/recognition tasks? Clarifying the relationships between these early electrophysiological effects and behavior may be a critical step towards our understanding of the nature of the face inversion effect. If there were no relationship between early electrophysiological parameters and behavioral effects of inversion, this would seriously cast doubt on the perceptual encoding hypothesis. However, if there were systematic relationships between the electrophysiological parameters of early visual components and behavior, this would clearly demonstrate the perceptual encoding basis of the face inversion effect, and would strengthen the interest of these measures to understand the nature of this effect (i.e. which perceptual processes are affected by inversion).

To clarify the time course of the relationships between behavioral effects of face inversion and early face processes identified in human ERP studies, the present study differs from previous work in several ways. First, while most ERP studies on this topic have examined the effect of face inversion on the P1 and N170 components during tasks that are not associated with a behavioral inversion effect (i.e. orientation judgments, unrelated target detection or passive viewing) (e.g. Bentin et al., 1996, de Haan et al., 2002, Eimer, 2000, Linkenkaer-Hansen et al., 1998, Milivojevic et al., 2003, Rebai et al., 2001, Rossion et al., 2000, Rossion et al., 2003, Sagiv and Bentin, 2001), we measured ERP responses to faces during a delayed individual face matching (same/different) task. Second, and most importantly, we recorded ERPs in response to photographs of faces presented at 12 different orientations from 0° to 360° in 30° steps, rather than contrasting only upright and inverted faces as in previous studies. Presenting face stimuli at multiple orientations allowed us to characterize and correlate both the patterns of modulation of the P1 and N170 parameters and the pattern of behavioral responses, as a function of angles of face rotation. Two previous ERP studies have used multiple face orientations to characterize the time-course of face processes, but did not address the questions raised here. Jeffreys (1993) observed a linear increase of the latency of the VPP as a function of face orientation from 0° to 120°, and then slightly decreased again between 120° and 180°. There was no amplitude modulation of the VPP with the angle of face rotation due to face orientation reported in that study. Most importantly, there were no quantitative data analysis, and no behavioral data to compare with the ERP latency modulations. More recently, Milivojevic et al. (2003) recorded ERPs during a sex classification task of ‘Thatcherized faces’1 presented at six angular departures from the upright but only analyzed their data in terms of an interaction between “thatcherization” and orientation, and did not report main effects of face orientation. Third, and finally, besides the analysis of specific ERP components (P1, N170), we took advantage of the variability in the ERP and behavioral responses introduced by our parametric manipulation of face orientation to relate the two measures, and search for spatio-temporal regions where the pattern of ERP modulations paralleled behavioral performance. More precisely, to determine the time-course of the face inversion effect at a global level, we correlated behavioral data with ERP signal at each time point and at each scalp electrode. Similar approaches of correlating behavioral and neurophysiological responses to characterize the relationships between face perceptual processes and decision making have been recently applied to single neuron recordings in the monkey inferior–temporal cortex (Keysers et al., 2001) or single-trial EEG analysis in humans (Philiastides and Sajda, 2006).

According to the perceptual encoding view, the effect of face orientation measured behaviorally should be directly related to early ERP differences between face orientations. We hypothesized that systematic relationships between behavior and ERPs would emerge during the N170 time window, following the P1 component. This is because the N170 is the first component that shows a reliable larger response to faces than objects (e.g. Bentin et al., 1996, Rossion et al., 2000, Rousselet et al., 2004) and is thought to reflect a structural encoding stage (Bentin et al., 1996, Eimer, 2000). That is, a stage at which faces are not only discriminated from other object categories, but also at which individual representations of faces are activated (Jacques and Rossion, 2006), and which should thus be particularly sensitive to inversion (Rossion and Gauthier, 2002).

Section snippets

Subjects

Ten paid volunteers (10 males, 2 left-handed, mean age 21.8 ± 1.8 years) participated in this study. All subjects had normal or corrected-to-normal vision.

Stimuli

Thirty front view face pictures (15 males) without glasses, facial hair and make-up, and with neutral expression were used in this study. All face photographs were edited to remove backgrounds, hair, and everything below the chin, and were equated for mean luminance within the same face–gender. All stimuli subtended approximately 2.8° × 3.7° of

Behavioral results

The relationship between face orientation and behavioral responses is plotted in Fig. 2. The points plotted at 360° are duplicates of the 0° point and are included for clarity. Analyses revealed a significant effect of orientation for response times (F(3.8,34) = 13.8, p < 0.001), error rates (F(3.9,34.9) = 15.57, p < 0.001) and inverse efficiency (F(3.5,31.4) = 24.14, p < 0.001). The pattern of modulation with orientation was an increase in response times, error rates and inverse efficiency as the face was

Discussion

To summarize our findings, we observed significant effects of face orientation on both the P1 and N170, in line with previous observations. However, the patterns of modulation of behavioral responses with orientation were clearly different from the patterns observed for the P1 parameters (amplitude and latency), while they were remarkably similar to the patterns observed for the N170 parameters. Point-by-point correlation analyses performed over the entire scalp showed that the effect of face

Acknowledgments

The authors are supported by the Belgian National Fund for Scientific Research (FNRS). We are grateful to Quentin Duvivier for his help during data collection, and to three anonymous reviewers for their helpful comments on a previous version of the manuscript.

References (83)

  • K. Linkenkaer-Hansen et al.

    Face-selective processing in human extrastriate cortex around 120 ms after stimulus onset revealed by magneto- and electroencephalography

    Neurosci. Lett.

    (1998)
  • N.K. Logothetis et al.

    Shape representation in the inferior temporal cortex of monkeys

    Curr. Biol.

    (1995)
  • A. Mazard et al.

    Recovery from adaptation to facial identity is larger for upright than inverted faces in the human occipito-temporal cortex

    Neuropsychologia

    (2006)
  • D.I. Perrett et al.

    Specialized face processing and hemispheric asymmetry in man and monkey: evidence from single unit and reaction time studies

    Behav. Brain Res.

    (1988)
  • D.I. Perrett et al.

    Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations

    Cognition

    (1998)
  • G. Rhodes et al.

    What’s lost in inverted faces?

    Cognition

    (1993)
  • B. Rossion et al.

    Spatio-temporal localization of the face inversion effect: an event-related potentials study

    Biol. Psychol.

    (1999)
  • B. Rossion et al.

    Early lateralization and orientation tuning for face, word, and object processing in the visual cortex

    NeuroImage

    (2003)
  • S. Watanabe et al.

    The spatiotemporal dynamics of the face inversion effect: a magneto- and electro-encephalographic study

    Neuroscience

    (2003)
  • G. Yovel et al.

    The neural basis of the behavioral face-inversion effect

    Curr. Biol.

    (2005)
  • T. Allison et al.

    Electrophysiological studies of human face perception: I. Potentials generated in occipitotemporal cortex by face and non-face stimuli

    Cereb. Cortex

    (1999)
  • J.J. Barton et al.

    Discrimination of spatial relations and features in faces: effects of inversion and viewing duration

    Br. J. Psychol.

    (2001)
  • Y. Benjamini et al.

    Controlling the false discovery rate – a practical and powerful approach to multiple testing

    J. R. Stat. Soc., B Met.

    (1995)
  • S. Bentin et al.

    Electrophysiological studies of face perception in humans

    J. Cogn. Neurosci.

    (1996)
  • J. Bodamer

    Die-Prosop-agnosie

    Arch. Psychiatr. Nervenkr.

    (1947)
  • R. Bruyer et al.

    Effect of disorientation on visual analysis, familiarity decision and semantic decision on faces

    Br. J. Psychol.

    (1993)
  • V.P. Clark et al.

    Identification of early visual evoked potential generators by retinotopic and topographic analyses

    Hum. Brain Mapp.

    (1995)
  • S.M. Collishaw et al.

    Featural and configurational processes in the recognition of faces of different familiarity

    Perception

    (2000)
  • S.M. Collishaw et al.

    Is there a linear or a nonlinear relationship between rotation and configural processing of faces?

    Perception

    (2002)
  • M. de Haan et al.

    Specialization of neural mechanisms underlying face recognition in human infants

    J. Cogn. Neurosci.

    (2002)
  • F. Di Russo et al.

    Cortical sources of the early components of the visual evoked potential

    Hum. Brain Mapp.

    (2002)
  • M.J. Farah et al.

    What is “special” about face perception?

    Psychol. Rev.

    (1998)
  • A. Freire et al.

    The face-inversion effect as a deficit in the encoding of configural information: direct evidence

    Perception

    (2000)
  • V. Goffaux et al.

    Faces are “spatial” – holistic face perception is supported by low spatial frequencies

    J. Exp. Psychol. Hum.

    (2006)
  • Goffaux, V., Rossion, B., in press. Face inversion disproportionately impairs the perception of vertical but not...
  • C.M. Gomez Gonzalez et al.

    Sources or attention-sensitive visual event-related potentials

    Brain Topogr.

    (1994)
  • C.G. Gross et al.

    Visual properties of neurons in inferotemporal cortex of the macaque

    J. Neurophysiol.

    (1972)
  • R.N. Henson et al.

    Electrophysiological and haemodynamic correlates of face perception, recognition and priming

    Cereb. Cortex

    (2003)
  • M.J. Herrmann et al.

    Source localization of early stages of face processing

    Brain Topogr.

    (2005)
  • J. Hochberg et al.

    Recognition of faces: an explanatory study

    Psychol. Sci.

    (1967)
  • C. Jacques et al.

    The speed of individual face categorization

    Psychol. Sci.

    (2006)
  • Cited by (134)

    View all citing articles on Scopus
    View full text