The effects of inversion and eye displacements of familiar and unknown faces on early and late-stage ERPs

Abstract

Objectives

The objective of this study is to examine whether configural alterations of faces affect early or late processing stages as a function of their familiarity and their level of representation in memory. We then sought to verify whether the structural encoding stage is susceptible to top-down influences.

Methods

Electrophysiologic and behavioral studies were undertaken, during which unknown and familiar faces were presented upright or upside-down with or without feature alterations. The subjects were asked to determine whether the faces were familiar or not.

Results

N170 and N360 amplitudes were larger for familiar faces as well as altered ones. A higher degree of familiarity decreased reaction times (RTs) and N360 latencies, but increased N170 latencies, whereas face alterations increased RTs and latencies of both components examined. However, familiarity interacted with altered face configurations only for RTs and the N170.

Significance

In the perceptual stage, familiar faces seem to develop a more elaborate type of processing because of top-down influences linked to the robust nature of their representations in memory. The more elaborate type of processing for familiar faces has advantageous consequences for the following steps of information processing, by facilitating access to structural representations in memory (N360) as well as the final step reflected by RTs. The fact that configural alterations cause different effects for familiar as opposed to unfamiliar faces indicate that these stimuli are processed in a qualitatively different manner and solicit different representations in memory.

Introduction

Human beings have developed a remarkable capacity of acquiring expertise in the perception and recognition of faces. Despite similar configurations, hundreds of faces can readily be identified. A hierarchy exists in regard to the importance of facial features, the eyes being particularly important (Barton et al., 2001, Shepherd et al., 1981).

It has been proposed that the ability to identify faces is more the result of the analysis of spatial relations existing between facial features than individual features per se (Maurer et al., 2002). Different terms have been used to describe this type of treatment. The terms “configural” and “relational” are used to designate the spatial relations existing between facial features and the term “holistic” to emphasize the processing of all types of information contained in a face. The face may be represented as unparsed gestalts or templates, in which features and their spatial relations are not separable sources of information (Farah et al., 1995, Tanaka and Farah, 1993). Other types of information appear to be just as important for face processing, such as differences in texture and relative differences in light–dark areas (Hancock et al., 1996, Calder et al., 2001).

The importance of configural information in face recognition is supported by findings of its impairment after inversion (Rhodes et al., 1993), in a more severe manner than for objects (Yin, 1969). Face recognition appears to depend on configural processing, as opposed to the analytic trait-by-trait processing involved in object recognition (Maurer et al., 2002, Moscovitch et al., 1997, Rossion and Gauthier, 2002). Moreover, each is represented in partially segregated neocortical regions (Kanwisher and Moscovitch, 2000), as indicated by neuropsychologic studies of agnosic patients (Damasio et al., 1990, de Renzi et al., 1991, Farah et al., 1995) and neuroimaging findings in normal subjects (Gauthier et al., 2000, Haxby et al., 2000, Kanwisher et al., 1997).

Since the work of Yin (1969), inversion effects have been a mainstay in various experimental paradigms. Even minimal changes in the space between facial features are easily detectable with upright but not with inverted faces (Kemp et al., 1990, Leder et al., 2001, Rhodes et al., 1993). Inverted faces are processed on the basis of individual features, because trait alterations do not affect this type of processing, as exemplified by the “Thatcher effect” (Thompson, 1980), whereby face inversion reduced the subjective impression of “monstrosity” caused by eye–mouth inversions in the upright position.

Electrophysiologic recordings indicate that face inversion augments latencies and amplitudes of the N170 component (Eimer, 2000, Itier and Taylor, 2002, Itier and Taylor, 2004, Rebaï et al., 2001, Rossion et al., 1999, Rossion et al., 2003). The same component was also affected by configural modifications, such as displaced (George et al., 1996), masked (Jemel et al., 1999), and isolated (Taylor et al., 2001) internal features. Because this component does not seem to be modulated by familiarity (Bentin and Deouell, 2000, Eimer, 2000, Schweinberger et al., 2002) and repetition (Schweinberger et al., 1995, Schweinberger et al., 2002), it may constitute the electrophysiologic manifestation of the structural encoding stage, as described by Bruce and Young (1986). However, it has recently been proposed that this wave is not exclusively perceptual and automated, but instead reflects numerous sub-stages of brain activity (Rossion et al., 2003), being influenced by category types (Rebaï et al., 2001, Rossion et al., 2003), types of task (Goffaux et al., 2003), attentional processes (Eimer, 2000), level of expertise attained by the subjects for different object categories (Tanaka and Curran, 2001), level of familiarity of faces (Caharel et al., 2002, Caharel et al., 2005, Jemel et al., 2003), and top-down processes (Goffaux et al., 2003, Jemel et al., 2003). However, in the studies by Caharel et al., 2002, Caharel et al., 2005, the familiarity effect on the N170 is manifested by larger amplitudes for familiar as opposed to unfamiliar faces, contrary to the findings of Jemel et al. (2003). These differences are likely due to methodologic factors, as Caharel et al. (2002) used passive viewing, Caharel et al. (2005) familiarity/emotional expression identification tasks, and Jemel et al. (2003) a priming task, whereby familiarity effect appears with successive presentations of the same familiar face.

Such results seem to indicate that the structural encoding stage as reflected by the N170 component is not a perceptual stage closed to experience, but rather permits discriminations between objects of expertise and others (Rossion et al., 2004, Tanaka and Curran, 2001) as well as faces based on their level of familiarity (Caharel et al., 2002, Caharel et al., 2005, Jemel et al., 2003). This processing stage may be the result of inter-categorical (object vs face) as well as intra-categorical (individualization of a specific face) discriminations (Rossion and Gauthier, 2002).

Subsequent negative components of face processing have been characterized. On one hand, the N250r, demonstrated in numerous priming paradigms, appears sensitive to the level of familiarity of faces and may be associated with the activation of stored structural representations of familiar faces (Herzmann et al., 2004, Martin-Loeches et al., 2004, Schweinberger et al., 2002, Schweinberger et al., 2004). On the other, during feature matching of famous faces and those devoid of verbal/semantic information, incongruity processing effects were described for N350 (Jemel et al., 1999) and N360 (Olivares et al., 2003) components, respectively. These components were observed in postero-temporal regions and identified by these authors as a N400-type component. In addition, a negative component in the 300–500 ms temporal window (N400) at centro-parietal sites was sensitive to the familiarity of faces, perhaps due to retrieval of semantic information (Bentin and Deouell, 2000, Eimer, 2000, Schweinberger, 1996).

The spatial relations existing between facial feature are considered to be of paramount importance for the structural encoding stage, which in subsequent steps permits optimal encoding of memory retrieval and access to semantic information. The orientation of the face appears to be crucial for optimal configural processing. According to Rossion and Gauthier (2002), the use of configural processing is the result of visual experiences for certain stimuli oriented in a specific way and that are perturbed by the inversion effect. The expertise gained by this category may be further developed by life experiences, enabling certain faces to become more often seen and thereby more familiar than others. This experience or learning may lead to even more specialized configural processing, permitting the individualization and identification of familiar faces. Thus, we may hypothesize that if faces imply configural processing that change over time, then configural modifications should be all the more important with respect to familiar faces. However, Tong and Nakayama (1999) suggest that very familiar faces should be more robustly represented than less familiar ones, and therefore less susceptible to the inversion effect.

In order to examine these postulates, we found out whether configural modifications alter ERPs depending on their level of familiarity and thereby their representation in memory. These effects were analyzed in successive stages of processing. More specifically, we sought to verify the Bruce and Young (1986) model, which states that familiar faces are processed in a qualitatively different fashion than unknown ones. Two levels of familiarity were used (personally important or totally unknown) for upright and inverted faces and with intact or displaced eyes (single or double) in a task asking subjects to identify familiarity. Cooper and Wojan (2000) showed that face recognition is more difficult with two displaced eyes than with a single one, explainable by the vulnerability of face recognition to changes in configural information as opposed to trait displacement as such. Our hypothesis is that the effects of the configural modification of faces seen at the N170 structural encoding stage have repercussions on subsequent ones and that these effects differ depending on the level of familiarity of faces. This would be due to the fact that configural alterations may perturb top-down influences on the formation of structural representations affecting the familiarity effect and subsequent stages of processing, in particular the one reflected by the N400.