Full Length ArticlesEarly sensitivity for eyes within faces: A new neuronal account of holistic and featural processing
Introduction
The eyes play an important role in face perception. Eye tracking studies suggest that regardless of task and initial fixation position, participants tend to fixate on or close to the eye region (e.g. Arizpe et al., 2012, Barton et al., 2006, Janik et al., 1978). Eyes are also the most attended feature regardless of face familiarity (Heisz and Shore, 2008) and this preference is seen in infants (Maurer, 1985), with a clear sensitivity to gaze direction already present at birth (Batki et al., 2000). The eyes seem to be the diagnostic feature used to recognize identity, several facial expressions, and gender (Dupuis-Roy et al., 2009, Schyns et al., 2007). Better expertise in face processing seems to be driven by better information extraction from the eye region (Vinette et al., 2004), a capacity that might go awry in some cases of prosopagnosia in which the eye region is not properly attended (Caldara et al., 2005). Eyes provide essential cues to others' attention and intention through gaze perception, putting them at the core of social cognition and its impairments as seen in Autism Spectrum Disorder (Itier and Batty, 2009 for a review). While electrophysiological studies also point toward a special status for eyes presented in isolation (Bentin et al., 1996, Itier et al., 2006, Itier et al., 2007), their role in the earliest stages of face encoding is unclear and the featural versus holistic nature of early face perception has puzzled cognitive neuroscientists for nearly two decades. The present paper provides new evidence supporting a particular sensitivity to eyes even in the context of the whole face and their role in early face encoding stages. These findings have important implications for our understanding of face perception and how it breaks down in various disorders, as well as more generally for visual perception. A new neuronal model is proposed that accounts for all the N170 modulations reported, for the holistic processing of upright faces and for the featural processing of inverted faces.
Scalp electrophysiological studies have identified a now well-known ERP component called the N170, the earliest reliable face-sensitive component occurring between 130 and 200 ms over occipito-temporal sites (Bentin et al., 1996). The N170 has been proposed to reflect early perceptual face encoding stages (Eimer, 2000, Sagiv and Bentin, 2001) where features are “glued” together in a holistic facial percept. However, the N170 is even larger for eye regions presented in isolation than for upright faces (Bentin et al., 1996, Itier et al., 2006, Itier et al., 2007) and this eye sensitivity is seen as early as 4 years of age (Taylor et al., 2001a). This featural sensitivity does not include other facial features as nose and mouth elicit delayed and smaller N170s than faces (Bentin et al., 1996, Nemrodov and Itier, 2011, Taylor et al., 2001b). However, eyeless faces elicit N170s with amplitudes similar to normal intact faces, albeit delayed (Eimer, 1998, Itier et al., 2007, Itier et al., 2011, Kloth et al., 2013) and this finding has been interpreted as supporting the view that the N170 reflects a holistic processing stage rather than an eye detector.
The idea that the N170 reflects holistic face processing is also supported by the inversion manipulation, which is known to disrupt face perception and recognition more so than perception and recognition of objects (Yin, 1969). Numerous behavioral studies have shown that objects are processed mostly in a piecemeal way while faces are perceived mostly holistically (e.g. Tanaka and Farah, 1993, Tanaka and Gordon, 2011) and that inversion disrupts this holistic processing (Rossion, 2009). In ERP studies, faces presented upside down trigger delayed but most importantly larger N170s compared to upright faces (Bentin et al., 1996, Itier and Taylor, 2002, Rossion et al., 1999), while upside down objects usually elicit only delayed responses (Itier et al., 2006, Kloth et al., 2013). Similar to objects, animal faces or impoverished human face stimuli, such as sketches or Mooney faces, also show delayed N170 with inversion but no increase in amplitude and sometimes even a slight amplitude reduction (de Haan et al., 2002, Itier et al., 2006, Itier et al., 2011, Latinus and Taylor, 2005, Sagiv and Bentin, 2001, Wiese et al., 2009). The amplitude increase with inversion, also termed the N170 “face inversion effect” (FIE), is thus believed to reflect the disruption of early holistic processing stages specific to human faces and has been used as a hallmark of face specificity. At the neuronal level, this increase has been explained by the recruitment, in addition to face-sensitive neurons, of object-sensitive neurons (Itier and Taylor, 2002, Rossion et al., 1999, Sadeh and Yovel, 2010, Yovel and Kanwisher, 2005), other face-sensitive neurons tuned to the inverted orientation (Eimer et al., 2010), or eye-sensitive neurons (Itier et al., 2007).
Itier et al. (2007) showed that, in contrast to intact faces, inversion of eyeless faces elicited a much reduced N170 FIE and thus proposed that eyes played an important role in this early face specific phenomenon, an idea reinforced by the replication of this finding in later studies (Itier et al., 2011, Kloth et al., 2013, Nemrodov and Itier, 2011). A model involving eye- and face-sensitive neuronal populations (Itier and Batty, 2009, Itier et al., 2007) tried to account for the N170 FIE as well as the larger N170 response to isolated eyes that is seen regardless of eye orientation (Itier et al., 2006, Itier et al., 2007, Itier et al., 2011). According to this model, upright intact faces trigger the activation of face-sensitive neurons which would inhibit eye-sensitive neurons in the context of a configurally correct face, a mechanism that accounts for the lack of amplitude change with upright eyeless faces. Presenting faces upside-down, however, would stop this inhibition, due to the disruption of holistic processing, and the N170-FIE would then reflect the co-activation of both neuronal populations. Both neuronal populations would also respond to isolated eyes, explaining the larger N170 amplitude for eyes than for upright faces.
ERP studies usually use a centrally presented cross to control for fixation position. In most face studies, this central fixation is situated close to the eyes, on the nasion or on the nose. In particular, the studies that reported larger N170 amplitude for isolated eyes or a lack of FIE for inverted eyeless faces presented the fixation on the nasion area (Bentin et al., 1996, Itier et al., 2007, Itier et al., 2011, Kloth et al., 2013, Taylor et al., 2001b). However, recent studies have reported changes in N170 amplitude as a function of fixation location on the face. Fixation on the nasion and mouth yielded larger N170 amplitudes than fixation on the nose in one study (McPartland et al., 2010), although eye-tracking was not employed to ensure participants were indeed fixating at these fixation locations during face presentation. Another study using a gaze contingent paradigm with eye tracking reported largest N170 response for fixation on the nasion when faces were presented upright and for fixation on the mouth when faces were presented inverted (Zerouali et al., 2013). This study suggested that the encoding of a face as indexed by the N170 arises from a general upper-visual field advantage rather than from sensitivity to eyes, whether the face is upright or inverted. These results raise the concern that what was taken as evidence for a specific role of the eyes in the FIE (Itier et al., 2007) might simply reflect an artifact of gaze position rather than a true eye sensitivity.
To address this concern and to probe further the role of eyes in early face encoding, we investigated whether fixation on various facial features modulated the N170 response and the N170 FIE. Crucially, in addition to intact faces we also tested eyeless faces. This new condition allowed us to confirm the potential sensitivity to eyes (or lack thereof) and to test the hypothesis that eyes are important in driving the N170 FIE (Itier et al., 2007). Intact and eyeless faces were presented upright and inverted with fixation locations on the middle of the forehead, nasion, left eye, right eye, tip of the nose, and mouth. To ensure a correct point of gaze, eye tracking was used with a fixation-contingent stimulus presentation and any trial in which gaze deviated by more than 1.8º of visual angle around that fixation location was excluded (Fig. 1). In addition, to prevent participants from using anticipatory strategies the fixation cross was always presented in the center of the screen, while faces were moved around it to obtain the desired fixation position (Fig. 1). House stimuli were used as a control category to ensure that our face stimuli elicited a proper face sensitive N170.
If Zerouali et al. (2013) were correct, we expected to find an interaction between face orientation and fixation location such that the largest N170 responses would be seen for nasion fixation with upright faces and the largest N170 responses would be seen for mouth fixation with inverted faces, reproducing the upper–lower visual field effect. This should also be seen regardless of whether eyes are present or absent from the face. Contrary to this expectation, we found that the N170 was largest when fixation was on the eyes compared to the other fixations and this was seen in both upright and inverted faces, ruling out a simple upper versus lower visual field effect and supporting a special sensitivity to eyes. Crucially, this eye sensitivity disappeared in eyeless faces, demonstrating that it was due to the presence of eyes at fovea. In other words, the sensitivity for eyes was present beyond the classic N170 FIE. However, for eyeless faces, the inversion effect interacted with fixation location such that when fixation was on the mouth, a normal inversion effect was seen while the FIE was maximally reduced when fixation was around the eyes, as reported before (Itier et al., 2007). These findings suggest that eyes do play a role in the inversion effect but only when they are in fovea. We propose a new mechanism to explain this set of data and the N170 FIE. We discuss these findings and their implication for understanding early face perception and in particular holistic versus featural processing.
Section snippets
Participants
Forty-one undergraduate students from the University of Waterloo (UW) were tested and received course-credit for their participation. They all reported normal or corrected-to-normal vision, no history of head-injury or neurological disease, and were not taking any medication. They all signed informed written consent and the study was approved by the Research Ethics Board at UW. Twenty-one participants were rejected, 10 for not completing the experiment and thus registering too few trials per
Behavioral results
Accuracy on this orientation detection task was overall high. Participants responded more accurately to faces than to houses (93.8% and 91.5%, respectively; effect of category, F(1,20) = 5.73, p = 0.027, ηp2 = 0.223). The category by orientation interaction (F(1,20) = 12.4, p < 0.005) was due to an effect of inversion for faces, with higher accuracy for upright than inverted faces (95% vs 92%, F = 10.5, p < 0.005) while no inversion effect was found for houses (F = 1.8, p = 0.195).
Responses were faster to faces
Discussion
In the present study we explored the effect of fixation location on the N170 ERP component, the earliest neural marker of face perception, to test the idea of an eye detector and of its possible role in driving the N170 face inversion effect. Using eye tracking to enforce correct fixation on the desired facial part we found that fixation location clearly affected the N170 amplitude and interfered with the face inversion effect when eyes were removed. We discuss below these effects in turn and
Conclusion
The N170 amplitude was largest when fixation was on the eyes compared to any other facial locations and this effect was seen regardless of face orientation and was lost when eyes were removed. These results suggest a special sensitivity of this ERP component to eye features and support the eye detector theory. Importantly, all the modulations of the N170 can be explained by lateral inhibitions of the neurons coding for the fixated feature in fovea by the neurons coding for features situated in
Acknowledgments
This study was supported by grants from the Natural Sciences and Engineering Research Council of Canada (NSERC Discovery Grant #418431), the Canada Foundation for Innovation (CFI), the Ontario Research Fund (ORF) and the Canada Research Chair program to RJI.
References (56)
- et al.
Is there an innate gaze module? Evidence from human neonates
Infant Behav. Dev.
(2000) - et al.
Acquired prosopagnosia abolishes the face inversion effect
Cortex
(2010) - et al.
EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
J. Neurosci. Methods
(2004) Effects of face inversion on the structural encoding and recognition of faces: evidence from event-related brain potentials
Cogn. Brain Res.
(2000)- et al.
Neural bases of eye and gaze processing: the core of social cognition
Neurosci. Biobehav. Rev.
(2009) - et al.
Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs
NeuroImage
(2002) - et al.
Face, eye and object early processing: what is the face specificity?
NeuroImage
(2006) - et al.
Species sensitivity of early face and eye processing
NeuroImage
(2011) - et al.
Combined effects of inversion and feature removal on N170 responses elicited by faces and car fronts
Brain Cogn.
(2013) - et al.
The many faces of configural processing
Trends Cogn. Sci.
(2002)