Elsevier

NeuroImage

Volume 97, 15 August 2014, Pages 81-94
NeuroImage

Full Length Articles
Early sensitivity for eyes within faces: A new neuronal account of holistic and featural processing

https://doi.org/10.1016/j.neuroimage.2014.04.042Get rights and content

Highlights

  • N170 is largest when fixation is on the eyes compared to other face locations.

  • This eye sensitivity is independent of orientation and disappears in eyeless faces.

  • The N170 face inversion effect varies with fixation location in eyeless faces.

  • We propose the Lateral Inhibition, Face Template and Eye Detector based model.

  • The LIFTED model accounts for N170 modulations, holistic and featural processing.

Abstract

Eyes are central to face processing however their role in early face encoding as reflected by the N170 ERP component is unclear. Using eye tracking to enforce fixation on specific facial features, we found that the N170 was larger for fixation on the eyes compared to fixation on the forehead, nasion, nose or mouth, which all yielded similar amplitudes. This eye sensitivity was seen in both upright and inverted faces and was lost in eyeless faces, demonstrating it was due to the presence of eyes at fovea. Upright eyeless faces elicited largest N170 at nose fixation. Importantly, the N170 face inversion effect (FIE) was strongly attenuated in eyeless faces when fixation was on the eyes but was less attenuated for nose fixation and was normal when fixation was on the mouth. These results suggest the impact of eye removal on the N170 FIE is a function of the angular distance between the fixated feature and the eye location. We propose the Lateral Inhibition, Face Template and Eye Detector based (LIFTED) model which accounts for all the present N170 results including the FIE and its interaction with eye removal. Although eyes elicit the largest N170 response, reflecting the activity of an eye detector, the processing of upright faces is holistic and entails an inhibitory mechanism from neurons coding parafoveal information onto neurons coding foveal information. The LIFTED model provides a neuronal account of holistic and featural processing involved in upright and inverted faces and offers precise predictions for further testing.

Introduction

The eyes play an important role in face perception. Eye tracking studies suggest that regardless of task and initial fixation position, participants tend to fixate on or close to the eye region (e.g. Arizpe et al., 2012, Barton et al., 2006, Janik et al., 1978). Eyes are also the most attended feature regardless of face familiarity (Heisz and Shore, 2008) and this preference is seen in infants (Maurer, 1985), with a clear sensitivity to gaze direction already present at birth (Batki et al., 2000). The eyes seem to be the diagnostic feature used to recognize identity, several facial expressions, and gender (Dupuis-Roy et al., 2009, Schyns et al., 2007). Better expertise in face processing seems to be driven by better information extraction from the eye region (Vinette et al., 2004), a capacity that might go awry in some cases of prosopagnosia in which the eye region is not properly attended (Caldara et al., 2005). Eyes provide essential cues to others' attention and intention through gaze perception, putting them at the core of social cognition and its impairments as seen in Autism Spectrum Disorder (Itier and Batty, 2009 for a review). While electrophysiological studies also point toward a special status for eyes presented in isolation (Bentin et al., 1996, Itier et al., 2006, Itier et al., 2007), their role in the earliest stages of face encoding is unclear and the featural versus holistic nature of early face perception has puzzled cognitive neuroscientists for nearly two decades. The present paper provides new evidence supporting a particular sensitivity to eyes even in the context of the whole face and their role in early face encoding stages. These findings have important implications for our understanding of face perception and how it breaks down in various disorders, as well as more generally for visual perception. A new neuronal model is proposed that accounts for all the N170 modulations reported, for the holistic processing of upright faces and for the featural processing of inverted faces.

Scalp electrophysiological studies have identified a now well-known ERP component called the N170, the earliest reliable face-sensitive component occurring between 130 and 200 ms over occipito-temporal sites (Bentin et al., 1996). The N170 has been proposed to reflect early perceptual face encoding stages (Eimer, 2000, Sagiv and Bentin, 2001) where features are “glued” together in a holistic facial percept. However, the N170 is even larger for eye regions presented in isolation than for upright faces (Bentin et al., 1996, Itier et al., 2006, Itier et al., 2007) and this eye sensitivity is seen as early as 4 years of age (Taylor et al., 2001a). This featural sensitivity does not include other facial features as nose and mouth elicit delayed and smaller N170s than faces (Bentin et al., 1996, Nemrodov and Itier, 2011, Taylor et al., 2001b). However, eyeless faces elicit N170s with amplitudes similar to normal intact faces, albeit delayed (Eimer, 1998, Itier et al., 2007, Itier et al., 2011, Kloth et al., 2013) and this finding has been interpreted as supporting the view that the N170 reflects a holistic processing stage rather than an eye detector.

The idea that the N170 reflects holistic face processing is also supported by the inversion manipulation, which is known to disrupt face perception and recognition more so than perception and recognition of objects (Yin, 1969). Numerous behavioral studies have shown that objects are processed mostly in a piecemeal way while faces are perceived mostly holistically (e.g. Tanaka and Farah, 1993, Tanaka and Gordon, 2011) and that inversion disrupts this holistic processing (Rossion, 2009). In ERP studies, faces presented upside down trigger delayed but most importantly larger N170s compared to upright faces (Bentin et al., 1996, Itier and Taylor, 2002, Rossion et al., 1999), while upside down objects usually elicit only delayed responses (Itier et al., 2006, Kloth et al., 2013). Similar to objects, animal faces or impoverished human face stimuli, such as sketches or Mooney faces, also show delayed N170 with inversion but no increase in amplitude and sometimes even a slight amplitude reduction (de Haan et al., 2002, Itier et al., 2006, Itier et al., 2011, Latinus and Taylor, 2005, Sagiv and Bentin, 2001, Wiese et al., 2009). The amplitude increase with inversion, also termed the N170 “face inversion effect” (FIE), is thus believed to reflect the disruption of early holistic processing stages specific to human faces and has been used as a hallmark of face specificity. At the neuronal level, this increase has been explained by the recruitment, in addition to face-sensitive neurons, of object-sensitive neurons (Itier and Taylor, 2002, Rossion et al., 1999, Sadeh and Yovel, 2010, Yovel and Kanwisher, 2005), other face-sensitive neurons tuned to the inverted orientation (Eimer et al., 2010), or eye-sensitive neurons (Itier et al., 2007).

Itier et al. (2007) showed that, in contrast to intact faces, inversion of eyeless faces elicited a much reduced N170 FIE and thus proposed that eyes played an important role in this early face specific phenomenon, an idea reinforced by the replication of this finding in later studies (Itier et al., 2011, Kloth et al., 2013, Nemrodov and Itier, 2011). A model involving eye- and face-sensitive neuronal populations (Itier and Batty, 2009, Itier et al., 2007) tried to account for the N170 FIE as well as the larger N170 response to isolated eyes that is seen regardless of eye orientation (Itier et al., 2006, Itier et al., 2007, Itier et al., 2011). According to this model, upright intact faces trigger the activation of face-sensitive neurons which would inhibit eye-sensitive neurons in the context of a configurally correct face, a mechanism that accounts for the lack of amplitude change with upright eyeless faces. Presenting faces upside-down, however, would stop this inhibition, due to the disruption of holistic processing, and the N170-FIE would then reflect the co-activation of both neuronal populations. Both neuronal populations would also respond to isolated eyes, explaining the larger N170 amplitude for eyes than for upright faces.

ERP studies usually use a centrally presented cross to control for fixation position. In most face studies, this central fixation is situated close to the eyes, on the nasion or on the nose. In particular, the studies that reported larger N170 amplitude for isolated eyes or a lack of FIE for inverted eyeless faces presented the fixation on the nasion area (Bentin et al., 1996, Itier et al., 2007, Itier et al., 2011, Kloth et al., 2013, Taylor et al., 2001b). However, recent studies have reported changes in N170 amplitude as a function of fixation location on the face. Fixation on the nasion and mouth yielded larger N170 amplitudes than fixation on the nose in one study (McPartland et al., 2010), although eye-tracking was not employed to ensure participants were indeed fixating at these fixation locations during face presentation. Another study using a gaze contingent paradigm with eye tracking reported largest N170 response for fixation on the nasion when faces were presented upright and for fixation on the mouth when faces were presented inverted (Zerouali et al., 2013). This study suggested that the encoding of a face as indexed by the N170 arises from a general upper-visual field advantage rather than from sensitivity to eyes, whether the face is upright or inverted. These results raise the concern that what was taken as evidence for a specific role of the eyes in the FIE (Itier et al., 2007) might simply reflect an artifact of gaze position rather than a true eye sensitivity.

To address this concern and to probe further the role of eyes in early face encoding, we investigated whether fixation on various facial features modulated the N170 response and the N170 FIE. Crucially, in addition to intact faces we also tested eyeless faces. This new condition allowed us to confirm the potential sensitivity to eyes (or lack thereof) and to test the hypothesis that eyes are important in driving the N170 FIE (Itier et al., 2007). Intact and eyeless faces were presented upright and inverted with fixation locations on the middle of the forehead, nasion, left eye, right eye, tip of the nose, and mouth. To ensure a correct point of gaze, eye tracking was used with a fixation-contingent stimulus presentation and any trial in which gaze deviated by more than 1.8º of visual angle around that fixation location was excluded (Fig. 1). In addition, to prevent participants from using anticipatory strategies the fixation cross was always presented in the center of the screen, while faces were moved around it to obtain the desired fixation position (Fig. 1). House stimuli were used as a control category to ensure that our face stimuli elicited a proper face sensitive N170.

If Zerouali et al. (2013) were correct, we expected to find an interaction between face orientation and fixation location such that the largest N170 responses would be seen for nasion fixation with upright faces and the largest N170 responses would be seen for mouth fixation with inverted faces, reproducing the upper–lower visual field effect. This should also be seen regardless of whether eyes are present or absent from the face. Contrary to this expectation, we found that the N170 was largest when fixation was on the eyes compared to the other fixations and this was seen in both upright and inverted faces, ruling out a simple upper versus lower visual field effect and supporting a special sensitivity to eyes. Crucially, this eye sensitivity disappeared in eyeless faces, demonstrating that it was due to the presence of eyes at fovea. In other words, the sensitivity for eyes was present beyond the classic N170 FIE. However, for eyeless faces, the inversion effect interacted with fixation location such that when fixation was on the mouth, a normal inversion effect was seen while the FIE was maximally reduced when fixation was around the eyes, as reported before (Itier et al., 2007). These findings suggest that eyes do play a role in the inversion effect but only when they are in fovea. We propose a new mechanism to explain this set of data and the N170 FIE. We discuss these findings and their implication for understanding early face perception and in particular holistic versus featural processing.

Section snippets

Participants

Forty-one undergraduate students from the University of Waterloo (UW) were tested and received course-credit for their participation. They all reported normal or corrected-to-normal vision, no history of head-injury or neurological disease, and were not taking any medication. They all signed informed written consent and the study was approved by the Research Ethics Board at UW. Twenty-one participants were rejected, 10 for not completing the experiment and thus registering too few trials per

Behavioral results

Accuracy on this orientation detection task was overall high. Participants responded more accurately to faces than to houses (93.8% and 91.5%, respectively; effect of category, F(1,20) = 5.73, p = 0.027, ηp2 = 0.223). The category by orientation interaction (F(1,20) = 12.4, p < 0.005) was due to an effect of inversion for faces, with higher accuracy for upright than inverted faces (95% vs 92%, F = 10.5, p < 0.005) while no inversion effect was found for houses (F = 1.8, p = 0.195).

Responses were faster to faces

Discussion

In the present study we explored the effect of fixation location on the N170 ERP component, the earliest neural marker of face perception, to test the idea of an eye detector and of its possible role in driving the N170 face inversion effect. Using eye tracking to enforce correct fixation on the desired facial part we found that fixation location clearly affected the N170 amplitude and interfered with the face inversion effect when eyes were removed. We discuss below these effects in turn and

Conclusion

The N170 amplitude was largest when fixation was on the eyes compared to any other facial locations and this effect was seen regardless of face orientation and was lost when eyes were removed. These results suggest a special sensitivity of this ERP component to eye features and support the eye detector theory. Importantly, all the modulations of the N170 can be explained by lateral inhibitions of the neurons coding for the fixated feature in fovea by the neurons coding for features situated in

Acknowledgments

This study was supported by grants from the Natural Sciences and Engineering Research Council of Canada (NSERC Discovery Grant #418431), the Canada Foundation for Innovation (CFI), the Ontario Research Fund (ORF) and the Canada Research Chair program to RJI.

References (56)

  • J. McPartland et al.

    Face-related ERPs are modulated by point of gaze

    Neuropsychologia

    (2010)
  • E.T. Rolls et al.

    The neuronal encoding of information in the brain

    Prog. Neurobiol.

    (2011)
  • B. Rossion

    Distinguishing the cause and consequence of face inversion: the perceptual field hypothesis

    Acta Psychol.

    (2009)
  • B. Rossion et al.

    Spatio-temporal localization of the face inversion effect: an event-related potentials study

    Biol. Psychol.

    (1999)
  • B. Sadeh et al.

    Why is the N170 enhanced for inverted faces? An ERP competition experiment

    NeuroImage

    (2010)
  • P.G. Schyns et al.

    Dynamics of visual information integration in the brain for categorizing facial expressions

    Curr. Biol.

    (2007)
  • M.J. Taylor et al.

    Direction of gaze effects on early face processing: eyes-only versus full faces

    Cogn. Brain Res.

    (2001)
  • C. Vinette et al.

    Spatio-temporal dynamics of face recognition in a flash: it's in the eyes

    Cogn. Sci.

    (2004)
  • H. Wiese et al.

    Configural processing of other-race faces is delayed but not decreased

    Biol. Psychol.

    (2009)
  • G. Yovel et al.

    The neural basis of the behavioral face-inversion effect

    Curr. Biol.

    (2005)
  • Y. Adini et al.

    Excitatory–inhibitory network in the visual cortex: psychophysical evidence

    Proc. Natl. Acad. Sci. U. S. A.

    (1997)
  • T. Allison et al.

    Category-sensitive excitatory and inhibitory processes in human extrastriate cortex

    J. Neurophysiol.

    (2002)
  • J. Arizpe et al.

    Start position strongly influences fixation patterns during face processing: difficulties with eye movements as a measure of information use

    PLoS One

    (2012)
  • J.J.S. Barton et al.

    Information processing during face recognition: the effects of familiarity, inversion, and morphing on scanning fixations

    Perception

    (2006)
  • M. Behrmann et al.

    Detailed exploration of face-related processing in congenital prosopagnosia: 1. Behavioral findings

    J. Cogn. Neurosci.

    (2005)
  • S. Bentin et al.

    Electrophysiological studies of face perception in humans

    J. Cogn. Neurosci.

    (1996)
  • M. Bindemann et al.

    Viewpoint and center of gravity affect eye movements to human faces

    J. Vis.

    (2009)
  • R. Caldara et al.

    Does prosopagnosia take the eyes out of face representations? Evidence for a defect in representing diagnostic facial information following brain damage

    J. Cogn. Neurosci.

    (2005)
  • Cited by (0)

    View full text