Abstract
Humans reliably recognize faces across a range of viewpoints, but the neural substrates supporting this ability remain unclear. Recent work suggests that neural selectivity to mirror-symmetric viewpoints of faces, found across a large network of visual areas, may constitute a key computational step in achieving full viewpoint invariance. In this study, we used repetitive transcranial magnetic stimulation (rTMS) to test the hypothesis that the occipital face area (OFA), putatively a key node in the face network, plays a causal role in face viewpoint symmetry perception. Each participant underwent both offline rTMS to the right OFA and sham stimulation, preceding blocks of behavioral trials. After each stimulation period, the participant performed one of two behavioral tasks involving presentation of faces in the peripheral visual field: (1) judging the viewpoint symmetry; or (2) judging the angular rotation. rTMS applied to the right OFA significantly impaired performance in both tasks when stimuli were presented in the contralateral, left visual field. Interestingly, however, rTMS had a differential effect on the two tasks performed ipsilaterally. Although viewpoint symmetry judgments were significantly disrupted, we observed no effect on the angle judgment task. This interaction, caused by ipsilateral rTMS, provides support for models emphasizing the role of interhemispheric crosstalk in the formation of viewpoint-invariant face perception.
SIGNIFICANCE STATEMENT Faces are among the most salient objects we encounter during our everyday activities. Moreover, we are remarkably adept at identifying people at a glance, despite the diversity of viewpoints during our social encounters. Here, we investigate the cortical mechanisms underlying this ability by focusing on effects of viewpoint symmetry, i.e., the invariance of neural responses to mirror-symmetric facial viewpoints. We did this by temporarily disrupting neural processing in the occipital face area (OFA) using transcranial magnetic stimulation. Our results demonstrate that the OFA causally contributes to judgments facial viewpoints and suggest that effects of viewpoint symmetry, previously observed using fMRI, arise from an interhemispheric integration of visual information even when only one hemisphere receives direct visual stimulation.
- face recognition
- interhemispheric crosstalk
- occipital face area
- transcranial magnetic stimulation
- viewpoint symmetry
- viewpoint-invariance
Introduction
The ability to recognize others based on their facial appearance is a key social skill in our day-to-day visual experience. Computationally, however, this task is challenging, because the retinal projection of the same face can change drastically with changes in illumination, viewing distance, or 3D viewpoint. Befitting the importance of face recognition, the human brain is equipped with a network of face-selective regions (Kanwisher et al., 1997; Gauthier et al., 2000; Haxby et al., 2000; Yovel and Kanwisher, 2005; Freiwald and Tsao, 2010; Pitcher et al., 2011). Despite our increasing understanding of the selectivity in distinct nodes of this network, it remains unclear exactly how interactions in the network support invariant facial recognition.
Recent electrophysiological recordings suggest a rather late emergence of a viewpoint-invariant code, located in the most anterior face patch in macaque monkey temporal cortex (Freiwald and Tsao, 2010). In a region posterior to this site, neurons responded strongest to selected viewpoints of faces, as well as to their mirror-symmetric counterparts (e.g., 30° and −30° rotated away from a front-on view). Based on this observation, it was suggested that viewpoint symmetry constitutes an intermediate computational step in building a fully viewpoint-invariant neuronal code. Indeed, markers of mirror-symmetric viewpoint representations have been documented reliably in human and nonhuman primates (Perrett et al., 1991; Freiwald and Tsao, 2010; Axelrod and Yovel, 2012; Kietzmann et al., 2012; Dubois et al., 2015). Despite this evidence for mirror-symmetric visual representations, the causal relevance of the involved cortical regions for judgments of facial viewpoints and the cortical mechanisms underlying our ability to perceive mirror-symmetric views remain unknown.
Here we address these questions by following a two-staged approach, focusing on the occipital face area (OFA), a constituent node of the human face processing network (Pitcher et al., 2007, 2011; Solomon-Harris et al., 2013) that exhibits effects of viewpoint symmetry (Kietzmann et al., 2012). First, we assessed the degree to which this region is involved in judgments of facial viewpoint and viewpoint symmetry. Having established this causal link, we then investigated the underlying cortical mechanisms by testing the hypothesis that interhemispheric transfer of information directly contributes to the processing of mirror-symmetric face views.
To test for causal contributions, we used functional magnetic resonance imaging (fMRI) to individually localize the right OFA in human participants and subsequently disrupted processing in this region by administering slow repetitive transcranial magnetic stimulation (rTMS). Immediately after stimulation, participants were asked to perform perceptual judgments about facial viewpoints. This performance was then compared against a baseline condition, in which a sham coil with no effect on cortical activity was used. Two tasks were tested: (1) a viewpoint symmetry judgment task; and (2) a viewpoint angle judgment task (Fig. 1). In the symmetry judgment task, participants viewed a pair of faces with different viewpoints, followed by a second pair, and were asked to report which pair of facial viewpoints was mirror symmetric. The angle judgment task was similar in design, with the exception that participants judged which of two face pairs depicted a larger difference in viewing angle. This experimental approach allowed us to (1) determine whether the OFA causally contributes to perceptual judgments of facial viewpoints and (2) learn whether the two types of viewpoint judgments, symmetry and angle, are affected differentially.
Stimuli and trial structure. A, Four identities, shown from 180 viewpoints, were used for the experiments. B, The symmetry judgment task required participants to report which of two pairs of faces were shown from mirror-symmetric viewpoints. The angle judgment task required reporting the pair with the larger difference in viewpoint. Shown here are examples from Experiment 1 (contralateral stimulation). In Experiment 2 (ipsilateral stimulation), the stimuli were presented to the right of fixation.
Materials and Methods
Participants
Inclusion in the TMS experiments was contingent on reliable functional localization of the right OFA in one of several previous fMRI studies. Of the available pool at Vanderbilt University, 11 participants were recruited for Experiment 1 (median age, 24 years) and 12 participants for Experiment 2 (median age, 25 years). Four participants contributed to both studies. Participants who exhibited large head movements away from the initial coil positioning (>0.5 cm) or reported discomfort during stimulation were excluded from additional analyses (one in Experiment 1, two in Experiment 2). Twnety-four participants (median age, 19 years) took part in a behavioral control experiment. Nine original TMS and eight additional participants (median age, 26 years) took part in an eye-tracking control experiment (three participants were excluded as a result of poor recording quality).
Participants had normal or corrected-to-normal visual acuity, were informed of their right to withdraw from the experiment at any time, and gave written informed consent to participate. The protocol was approved by the Vanderbilt University Institutional Review Board.
Experimental stimuli
The stimulus set was created using the face-modeling software FaceGen (Singular Inversions). It included two female and two male individuals depicted from left profile to right profile across 180 viewpoints, each separated by 1° of rotation in the depth plane. Viewpoints facing right were created by horizontally mirroring viewpoints facing left (Fig. 1A).
Design and procedure
TMS experiments.
Each session followed a 2 × 2 design, consisting of four experimental blocks. Each block consisted of a stimulation period, followed by a behavioral test. After either offline rTMS or sham TMS, applied for 5 min, performance was tested in two tasks: (1) angle judgment; and (2) symmetry judgment. The first two blocks were both either sham or rTMS, with the respective other stimulation condition occurring before blocks 3 and 4; the order of tasks in either TMS condition was randomized across participants. This non-interleaved sham/rTMS design reduced the overall number of setup changes and minimized potential carryover effects from rTMS to sham conditions. A resting break of 1 min was included between experimental blocks.
Both tasks followed a two-interval forced-choice discrimination design. In each trial, two pairs of faces shown from different viewpoints were presented sequentially, separated by a brief interval (pair 1 consisted of stimuli 1 and 2, pair 2 consisted of stimuli 3 and 4). Stimuli 1 and 3 were identical and served as the baseline angle. In the angle judgment task, participants judged which pair of faces exhibited a larger change in viewpoint angle. The symmetry judgment task required reporting which of the two pairs was viewpoint symmetric. Each block contained 60 trials and took ∼4 min to complete.
To render difficulty levels comparable across tasks and individuals, each participant initially performed several QUEST (Watson and Pelli, 1983) sequences (at least two blocks of 60 trials per task), in which the difficulty of both tasks was adjusted to target 80% correct trials (d′ = 1.2). In angle judgment blocks, one of the two pairs always differed by 10°, allowing us to change the difficulty of the task by altering the viewpoint difference of the other pair. For each of the two experiments, an average viewpoint difference of 16.5° was required to reach the predefined performance level. In the symmetry judgment blocks, the viewpoints of one of the two face pairs were mirror symmetric. The complexity of the task was adjusted by changing the viewpoint angle of the other pair away from symmetry. An average change of ∼10° (10.2° in Experiment 1, 9.3° in Experiment 2) was used to reach 80% accuracy. That is, when the symmetric pair of faces was set to −50° and 50°, the nonsymmetric pair was set to −50° and 60°. On each trial in either task, the base viewpoint was randomly selected between ±50° and ±70°.
Each trial started with a fixation dot in the center of the screen. Then, each face stimulus appeared for 500 ms with a 200 ms blank interval between faces of the same pair. The two pairs were separated by a 400 ms interval. After the presentation of both pairs, the participants had 800 ms to respond, followed by a 400 ms intertrial interval (Fig. 1B). Trials in which the participant failed to respond in time were excluded from the analyses.
Stimuli were generated using MATLAB (MathWorks) and the Psychophysics Toolbox 3 (Kleiner et al., 2007). Observers viewed the stimuli on a CRT monitor (22-inch HP 1230 pixels; resolution, 1024 × 768; refresh rate, 85 Hz), with their heads stabilized by a chin and forehead rest (viewing distance, 57 cm). The stimulus width ranged from 4.7° to 6.9° visual angle, depending on the viewpoint shown. In Experiment 1, all stimuli were presented in the left visual field, contralateral to the side of TMS stimulation. The distance between the central fixation dot and the nearest edge of the stimulus was 1.5° visual angle. Experiment 2 followed the same procedure as Experiment 1, but stimuli were presented in the right visual field, ipsilateral to the TMS stimulation. Ipsilateral presentation ensured that low-level retinotopic effects could not account for the disruptive effects of TMS. Before each experimental block, participants were reminded to maintain fixation on the centrally presented fixation dot via on-screen instructions.
Behavioral control.
Each participant performed four experimental blocks, estimating the task performance in angle and symmetry judgments in the contralateral and ipsilateral visual fields. Conditions were balanced across participants, and task thresholds were set to the rounded average thresholds used in the TMS experiments (ipsilateral symmetry, 10°; ipsilateral angle, 17°; contralateral symmetry, 9°; contralateral angle, 17°).
Eye-tracking control.
Data were recorded using an Eyelink 1000 eye tracker (SR Research), following the same protocol as the behavioral control. To correct for slow artifactual drifts in estimated position, eye-tracking data were adjusted with respect to the average fixation position during the pretrial baseline period. We calculated the average fixation positions during the presentation of the four stimuli, excluding data recorded during blinks and saccades, and computed the percentage of trials in which fixations at the stimulus location lasted at least 150 ms.
TMS stimulation and site localization
TMS was administered using a Magstim 2T Rapid stimulator (peak discharge, 1.8 kV; Magstim) and a 70 mm air-cooled figure-eight coil. Cortical activity was depressed using a low-frequency (1 Hz) repetitive stimulation protocol in which brief TMS pulses were applied for 5 min, before behavioral tests. This approach, known as offline rTMS, has the advantage that task performance is estimated without the disruption of concurrent TMS pulses (Ling et al., 2009; Bolognini and Ro, 2010; Hirnstein et al., 2011). The intensity of the TMS pulses was set based on participants' individual tolerance, ranging from 75% to 90% of maximum stimulation. Stimulation sites were located using the Brainsight TMS neuronavigation system (Rogue Research), which allows for a coregistration of TMS and a 3D structural MRI scan of an individual. We targeted the region of peak activation corresponding to the OFA of each individual, focusing on the right hemisphere because of more reliable localization (Gauthier et al., 2000; Rossion et al., 2003).
Functional imaging
Functional localizer data were collected using either a 3T Philips Intera Achieva or 7T Philips Achieva MRI scanner (Philips) at Vanderbilt University. All localizer runs included stimulation blocks using faces, houses, objects, and corresponding scrambled stimulus versions. Processing of the fMRI data followed the procedure described by Kietzmann et al. (2012). The right OFA was localized in each person, based on linear contrasts between the resulting GLM β estimates, as a cluster of voxels in the occipital lobe exhibiting significantly larger BOLD activation in response to faces compared with houses/objects and corresponding scrambled stimuli (Fig. 2).
OFA targeting. The right OFA was defined functionally using individual fMRI localizer data, contrasting BOLD responses to faces versus houses/objects and corresponding scrambled stimuli (example participant shown).
Results
In Experiment 1, we sequentially presented pairs of faces in the left visual field, contralateral to the right hemisphere receiving rTMS. This allowed us to test whether rTMS stimulation of OFA impairs viewpoint processing and to verify that it causally contributes to both angle and symmetry judgments. In line with previous work suggesting the causal involvement of the OFA in the processing of faces (Pitcher et al., 2007, 2011), we found that performance of both tasks was impaired after rTMS (Fig. 3A,C) relative to performance measured after the sham control. A repeated-measures ANOVA revealed a significant main effect of TMS (F(1,9) = 6.54, p = 0.03) but no significant effects of task (F(1,9) = 0.57, p = 0.47) or interaction (F(1,9) = 0.018, p = 0.89). Hence, we observed a significant effect of rTMS but no significant difference across the two tasks when faces appeared in the visual field contralateral to the stimulated hemisphere.
Results of Experiments 1 and 2. Performance in the angle and symmetry tasks after sham and rTMS in the contralateral experiment (A) and ipsilateral experiment (B). C and D show the effect of TMS on both tasks when performed in the contralateral and ipsilateral visual fields, respectively. Error bars indicate ±SEM.
Having demonstrated a causal link between TMS applied to the right OFA and performance across both judgments of facial viewpoints, we conducted a second experiment to isolate the mechanistic underpinnings of viewpoint symmetric selectivity. Inspired by prominent theories on mirror-image confusion (Corballis and Beale, 1976), we investigated whether interhemispheric interactions might contribute to effects of viewpoint symmetry. This was accomplished by testing the visual stimulus location ipsilateral to the stimulated right hemisphere, while otherwise following the same procedure as in the first experiment. With regard to the impairment through rTMS, we predicted that basic judgments of facial viewpoints should not be impaired in the ipsilateral hemifield, because core aspects of face processing should proceed unimpeded in the contralateral OFA. In contrast, judgments requiring interhemispheric exchange of information should be affected by ipsilateral rTMS.
When stimuli were presented ipsilateral to the stimulated right OFA, viewpoint symmetry judgments were impaired significantly by rTMS, whereas face angle judgments were unaffected by ipsilateral stimulation (Fig. 3B,D). A repeated-measures ANOVA contrasting task (symmetry/angle) and TMS type (rTMS/sham) showed no significant effect of task (F(1,9) = 0.38, p = 0.55) or TMS (F(1,9) = 1.27, p = 0.29) but did reveal a significant interaction effect (F(1,9) = 5.29, p = 0.047). Post hoc comparisons revealed a significant decrease in performance after rTMS for symmetry judgments (t(9) = 2.43, p = 0.038), whereas angle judgments were no more affected by rTMS than by sham TMS (t(9) = −0.86, p = 0.40). The post hoc test of the interaction revealed that the effect of rTMS over OFA was significantly greater for symmetry judgments than for angle judgments (t(9) = 2.32; p = 0.046). Furthermore, this ipsilateral interaction is significantly larger than the nonsignificant interaction effect observed contralaterally (second-level interaction, tested via one-sided, unpaired t test, t(18) = 1.91, p = 0.036). Together, these results imply that viewpoint-symmetry judgments are dependent on interhemispheric interactions. In contrast, the results for angle judgments suggest that this task can be performed solely in the contralateral hemisphere.
Although no significant differences in baseline performance were found across experimental conditions [mixed-design ANOVA, task (angle/symmetry) as within-subject factor, experiment (contralateral/ipsilateral) as between-subject factor, main effect and interaction: all p > 0.05], we noted slightly elevated baseline performance in the sham condition for ipsilateral compared with contralateral symmetry judgments. The most likely explanation for this effect is that the angular differences used were on average slightly larger for participants in the ipsilateral experiment than for those in the contralateral TMS experiment (10.2° compared with 9.3°, respectively), rendering the ipsilateral presentation task slightly easier overall. This hypothesis was verified by a behavioral follow-up experiment in which the same angular differences were used to measure task performance in a separate participant group. We observed the same pattern of results, indicating that modest differences in baseline performance can be ascribed to slight differences in task difficulty. In line with this observation, a mixed-design ANOVA, comparing behavioral control and sham performance across positions and tasks, revealed no main effects (all p > 0.05) but a significant interaction of task and position (F(1,64) = 6.91, p = 0.01).
In the TMS and behavioral experiments, participants were explicitly asked to fixate continuously throughout the experiment. To ensure that the overall task is performed while maintaining fixation, we ran another follow-up experiment including eye tracking. All participants (seven original TMS participants and seven new participants) fixated well during the stimulus presentations, while performing well above chance (all t(13) > 9.67, all p < 0.001). The average ± SEM fixation position deviated by only 0.13 ± 0.02° from the central fixation dot (Fig. 4). Overall, only 0.2% of trials included fixations on at least one stimulus of each stimulus pair, and 1.1% included a fixation on at least one of the four individual stimuli (average difference across conditions, 0.75%; maximum, 1.4%). Compared with the average performance of the behavioral control participants, who were also tested in both tasks and visual fields, we find no significant performance differences (t(36) = −0.76, p = 0.45). Moreover, we found no correlation between task performance and horizontal fixation accuracy in any of the four conditions (all p > 0.05). This suggests that our ipsilateral stimulation results truly emerged as a result of interhemispheric crosstalk.
Eye-tracking control experiment. Shown are the average fixation positions, recorded during the presentation of the experimental stimuli, for both visual fields and tasks. Gray dots show single trial data; red dots and orange dots indicate average positions for individual TMS and control participants, respectively. Black dots indicate group averages of fixation positions.
Discussion
This study demonstrates that the right OFA causally contributes to two different types of facial viewpoint judgments. Compared with sham baseline, rTMS impaired judgments of both angular differences and viewpoint symmetry for faces appearing in the contralateral, left visual field. In search of possible mechanisms underlying effects of viewpoint symmetry, we turned to the highly related effects of left–right mirror-image confusion (Bornstein et al., 1978; Rollenhagen and Olson, 2000; Baylis and Driver, 2001; Dehaene et al., 2010; Dilks et al., 2011; Pegado et al., 2011), and tested for the contribution of interhemispheric interactions. In contrast to the first experiment, the effects of rTMS on task performance in the ipsilateral, right visual field exhibited task-specific differences. Here, only viewpoint symmetry performance was disrupted by rTMS, whereas angle judgments were unimpaired.
This pattern of results provides insight into the cortical mechanisms underlying the progressively invariant face codes, observed when ascending the ventral visual pathway (DiCarlo and Cox, 2007; Freiwald and Tsao, 2010), to which viewpoint symmetry is thought to critically contribute (Vetter et al., 1994; Dehaene, 2010; Freiwald and Tsao, 2010). Although effects of viewpoint symmetry have been reliably documented in human and nonhuman primates (Perrett et al., 1991; Freiwald and Tsao, 2010; Axelrod and Yovel, 2012; Kietzmann et al., 2012; Dubois et al., 2015), the cortical mechanisms underlying this specific computation remained elusive. The current results not only reveal a causal contribution of the right OFA but importantly also demonstrate that perception of viewpoint symmetry directly benefits from interhemispheric crosstalk. Although our visual stimuli were laterally presented so that only the contralateral hemisphere would be directly activated, viewpoint symmetry judgments were still affected by rTMS applied ipsilaterally. Critically, these results cannot be explained by either general effects of TMS on visual processing, which would affect both tasks equally, or by low-level retinotopic effects, which predict that rTMS should not impair processing of ipsilateral visual stimuli.
Although our administration of TMS directly targeted the right OFA, rendering it the most likely source of the observed effects, we realize that TMS can have effects that ramify throughout the cortical network connected to that targeted area (Pitcher et al., 2014). For example, the OFA has strong connections with the fusiform face area, which also exhibits viewpoint symmetric responses (Axelrod and Yovel, 2012; Kietzmann et al., 2012). Although part of the effects we observed could be driven by an inhibition of these ipsilaterally connected regions, the present results provide unequivocal evidence of the contribution of interhemispheric communication to processing mirror-symmetric face views.
It would be of interest for future studies to test for such causal effects in nonhuman primates, because both single-unit and fMRI work has found viewpoint symmetric representations at intermediate levels of the face-patch network but not in posterior sites (Freiwald and Tsao, 2010; Dubois et al., 2015). Although there are strong parallels in the functional organization of the face networks across humans and monkeys (Kriegeskorte et al., 2008; Tsao et al., 2008), our previous fMRI results (Kietzmann et al., 2012) and the current evidence could suggest possible differences across the two systems.
The current set of experiments focused on the neural underpinnings of facial viewpoint encoding, with an emphasis on effects of viewpoint symmetry. For faces, viewpoint symmetry effects have been demonstrated reliably, and they can benefit from a common representation of symmetric viewpoints, given their highly symmetrical 3D structure. Although the current set of experiments does not tell us whether other categories exhibit similar viewpoint mechanisms, future work can reveal the degree to which similar crosstalk interactions transpire within other categories and object-selective cortical regions. The effects of viewpoint symmetry observed in the human dorsal and ventral streams (Kietzmann et al., 2012), the distributed effects of mirror-image confusion, including the results of a previous TMS experiment on letter identification (Nakamura et al., 2014), suggest that similar cortical computations might indeed underlie object viewpoint invariance in general.
In summary, our results provide causal evidence for the involvement of the right OFA in facial viewpoint processing, with an emphasis on interhemispheric crosstalk for judgments of viewpoint symmetry: viewpoint symmetry processing relies on both hemispheres, even if only one hemisphere receives direct visual stimulation.
Footnotes
The work was supported by National Science Foundation Grants BCS-0642633 and BCS-1228526 (F.T.), a Fulbright scholarship (T.C.K.), European Research Council Grants FP7-ICT-270212 and ERC-2010-AdG 269716 (P.K.), and National Institutes of Health Grant P30-EY008126 (to Vanderbilt Vision Research Center). We thank Isabel Gauthier and Katie Ryan for making OFA localizer data available, as well as Jascha Swisher and Niklas Wilming for their valuable input during the design phase of this study.
The authors declare no competing financial interests.
- Correspondence should be addressed to Tim C. Kietzmann, Institute for Cognitive Science, University of Osnabrück, Albrechtstrasse 28, D-49076 Osnabrück, Germany. tkietzma{at}uos.de
