Face-selective neural responses in the human fusiform gyrus have been widely examined. However, their causal role in human face perception is largely unknown. Here, we used a multimodal approach of electrocorticography (ECoG), high-resolution functional magnetic resonance imaging (fMRI), and electrical brain stimulation (EBS) to directly investigate the causal role of face-selective neural responses of the fusiform gyrus (FG) in face perception in a patient implanted with subdural electrodes in the right inferior temporal lobe. High-resolution fMRI identified two distinct FG face-selective regions [mFus-faces and pFus-faces (mid and posterior fusiform, respectively)]. ECoG revealed a striking anatomical and functional correspondence with fMRI data where a pair of face-selective electrodes, positioned 1 cm apart, overlapped mFus-faces and pFus-faces, respectively. Moreover, electrical charge delivered to this pair of electrodes induced a profound face-specific perceptual distortion during viewing of real faces. Specifically, the subject reported a “metamorphosed” appearance of faces of people in the room. Several controls illustrate the specificity of the effect to the perception of faces. EBS of mFus-faces and pFus-faces neither produced a significant deficit in naming pictures of famous faces on the computer, nor did it affect the appearance of nonface objects. Further, the appearance of faces remained unaffected during both sham stimulation and stimulation of a pair of nearby electrodes that were not face-selective. Overall, our findings reveal a striking convergence of fMRI, ECoG, and EBS, which together offer a rare causal link between functional subsets of the human FG network and face perception.
The neural mechanisms of face recognition involve a network of brain regions in which the fusiform gyrus (FG) is a crucial component. Damage to the FG can cause impairments in perceiving or naming faces (Damasio et al., 1982; Barton, 2008; Konen et al., 2011). Intracranial recordings (Allison et al., 1999; McCarthy et al., 1999; Puce et al., 1999; Mundel et al., 2003; Vidal et al., 2010) and functional magnetic resonance imaging (fMRI) studies (Kanwisher et al., 1997) in humans reveal stronger FG activations to faces compared with nonface stimuli. Imaging studies have shown that FG face-selective activations are greater during successful perception and identification of faces (Tong et al., 1998; Hasson et al., 2001; Moutoussis and Zeki, 2002; Grill-Spector et al., 2004; Rotshtein et al., 2005) and are modulated by variations in face identity (Grill-Spector et al., 1999; Winston et al., 2004; Jiang et al., 2006). Recent imaging studies reveal multiple face-selective clusters along the FG (Tsao et al., 2008; Pinsk et al., 2009; Weiner and Grill-Spector, 2010) that are 10–15 mm apart from each other and have a consistent spatial arrangement relative to gross anatomical landmarks, retinotopic maps, and other functional regions (Weiner and Grill-Spector, 2012). These findings raise the possibility that different face-selective regions are involved in different components of face processing.
One method to examine the roles of a given region in face perception and recognition is electrical brain stimulation (EBS) during which a volley of electrical charge is delivered to a focal brain area to perturb its function (Selimbeyoglu and Parvizi, 2010). However, EBS in the human FG remains scarce due to its invasive nature. Indeed, only three studies have examined the effects of stimulating the FG on face perception with variable findings. Two studies reported that EBS of FG face-selective sites produces deficits in naming, but not perceiving, faces (Allison et al., 1994; Puce et al., 1999). In contrast, one study reported that EBS of a right FG electrode caused the subject to report that “all faces in the room looked the same” consistent with the subject's typical seizure auras (Mundel et al., 2003). It is unknown from these studies which FG cluster was stimulated. Consequently, discrepant findings may stem from stimulation of different regions across studies, underscoring the necessity of precise anatomical and functional localization of EBS sites for understanding their key role in face processing.
Using methodological advancements in localizing the anatomical position of intracranial electrodes in the subject's own native neuroanatomical space (Hermes et al., 2010) and in defining fMRI activations using high-resolution voxels (1.8 mm isotropic), we precisely colocalized the sites of EBS with both fMRI and electrophysiological data. We report face-specific perceptual deficits when electrical charge was delivered through two electrodes overlapping with pFus and mFus (posterior and mid fusiform, respectively) face-selective regions [referred to as pFus-faces (FFA-1) and mFus-faces (FFA-2), respectively] and provide a causal link between localized functional subsets of the FG and face perception.
Materials and Methods
The subject was a 45-year-old man implanted with intracranial electrodes to localize the source of medication-resistant seizures who provided written informed consent to participate in the study and to publish the video of the EBS procedure. The procedure was approved by the Stanford Institutional Review Board. Clinically, his seizures always started with seeing phosphenes and colors with scalp video-EEG monitoring confirming a right posterior quadrant seizure focus. Standard presurgical evaluation revealed normal intellectual abilities and visuospatial functioning without any psychiatric comorbidities or visual field abnormalities. Continuous video-EEG recording for several days revealed epileptic source in the right pericalcarine area. The inferior temporal region was devoid of pathological electrophysiological activity.
Imaging data were obtained using a GE 3-Tesla Signa scanner at Stanford University.
A high-resolution anatomical volume of the whole brain was acquired with a head coil using a T1-weighted SPGR pulse sequence. Data were aligned to the AC-PC plane and resampled to 1 mm isotropic voxels. Both fMRI data and electrocorticography (ECoG) electrode locations were aligned to this brain volume. This volume was segmented to separate gray from white matter, which was used to reconstruct the subject's cortical surface.
Twenty-eight slices with 1.8 mm isotropic voxels were acquired with a T2* GE EPI sequence (FOV = 192 mm, TE = 30 ms, TR = 2000 ms, flip angle = 77° and bandwidth = 128 kHz) using a 32-channel surface coil (Nova Medical Inc.). Smaller voxels result in improved localization of functional activations and reduce the effects of susceptibility artifacts, such as those produced by the ear canal, which extend into the fusiform gyrus.
The subject participated in 2 runs of a block design experiment during which images of faces, limbs, flowers, houses, cars, guitars, and scrambled objects were shown in 12 s blocks (Weiner and Grill-Spector, 2010). Each run consisted of 4 blocks of each condition and 6 blank blocks. The subject responded by button press when two consecutive images were identical while maintaining fixation.
We scanned the subject in 4 runs of standard retinotopic mapping. The subject viewed checkerboard bar stimuli that swept the visual field while fixating (Dumoulin and Wandell, 2008).
Data were analyzed with MATLAB (MathWorks) using the mrVista toolbox (http://white.stanford.edu/software) as in our prior publications (Weiner and Grill-Spector, 2010, 2011). Data were motion corrected, detrended, and aligned to the subject's whole brain anatomy. For the localizer, we conducted standard general linear model analyses in each voxel to identify higher responses to faces vs other stimuli (Fig. 1b,c). We also independently extracted the fMRI signals from a gray matter disk of radius 2 mm that was centered on each of the stimulated electrodes (Fig. 2c). Retinotopy data were used to determine boundaries of retinotopic visual areas to further assess the topological location of face-selective regions.
Anatomical localization of electrodes.
Post-implant CT images were aligned to the preop MRI anatomical brain volume (Hermes et al., 2010). Electrodes were visualized on the subject's own brain volume and reconstructed 3D cortical surface allowing for accurate anatomical localization of electrodes (Fig. 1).
Electrophysiological data were obtained, as described in our recent publication (Foster and Parvizi, 2012) while the subject viewed grayscale images of faces, limbs, cars, and houses across 9 runs. In each run, 48 images (12 from each category) were shown once. Eight additional images, 2 from each category, were repeated 6 times each. Here, we only show data for nonrepeated images. Each image was presented for 1 s with a blank intertrial interval of variable duration (ranging from 600 to 1400 ms; uniform distribution). Trial order was counterbalanced such that repeated images were equally likely to follow repeated or nonrepeated images. Images were not repeated across experimental runs.
Data processing was performed using custom routines in MATLAB (MathWorks). Continuous data were filtered with a 60 Hz notch filter, epoched in −1.8 to 1.8 s stimulus-locked time windows and re-referenced using an average of 22 (of 112) nonvisually responsive electrodes.
We calibrated the onset of each stimulus trial to a photodiode signal generated by a luminance change in the stimulus display associated with the onset of each trial. We estimate the upper bound of timing inaccuracies to be 17 ms. To further validate the timing of our responses, we also examined the latency of visual event-related potentials (ERPs) in electrodes overlapping early visual areas around the calcarine sulcus. These electrodes show, as expected, a negative visual ERP peaking around 80 ms after stimulus onset (data not shown) in line with the latency findings in other publications (Yoshor et al., 2007).
Power analyses were performed on the same −1.8 to 1.8 s epoch and baseline corrected by computing the percentage power change relative to the mean power in the −700 ms to −300 ms prestimulus window (during which the subject viewed a blank screen) and then averaged for each condition separately (113–125 trials per condition). Power change relative to baseline was plotted and analyzed for several narrow band frequencies (Fig. 2a). We found similar responses across low gamma (40–80 Hz) and higher frequencies (80–160 Hz), thus, data were analyzed in a broadband frequency range of 40–160 Hz. Based on the log power plots, the broadband increase of power after the onset of stimuli started before 40 Hz range. To calculate the signal-to-noise ratio (SNR) for each electrode across the task, we first calculated for each category the mean power in the broadband frequency range (40–160 Hz) over a time window of 100–350 ms after stimulus onset and the across-trial SE. Then we averaged the power across categories and divided this number by the average SE across categories to generate an SNR per electrode. Diameters of pie charts in Figure 1a are scaled to reflect the SNR. It should be noted that the SNR measure in our study is not about the signal quality (degree of noise) for each electrode, but a measure referring to the power of signal in a specific frequency band during a specific task condition. This is important because there may be some electrodes that show a trivial or random increase to only faces (e.g., 3% increase to faces, 0% to all other conditions), that would otherwise show a ‘face selective’ pie-chart. Therefore, additionally characterizing the magnitude of event-related changes is an important part of identifying condition selectivity and conveying how large such effects are.
Bipolar electrical charge was delivered between two adjacent electrodes using sham (0 mA) and real trials (2–4 mA). Alternating square wave electrical pulses with 200 μs width were delivered at 50 Hz (i.e., the volley of electrical charge alternated in polarity 50 times per second; thus, equal charge was delivered to both electrodes). During all sessions, the EEG was monitored for the presence of after-discharges or seizures. None of the stimulation sessions led to any after-discharges or epileptic activity in any implanted electrodes.
The EBS procedure stimulating a pair of face-selective electrodes (Figs. 1, 2) was performed during two conditions: when the subject was viewing A) the face of specific individuals (doctor or technician) or objects (TV or “get well” balloon) in the room, and B) photographs of famous faces (n = 10, 5 sham EBS) or famous scenes/monuments (n = 10, 5 during sham EBS) on a computer screen at a visual angle of ∼4°. We call this task the famous face/place naming task. The electrical current during all real EBS trials was set at 3 mA (except one 4 mA trial) with duration of ∼1 s. Individuals in the room differed in their gender and distance to the patient. All photographs had been identified and named correctly by the subject before the EBS procedure.
We ran a similar EBS procedure on a pair of control electrodes located 1.7 cm medially to electrodes 1 and 2 (Fig. 1, dotted white circles). We conducted 3 EBS trials (0 sham) during viewing of faces of people in the room and 20 trials of the famous face/place naming task (10 faces trials and 10 place trials, 50% during sham EBS) using a different set of images.
It should be noted that the aim of the famous face/place naming task was to assess the subject's naming retrieval. That is, the aim of the task was not to compare the effect of EBS on face perception during viewing of real faces and photos of famous faces because these two conditions differ in three key components: (1) the electrical charge during the real face condition was delivered while the subject was already looking at the person's real face (so the transformation could be more clear), whereas the stimulation in the famous face/place naming task entailed presentation of the face photograph while the electrical stimulation preceded and outlasted the presentation of each photograph; (2) during the famous face/place naming task, the subject was asked to name the visual images on the screen whereas during the real face condition, this was not required; and (3) before the famous face/place naming task, the subject was not instructed to pay attention to his perceptual change, but instead, to efficiently retrieve the names of the objects, whereas during the real face condition, the subject was instructed to report the presence of any perceptual changes.
Convergence of fMRI and ECoG data
The electrode grid coverage of right ventral temporal cortex (VTC) enabled examination of ECoG responses over a wide cortical expanse. ECoG analyses of the broadband power revealed some VTC electrodes with comparable visual responses across categories, and others that varied in their selectivity to particular visual categories (Fig. 1a). Face-selective electrodes were clustered on the FG, where two electrodes on the lateral FG (Fig. 1, marked 1 and 2) showed the most robust face-selective ECoG responses, with significantly higher power (starting 100 ms after stimulus onset) to faces compared with limbs, cars, or houses sustained for the entire duration of stimulus presentation for frequencies >40 Hz (Fig. 2a). Face selectivity was also apparent during the transient visual response (100–350 ms after stimulus onset) at lower frequency bands (Fig. 2a), and the same electrodes showed a face-selective ERP manifesting as a larger negative component for faces than other stimuli peaking ∼130–147 ms after stimulus onset (Fig. 2b). While the latency of this negative peak in this subject is faster than several prior publications (Allison et al., 1994; Puce et al., 1999) it is likely the same ERP, as there are interindividual differences in response latencies.
Importantly, the spatial localization of face-selective ECoG responses on the FG showed a striking anatomical correspondence with face-selectivity measured with fMRI (Fig. 1b,c). In line with our recent fMRI findings in healthy individuals (Weiner and Grill-Spector, 2010, 2011, 2012), high-resolution fMRI data in this subject revealed anatomically distinct face-selective regions on the lateral FG with separate functional boundaries. These clusters showed the typical topological relationship to nearby retinotopic regions where pFus-faces is located lateral to the ventral occipital (VO) visual field map cluster and mFus-faces is located ∼1 cm more anterior to both pFus-faces and the VO cluster (Fig. 1b). Notably, the location of electrodes 1 and 2 overlapped with the anatomical location of fMRI-localized pFus-faces and mFus-faces, respectively (Fig. 1b,c). Indeed, extracting fMRI signals from the gray matter under these electrodes revealed a response to faces nearly twice as strong as that elicited by other categories (Fig. 2c). Thus, converging evidence from ECoG and fMRI measurements illustrate robust face selectivity in these two cortical sites along the FG.
EBS and distortion of conscious visual face perception
Multimedia material online (Movie 1) shows the video of the EBS procedure when the subject was viewing real faces. The subject described vivid distortions during the perception of real faces when electrical charge was delivered through FG electrodes 1 and 2 (7 trials), but not during sham stimulations of the same electrodes (4 trials) or EBS through other pairs of nearby electrodes (3 trials). When EBS was applied through FG electrodes 1 and 2 while looking at the examiner's face, the subject described the striking nature of his visual distortion: “You just turned into somebody else. Your face metamorphosed.” When probed further, he reported that features appeared distorted: “You almost look like somebody I've seen before, but somebody different. That was a trip…. It's almost like the shape of your face, your features drooped” (Movie 1). In subsequent discussion post-EBS, the subject reiterated that the face did not morph into an intact face of someone else, but rather it became distorted.
Electrical stimulation of the fusiform face-selective regions in a patient implanted with intracranial electrodes. This video file contains portions of the electrical brain stimulation procedure where the patient reports metamorphosis of the real faces in the room only when electrical charge is delivered to mFus- and pFus-faces (FFA-2 and FFA-1, respectively).
Importantly, similar perceptual deficits were not elicited when the subject was viewing objects in the room, such as a TV (1 trial) or reading words written on a balloon (1 trial) at the same distance. It is important to note that the subject reported small changes in vision when viewing these nonface objects and words, but he could not describe them fully and these were unspecific to particular stimuli. In contrast, the selective distortion to real faces was striking. For instance, when probed by the examiner to determine the appearance of anything else changed, the subject reported: “Only your face changed. Everything else was the same.” Furthermore, the subject reported similar distortions when viewing another person's face: “The bottom of her face sort of metamorphosed up. Kind of stretched up to give her a different look. Um… it wasn't pretty” (Movie 1).
A series of controls illustrated the specificity of the perceptual deficit to perception of faces. First, stimulating a pair of nearby electrodes (Fig. 1a, electrodes with dotted white circles) did not cause any change in his perception of faces in the room (3 trials). Second, EBS of FG electrodes 1 and 2 did not induce selective impairment during naming of famous faces. Specifically, naming photographs of famous people was incorrect in 1/5 and 2/5 occasions during real and sham conditions, respectively, while naming of photographs of famous places/monuments was incorrect in 2/5 and 0/5 occasions during real and sham conditions, respectively. In a similar naming experiment, while stimulating the pair of control electrodes, the subject named correctly all face (10/10) and nonface (10/10) photographs even though during real (but not sham trials) he reported a nonspecific visual sensation (“pictures were a little bit rough, lines on them? Wouldn't have been there if I saw them on a newspaper”).
The subjective perceptual change reported by the patient in the present study clearly suggests that electrical perturbation of human pFus-faces and mFus-faces (also referred to as FFA-1 and FFA-2, respectively; Pinsk et al., 2009) leads to selective distortion during the conscious visual perception of real faces. These findings provide evidence for the causal role of these fusiform face-selective regions in face perception, in agreement with studies in typical populations showing correlations between fMRI responses in these regions and behavioral measures of face perception (Tong et al., 1998; Hasson et al., 2001; Moutoussis and Zeki, 2002; Grill-Spector et al., 2004; Rotshtein et al., 2005).
As shown in the multimedia material online (Movie 1), the specificity of perceptual distortions in the appearance of real faces following electrical perturbation of mFus-faces and pFus-faces is particularly striking given that both ECoG and fMRI data in these regions show above baseline responses to nonface stimuli. Further, the disruption in neural activity caused by electrical brain stimulation occurs by driving current into a large neural population under each electrode with an estimate of ∼500,000 neurons under the 2.3 mm diameter space of each electrode (Pakkenberg and Gundersen, 1997). This likely affects local responses, as well as distal cortical and subcortical sites via propagation of current along the afferent and efferent axons of this neuronal population. The latter effect may explain why the subject had some nonspecific visual changes when viewing the TV and the balloon (perhaps by antidromical stimulation driving current to early visual areas). Nevertheless, the specificity of our effects showing pronounced distortion in the appearance of real faces, but not to other real objects, or when stimulating nearby electrodes, suggests the existence of a specific neural circuit involved in visual perception of faces. This neural circuit may be implemented locally within pFus or mFus face-selective regions, or may include these regions as nodes in a more extended face-processing network. Indeed, a recent preliminary report suggests that EBS of a more lateral and posterior face-selective region located on the inferior occipital gyrus may also produce distorted perception of faces (Jonas et al., 2012). Future studies with methods affecting only local neuronal responses (e.g., by cooling electrodes) will provide additional evidence of the contribution of local neural responses within these FG sites during the conscious perception of faces.
While our results show that EBS of face-selective FG electrodes produce perceptual distortions during face viewing, two prior studies reported that EBS of FG electrodes impaired naming of famous faces (Allison et al., 1994; Puce et al., 1999). Differences across studies may be due to differences in the direction of stimulation across cortex, and differences in the cortical sites that were stimulated. Related to the first point, studies that showed naming deficits stimulated electrode pairs along a medial-to-lateral axis, whereas the present study stimulated along a posterior-to-anterior axis. Since there is evidence that the direction of charge delivery is aligned to the axons that are most perturbed (Ranck, 1975), it is possible that the spread of the current differs under medial-lateral vs posterior-anterior stimulation, thereby producing differential behavioral consequences. Related to the second point, it is possible that previous investigators stimulated different FG sites than those in the present study. Indeed, the present data benefit from recent methodological advancements in the anatomical localization of fMRI blood oxygenation level-dependent (BOLD) and subdural recording sites on the patient's cortical surface, which were not available at the time of the prior studies.
As grid placements on the FG are rare, the present findings propose a clear method and set of questions to guide future research. First, a finding of the present study is that EBS to pFus-faces and mFus-faces induced much more pronounced and specific distortions when viewing real faces as opposed to impairments in viewing or naming famous faces. Second, it is unknown whether the stimulation of pFus-faces and mFus-faces together is necessary to induce perceptual distortions of faces or whether the stimulation of only one region (while also considering the axis of stimulation) is sufficient to induce a comparable perceptual deficit. We were unable to probe these questions further due to safety considerations limiting the number of EBS trials. Although our EBS findings clearly suggest face-selective perceptual processing in the FG, we are mindful that they do not explain the exact nature of such processing. Future multimodal approaches with comparable rigor in the localization of EBS sites are needed to establish the functional specificity of the EBS effect by testing a larger number of trials on a broader range of stimuli, to clarify the kind of face processing that occurs in the FG, and to explain why EBS to these regions disrupts processing of real faces, but does not cause similar deficits in naming photographs of faces.
Our finding of spatial overlap between fMRI BOLD and ECoG broadband power is in accord with our recent study of human area MT (Rauschecker et al., 2011) and with a large body of evidence in the literature suggesting that changes across the broadband power are spatially restricted (Flinker et al., 2011) and correlated with both unit firing rate (Ray et al., 2008; Manning et al., 2009), and fMRI responses (Hermes et al., 2012). Importantly, we underscore the utility of using high-resolution fMRI and multiple fMRI measurements of category-selectivity and retinotopy to determine precisely which cortical sites are stimulated. Increased fMRI resolution improves the localization of activations and decreases the spatial extent of susceptibility artifacts, such as those produced by the ear canal. Obtaining complementary fMRI measurements and aligning data to the subject's native anatomy improves the precision of localization by taking into consideration the topological relation among activations as well as their anatomical location. This improved precision is especially significant for clinical applications where the identification of activations needs to be determined on an individual subject basis and with the best precision possible.
In sum, we show the causal role of FG face-selective regions in face perception. Our multimodal approach illustrates the convergence of three modalities of neuroscience research: ECoG, fMRI, and EBS. Such an approach provides precise information and a unique method to understand the causal role of specific cortical sites in perception and to link EBS data to a plethora of results from noninvasive imaging studies in humans.
This research was funded by Stanford NeuroVentures Program and NIH Grant R01 NS078396-01 to J.P.; the Belgian Fund for Scientific Research (FNRS) to C.J., and National Science Foundation Grants BCS 0920865 and R01 EY019279-01A1 to K.G.-S. We thank the patient for his involvement in the study; Stanford Epilepsy Monitoring Unit staff for help with electrophysiological recordings; and Jon Winawer for collecting and analyzing the retinotopy data.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Josef Parvizi, Laboratory of Behavioral and Cognitive Neurology, Department of Neurology and Neurological Sciences, 300 Pasteur Drive, Stanford, CA 94305.
This article is freely available online through the J Neurosci Open Choice option.