Abstract
Neuroimaging and electrophysiological studies across species have confirmed bilateral face-selective responses in the ventral temporal cortex (VTC) and prosopagnosia is reported in patients with lesions in the VTC including the fusiform gyrus (FG). As imaging and electrophysiological studies provide correlative evidence, and brain lesions often comprise both white and gray matter structures beyond the FG, we designed the current study to explore the link between face-related electrophysiological responses in the FG and the causal effects of electrical stimulation of the left or right FG in face perception. We used a combination of electrocorticography (ECoG) and electrical brain stimulation (EBS) in 10 human subjects implanted with intracranial electrodes in either the left (5 participants, 30 FG sites) or right (5 participants, 26 FG sites) hemispheres. We identified FG sites with face-selective ECoG responses, and recorded perceptual reports during EBS of these sites. In line with existing literature, face-selective ECoG responses were present in both left and right FG sites. However, when the same sites were stimulated, we observed a striking difference between hemispheres. Only EBS of the right FG caused changes in the conscious perception of faces, whereas EBS of strongly face-selective regions in the left FG produced non-face-related visual changes, such as phosphenes. This study examines the relationship between correlative versus causal nature of ECoG and EBS, respectively, and provides important insight into the differential roles of the right versus left FG in conscious face perception.
Introduction
Electrophysiological evidence in humans (McCarthy et al., 1999; Puce et al., 1999; Murphey et al., 2009; Davidesco et al., 2013) and nonhuman primates (Afraz et al., 2006; Tsao et al., 2008) along with a large body of evidence from neuroimaging studies (Sergent et al., 1992; Kanwisher et al., 1997; Golarai et al., 2010; Weiner and Grill-Spector, 2013) suggest the bilateral presence of face-selective regions within the ventral temporal cortex (VTC) including the fusiform gyrus (FG). Functional magnetic resonance imaging (fMRI) studies of the FG have also identified a positive correlation between the strength of fMRI responses and the successful perception or identification of faces (Kanwisher et al., 1997; Tong et al., 1998; Moutoussis and Zeki, 2002; Grill-Spector et al., 2004) without a clear lateralization effect—although some imaging studies have reported stronger responses in the right, compared with the left FG (for review, see Rossion et al., 2000) and behavioral studies have shown a right hemisphere advantage in face recognition (Rhodes, 1993). Intracranial electrophysiological studies in humans have reported face-selective FG responses (Allison et al., 1994a,b; Puce et al., 1999; Murphey et al., 2009; Parvizi et al., 2012) that are sensitive to stimulus properties (Davidesco et al., 2014) and attention (Engell and McCarthy, 2011; Davidesco et al., 2013) without clearly addressing the issue of lateralization.
Because imaging and electrophysiological studies use correlative measurements, evidence from lesion studies has been used to probe whether the FG is causally involved in face perception. Specifically, bilateral (Meadows, 1974; Damasio et al., 1982; Rossion et al., 2003) or only right (De Renzi, 1986; De Renzi et al., 1994) lesions in the VTC result in perceptual deficits in face processing. Although lesion studies in humans have provided invaluable causal evidence for the importance of VTC (especially in the right hemisphere) in face recognition (Barton et al., 2002; de Gelder et al., 2003; Fox et al., 2011), it is often unclear the extent to which the clinical deficits are due to the compromise of the FG itself or the adjacent cortical gray or white matter structures included in the lesions.
In the present study, we used electrocorticography (ECoG) combined with electrical brain stimulation (EBS) to confirm the presence of face-selective electrophysiological activity in the right and left FG, and determine the perceptual distortions when face-selective and nonselective FG sites were electrically perturbed during a clinical EBS procedure. We also aimed to investigate whether the degree of face selectivity in the VTC, measured in terms of high-frequency broadband (HFB) activity or event related potentials (ERP), could predict perceptual changes caused by EBS. Studies with combined ECoG and EBS in the VTC are extremely rare and limited to clinical settings (Selimbeyoglu and Parvizi, 2010). The stimulation reported here was a clinical functional mapping procedure that was performed at the patients' bedside, and therefore, was solely dictated by its clinical nature (e.g., a limited number of EBS trials and the use of real-life faces present at the bedside).
Materials and Methods
Participants.
Ten subjects (3 female) were implanted with subdural intracranial electrodes (AdTech) over the right (5 subjects) or left (5 subjects) VTC. In half the subjects Full-4 IQ were measured; the mean Full-4 IQ was 112 for right hemisphere (RH; 3 of 5) and 86.5 for left hemisphere (LH; 2 of 5) participants (all subjects completed high school or some college level education). All RH and 4 of 5 LH participants were right-handed. The single left-handed subject had LH language dominance. Electrode implantation was performed at Stanford Medical Center to localize the source of seizures in subjects with intractable epilepsy. The location of electrodes was determined solely by clinical needs. All participants gave their written informed consent to participate in our experiments approved by the Stanford University IRB. None of the participating subjects had a clinical seizure focus within the FG as determined by intracranial monitoring, but electrodes were implanted in this area to monitor seizure propagation and for surgical planning.
Electrode localization and anatomical classification.
Preimplant high-resolution anatomical MRIs were acquired on a GE Discovery MR750, 3-tesla scanner. The whole-brain scans were acquired as 0.9 mm axial slices using a T1-weighted SPGR sequence and data were resampled to 1 mm isotropic voxels. Postimplant computed tomography (CT) images were coregistered to the preimplant anatomical MRIs to visualize electrode locations and account for surgical brain shift (Hermes et al., 2010). Fused single-subject CT/MRIs were used to isolate electrodes falling within the FG; in total, 56 electrodes were located within this neuroanatomical region-of-interest (26 RH and 30 LH electrodes) with all participants having an average (± SD) of 5.11 (± 3.51) electrodes in this region.
Experimental task design.
To identify face-selective areas within the FG, participants were presented with one of two tasks during which ECoG data were recorded. In Task 1, grayscale images of faces, houses, cars, and limbs were presented for 1000 ms each with a variable interstimulus interval of 600–1400 ms. Subjects fixated on a central black cross and pressed a key when its color changed to red. In Task 2, rows of either six faces, six digit numbers, six letter words, six scrambled or foreign symbols were presented in a 12 s block design using a 700 ms stimulus duration and a 300 ms interstimulus interval and a 12 s interblock rest interval. The stimuli were also grayscale and restricted to the horizontal meridian to control for low-level differences among faces, words, and numbers. Participants were asked to fixate on a dot centered on the screen and to press a button when it changed color. Two LH and two RH subjects performed Task 1 (subjects S2, S4, S8, S10), three LH and two RH subjects performed Task 2 (subjects S1, S3, S6, S7, S9). One RH subject (S5) did not undergo ECoG testing, and is excluded from task analyses. Data from S2 were previously reported (Parvizi et al., 2012).
ECoG data acquisition and analysis.
Subdural electrode signals were recorded at 3052 Hz sampling rate referenced to a selected electrographically silent intracranial electrode using a Tucker Davis Technologies multichannel recording system. S4 signals were recorded at 1526 Hz sampling rate. Data were digitally bandpass filtered from 0.5 to 300 Hz at the time of acquisition. Before data processing, pathological channels identified by the physician (J.P.) and artifactual channels with greater than five times the mean variance were excluded. The remaining data were notch filtered at 120 Hz and re-referenced to a common average of all remaining implanted channels (mean: 98 ± 16.69 channels/subject) to eliminate shared neural noise. Data were then down-sampled to 1000 Hz to reduce computational load and bandpass filtered between 70 and 150 Hz in 5 Hz nonoverlapping bins (e.g., 70–75, 75–80… 145–150). A Hilbert transform was then applied to each of the bands between 70 and 150 Hz to obtain an estimate of the band-limited power (envelope squared). A log transform was applied to each power time series and the mean for the entire log power time series was subtracted. All transformed power times series were then averaged and downsampled to 100 Hz. These steps result in a time series of HFB activity (70–150 Hz) for each electrode that has been corrected for the 1/frequency power decay over the 70–150 Hz range. The HFB range studied could have been defined across a wider frequency range, but our prior observations have shown similar response profiles across different HFB bands [e.g., 70–180 Hz (Foster et al., 2013), 40–180 Hz (Parvizi et al., 2012), 70–110 Hz (Dastjerdi et al., 2013)]. For each electrode, HFB power was normalized (Z-scored) with respect to the average of the 150 ms prestimulus baseline periods by category. All analysis was done using custom MATLAB (MathWorks) routines.
Using the mean HFB power over the 100–350 ms window after stimulus onset, paired two-sided t tests were calculated to determine which electrodes were statistically face-selective (p < 0.01; see Fig. 2). A false discovery rate (FDR; Benjamini-Hochberg procedure; Benjamini et al., 2001) multiple-comparison correction was applied (α = 0.05). The same mean value was used to calculate the selectivity index (d′; see Fig. 3). d′ was calculated using the following formula: where A = mean HFB for faces, B = mean HFB for non-faces, and σ2 = variance.
ERP analysis.
Before ERP processing, channels containing pathological and artifactual data were excluded as previously described. The remaining data were notch filtered at 60 Hz, downsampled to 100 Hz for computational load, and baseline corrected by subtracting the mean voltage for 100 ms prestimulus. To define sites with an N200 ERP, the average ERP, for all face and non-face conditions grouped together, was calculated. N200 ERPs were defined as electrodes in which the mean ERP had a large negative deflection between 100 and 250 ms (Allison et al., 1994b; Rosburg et al., 2010; Rossion and Jacques, 2011). Based on this average ERP, the latency (Lerp) was calculated as the largest negative deflection in this time window. The Lerp ranged from 140 to 210 ms after the stimulus presentation. We subsequently calculated the ERP amplitude (Aerp,t) for each trial by taking the mean amplitude between Lerp−20 ms:Lerp+20 ms (Luck, 2005). In the same manner as the HFB face selectivity, the Aerp,t measurement was used to calculate the selectivity index (d′): where A = mean ERP for faces, B = mean ERP for non-faces, and σ2 = variance.
EBS.
In a clinical paradigm, electrical current was delivered directly to a brain region in alert subjects to identify cortical areas whose stimulation causes typical seizure sensations, thereby mapping the location of potential seizure zones (Selimbeyoglu and Parvizi, 2010). EBS was also used to map the function of cortical areas near or within the planned resection area. During stimulation, a symmetric bipolar square wave was delivered between two adjacent electrodes on the inferior cortical surface. Only one of two electrodes needed to be in the FG to be included in the study. Real trials (2–8 mA), interwoven with sham control trials (0 mA), used a 200 μs pulse-width at a frequency of 50 Hz for 1–3 s durations. Subjects were asked to recount any perceptual effects or changes they experienced during or after each trial while directed to attend to a specific person, face, object, or sensation. No computer stimuli were used. Instead, all faces and objects were nonmoving natural stimuli in the room. The participants' subjective reports were classified into three categories: (1) stimulations that caused face-related perceptual changes, (2) non-face-related visual changes, and (3) no perceptual change. Independent reviewers transcribed and time-stamped video record-ings of the EBS procedure for all ventral sites and classified the participant responses. Simultaneous ECoG was monitored for the presence of after-discharges, seizures or meningeal stimulation and any such trials were excluded from behavioral analysis. The instructions given to each participant were the same for the LH and RH cohorts, and the same physician (J.P.) performed the procedure in all participants.
Results
Data from 10 subjects undergoing invasive intracranial monitoring for intractable epilepsy were included in the study. Subjects had unilateral intracranial electrodes either in the RH (5 subjects) or the LH (5 subjects). Due to routine clinical procedures, no subject had bilateral coverage with subdural electrodes. Utilizing the anatomical boundaries of the FG (Fig. 1a), 56 electrodes (26 right, 30 left) were included in our analysis.
HFB activity was used as a measure of neuronal population activity during the visual presentation of faces, houses, words, and other stimuli (Fig. 1b). Thirteen RH and 12 LH face-selective electrodes, with significantly (p < 0.01) greater HFB responses to faces compared with all other conditions, were identified in nine participants. One participant did not perform either ECoG task. We calculated the strength of face selectivity in each electrode and visualized the spatial distribution of face responses in individual and normalized brain space across all participants, corrected for electrode density (Fig. 2a). Sorted t statistic values revealed highly face-selective sites within the FG in both hemispheres (Fig. 2b).
Next, we surveyed the effects of EBS in the FG as participants were instructed to look at faces and objects and describe any perceptual changes they experienced. Participants were not prompted to pay attention to any specific perceptual changes. The stimulation was controlled in three ways by verifying: (1) the same perceptual change with repeated stimulations, (2) a lack of similar perceptual changes with sham stimulations, and (3) by comparing the EBS effect when the subject was instructed to look at face and non-face objects. All participants underwent EBS and all but one participant (S10) reported some perceptual change (Fig. 3a). Our findings suggest a striking lateralization effect in the stimulation responses. All electrodes that produced a face-related perceptual change were in the right FG (Fig. 3a); these electrodes were found in all 5 RH participants (Fig. 4 shows single subject results). No face-related perceptual change was seen with EBS in the left FG sites, even though the left FG was sampled similarly to the right (30 sites on the left and 26 on the right).
All five RH participants described face-related perceptual changes when instructed to look directly at a face and gave the following self-reports (Table 1). For instance, S1 reported that the face changed as if “you weren't who you were.” S2 explained, “You just turned into somebody else, your face metamorphosed. Your nose got saggy and went to the left.” S3 mentioned, “maybe the shape of your face was different. It wasn't wrong, it was more defined, more masculine I guess… nose looked different, larger.” S4 described, “the middle of the eyes twist…chin looks droopy.” S5 said, “Something perceivable about your face. It was almost like you were a cat.” Contrastingly, EBS of the left FG caused non-face-related visual changes such as twinkling and sparkling (S6), blue and white balls traveling up and down (S7), lights in the participant's right eye (S8), and lights crossing the participant's vision (S9). S10 (LH) described no perceptual changes from EBS (Fig. 3). Repeated-stimulation trials produced almost identical perceptual changes, whereas sham trials (current set to 0 mA) did not produce the same distortions.
To compare face selectivity with EBS results, we clustered the face selectivity d′ index by stimulation effect (Fig. 3c). A Wilcoxon rank sum test between the HFB d′ seen in the stimulation sites with face-related changes (red) versus those that did not induce such changes (blue and yellow) showed a significant effect (p = 0.006). Additionally, a small but significant effect was observed between the d′ values in sites eliciting a face-related change (red) versus those with nonspecific visual (yellow, p = 0.0298) or no perceptual changes (blue, p = 0.0121; Fig. 3c). Thus, on average, sites with larger HFB d′ were more likely to produce a face-related EBS effect.
ERP analysis by channel showed that not all FG sites produced the N200 (Fig. 3d) associated with face selectivity. Of those electrodes that showed the N200, large negative face-selective deflections were observed bilaterally (Fig. 3e). However, there was no significant difference (p = 0.2841, Wilcoxon rank sum test) between the ERP face selectivity index (d′) measured in the stimulation sites with face-related changes (red) versus those that did not induce such changes (blue and yellow). Additionally, there was no significant difference between the ERP d′ values in sites eliciting a face-related change (red) versus those with nonspecific visual (yellow, p = 0.3277) or no perceptual changes (blue, p = 0.3681; Fig. 3f). Furthermore, the ERP effect sizes (measured with d′) in our findings were smaller than the HFB effect sizes (HFB max: 5.3254; ERP max: 1.7174). Overall, there was no significant relationship between the effect of EBS and the ERP face selectivity (Fig. 3f).
In summary, the most striking result from our data were the effect of hemisphere in inducing face-related perceptual distortions. As noted, all electrodes whose stimulation caused distortions in face perception were located in the right FG. Although these electrodes had, on average, higher HFB face selectivity, not all of these sites showed task-related face selectivity in ECoG as indicated by red electrodes without a white halo in Figure 3a (t test, p < 0.01; FDR corrected for all electrodes).
Discussion
In the present study, we examined how electrical stimulation of face-selective or non-face-selective neural populations in the right versus left FG influences face perception. This approach revealed three key findings. First, both the left and right FG contained face-selective electrophysiological responses measured with HFB and ERP. Second, EBS of the left and right FG had strikingly different effects on human face perception: EBS of the right FG caused distortions of faces, while EBS of the left FG led to nonspecific visual changes such as color distortion or flashing phosphenes. Third, at the single electrode level, the face-related perceptual changes induced by EBS could not be predicted by the degree of face selectivity. However, at the group level, sites whose stimulation caused face-related changes were, on average, more face-selective. In the sections below, we discuss these findings in more detail.
Varied EBS effects in the right and left FG
Our EBS results indicate a clear functional dissociation between FG face-selective sites across hemispheres. We documented perceptual deficits in real face perception with multiple right, but not left, FG stimulations across several individual subjects. Our findings show that stimulation of ECoG face-selective sites in the right FG often caused a clear distortion in face perception whereas the stimulation of face-selective electrodes in the left hemisphere did not cause face-distortions. Our findings are clearly in agreement with past behavioral studies that have shown a RH advantage in face recognition (Rhodes, 1993) and clinical reports that have supported a predominance of RH lesions in patients with prosopagnosia (De Renzi, 1986, 1994; Landis et al., 1988; Sergent and Signoret, 1992; Wada and Yamamoto, 2001; Barton et al., 2002; de Gelder et al., 2003; Barton, 2008; Busigny et al., 2010; Fox et al., 2011). However, our study is the first of its kind to document perceptual deficits in real face perception with multiple right, but not left, FG stimulations across several individual subjects. It should be noted that transcranial magnetic stimulation (TMS) has been used in the past to successfully disrupt face perception (Pitcher et al., 2007, 2012) by targeting the more accessible cortical tissue on the inferior occipital gyrus. However, TMS has thus far been unable to access the deeper structures such as the FG, which makes the current dataset quite distinctive. It is also noteworthy that in two of our subjects, S3 and S4, EBS of FG sites produced face hallucinations. This is in line with the previous reports in which partial or whole-face hallucinations are reported during electrical stimulation (Penfield and Perot, 1963; Puce et al., 1999; Jonas et al., 2014).
We are mindful that there are differences between IQ measures for our LH and RH cohorts. Although this issue is important, we believe that IQ discrepancy cannot explain the lateralized EBS effect for the following reasons. First, IQ measurements in our cohort were obtained in only half the subjects (3 of 5 RH and 2 of 5 LH subjects). Therefore, the apparent IQ disparity may be an artifact of the extremely small (and unrepresentative) sample sizes. Second, all five LH subjects completed at least high school, two of them had some college level education and none had difficulties describing their normal perceptions or EBS induced perceptual changes within the FG (Table 1) or in other cortical regions. Third, there were strongly face-selective electrodes in all LH cases, regardless of IQ.
Metrics of face selectivity
One key finding of our study is that face-selective responses are seen bilaterally and on average, RH electrodes, which caused EBS effects were more face-selective. As a measure for selectivity of electrophysiological response, many studies have previously relied on ERP. In our intracranial study, we focused on face selectivity measured with HFB power. Unlike ERP, HFB does not depend on the presence of precisely time-locked responses, is not influenced heavily by the power of ongoing slow oscillations, and provides a regionally invariant measurement of electrocortical activation (Miller et al., 2014) which is highly correlated with fMRI BOLD responses and reflects local neuronal population activity (Logothetis et al., 2001; Ray and Maunsell, 2011; Hermes et al., 2012). Given these differences, ERP and HFB measures may not always be in agreement. For instance, Engell and McCarthy (2011) showed dissociation between the ERP and the gamma power index of selectivity, and that gamma activity was sensitive to attentional manipulations, whereas ERP activity was not (Engell and McCarthy, 2010; Davidesco et al., 2013). Consistent with these findings, not all sites with face-induced HFB responses in our study produced N200 ERP effects and vice versa (Fig. 3). We found that the HFB effect sizes (d′) were substantially larger than the ERP effects (HFB max: 5.3254; ERP max: 1.7174) and thereby may be a more sensitive measure for discriminating between face-selective and non-face-selective responses.
RH lateralization of face perception
A recent hypothesis suggests that the functional lateralization of face perception to the right VTC may occur as a byproduct of language development, which co-opts the left VTC region (Allison et al., 2002; Dehaene et al., 2010; Dundas et al., 2013). Consistent with this hypothesis, causal evidence for the lateralization of FG functions has not been seen in nonhuman primates who presumably do not have the same language faculties as humans. For instance, in a study by Afraz et al. (2006) using intracortical recordings in two rhesus monkeys, one left and one right hemisphere, the degree of face selectivity predicted the effect of microstimulation regardless of the hemisphere. In a series of studies, Tsao et al. have also found evidence for nonlateralized microstimulation effects in the rhesus inferior temporal cortex. For instance, they stimulated clusters of face-selective neurons in either the left or the right hemisphere of the same monkey, and have found significant effects on identity discrimination in both hemispheres (D. Tsao, personal communication). Thus, it is reasonable to postulate that in the course of human brain evolution, along with the lateralization of language functions to the LH, face-selective regions within the left FG may have acquired language-related or semantic functions in face processing (e.g., retrieving names of faces rather than visual perception of whole faces). Although future systematic studies are needed to test this hypothesis, a few studies in humans have already reported deficits in face-naming with the stimulation of left FG sites (Allison et al., 1994a; Puce et al., 1999) which have been segregated from perceptual deficits in the RH (Puce et al., 1999; Parvizi et al., 2012). Thus, had we used a face-naming paradigm, it is possible that we could have found significant face-naming deficits during EBS of both right and left FG sites; RH naming deficits may, or may not (Puce et al., 1999), be accompanied by perceptual changes in the faces presented and LH naming deficits may result from disrupting semantic functions without causing associated face distortions. Additionally, the effects of stimulation may be task and instruction-dependent resulting in naming deficits when asked to name a face image on a screen (Allison et al., 1994a; Puce et al., 1999), hallucinations when looking at a blank surface (Penfield and Perot, 1963; Jonas et al., 2014), or face distortions when looking at a real face (Parvizi et al., 2012). As some have argued that face-selective regions in the FG consist of several different nodes, all involved in various levels of face perception (Tsao et al., 2008; Weiner and Grill-Spector, 2013), it is possible that our stimulation paradigm disrupted a specific portion of this network, resulting in particular facial distortions (e.g., “I think your face got rounder”) while maintaining other network features (e.g., face-naming) which may reside in segregated, yet partially overlapping, neural regions.
In future studies, subjective reports under naturalistic viewing conditions should be complemented by behavioral measures during the categorization of carefully controlled images. While the latter allows for a comparison across species using comparable methodologies, the former enables us to probe face perception in a manner unique to humans. Moving forward, both approaches are critical in assessing the causal role of functionally specialized VTC regions in human perception.
Footnotes
This work was supported by Stanford NeuroVentures Program, R01 NS078396-01 and BCS1358907 to J.P., NSF BCS0920865 NIH 1 RO1 EY 02231801A1 to K.G.S., and Belgian Fund for Scientific Research and Belgian Federal Science Policy Office to C.J. We thank Dr Gayle Deutsch for neuropsychological evaluations of the patients; Drs Larry Shuer and Hong Yu for neurosurgical implantation of the electrodes; Mark Burdelle, Harinder Kaur, Thi Pham, and Liudmila Schumacher for assistance and collaboration on the ECoG and EBS procedures; Charlotte Zejilon and Nikita Desai for help with data screening; Sandra Gattas for help with data collection; and Nathan Witthoft for coding ECoG Task 1.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr Josef Parvizi, Laboratory of Behavioral and Cognitive Neurology, Stanford Human Intracranial Cognitive Electrophysiology Program, Department of Neurology and Neurological Sciences, 300 Pasteur Drive, Stanford, CA 94305. jparvizi{at}stanford.edu