Abstract
Neuroimaging studies have identified a face-selective region in the right posterior superior temporal sulcus (rpSTS) that responds more strongly during facial expression recognition tasks than during facial identity recognition tasks, but precisely when the rpSTS begins to causally contribute to expression recognition is unclear. The present study addressed this issue using transcranial magnetic stimulation (TMS). In Experiment 1, repetitive TMS delivered over the rpSTS of human participants, at a frequency of 10 Hz for 500 ms, selectively impaired a facial expression task but had no effect on a matched facial identity task. In Experiment 2, participants performed the expression task only while double-pulse TMS (dTMS) was delivered over the rpSTS or over the right occipital face area (rOFA), a face-selective region in lateral occipital cortex, at different latencies up to 210 ms after stimulus onset. Task performance was selectively impaired when dTMS was delivered over the rpSTS at 60–100 ms and 100–140 ms. dTMS delivered over the rOFA impaired task performance at 60–100 ms only. These results demonstrate that the rpSTS causally contributes to expression recognition and that it does so over a longer time-scale than the rOFA. This difference in the length of the TMS induced impairment between the rpSTS and the rOFA suggests that the neural computations that contribute to facial expression recognition in each region are functionally distinct.
Introduction
A face is a rich source of social information that simultaneously conveys someone's identity, attentional focus, and emotional state. Our brains are so efficient that, to us, processing this wealth of information seems effortless. Yet the simplest functions, such as recognizing your mother or judging her mood, require the interaction of multiple specialized brain regions (Bruce and Young, 1986; Haxby et al., 2000; Adolphs, 2002; Calder and Young, 2005). The locations of these regions are well established, but our understanding of the functions performed in each region and how these regions interact to recognize faces is limited.
One successful approach has been to use fMRI to establish how each face-selective region selectively responds to different facial aspects, such as unique identity, facial expression, or eye gaze (Hoffman and Haxby, 2000; Winston et al., 2003; Andrews and Ewbank, 2004). Such studies have identified a region in the right posterior superior temporal sulcus (rpSTS) that exhibits a greater response during facial expression recognition tasks than during facial identity recognition tasks (Puce et al., 1998; Hoffman and Haxby, 2000). Although this finding has proven to be robust (for review, see Allison et al., 2000; Nummenmaa and Calder, 2009), it is important to note that there is no human neuropsychological evidence that the rpSTS is causally essential for accurately discriminating facial expressions. The present study addressed this lack of causal evidence by selectively disrupting the rpSTS with transcranial magnetic stimulation (TMS) while healthy human participants performed facial expression and facial identity recognition tasks.
Prior TMS studies have demonstrated that the rpSTS is causally engaged in eye gaze discrimination (Pourtois et al., 2004), the McGurk effect (Beauchamp et al., 2010), and in judgments of facial trustworthiness (Dzhelyova et al., 2011), but no TMS studies of the rpSTS have reported performance deficits on a facial expression task. In the present study, participants performed delayed match-to-sample facial expression and facial identity tasks while repetitive TMS was delivered, at a frequency of 10 Hz for 500 ms, over the functionally localized rpSTS or over the vertex control site. In Experiment 2, double pulse TMS (dTMS), separated by 40 ms, was delivered over the rpSTS and over the right occipital face area (rOFA) at different latencies after stimulus onset while participants performed the expression task only. A prior study (Pitcher et al., 2008) demonstrated that dTMS delivered over the OFA (Gauthier et al., 2000), a face-selective region in the inferior occipital gyrus, impairs facial expression recognition when delivered at 60–100 ms after stimulus onset. Comparing the onset and duration of the latency that TMS impairs the rpSTS with the rOFA is a novel method for understanding when category-selective brain regions are functionally active in relation to each other (Pitcher et al., 2012). Such temporal information can be used to better understand the functional connections between regions and the computational differences across different components of the cortical face network (Haxby et al., 2000; Calder and Young, 2005; Pitcher et al., 2011b).
Materials and Methods
Participants.
Fourteen right-handed participants with normal or corrected-to-normal vision (8 males, 6 females, age 22–51 years) gave informed consent as directed by the University College London ethics committee. Ten participated in Experiment 1, and 12 participated in Experiment 2. Eight participated in both experiments.
Brain imaging.
An fMRI localizer using dynamic and static face and object stimuli was used to individually identify the TMS target sites (rOFA and rpSTS) in each participant (Pitcher et al., 2011a). Whole-brain imaging was performed on a Siemens 1.5 Tesla MR scanner using a 32-channel head coil at the Birkbeck-UCL Neuroimaging Centre. Functional data were acquired over 4 blocked-design functional runs, each lasting 234 s with a gradient-echo EPI sequence (TR = 2000 ms; TE = 50 ms, FOV = 192 × 192, matrix = 64 × 64) giving a notion resolution of 3 × 3 × 3 mm. In addition, a high-resolution anatomical scan was acquired (T1-weighted FLASH, TR = 12 ms; TE = 5.6 ms; 1 mm3 resolution) for anatomically localizing activations and to accurately target TMS stimulation sites in each participant using a frameless stereotaxic system (BrainSight, Rogue Research). Each functional run contained two sets of five consecutive stimulus blocks (faces, bodies, scenes, objects, or scrambled objects) sandwiched between rest blocks, to make two blocks per stimulus category per run. Each of the five stimulus categories was presented as dynamic 3 s movies or as static 1 s static images taken from the beginning, middle, and end of each movie. Each block lasted 18 s and contained stimuli from one of the five stimulus categories.
Functional imaging data were analyzed using FSL (www.fmrib.ox.ac.uk/fsl). After deleting the first three volumes of each run to allow for T1 equilibrium, the functional images were realigned to correct for small head movements (Jenkinson et al., 2002). The images were then smoothed with a 5 mm FWHM Gaussian filter and prewhitened to remove temporal autocorrelation (Woolrich et al., 2001). The resulting images were entered into a participant-specific GLM with five conditions of interest corresponding to the five categories of visual stimuli. Blocks were convolved with a double γ “canonical HRF” (Glover, 1999) to generate the main regressors. In addition, the estimated motion parameters were entered in as covariates of no interest to reduce structured noise due to minor head motion.
Significance maps of the brain were computed using the same statistical threshold for each TMS target site (p = 10−3, uncorrected). Both sites (rOFA and rpSTS) were identified using a contrast of dynamic and static faces greater than dynamic and static objects. The rpSTS was located in the posterior region of the superior temporal sulcus in all participants (group mean MNI coordinates = 51, −70, 2). The rOFA was located on the lateral surface of the occipital lobe in all participants (group mean MNI coordinates = 42, −79, −10). The coordinates and strength of the peak responses varied across participants, but the rOFA and rpSTS were successfully identified in each participant. The location of the rOFA and rpSTS in a typical participant is shown in Figure 1.
Locations, in one participant, of the rpSTS and rOFA.
Materials.
Stimuli were presented centrally on an SVGA 20 inch monitor (Resolution 1024 by 768, refresh rate 100 Hz). The faces were six female models (C, MF, MO, NR, PF, and SW) from Ekman and Friesen's (1976) facial affect series expressing one of six emotions: happy, sad, surprise, fear, disgust, and anger. Each grayscale picture was cropped with the same contour using Adobe Photoshop to cover the hair and neck. The same set of faces was used for both the identity and expression recognition blocks. This task was used in a previous TMS study of facial expression recognition (Pitcher et al., 2008) as well as in neuropsychological (Garrido et al., 2009; Banissy et al., 2011) and neuroimaging studies (Germine et al., 2011).
For the expression recognition task, half the trials showed picture pairs with the same expression and half showed pairs with different expressions. Identity always changed between match and target faces. The six expressions were presented an equal number of times.
For the identity recognition task, half the trials showed pairs with the same identity and half showed pairs with different identities. Expression always changed between the match and target faces. The six models were presented an equal number of times.
TMS stimulation and site localization.
TMS was delivered at 60% of maximal stimulator output, using a Magstim Super Rapid Stimulator (Magstim) and a 70 mm figure-of-eight coil, with the coil handle pointing upward and parallel to the midline. A single intensity was used on the basis of previous studies (Pitcher et al., 2007, 2008) and for ease of comparison with related studies, the majority of which have used a single intensity. The use of a within-site control task (e.g., the identity task) means any task-specific effects cannot be explained by induced TMS intensity differences across participants.
During Experiment 1, stimuli were presented while right TMS (rTMS) was delivered over the target site at a frequency of 10 Hz for 500 ms. rTMS onset was concurrent with the onset of the probe stimulus. During Experiment 2, dTMS with 40 ms between pulses (Pitcher et al., 2007, 2008) was delivered at one of five different latencies after the probe stimulus onset: 20–60, 60–100, 100–140, 130–170, and 170–210 ms. These latencies were chosen to cover the most likely times of rOFA and rpSTS involvement in facial expression recognition (Bentin et al., 1996; Eimer and Holmes, 2002; Pourtois et al., 2004; Pitcher et al., 2007, 2008).
TMS sites were located using the Brainsight TMS–MRI coregistration system, using individual high resolution MRI scans for each participant. The rOFA and rpSTS were localized by overlaying individual activation maps from the fMRI localizer task onto the structural scan and the proper coil locations were marked on each participant's head. The voxel exhibiting the peak activation in both of the functionally defined regions was used as the target.
Procedure.
Experiment 1 delivered rTMS over the rpSTS and vertex while participants performed the facial identity and facial expression tasks. The vertex condition served as a control for nonspecific effects of TMS. The identity component acted as a control task based on the proposed functional role of the pSTS in facial expression recognition but not identity recognition (Winston et al., 2003). Figure 2 displays the trial procedure. Participants sat 57 cm from the monitor with their heads stabilized in a chin rest and indicated, by a right hand key press, whether the match face showed the same facial expression as the target face (expression task) or the same person as the target face (identity task). They were instructed to respond as accurately and quickly as possible. Participants were not given feedback on their performance. Two blocks of 36 trials were presented for each task (expression and identity), and task order was balanced across participants. During each task, rTMS was delivered over either rpSTS or vertex. Block order was balanced across participants. Within each block, the trial order was randomized.
The timeline of the experimental procedure in Experiments 1 and 2. In Experiment 1, participants had to judge whether the match face and target face had the same identity or the same expression. In Experiment 2, participants performed the expression recognition task only.
Experiment 2 required participants to perform only the expression task. dTMS was delivered over the rOFA, rpSTS, and vertex at one of five different latencies after stimulus onset (20–60, 60–100, 100–140, 130–170, and 170–210 ms). There were 36 trials per timing condition block. Timing condition order and TMS stimulation site were balanced across participants.
Results
Experiment 1: rTMS delivered over rpSTS impairs expression but not identity recognition
The aim of Experiment 1 was to establish whether TMS delivered over the rpSTS would selectively impair facial expression but not facial identity recognition. Participants performed expression and identity recognition tasks while rTMS (10 Hz for 500 ms) was delivered over the rpSTS or the vertex control site. There was no difference in performance between the expression and identity tasks when TMS was delivered over the vertex, demonstrating that the tasks were matched for difficulty. Figure 3 shows that rTMS delivered over the rpSTS impaired performance on the facial expression recognition task compared with the vertex control site. As predicted, rTMS had no effect on the identity task.
Mean accuracy performance for the expression and identity recognition tasks. rTMS delivered over the rpSTS selectively impaired the facial expression task but had no effect on the facial identity task. *Significant effects in planned Bonferroni-corrected comparisons. Error bars indicate SEs.
Accuracy data were entered into a 2 × 2-way repeated-measures ANOVA with TMS site (rpSTS and vertex) and recognition task (expression or identity) as independent variables. Results showed main effects of TMS site (F(1,9) = 15.6, p = 0.003) and task (F(1,9) = 37.3, p = 0.007) as well as a significant interaction between site and task (F(1,9) = 39.3, p < 0.001). Planned Bonferroni-corrected post hoc tests showed that rTMS delivered over the rpSTS significantly impaired accuracy on the expression task compared with the identity task (p < 0.0001). Further tests showed that task performance for the expression task when rTMS was delivered over the rpSTS was significantly lower than task performance for both the expression (p = 0.003) and identity (p < 0.0001) tasks when rTMS was delivered over the vertex control site. No other tests approached significance (p > 0.24).
A 2 × 2 repeated-measures ANOVA on the reaction time (RT) data produced no significant main effects (p > 0.3) and no significant interaction (p = 0.55). There was a trend toward a slowing down of RT for the expression task when TMS was delivered over the rpSTS, but this did not reach significance (expression rpSTS = 864 ms, expression vertex = 822 ms, identity rpSTS = 819 ms, identity vertex = 809 ms). This absence of significant effects in RT data is consistent with similar prior TMS studies that used the same or similar stimuli (Pitcher et al., 2007, 2008, 2012).
Experiment 2: dTMS reveals the temporal specificity of the causal contribution of the rOFA and rpSTS to expression recognition
The aim of Experiment 2 was to establish when the rOFA and rpSTS causally contribute to facial expression recognition. dTMS, separated by 40 ms, was delivered over the rpSTS, rOFA, and vertex control site at five different latencies after stimulus onset: 20–60, 60–100, 100–140, 130–170, or 170–210 ms. Previous studies suggested that this 190 ms period would encompass the involvement of both rOFA and rpSTS in facial expression processing (Bentin et al., 1996; Eimer and Holmes, 2002; Pourtois et al., 2004; Pitcher et al., 2007, 2008). As Figure 4 shows, accuracy was selectively and significantly reduced when dTMS was delivered over rOFA in a pulse pair delivered at 60–100 ms after stimulus onset only. dTMS delivered over the rpSTS produced a different impairment pattern in that it selectively impaired expression accuracy at two latency windows, both 60–100 ms and 100–140 ms after stimulus onset.
Mean accuracy performance for the expression recognition task when dTMS was delivered over the rpSTS and rOFA. *Significant effects in planned Bonferroni-corrected comparisons. Error bars indicate SEs.
Accuracy data were entered into a 3 × 5-way repeated-measures ANOVA with TMS site (rOFA, rpSTS, and vertex) and dTMS latency (20–60, 60–100, 100–140, 130–170, and 170–210 ms) as independent factors. Results showed a main effect of TMS site (F(2,22) = 5.6, p = 0.01) but not of latency (F(4,44) = 2.1, p = 0.096). TMS site and latency combined in a significant two-way interaction (F(8,88) = 2.9, p = 0.007). Planned Bonferroni-corrected post hoc tests showed that, when dTMS was delivered at 60–100 ms, there was a significant difference between rOFA and vertex (p = 0.019) and between rpSTS and vertex (p = 0.026), but there was no significant difference between rOFA and rpSTS (p = 1). When dTMS was delivered at 100–140 ms, there was a significant difference between the rOFA and rpSTS (p = 0.003) and between rpSTS and vertex (p = 0.011), but there was no significant difference between rOFA and vertex (p = 1). No other tests approached significance (p > 0.3).
A 3 × 5-way repeated-measures ANOVA on the RT data showed a main effect of dTMS latency (p = 0.04) but not of TMS site (p = 0.4), and there was no significant interaction (p = 0.8).
Discussion
In the present study, rTMS was delivered over the face-selective region in the rpSTS, or the vertex control site while participants performed facial expression and facial identity match-to-sample recognition tasks. rTMS selectively impaired performance on the expression, but not the identity task, a result consistent with an extensive body of fMRI evidence that the rpSTS is a cortical locus for facial expression recognition (Allison et al., 2000; Nummenmaa and Calder, 2009). In Experiment 2, dTMS, separated by 40 ms, was delivered over the rpSTS and the rOFA at different latencies, ranging from 20 ms to 210 ms, after stimulus onset while participants performed the expression recognition task only. dTMS selectively impaired task performance when delivered at 60–100 ms and 100–140 ms over the rpSTS and at 60–100 ms only when delivered over the rOFA. These results demonstrate that the rpSTS and the rOFA were impaired by dTMS delivered at the same early latency and that the rpSTS impairment was twice the length of the impairment at the rOFA.
The finding that the rOFA and rpSTS are functionally impaired by dTMS delivered at the same latency (60–100 ms after stimulus onset) suggests that the rOFA and rpSTS may begin to process facial expressions at the same time. This hypothesis is consistent with electrophysiological data recorded intracranially from neuropsychological patients that shows a range of face tasks generate event-related potentials with the same latencies in both the ventral and lateral occipitotemporal cortices (Allison et al., 1999; McCarthy et al., 1999; Puce et al., 1999). However, suggesting that the rOFA and rpSTS begin to process facial expressions at the same time is seemingly inconsistent with existing cortical models of face perception (Haxby et al., 2000; Calder and Young, 2005; Pitcher et al., 2011b). These models propose that the OFA represents facial information before further analysis in downstream face-selective regions, such as the pSTS and the fusiform face area (Kanwisher et al., 1997; McCarthy et al., 1997). Rather, it seems possible that the rOFA and rpSTS may receive input via independent cortical pathways beginning in early visual cortex, as suggested by a recent human diffusion study (Gschwind et al., 2012) and by nonhuman primate neuroanatomy (Kravitz et al., 2013).
The difference in the length of the TMS-induced impairment window between the rpSTS and the rOFA suggests that the neural computations that contribute to facial expression recognition in each region are qualitatively different. What are the likely characteristics of these differences? One possible hypothesis is suggested by evidence that the rpSTS shows a threefold greater response to dynamic faces than to static faces, whereas the OFA shows no significant difference between the responses to dynamic and static faces (Pitcher et al., 2011a). Although the present study used only static stimuli, it still seems likely that the expression, but not the identity, task better reflected the functional role of the rpSTS because the rpSTS shows a greater response to visual input that changes over time (e.g., facial expressions). By contrast, prior fMRI and TMS studies demonstrated that the OFA represents the component parts of a face that contribute to identity recognition (Pitcher et al., 2007; Liu et al., 2010), and this function could be accomplished over a comparatively shorter duration. This hypothesis is also consistent with a prior TMS study of facial expression that reported a longer dTMS-induced impairment window in a higher face-selective region, the face region of the right somatosensory cortex, than in the rOFA (Pitcher et al., 2008).
The difference in the length of the TMS impairment window between the rpSTS and rOFA is also consistent with physiological evidence from nonhuman primates that demonstrates that the response profile of neurons in higher cortical regions is longer than the response in earlier cortical regions (Kovács et al., 1995). Human fMRI evidence also shows that cortical regions, in and around the rpSTS, show a longer temporal response window to movie clips than regions in early visual cortex (Hasson et al., 2008). The duration of the TMS-disruptive effect reported in Experiment 2 could reflect the duration that the neurons in each stimulated region are causally engaged in contributing to the expression task. One prediction from this hypothesis is that stimulating the same region (e.g., the rpSTS) while participants perform tasks that differ in complexity should show impairment duration windows commensurate with the level of complexity of each task.
In conclusion, the present study demonstrates that the rpSTS and the rOFA are causally engaged in processing facial expressions and that the rpSTS does so for a longer duration than the rOFA. These results suggest that the neural computations that contribute to facial expression recognition in each region are functionally distinct.
Footnotes
I thank Brad Duchaine, Lucia Garrido, David Leopold, Leslie Ungerleider, and Vincent Walsh for advice and support.
The author declares no competing financial interests.
- Correspondence should be addressed to Dr. David Pitcher, Laboratory of Brain and Cognition, National Institute for Mental Health, Bethesda, MD 20892. david.pitcher{at}nih.gov
