Abstract
The late positive potential (LPP) is a reliable electrophysiological index of emotional perception in humans. Despite years of research, the brain structures that contribute to the generation and modulation of LPP are not well understood. Recording EEG and fMRI simultaneously, and applying a recently proposed single-trial ERP analysis method, we addressed the problem by correlating the single-trial LPP amplitude evoked by affective pictures with the blood oxygen level-dependent (BOLD) activity. Three results were found. First, relative to neutral pictures, pleasant and unpleasant pictures elicited enhanced LPP, as well as heightened BOLD activity in both visual cortices and emotion-processing structures such as amygdala and prefrontal cortex, consistent with previous findings. Second, the LPP amplitude across three picture categories was significantly correlated with BOLD activity in visual cortices, temporal cortices, amygdala, orbitofrontal cortex, and insula. Third, within each picture category, LPP–BOLD coupling revealed category-specific differences. For pleasant pictures, the LPP amplitude was coupled with BOLD in occipitotemporal junction, medial prefrontal cortex, amygdala, and precuneus, whereas for unpleasant pictures significant LPP–BOLD correlation was observed in ventrolateral prefrontal cortex, insula, and posterior cingulate cortex. These results suggest that LPP is generated and modulated by an extensive brain network composed of both cortical and subcortical structures associated with visual and emotional processing and the degree of contribution by each of these structures to the LPP modulation is valence specific.
Introduction
The event-related potential (ERP) method is used extensively in affective neuroscience. A key feature observed in ERPs evoked by emotionally engaging stimuli is the late positive potential (LPP), which is characterized by an amplitude enhancement for pleasant and unpleasant stimuli, relative to neutral stimuli, and has a centroparietal maximum topography. For affective picture viewing, LPP starts ∼300–400 ms after picture onset and is often sustained throughout the duration of picture presentation (Cuthbert et al., 2000). LPP amplitude has been shown to vary systematically with the experienced intensity of the affective picture content (Schupp et al., 2000; Keil et al., 2002) and exhibit abnormal patterns in mood disorders and other psychiatric conditions (Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Jaworska et al., 2012). In parallel, functional magnetic resonance imaging (fMRI) has found that viewing of affective pictures is associated with increased blood oxygen level-dependent (BOLD) activity in widespread brain regions, including occipital, parietal, inferotemporal cortices, and amygdala (Breiter et al., 1996; Lang et al., 1998a; Bradley et al., 2003; Norris et al., 2004; Sabatinelli et al., 2005, 2009), suggesting that emotionally salient content enhances visual stimulus processing by attracting attentional resources (Lang et al., 1998b; Lang and Bradley, 2010). Together, if enhanced LPP and BOLD reflect a common underlying mechanism, one might expect a coupling between LPP amplitude and BOLD activity in the above reported regions.
A prior study recording EEG and fMRI from the same subjects but in separate sessions has found that LPP amplitude was positively correlated with BOLD responses in lateral occipital, parietal, and inferotemporal cortices (Sabatinelli et al., 2007a). This study did not examine LPP–BOLD coupling in other higher-order emotional processing areas such as prefrontal cortex and deep subcortical structures known to be involved in emotional perception (Sabatinelli et al., 2009). A more recent study using a between-subjects design observed coupling between LPP amplitude and BOLD activity in both deep and anterior structures (Sabatinelli et al., 2012), but it is still unclear whether these structures are engaged with the LPP in a category-specific way based on trial-by-trial information within each picture category. The advent of the simultaneous EEG–fMRI recording technique, together with reliable estimation of single-trial ERPs, opens new avenues to address this problem.
We recorded simultaneous EEG–fMRI while subjects passively viewed emotionally arousing and neutral pictures. The single-trial LPP amplitudes were estimated using a recently proposed method (Xu et al., 2009) and then correlated with the single-trial evoked BOLD responses across the entire brain to identify brain structures whose activity is linearly related to the trial-by-trial variation of the scalp-recorded LPP. In addition, in light of a host of prior studies reporting differential engagement of cortical and subcortical structures in appetitive versus aversive processing (Sabatinelli et al., 2007a), we investigated whether trial-by-trial LPP amplitude fluctuations are mediated by different neural generators during different affective states by examining the coupling between LPP amplitude and BOLD within each picture category (pleasant, neutral, unpleasant).
Materials and Methods
Participants.
Fifteen healthy volunteers participated in the experiment in exchange for either course credits or a financial incentive of $30. One participant withdrew from the experiment. In addition, data from three participants were discarded due to artifacts generated by excessive movement inside the scanner. The remaining 11 participants (7 females; mean age, 20; SD, 2.65) had normal or corrected-to-normal vision. The experimental protocol was approved by the Institutional Review Board of the University of Florida. Informed consent was obtained from all participants before the experiment.
Stimuli and procedure.
The stimuli consisted of 20 pleasant, 20 neutral, and 20 unpleasant pictures selected from the International Affective Picture System (IAPS) (Lang et al., 2008) based on their normative valence and arousal levels. The IAPS picture numbers used in this study are as follows: pleasant: 4311, 4599, 4610, 4624, 4626, 4641, 4658, 4680, 4694, 4695, 2057, 2332, 2345, 8186, 8250, 2655, 4597, 4668, 4693, 8030; neutral: 2398, 2032, 2036, 2037, 2102, 2191, 2305, 2374, 2377, 2411, 2499, 2635, 2347, 5600, 5700, 5781, 5814, 5900, 8034, 2387; unpleasant: 1114, 1120, 1205, 1220, 1271, 1300, 1302, 1931, 3030, 3051, 3150, 6230, 6550, 9008, 9181, 9253, 9420, 9571, 3000, 3069. The selected pictures cover a wide range of contents and normative ratings. The pleasant pictures in general included sport scenes, romance, and erotic couples, whereas the unpleasant pictures incorporated threat, attack scenes, and bodily mutilations. The neutral pictures included landscapes and neutral human beings. The mean pleasure (valence) rating for pleasant, neutral, and unpleasant pictures was 7.0, 6.3, and 2.8, respectively. The pleasant and unpleasant pictures had similar mean arousal levels (pleasant, 5.8; unpleasant, 5.9), both being higher than neutral pictures (4.2). Pictures were chosen to be similar overall in composition, matched in jpeg size across categories, and comparable in rated complexity, to minimize confounds.
The experimental paradigm was implemented in an event-related fMRI design. Each IAPS picture was centrally displayed on a monitor for 3 s followed by a variable (2800 or 4300 ms) interstimulus interval. All participants completed five experimental sessions in which the pictures were repeated in different random orders. The order of picture presentation was also randomized across different participants. A cross was displayed at the center of the screen during the entire experiment to aid fixation. Stimuli were presented on an MR-compatible monitor using E-Prime software (Psychology Software Tools). The monitor was placed outside the scanner bore over the head of the subject. Participants viewed the task presentation in the scanner via a reflective mirror. Before the start of the first experimental session, participants were instructed to maintain eye fixation whenever the fixation cross is present and viewed the pictures without moving their eyes. After the experiment, as a validation, participants were asked to provide their ratings of 12 representative pictures (4 pictures within each category) they had not seen during the experiment along the scales of valence and arousal using a paper and pencil version of the self-assessment manikin (Bradley and Lang, 1994). The entire experiment lasted ∼40 min.
Simultaneous EEG–fMRI acquisition.
MRI data were collected on a 3 T Philips Achieva scanner (Philips Medical Systems). Two hundred twelve volumes of functional images were acquired using a gradient-echo echo-planar imaging sequence during each session [echo time (TE), 30 ms; repetition time (TR), 1.98 s; flip angle, 80°; slice number, 36; field of view, 224 mm; voxel size, 3.5 × 3.5 × 3.5 mm; matrix size, 64 × 64]. The slices were acquired in ascending order and oriented parallel to the plane connecting the anterior and posterior commissure. Slice acquisition was performed within an interval of 1850 ms during each TR, leaving an interval of 130 ms toward the end of the TR where no image acquisition was performed. This image acquisition approach allowed us to visually monitor the EEG recording within each volume during the no-scan period in which EEG was not contaminated by gradient switching. A T1-weighted high-resolution structural image was also obtained.
EEG data were recorded during the experiment using a 32 channel MR-compatible EEG system (Brain Products). Thirty-one sintered Ag/AgCl electrodes were placed according to the 10–20 system, and one additional electrode was placed on subject's upper back to monitor electrocardiograms (ECGs). The recorded ECG will be used to detect heartbeat events to be used for the removal of the cardioballistic artifact. The EEG channels were referenced to site FCz during recording. The impedance from all scalp channels was kept below 10 kΩ during experiment as suggested by the manufacturer. EEG signal was recorded with a built-in 0.1∼250 Hz bandpass filter and digitized to 16 bit at a sampling rate of 5 kHz. The digitized EEG signal was then transferred to the recording computer via a fiber-optic cable. The EEG recording system was synchronized with the scanner's internal clock throughout the recording session to ensure the successful removal of the gradient artifact in subsequent analyses.
EEG data preprocessing.
Brain Vision Analyzer 2.0 (Brain Products) was used for data preprocessing. Gradient artifacts in the EEG data were removed using a modified version of the original algorithms proposed by Allen et al. (2000). Briefly, an artifact template was created by segmenting and averaging the data according to the onset of each volume within a sliding window consisting of 41 consecutive volumes, and subtracted from the raw EEG data. To remove the cardioballistic artifact, an average artifact subtraction method (Allen et al., 1998) was used, in which R peaks were detected in the low-pass-filtered ECG signal and used to construct a delayed average artifact template over 21 consecutive heartbeat events in a sliding-window approach, which was subtracted from the original EEG signal. The resulting EEG data were then low-pass filtered with the cutoff set at 50 Hz, downsampled to 250 Hz, and rereferenced to the average reference. These data were then exported to the EEGLAB (Delorme and Makeig, 2004). Second-order blind identification (SOBI) (Belouchrani et al., 1993) was performed to further correct for eye blinking, residual cardioballistic, and movement-related artifacts. Recent work has shown that SOBI is effective in removing the residual cardioballistic artifact (Vanderperren et al., 2010), as well as in separating EEG data into physiologically interpretable components (Tang et al., 2005; Klemm et al., 2009). The artifacts-corrected data were then epoched from −300 to 2000 ms with 0 ms being the onset of affective pictures. The prestimulus baseline was defined as −300 to 0 ms. The EEG epochs were averaged within each condition separately to produce the average ERP (AERP). The AERP coming from each subject was further averaged across subjects to produce the grand average ERP.
Single-trial estimation of LPP.
Channel Pz was chosen to guide our subsequent EEG-informed fMRI analysis as it showed strong LPP difference between both emotional conditions and the neutral condition (Fig. 1B,C). The ERP of each trial at Pz was estimated using the analysis of single-trial ERP and ongoing activity (ASEO) method (Xu et al., 2009). ASEO has the following basic steps. First, according to the VSPOA (variable signal plus ongoing activity) generative model (Chen et al., 2006), the recorded EEG data for the rth trial (r = 1, 2, …, R) are expressed as follows: where sn(t) (n = 1, 2, …, N) is the nth ERP component and zr(t) is an AR (autoregressive) process modeling the ongoing activity. Within each individual trial, the nth ERP component is characterized by an amplitude scaling factor βrn and a latency shift τrn to account for trial-to-trial variability. Second, using a proper initial condition, the ASEO algorithm estimates the waveforms of the ERP components and their associated amplitude scaling factors and latency shifts in an iterative fashion. Third, from scaled and latency-adjusted ERP component estimates, the single-trial ERP was reconstructed on a trial-by-trial basis. Fourth, the LPP amplitude on each trial was obtained by averaging the single-trial ERP amplitude within a time interval around the peak of LPP (Figs. 2, 3). To date, ASEO has been applied to study both monkey local field potential data and human EEG data (Wang et al., 2008; Wang and Ding, 2011; Liu et al., 2012). See Xu et al. (2009) for a more detailed description of the ASEO algorithm.
MRI data analysis.
The fMRI data were processed using SPM5 (http://www.fil.ion.ucl.ac.uk/spm/). The first five volumes in an experimental session were discarded to allow the scanner to stabilize. Slice timing was corrected using sinc interpolation to account for differences in acquisition time. The images were then corrected for head movement by spatially realigning the images to the sixth image of each session. Images were further normalized and registered to a standard template within SPM [the Montreal Neurological Institute (MNI) space]. The functional volume images were resampled to a spatial resolution of 3 × 3 × 3 mm. The transformed images were then smoothed by a Gaussian filter with a full-width at half-maximum of 8 mm. The low-frequency temporal drifts were removed from the functional images by applying a high-pass filter with a cutoff frequency of 1/128 Hz, and the global signal was removed by dividing every voxel in a slice by the estimated global signal value.
Two separate general linear models (GLMs) with parametric modulation were constructed to model the relationship between LPP amplitude and BOLD. In the first model, we were mainly interested in examining the overall LPP–BOLD coupling across all three picture categories (i.e., pleasant, neutral, and unpleasant). Therefore, in this model, we combined all three picture categories and modeled the resulting single experimental condition with two task-related regressors. The first regressor described the combined condition and consisted of a sequence of boxcar functions with unit height synchronized with the onset of pictures. The width of each boxcar function was set to the duration of picture presentation (3 s). For the second regressor, the height of the boxcar functions in the first regressor varied according to the ASEO-estimated single-trial LPP amplitude, with the mean level of the single-trial LPP amplitudes removed (see Fig. 3C). This regressor was intended to account for both the between-category and the within-category variability in LPP amplitude and its correlation with BOLD. The two regressors were further convolved with a canonical hemodynamic response function (HRF) before being incorporated into the design matrix. Six regressors describing the subjects' head movement obtained from image preprocessing were further introduced to account for any movement-related artifacts during scan. We referred to our first model as the “full model.” A single contrast was performed based on this model resulting in one statistical map for each subject.
In the second model, we included six task-related regressors, with three corresponding to the three picture categories (pleasant, neutral, and unpleasant), and the other three modeling the relationship between LPP amplitudes and BOLD within each picture category. Since this model captures trial-by-trial coupling between LPP amplitude and BOLD within each picture category, we refer to it as the “within-category model.” The three regressors modeling the three picture categories consisted of sequences of boxcar functions with unit height placed according to the picture onset within the corresponding categories. The width of the boxcar functions remained the same as those in the “full model” (3 s). For regressors modeling trial-by-trial LPP–BOLD coupling, we further scaled the height of the boxcar functions with the corresponding mean removed single-trial LPP amplitudes within each picture category. Similar to the full model described above, the six task-related regressors were convolved with a canonical HRF. Additional six regressors describing subjects' head movement were also introduced as covariates in the model. The following contrasts were performed based on this model: pleasant versus neutral, unpleasant versus neutral, LPP–BOLD coupling within each category (pleasant, neutral, and unpleasant).
Second-level analyses were performed using random-effects models based on the statistical maps obtained from the within-subjects analyses to examine reproducible effects across all subjects. For conventional BOLD contrasts, the group level T maps were thresholded at p < 0.05 [false discovery rate (FDR) corrected]. For LPP–BOLD coupling, because it is derived from the trial-to-trial variability signal on top of the large picture-evoked response, and this residual variability is generally smaller than the large picture-evoked response and may contain other ongoing brain processes that are not related to the experimental task, the correlation was generally smaller and required a more relaxed statistical threshold. In line with recent studies using EEG-informed fMRI analysis (Debener et al., 2005; Eichele et al., 2005; Bénar et al., 2007; Scheeringa et al., 2011), for LPP–BOLD coupling effects, the group-level T maps were thresholded at p < 0.003 (uncorrected). A cluster-level threshold of k = 5 voxels was further imposed.
Results
ERP analysis
Postexperiment ratings of 12 representative pictures indicate that subjects correctly distinguished the three categories of pictures (valence: pleasant, 6.5; neutral, 5.3; unpleasant, 2.6; arousal: pleasant, 4.7; neutral, 2.9; unpleasant, 4.0). Figure 1A shows enhanced positivity for both pleasant and unpleasant pictures, relative to neutral pictures, in the grand average ERP at Pz, starting from ∼300 ms after picture onset. Since the time interval during which LPP reached a maximum was relatively broad, the LPP amplitude was measured by taking the mean within 300–600 ms. A one-way ANOVA on LPP amplitudes with repeated measures identified a significant picture category-related difference in LPP amplitudes (F(2,20) = 23.11; p < 0.05). As further indicated by the results of post hoc tests with Bonferroni adjusted significance level, the mean (M) LPP amplitudes for both the pleasant (M, 3.153; SD, 1.733) and unpleasant (M, 3.090; SD, 2.048) pictures were significantly larger than that for the neutral pictures (M, 1.523; SD, 1.684; pleasant vs neutral: t(10) = 6.26, p < 0.001; unpleasant vs neutral: t(10) = 5.65, p < 0.001). However, no significant difference was found in LPP amplitudes between the pleasant and unpleasant categories (t(10) = 0.227; p = 0.825). The ERP difference topography further confirmed that the positivity is strongest among parietal channels for both pleasant and unpleasant conditions (Fig. 1B,C), agreeing with prior ERP studies of emotion and motivation (Lang and Bradley, 2010). The enhanced positivity was sustained throughout the duration of picture presentation for both pleasant and unpleasant pictures.
Artifact-removed raw EEG data and ASEO-estimated single-trial ERPs at Pz are shown in Figure 2, A and B, for a representative subject. Displayed as raster plots in Figure 2, C and D, the ASEO-estimated single-trial ERPs improved signal-to-noise ratio, and preserved the trial-by-trial dynamics of the LPP amplitude, which is important because the single-trial LPP amplitudes were used to correlate with BOLD response in subsequent analyses. The validity of the single-trial ERPs can be further supported by averaging the data in Figure 2, A and B (Wang et al., 2008; Xu et al., 2009; Wang and Ding, 2011). The similarity between ASEO AERP and the original AERP indicated that the algorithm accurately estimated the single-trial ERPs from the raw data (Fig. 2E). Figure 3, A and B, displays single-trial LPP amplitudes as functions of trial index. From these two figures, one can see that, on average, the single-trial LPP amplitudes for both pleasant and unpleasant pictures are higher than those for the neutral condition, yielding further support for the grand average ERP result in Figure 1. The estimated single-trial LPP amplitudes were used to scale the boxcar functions to examine the relationship between LPP amplitude and BOLD (Fig. 3C).
fMRI analysis
The traditional fMRI group level activation maps contrasting (1) pleasant against neutral and (2) unpleasant against neutral picture categories are shown in Figure 4, A and B. Both pleasant and unpleasant pictures activated the emotion-processing network, encompassing the visual cortices and deep structures. Specifically, relative to neutral pictures, pleasant pictures mainly activated areas in bilateral occipito-temporal junctions, bilateral posterior parietal cortices, medial prefrontal cortex, and left orbital frontal cortex. Other activated areas included fusiform gyrus, lingual gyrus, middle frontal gyrus, supramarginal gyrus, parahippocampal gyrus, and temporal pole (Fig. 4A). Unpleasant pictures mainly activated areas such as the bilateral occipito-temporal junctions, bilateral posterior parietal cortices, bilateral ventral lateral prefrontal cortices, left orbital frontal cortex, bilateral amygdalae/hippocampi, insula, and supplementary motor area. Other activated areas included fusiform gyrus, lingual gyrus, supramarginal gyrus, temporal pole, and postcentral cortex (Fig. 4B). In general, the activation results agree with a previous report using a similar experimental protocol (Sabatinelli et al., 2007a) and serve to demonstrate that the quality of the fMRI data is preserved despite the presence of the EEG recording system in the scanner.
Trial-by-trial coupling of LPP and BOLD
To assess the coupling between the LPP amplitude and BOLD, the coefficients for regressors associated with LPP amplitude variations were examined. Using the “full” model, which combined the neutral, pleasant, and unpleasant picture categories as a single regressor to describe the effect of both between- and within-category LPP amplitude variations on BOLD, the single-trial LPP amplitude was positively correlated with evoked BOLD responses in bilateral occipito-temporal junctions, insula, amygdala/hippocampus, temporal poles, and left orbital frontal cortex (Fig. 5A).
Using the “within-category” model, which allowed us to examine the coupling between LPP and BOLD within each picture category, it was found that, for the neutral condition, no significant coupling between single-trial LPP amplitude and BOLD existed among subjects. For the pleasant condition, the single-trial LPP amplitude was positively correlated with BOLD responses in bilateral occipito-temporal junctions, amygdala, temporal poles, precuneus, right nucleus accumbens (NAcc), medial prefrontal cortex (MPFC), and cerebellum (Fig. 5B). For the unpleasant condition, the single-trial LPP amplitude was positively correlated with BOLD responses in bilateral ventral lateral prefrontal cortices, bilateral insula, temporal poles, precuneus, left middle temporal cortex, and left postcentral cortex (Fig. 5C). Table 1 listed the MNI coordinates of these regions. It is worth noting that we did not find any structures in which BOLD was negatively correlated with LPP amplitude under the same significance level.
Discussion
Emotional stimuli evoke a LPP that is interpreted to signify enhanced attention and visual processing (Bradley, 2009). This signature ERP response is known to be altered in mood disorders and other related psychiatric illnesses (Foti et al., 2010; Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Jaworska et al., 2012; Weymar et al., 2012). Despite the importance of LPP, its neural substrate is not clear. ERP source localization is only partly successful (Keil et al., 2002; Sabatinelli et al., 2007a). This problem is addressed here by recording simultaneous EEG and fMRI while subjects viewed IAPS affective pictures. Extracting LPP on a trial-by-trial basis, the overall LPP amplitude variability across three picture categories (pleasant, neutral, and unpleasant) was found to be correlated with BOLD responses in an extensive cortical and subcortical network, including visual cortices and deep emotion-processing structures. In addition, consistent with the notion that appetitive and aversive information may engage different neural substrates, the brain areas in which BOLD activity was correlated with LPP amplitude during pleasant picture viewing were not the same as those during unpleasant picture viewing.
Methodological considerations
Prior investigation of the association between LPP and evoked BOLD responses relied on recording EEG and fMRI over separate sessions and correlating averaged responses across subjects (Sabatinelli et al., 2007a, 2012). One potential drawback of such an approach is that it is difficult to keep the psychological and biological conditions exactly the same in different recording sessions, and moreover, the correlation between average LPP and BOLD does not reflect their trial-by-trial covariations and coupling toward individual pictures within each subject. A new technology, simultaneous EEG–fMRI, has become available over the past few years. As has been recently demonstrated (Nagai et al., 2004; Debener et al., 2005; Eichele et al., 2005; Scheeringa et al., 2011), simultaneous EEG–fMRI is capable of overcoming these limitations and has the potential to allow the interrogation of trial-by-trial associations between the two recording modalities. The present study benefits from another methodological development. Applying a recently proposed single-trial analysis algorithm called ASEO, we were able to estimate the LPP amplitude on a trial-by-trial basis. The improved signal-to-noise ratio helps to more fully reveal the brain areas whose BOLD responses are correlated with LPP fluctuations. The ASEO algorithm has been tested in both monkey local field potential data (Wang et al., 2008; Xu et al., 2009) and human scalp EEG data (Fogelson et al., 2009; Wang and Ding, 2011; Liu et al., 2012), and proven useful to address questions arising in a number of contexts, ranging from the proper preprocessing of event-related data for functional connectivity analysis to the temporal dynamics of emotional conditioning.
LPP–BOLD coupling and its theoretical significance
The brain regions in which BOLD activities covary with single-trial LPP amplitude across three picture categories reflected a joint involvement of the visual system and a network of structures known to be associated with emotional processing. Past source-space modeling of LPP has only been able to identify generators in the visual system, including occipito-temporal, parietal, and inferior temporal cortices (Keil et al., 2002; Sabatinelli et al., 2007a), despite the fact that the amplitude of LPP is closely related to the rated intensity of emotion (Schupp et al., 2000; Keil et al., 2002). In line with a recent study (Sabatinelli et al., 2012), the present study extends the prior findings by showing that deep structures such as the insula and the amygdala, along with visual structures, contribute to the generation of LPP and its amplitude modulation. These results provide further evidence supporting the view that emotional pictures naturally attract attentional resources as a result of the engagement of the fundamental motivational system (Cacioppo et al., 1993, 1994; Lang et al., 1997, 1998b; Palomba et al., 1997; Schupp et al., 2000; Pastor et al., 2008; Lang and Bradley, 2010).
The contribution to scalp-recorded potentials by the emotional processing areas may be modulatory and mediated by the visual cortex. It has been hypothesized that, when observers view emotionally engaging scenes, cortical and deep subcortical structures modulate visual cortex in a reentrant fashion (Keil et al., 2009; Pessoa and Adolphs, 2010). These structures include the amygdala, insula, and prefrontal cortex (Rotshtein et al., 2001; Adolphs, 2002; Phan et al., 2002; LeDoux, 2003; Zald, 2003; Luo et al., 2007). As evidenced by recent human intracranial studies, amygdala and orbitofrontal cortex show fast responses to the emotional content of stimuli, which would enable them to provide reentrant feedback to the visual cortices (Oya et al., 2002; Krolak-Salmon et al., 2004), to potentially affect the gain of visual neurons. In addition, the activation of emotion-related BOLD modulations in the amygdala is found to precede that in the fusiform gyrus, the medial occipital gyrus, and the calcarine fissure, consistent with the putative role of the amygdala in initiating the reentrant interaction (Sabatinelli et al., 2009). Together, the positive correlation found in the current study between single-trial LPP amplitude and BOLD activity in the amygdala, insula, and areas in prefrontal cortex is supportive of the reentrant hypothesis of emotional perception.
Category-specific network processing
Restricting to pictures within each valence category revealed category-dependent differences in regions showing LPP and BOLD coupling. For pleasant pictures, LPP amplitude variability was found to be linearly related to BOLD activity in bilateral amygdalae, whereas for unpleasant pictures, this correlation was absent. This finding was further corroborated by a contrast of LPP–BOLD coupling maps between pleasant and unpleasant conditions showing that the amygdala was preferentially engaged in LPP modulation in the pleasant condition. For the unpleasant pictures, the amygdala was found to be activated by a traditional fMRI contrast between unpleasant and neutral conditions, suggesting a rather constant level of amygdala activation on a trial-by-trial basis. Specifically, while the amygdala activity is clearly enhanced by unpleasant pictures as a whole, it does not covary with trial-by-trial LPP changes for different pictures within this category. This may reflect a limited response variability of the amygdala for unpleasant scenes or the presence of additional sources of variance that govern the modulation of LPP. A wider selection of unpleasant scenes may help to identify the potential sources of covariation between electrophysiological measures and amygdala BOLD activity during aversive/defensive engagement.
Whether amygdala responds to pleasant stimuli is debated, although several studies reported amygdala activation during processing of pleasant stimuli, especially when these stimuli have highly arousing erotica content (Lane et al., 1997; O'Doherty et al., 2001; Hamann et al., 2002; Zald, 2003). In the present study, the fact that the amygdala was not activated in a traditional fMRI analysis by contrasting the pleasant with the neutral category may indicate that the overall mean amygdala BOLD response to pleasant pictures was small. Yet, the positive correlation between LPP amplitude and BOLD activities in bilateral amygdalae suggests that the BOLD fluctuations in the amygdala are parametrically related to intensity variations of pleasant emotional content, as measured by the LPP. This finding further demonstrates that combining electrophysiological recordings and functional imaging can yield information not possible with either modality alone.
The LPP–BOLD coupling was found in MPFC and NAcc only for pleasant pictures. Contrasting LPP–BOLD coupling maps between pleasant and unpleasant conditions showed that the MPFC is preferentially coupled with LPP in the pleasant condition. In addition, the MPFC was activated by contrasting pleasant with neutral pictures, but the same region was not activated when comparing unpleasant and neutral pictures. NAcc and MPFC are densely interconnected (Ferry et al., 2000; Roberts et al., 2007) and often show correlated activities in human reward studies (Knutson et al., 2003; Rogers et al., 2004), leading to the view that both NAcc and MPFC are part of the human reward system mediating appetitive behaviors. Several studies have reported involvement of both structures in the perception of pleasant emotional stimuli including attractive faces, romance, and erotica (Aharon et al., 2001; Karama et al., 2002; O'Doherty et al., 2003; David et al., 2005; Ferretti et al., 2005; Sabatinelli et al., 2007b) as well as in vivid imagery of pleasant scenes (Costa et al., 2010). Hence, the observed positive correlation between LPP amplitude and key structures in the reward system may reflect the contribution to the cortical potential by the appetitive system.
For unpleasant pictures, the LPP was correlated with BOLD in insula and adjacent temporal and ventrolateral prefrontal cortices (VLPFC). The same regions were found to be active when contrasting unpleasant with neutral pictures. It has been shown repeatedly that the human insula is involved in tasks that challenge the representation of bodily states as well as processing of emotions (Craig, 2009; Gu et al., 2010; Fan et al., 2011), especially for aversive emotions such as disgust and threat (Phillips et al., 1997, 1998; Adolphs, 2002; Straube and Miltner, 2011). Reliable covariation of the insula and periinsula with the LPP during aversive perception demonstrates that these structures contribute to the modulation of cortical potential during aversive events. It also suggests that the aversive/defensive circuitry involved in processing unpleasant pictures is not engaged in an all-or-none fashion, but varies parametrically as a function of aversive motivation, indexed by the LPP amplitude. It is worth noting that insula was not activated when LPP–BOLD coupling maps were contrasted between pleasant and unpleasant conditions. In light of the finding that insula is activated during viewing of highly arousing pleasant stimuli (e.g., erotica) (Karama et al., 2002), this may suggest that, for pleasant pictures, insula is engaged in LPP modulation but the degree of modulation did not reach the level of statistical significance.
Finally, for both pleasant and unpleasant pictures, BOLD activity in regions within midline parietal cortex is linearly correlated with the LPP. For pleasant pictures, the region that was most correlated with LPP amplitude was within the precuneus, whereas for unpleasant pictures, such correlation occurred in more ventral regions, particularly the posterior cingulate cortex and precuneus. The involvement of these parietal regions is in line with electrophysiological data and with the theoretical notion of “motivated attention” in which features of emotionally arousing scenes attract perceptual processing resources (Keil et al., 2002, 2012).
Footnotes
This work was supported by NIH Grant MH097320.
- Correspondence should be addressed to Mingzhou Ding, The J. Crayton Pruitt Family Department of Biomedical Engineering, J-285 Biomedical Sciences Building, P.O. Box 116131, University of Florida, Gainesville, FL 32611. mding{at}bme.ufl.edu