Abstract
Although the amygdala is a major locus for hedonic processing, how it encodes valence information is poorly understood. Given the hedonic potency of odor stimuli and the amygdala's anatomical proximity to the peripheral olfactory system, we combined high-resolution fMRI with pattern-based multivariate techniques to examine how valence information is encoded in the amygdala. Ten human subjects underwent fMRI scanning while smelling 9 odorants that systematically varied in perceived valence. Representational similarity analyses showed that amygdala codes the entire dimension of valence, ranging from pleasantness to unpleasantness. This unidimensional representation significantly correlated with self-reported valence ratings but not with intensity ratings. Furthermore, within-trial valence representations evolved over time, prioritizing earlier differentiation of unpleasant stimuli. Together, these findings underscore the idea that both spatial and temporal features uniquely encode pleasant and unpleasant odor valence in the amygdala. The availability of a unidimensional valence code in the amygdala, distributed in both space and time, would create greater flexibility in determining the pleasantness or unpleasantness of stimuli, providing a mechanism by which expectation, context, attention, and learning could influence affective boundaries for guiding behavior.
SIGNIFICANCE STATEMENT Our findings elucidate the mechanisms of affective processing in the amygdala by demonstrating that this brain region represents the entire valence dimension from pleasant to unpleasant. An important implication of this unidimensional valence code is that pleasant and unpleasant valence cannot coexist in the amygdale because overlap of fMRI ensemble patterns for these two valence extremes obscures their unique content. This functional architecture, whereby subjective valence maps onto a pattern continuum between pleasant and unpleasant poles, offers a robust mechanism by which context, expectation, and experience could alter the set-point for valence-based behavior. Finally, identification of spatial and temporal differentiation of valence in amygdala may shed new insights into individual differences in emotional responding, with potential relevance for affective disorders.
Introduction
An important function of the brain is to extract hedonic value, or valence, from external stimuli, as a key first step in guiding emotional responses, decision-making, and behaviors. Extensive animal research has implicated the amygdala in neural coding of both positive (Nishijo et al., 1988; Holland and Gallagher, 1999; Baxter et al., 2000; Setlow et al., 2002; Sugase-Miyamoto and Richmond, 2005; Shabel and Janak, 2009) and negative (Paton et al., 2006; Belova et al., 2008) stimulus values, and human imaging studies have shown that the amygdala responds to both pleasant and unpleasant emotional valence (Hamann et al., 1999; Costafreda et al., 2008; Sergerie et al., 2008; for review, see Zald, 2003; Phelps and LeDoux, 2005; Barrett and Wager, 2006; Ball et al., 2009). Interestingly, despite the well-accepted role of the amygdala in stimulus processing of emotional content, the form in which this information is encoded in the human brain remains poorly understood.
Whether pleasant and unpleasant valence is differentially represented in the human amygdala has not been definitively established (Ball et al., 2009; Lindquist et al., 2015). The absence of valence-related differences in neuroimaging data likely reflects the limitations of conventional functional imaging methods. For example, if regional representations of positive and negative valence were interspersed and overlapping within the amygdala, univariate fMRI techniques would be insensitive to teasing apart these unique populations due to spatial smoothing and normalization (Norman et al., 2006; Mur et al., 2009). Furthermore, if positive and negative valence evoked temporally distinct time-series profiles, then conventional imaging approaches that rely on hemodynamic convolution models might reduce sensitivity to temporal differences. Thus, although human neuroimaging research has yielded important insights into where negative and positive valences are coded in the brain, important questions remain unanswered: How is the valence of a stimulus uniquely represented in the amygdala if it is responsive to both positive and negative valence? Do amygdala representations of valence map onto self-reported experience of valence?
Considering that the vast number of stimuli in the environment are associated with a potentially limitless and finely graded range of valences, it would be highly inefficient for the brain to represent unique valence content for each and every stimulus. A more plausible strategy is that neuronal populations in the amygdala support a second-order representation of valence, based on stimulus similarity within an abstract space ranging from unpleasant to pleasant (Chikazoe et al., 2014). Behaviorally, it is well established that emotional experience is characterized along a valence dimension ranging from unpleasant to pleasant and an arousal dimension ranging from low to high (Russell, 1980; Lang et al., 1993). It is also evident that unpleasant (vs pleasant) stimuli are perceived as being more potent and perceptually more distinct from each other (Kahneman and Tversky, 1979; Rozin and Royzman, 2001). Finally, behavioral, physiological, and neural data indicate that the temporal dynamics of an emotional response can vary depending on both stimulus features and individual differences (Davidson, 1998; Blackford et al., 2009; Schuyler et al., 2014).
Here, we asked whether multivariate patterns of fMRI ensemble activity in the amygdala support continuous dimensions of subjectively experienced valence or arousal (indexed as stimulus intensity) and whether there are within-trial temporal differences between valence extremes. To this end, we measured odor-evoked distributed patterns of fMRI activity in the amygdala in healthy subjects as they smelled a series of nine odors that systematically varied in perceived pleasantness. Olfaction is an ideal modality to study valence coding in the amygdala because valence is a primary perceptual dimension of smell (Berglund et al., 1973; Schiffman, 1974) and because the amygdala is only two synapses removed from odor receptors in the nose (Carmichael et al., 1994; Shipley and Ennis, 1996). Our main prediction was that, across odor stimuli, pairwise similarity ratings of perceived valence would correspond to pairwise similarity measurements (fMRI correlations) of population-level amygdala activity, such that ensemble activity patterns for highly unpleasant and pleasant valence are furthest apart.
Materials and Methods
Participants
Eleven participants (7 females, mean ± SD age, 25.4 ± 2.0 years) gave informed consent to participate in an olfactory fMRI study approved by the Northwestern University Institutional Review Board. Participants reported no abnormal sense of smell or taste, history of neurological or psychiatric disease, history of nasal disorders, allergic rhinitis or sinusitis, or other fMRI counterindications. One subject was excluded from fMRI data analyses due to a lack of behavioral data, which were lost due to a technical problem, resulting in 10 subjects.
Odor stimuli
Nine different odorants were chosen to span the range of perceived valence: 0.05% ammonium sulfide, 0.25% trimethylamine, 0.3% isovaleric acid, 10% butanol, 3% acetophenone, 10% β-ionone, 100% vanillin, 50% limonene, and 15% methyl salicylate (see Fig. 1a). The odorants were diluted in either mineral oil or pure water (depending on odorant solubility). Odorants were delivered to subjects using an MRI-compatible, computer-controlled 10-channel olfactometer, based on a design by Lorig et al. (1999). Airflow was maintained at 2.5 L/min, and control of air valves was achieved using Cogent 2000 software (Welcome Department of Imaging Neuroscience, London), as implemented in MATLAB (The MathWorks).
Behavioral ratings
Before scanning, subjects rated odor valence using a visual analog scale (with anchors “extremely unpleasant” and “extremely pleasant”) as well as odor intensity (anchors “extremely weak” to “extremely strong”). All ratings were recorded via MATLAB and were converted to a scale ranging from −10 to 10 for analyses purposes. Mean differences in valence and intensity ratings were analyzed in SPSS using repeated-measures ANOVA. If Mauchly's test indicated that the assumption of sphericity had been violated, degrees of freedom were corrected using Greenhouse-Geisser estimates of sphericity.
Experimental design
The experiment consisted of 12 fMRI scanning sessions per day for 2 consecutive days (total of 24 sessions). Each odorant was presented once per scanning session (total of 9 trials per session) for 3 s, with a 12 s intertrial interval between odorants. Each trial consisted of a red fixation cross for 2 s that prepared subjects for odor onset, followed by a 3 s sniff cue (green fixation cross) that coincided with odor delivery. Each trial terminated with a black fixation cross that appeared at odor offset and stayed on until the next trial. The stimulus sequence of odorant presentation was counterbalanced across sessions.
fMRI data acquisition
fMRI data were acquired on a Siemens Trio 3T MRI scanner, using a 12-channel head coil and an integrated parallel acquisition technique known as Generalized Autocalibrating Partially Parallel Acquisition to improve signal recovery in medial temporal and basal frontal regions. Imaging parameters included: repetition time (TR), 1.51 s; echo time (TE), 20 ms; slice thickness, 2 mm with a gap of 1 mm; in-plane resolution, 1.72 × 1.72 mm; field of view, 220 × 206 mm, matrix size, 128 × 120 mm. To reduce susceptibility artifact in olfactory areas, a 30° tilt angle was applied to the image acquisition plane (Deichmann et al., 2003). A total of 24 slices per volume was collected ensuring an adequate coverage of olfactory brain regions. Additionally, a 1 mm3 T1-weighted MRI scan was acquired to define anatomical ROIs. An additional lower-resolution anatomical scan was acquired with the same slice protocol as the functional scans for image preprocessing and realignment.
Respiratory monitoring
For both the behavioral ratings and fMRI tasks, subjects were instructed to try and make the same-sized sniff on each and every trial. Respiration was monitored during fMRI scanning via breathing belts placed around the chest and abdomen. Sniff peak amplitude, duration, and inspiratory volume were computed for each trial, averaged across runs, and normalized within subjects by subtracting the mean parameter value (across conditions) from each condition-specific value. These data were then entered into individual repeated-measures ANOVA for statistical testing. One subject was excluded from the sniff data analyses due to technical problems in respiration data recording but was included in all other analyses.
Univariate fMRI analyses
fMRI data were analyzed with SPM8 software (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). For univariate analysis only, the functional images for each participant were spatially realigned, coregistered to that participant's first functional image and high-resolution anatomical T1 scan, spatially normalized using the MNI template brain, and smoothed with an isotropic Gaussian kernel of FWHM of 6 mm3. The first 6 “dummy” volumes of each run were discarded before statistical analyses.
For each subject, the BOLD contrast data were modeled separately for each odorant for each session using stick (δ) functions convolved with the canonical hemodynamic response function. The model included a 128 s high-pass filter to remove low-frequency fluctuations and an AR(1) model to account for temporal nonsphericity due to autocorrelations. Model estimation in SPM resulted in 216 (9 odorants × 24 sessions) parameter estimates (β) for each and every voxel, representing the magnitude of activation associated with each odorant. The odorant-related parameter estimates were then entered into a mixed-model ANOVA in SPM8 with subjects as a random factor and valence (pleasant, neutral, and unpleasant) as a fixed factor. The odorant-related parameter estimates were assigned to pleasant, neutral, or unpleasant condition based on each subject's ratings of the odorants. Hence, this analysis yielded regions that were significantly modulated by subjective valence. The fixation periods between trials constituted the implicit baseline used to investigate the independent effects of pleasant, neutral, and unpleasant odors. Conjunction “null” analyses (Friston et al., 2005; Nichols et al., 2005) were also conducted in SPM8 to examine whether amygdala activation differentiates between pleasant, neutral, and unpleasant stimuli. Conjunction contrasts of interest were pleasant > baseline, neutral > baseline, and unpleasant > baseline, and significance was set at p < 0.001, uncorrected, whereby each individual contrast was thresholded at p < 0.001.
Additionally, we examined whether univariate analyses showed differences in encoding valence in the amygdala at different time points following odorant onset by using finite impulse-response models. In the finite impulse-response model, each odorant was modeled using a set of impulse response function (7 bins of 1.5 s duration), with the first bin aligned to the initial onset of each event. With a TR of 1.5 s, these time bins corresponded to successive TRs (TR0 = stimulus onset, TR1 = 1.5 s, TR2 = 3 s, TR3 = 4.5 s, TR4 = 6 s, TR5 = 7.5 s, and TR6 = 9 s). This finite impulse-response model resulted in a map of 1512 (9 odorants × 7 time bins × 24 sessions) per-voxel β parameter estimates representing the magnitude of activation associated with each odorant. The following analyses focused on the time bins spanning the period between signal take-off and signal peak of the BOLD response (3–7.5 s) (Boynton et al., 1996; Aguirre et al., 1998), typically when emotional arousal-related differences emerge (Sabatinelli et al., 2009). The odorant-related parameter estimates were entered into a 3 × 4 full factorial ANOVA in SPM8 with two factors: valence (pleasant, neutral, and unpleasant) and time bin (4 time bins spanning TR2-TR5, i.e., 3–7.5 s, after odor onset). The odorant-related parameter estimates were assigned to pleasant, neutral, or unpleasant condition for each subject based on the subject's own ratings of the odorants. Hence, this analysis yielded regions that were significantly modulated by emotional valence at different time points after odor onset. All results were examined with (1) a whole-brain threshold of p < 0.05, family-wise-error (FWE) corrected; (2) an amygdala ROI generated using the AAL atlas (Tzourio-Mazoyer et al., 2002) via the WFU PickAtlas software (Maldjian et al., 2003; Maldjian et al., 2004), at a threshold of p < 0.05, FWE corrected for amygdala volume; and (3) a more liberal exploratory threshold of p < 0.005, uncorrected.
We also examined whether the mean activity in the amygdala ROI for the 9 stimuli differed based on valence and intensity ratings. After baseline correction (subtracting the mean activation across 3 TRs prestimulus onset), we computed the mean activation intensity for each odorant at each TR for each subject. Then, to examine whether the stimuli differed when ordered according to the subjective valence ratings, or showed a valence-related linear trend or intensity-related quadratic trend, we conducted a 9 odorant × 4 time-bin (TR) repeated-measures ANOVA in SPSS and tested the polynomial contrasts of linear, quadratic, and cubic transformations of the valence rating variable at each TR. To test whether mean amygdala activity differed based on the subjective intensity of the odorants, we ordered the stimuli according to subjective intensity ratings and tested whether there was a significant linear trend (from low to high intensity) in the mean (univariate) level of amygdala activity. Additionally, we also performed a direct test of the mean difference in amygdala activity between the most intense and the least intense stimuli.
Multivariate fMRI analyses
Data preprocessing for the multivariate pattern-based analysis (MVPA) followed a slightly different procedure compared with the univariate analysis. For each scanning session, we discarded the first six “dummy” volumes and then spatially realigned all functional images to the first volume of the first session using SPM8. No image normalization or smoothing was performed to preserve the subject-level fidelity of the fMRI signal. Subsequent steps, including extraction of brain activity pattern from each voxel, baseline correction, and assembly into linear vectors of odor-specific voxel activity, were done using customized scripts in MATLAB.
Multivoxel ensembles of odor-evoked fMRI activity patterns were extracted from left and right amygdala. For each subject, the left and the right amygdala were combined to yield one amygdala ROI. Amygdala ROIs were manually drawn on each subject's T1-weighted MRI scan using MRIcron software (http://www.mccauslandcenter.sc.edu/mricro/mricron/), with reference to a human brain atlas (Mai et al., 1997). To exclude voxels with excessive noise, such as voxels centered on large blood vessels, for each subject and each TR, we computed the SD of the response at each voxel and excluded voxels whose response was beyond 2 SDs of the mean across all the voxels within the ROI (Zelano et al., 2011).
To examine pattern coding of valence in the amygdala, we used an analysis strategy referred to as representational similarity analysis (RSA) (see Fig. 4). In RSA, multivoxel measures of fMRI activity are quantitatively related to each other and to computational models and behavioral data by comparing representational dissimilarity matrices (RDMs) (Kriegeskorte et al., 2008a,b). Importantly, before the RDMs were computed, for each subject, the odorants were ordered based on each subject's own valence ratings. Thus, rather than imposing an arbitrary ordering of stimuli across subjects, this procedure allowed for computation of RDMs based on individual perceptual ratings, which is particularly relevant when examining valence. To this end, for the 9-odorant set in our study, we created a 9 × 9 perceptual RDM, where each cell contained a value reflecting the dissimilarity (absolute value of rating differences) of valence or intensity associated with a given pair of odorants. We also created an amygdala RDM, where each cell in the 9 × 9 matrix contained a value reflecting the dissimilarity between brain activity patterns associated with a given pair of odorants. Details regarding how the amygdala RDMs were computed are provided below.
Data extraction for RSA.
For each subject and each of the 9 odorants presented in each of the 24 sessions, time-series data were extracted from each voxel within the amygdala (see Fig. 4a). Because single-trial data can be noisy (Howard et al., 2009; Zelano et al., 2011), we then averaged the time-series data across all trials within each day (12 trials per day for all subjects, except one subject who had 11 trials on the second day) for each odorant. The averaged fMRI time-series were then baseline-corrected by subtracting the mean activity averaged across the 3 TRs before onset of the stimulus. Because baseline activity of each voxel was subtracted from its own averaged time series, this baseline correction procedure does not affect the overall spatiotemporal pattern of the response.
Computation of the RDM.
For each subject and each TR, the averaged time-series data corresponding to each odorant were ordered based on the subject's valence ratings from the least to the most pleasant (Fig. 4a). Next, we generated a 9 × 9 RDM in which each cell reflects the dissimilarity (measured as 1 minus Pearson's r) between activity patterns for a pair of odorants arranged based on the subjective valence rankings (Fig. 4b,c) (Kriegeskorte et al., 2008a,b). In total, for each subject, there were eight RDMs: four corresponding to TR 2 to TR 5 for day 1 and four corresponding to TR 2 to TR 5 for day 2. Then, we averaged the RDMs across the 2 days, yielding four RDMs per subject. The RDMs were computed using customized scripts in MATLAB and the RSA toolbox (http://www.mrc-cbu.cam.ac.uk/methods-and-resources/toolboxes/).
For visualizing patterns in the RDMs, the amygdala activity pattern RDMs were submitted to multidimensional scaling (MDS; criterion: stress1, nonmetric) and displayed in a 2D space (see Fig. 5d). The stress value indicates how well the MDS plots reconstruct the dissimilarity matrices. A stress value <0.15 suggests a reliable representation of the matrix. The RDMs submitted to the MDS were group-level RDMs computed by averaging across all subjects for each TR. For each MDS plot, we computed the Pearson's correlation coefficients between the projections on the best-fitting axis (first dimension) and the mean valence ratings averaged across all subjects for each of the 9 conditions.
Comparing brain and perceptual dissimilarity matrices.
To statistically examine the correspondence between neural and perceptual coding of valence, Spearman's rank correlation coefficients were computed between RDMs for fMRI ensemble patterns in the amygdala and behavioral measures of stimulus similarity for each TR. Because RDMs are symmetric about a diagonal of zeros, the correlation coefficients were computed using values above the diagonal. To determine coherence between perceptual and amygdala RDMs, statistical inferences were performed on the RDM correlations. As classical inferential tests assume independence between each pair of measurements (which is not the case for RDMs, in which each cell also determines part of the rest of the matrix), we tested significance using permutation tests by randomizing the odor labels, in accordance with prior techniques (Kriegeskorte et al., 2008a,b). For each brain-behavioral RDM pair, we performed a conservative Monte Carlo random permutation 10,000 times, computing a Spearman's rank correlation coefficient in each iteration. These estimates served as a null distribution against which the actual correlation coefficient was compared. If the actual correlation coefficient fell within the top 5% of the simulated null distribution, we rejected the null hypothesis that the brain and perceptual RDMs are unrelated (Kriegeskorte et al., 2008a). The same analysis strategy was applied to test the match between the brain RDMs and both the valence and the intensity RDMs.
Next, to examine relative contribution of perceptual valence and intensity RDMs to variance in the amygdala RDM, we conducted a GLM using the vectorized valence and intensity RDMs as the linear predictors, and the vectorized amygdala RDM as the dependent variable. The estimated β coefficients for the predictor variables served as measures of the degree of match between the valence and intensity predictors and the amygdala ensemble pattern activity. To test for significance, for each TR we conducted 10,000 random permutation tests to yield null distributions of β coefficients for valence and intensity predictors. If observed βs fell beyond the 95th percentile of the null distributed βs, we rejected the null hypothesis that there is no match between the valence and intensity RDMs and the amygdala RDM.
Finally, we conducted a leave-one-out 10-fold cross-validation test to examine the consistency between brain and perceptual RDM match across subjects. In each iteration of the 10-fold cross-validation, we averaged the fMRI RDMs from 9 of 10 subjects, and then computed the Spearman's correlation between this averaged brain RDM and the 10th subject's perceptual RDM (valence and intensity RDMs separately). This was done 10 times, leaving one subject out each time. The group-level significance was determined by using a one-sample t test compared with 0 correlation.
Additional RDM analyses.
In addition to our main analyses based on subjective valence ratings, we conducted two other RDM analyses. The first of these involved creating a perceptual RDM based on objective valence ratings (see Fig. 1b,c), rather than subjective valence ratings. This was compared with a group-level amygdala RDM in which stimuli were ordered based on the objective valence rankings, and enabled us to determine whether there were differences between subjective and objective coding of valence in the amygdala. The second analysis involved creating a perceptual RDM based on subjective intensity ratings, rather than valence-sorted intensity ratings. Specifically, odorants were arranged based on each subject's own intensity rankings, where each cell contained a value reflecting the dissimilarity between subjective ratings of intensity for each pair of odorants. This was compared with a group-level amygdala RDM with stimuli ordered based on subjective intensity ranking, and otherwise followed the same RDM procedures described above for the main analyses.
Finally, to test whether subtle valence-specific changes in sniffing could have accounted for the effects in the amygdala, we computed three separate group-level sniff-related RDMs for sniff volume, peak amplitude, and sniff duration, using the same procedures above for the subjective valence RDMs. For each subject, the stimuli were ordered based on subjective valence rankings, and then RDMs were created so that each cell contained a pairwise difference in a sniff measure between each pair of stimuli. Group-level RDMs were then created by averaging across all subjects' RDMs. Random permutation tests were applied to test the significance of the correlation between sniff RDMs and the amygdala RDM for each TR.
Temporal and valence-specific differences in valence coding.
To examine whether valence representations in the amygdala are asymmetric for the two extremes of valence, we used two conceptual RDM models (see Fig. 6a). These models were created as 9 × 9 matrices, where each cell represented the dissimilarity between a pair of stimuli, on a color scale from yellow (dissimilar) to blue (similar). Model 1 captures the distinctiveness of unpleasant stimuli such that each stimulus shows greater dissimilarity along a gradient from pleasant to unpleasant. Hence, in Figure 6a, in the upper right triangle of the Model 1 RDM (symmetric about the diagonal), each column contains the dissimilarities between a stimulus and all the other stimuli that are unpleasant relative to itself (e.g., column 7 contains dissimilarities between stimulus ranked seventh with stimuli ranked first-sixth), and each row contains the dissimilarities between a stimulus with all the other stimuli that are pleasant relative to itself (e.g., row 7 contains dissimilarities between stimulus ranked seventh with stimuli ranked eighth-ninth). As a result, in Model 1, columns reflect dissimilarities (or differentiation) from relatively unpleasant stimuli and rows reflect no differentiation from relatively pleasant stimuli. Conversely, Model 2 captures the distinctiveness of pleasant stimuli, such that rows reflect differentiation from relatively pleasant stimuli and columns reflect lack of differentiation from relatively unpleasant stimuli.
To examine whether these two models (one emphasizing unpleasant differentiation and another emphasizing pleasant differentiation) accounted for unique variance in the amygdala RDMs across time, the models were vectorized and used as two predictors in a linear regression model, with the vectorized amygdala RDMs for each time point (TR2-TR5) as the dependent variable. Each regression yielded a β coefficient for each model. These β coefficients are a measure of the match between the conceptual models and the amygdala activity RDMs. Again, a random permutation test was used to test the significance of the β coefficients, as described above.
RDM analyses in additional regions.
To examine whether valence coding is specific to the amygdala, we conducted RDM analyses in two control regions: the piriform cortex (PC), which receives olfactory afferent input from the olfactory bulb, and the orbitofrontal cortex (OFC), which receives projections from both piriform cortex and amygdala and is extensively implicated in value-based coding. As in the construction of the amygdala ROI, the PC and OFC ROIs were manually drawn on each subject's T1-weighted scan. The olfactory OFC was defined based on a previous meta-analysis (Gottfried and Zald, 2005), describing its locus near the transverse orbital sulcus. The piriform cortex was drawn with reference to a human brain atlas (Mai et al., 1997). The time-series for each stimulus was extracted and ordered based on each subject's own valence rankings. Following exactly the same procedures applied to the amygdala analysis, we generated group-level RDMs of the PC and OFC by averaging across subjects separately for each TR. We tested the significance of correlation between the perceptual RDM (based on subjective valence ratings) and the PC or OFC RDM by using the Monte Carlo random permutation test. We also tested the pleasant and unpleasant models on the PC and OFC RDMs.
Results
Behavioral
We began by characterizing the perceptual space of the nine odors that included smells, such as vanilla, pine, and rotting fish (Fig. 1a). Repeated-measures ANOVAs were conducted to examine valence and intensity-related differences. Behavioral ratings of perceived valence were effectively distributed across the full spectrum of pleasantness (Fig. 2a) and exhibited a significant effect of valence (F(2.4,21.51) = 95.95, p < 0.0001), with a linear trend from least to most pleasant (F(1,9) = 212.27, p < 0.0001). By comparison, individual ratings of perceived intensity (F(8,72) = 4.68, p < 0.0001) spanned a range between “moderate” and “extremely” intense (Fig. 2b), exhibiting a significant quadratic trend for intensity (F(1,9) = 14.76, p = 0.004), in keeping with prior reported arousal profiles for visual stimuli (Kuppens et al., 2013). Overall, these findings validate the use of this odor stimulus set to evaluate the fMRI response space for valence.
Given that the perceptual ratings were collected only once before scanning, we also conducted several analyses to examine rating consistency both across subjects and across rating measures. Objective valence ratings were computed as the average rating across subjects for each of the odorants, and objective valence rankings were based on these objective valence ratings. First, the between-subject Pearson's correlation coefficients of objective valence ratings for the set of 9 odorants had a mean of 0.85, ranging from 0.63 to 0.98. Second, when stimuli were arranged according to subjective valence ratings, the between-subject correlation coefficients for stimuli from most unpleasant to most pleasant had a mean of 0.95, ranging from 0.85 to 0.99. Third, we compared each subject's subjective ratings from most unpleasant to most pleasant with the group-level objective ratings, which were the averaged ratings of each odorant ranked from low to high (Fig. 1b, red dots). The results had a mean correlation (r) of 0.98, ranging from 0.95 to 0.99. This strong inter-rater agreement, across three different analyses, provides some confidence in the perceptual robustness and stability of our stimulus set.
Analysis of the respiratory data, collected online during scanning, did not reveal any significant differences across the odor stimuli. Specifically, repeated-measures ANOVA results showed no systematic differences in sniff peak amplitude (F(2.53,20.24) = 1.06, p = 0.38), duration (F(3.77,30.17) = 1.2, p = 0.32), or inspiratory volume (F(8,64) = 1.47, p = 0.19) between odorants. One subject was excluded from sniff data analysis due to loss of data caused by technical problems.
Univariate fMRI results
Our initial imaging analysis explored whether the mean level of amygdala activity (as estimated using univariate fMRI approaches) significantly discriminated among different valence levels. The amygdala showed robust activation for pleasant, neutral, and unpleasant stimuli. A mixed-model ANOVA with three valence categories (pleasant, unpleasant, and neutral) as a factor showed no significant effect of valence and no significant pairwise differences (pleasant vs unpleasant, unpleasant vs neutral, pleasant vs neutral) in the amygdala, either when considered across the whole brain or within an amygdala ROI (at either FWE-corrected thresholds or a more liberal whole-brain p < 0.005, uncorrected threshold). Correspondingly, a conjunction “null” analysis (Friston et al., 2005; Nichols et al., 2005) showed greater activation for pleasant > baseline, neutral > baseline, and unpleasant > baseline in the amygdala (p < 0.001, uncorrected).
Next, to examine whether there was temporal difference in the amygdala's response to valence, we conducted a 3 × 4 repeated-measures ANOVA with valence categories (pleasant, unpleasant, and neutral) and time bins (4 time bins at 1.5 s intervals corresponding to 3–7.5 s after odor onset) as factors. Results showed no significant effects of valence or a valence × time interaction in the amygdala (all p values >0.005, uncorrected). Finally, when mean activity in the amygdala ROI was submitted to a repeated-measures 9 × 4 ANOVA, with the 9 odorants (arranged by subjective valence ratings from least to most pleasant) and 4 time bins (TR2-TR5) as factors, no significant linear trend (suggestive of valence-based differences, F(1,9) = 0.268, p = 0.617) or quadratic trend (suggestive of arousal-based differences, F(1,9) = 3.901, p = 0.08) was observed, nor was there an overall effect of valence (F(8,72) = 0.788, p = 0.615) (Fig. 3). Together, these univariate results show robust activation of the amygdala to all three types of odor stimuli with no evidence of differential coding of valence in the amygdala. Finally, we examined intensity-related differences in univariate amygdala activity. To do so, we submitted the mean activity of each stimulus in the amygdala ROI to a 2-way (9 stimuli × 4 time-bins) repeated-measures ANOVA, with the 9 stimuli ordered by subjective intensity ratings from low to high. There was no significant main effect of intensity (F(8,72) = 0.82, p = 0.588) or an interaction between intensity and time (F(24,216) = 0.981, p = 0.492). Additionally, when comparing the most intense and the least intense odors across subjects, no significant main effect of intensity (F(1,9) = 0.005, p = 0.947) or interaction between intensity and time bin (F(3,27) = 0.27, p = 0.847) was observed.
Multivariate fMRI results and amygdala ensemble coding of valence
In the absence of mean response differences across valence categories, we examined whether valence is encoded in finer-grained fMRI ensemble activity patterns in the amygdala. Ratings of odor valence were used to yield a 9 × 9 group-level, pairwise valence dissimilarity matrix (see Fig. 5a, left). Visual inspection of the valence RDM confirmed that odorants with similar valence show higher correlations along the diagonal than those with dissimilar valence on the off-diagonal. An intensity dissimilarity matrix was also generated, suggesting that intensity ratings of unpleasant and pleasant odorants were similar to each other and different from the neutral odorants (see Fig. 5a, right). The odorant stimuli in both the valence and intensity dissimilarity matrices were ordered on the basis of the subjective valence ratings (from least to most pleasant, Fig. 2a), as opposed to the objective valence ratings shown in Figure 1b.
Using the same stimulus order based on subjective ratings, we also generated a 9 × 9 amygdala RDM in which each cell represents the fMRI ensemble pattern dissimilarity for a given pair of stimuli (Figs. 4b, 5b) (Kriegeskorte et al., 2008b). Visual inspection of amygdala RDMs across time points showed a representation structure similar to the valence rating RDM, with stronger correlations between ensemble patterns for perceptually similar odors and weaker correlations for perceptually dissimilar odors (Fig. 5b). Multidimensional scaling of these RDMs further confirmed that amygdala activity patterns are organized based on valence (Fig. 5d; see Materials and Methods). Plots show that perceptually similar odorants tend to cluster together, consistent with their evoking similar fMRI ensemble patterns.
We then compared the perceived valence similarity RDM with the amygdala activity RDMs at each poststimulus time point. Random permutation tests confirmed that the amygdala RDM significantly mapped onto the valence RDM but not onto the intensity RDM (Fig. 5c). This analysis revealed a significant match between the perceived valence similarity RDM and the amygdala RDM from 3 s to 7.5 s after odor onset (r = 0.444, p = 0.0057 for TR2; r = 0.570, p = 0.0007 for TR3; r = 0.349, p = 0.0214 for TR4; r = 0.445, p = 0.0018 for TR5) (Fig. 5c). This result indicates that odors sharing more similarity in perceived valence are associated with greater pattern overlap of amygdala ensemble activity. In contrast to the strong relationship between amygdala activity RDMs and valence RDM, there was no significant match between amygdala activity RDMs and the intensity RDM at any time point (Fig. 5c; r = 0.2903, p = 0.0970 for TR2; r = 0.1627, p = 0.2596 for TR3; r = 0.0118, p = 0.4763 for TR4; r = −0.1797, p = 0.7443 for TR5).
Next, we tested the correlation between amygdala RDMs and the objective valence RDM at each TR. The results were similar to that of subjective valence RDMs, with significant correlations from TR2 to TR5 (TR2, r = 0.35, p = 0.02; TR3, r = 0.46, p = 0.03; TR4, r = 0.40, p = 0.01; TR5, r = 0.30, p = 0.04). Given that the odor stimuli were ranked very similarly by all participants (Fig. 1c), these results were not too surprising. We also tested the correlation between amygdala RDMs (with stimuli arranged based on subjective intensity rankings) and the subjective intensity RDM. The results showed no significant match at any of the time points: TR2, r = 0.25, p = 0.10; TR3, r = 0.29, p = 0.09; TR4, r = 0.13, p = 0.27; TR5, r = 0.05, p = 0.40. Finally, we examined the correlation between amygdala RDMs and sniff volume, peak amplitude, and duration RDMs for each TR (see Materials and Methods). Random permutation tests on the resulting Spearman's correlation coefficients revealed no significant correlation for sniff volume (TR2, r = −0.18, p = 0.80; TR3, r = −0.22, p = 0.81; TR4, r = 0.11, p = 0.30; TR5, r = 0.31, p = 0.10), sniff peak amplitude (TR2, r = −0.17, p = 0.80; TR3, r = −0.17, p = 0.76; TR4, r = −0.10, p = 0.66; TR5, r = −0.05, p = 0.59), or sniff duration (TR2, r = −0.28, p = 0.94; TR3, r = −0.16, p = 0.77; TR4, r = −0.08, p = 0.65; TR5, r = 0.21, p = 0.14).
To assess statistically whether the similarity structure of perceived valence or intensity independently explains the amygdala RDM, we conducted a GLM using vectorized valence and intensity RDMs as the independent variables and the vectorized amygdala activity RDM as the dependent variable. The results revealed that the perceptual valence matrix significantly correlated with the brain data from TR2 to TR5 (GLM coefficients or β coefficient for different TRs: β = 0.0037, p = 0.0096 for TR2; β = 0.0063, p = 0.0006 for TR3; β = 0.0033, p = 0.0087 for TR4; β = 0.0045, p = 0.0003 for TR5). In contrast, no significant match was found for the perceptual intensity matrix (GLM coefficients: β = 0.0044, p = 0.31 for TR2; β = 0.0033, p = 0.40 for TR3; β = 0, p = 0.53 for TR4; and r = −0.0077, p = 0.23 for TR5). Additionally, we used a leave-one-out analysis to assess the coherence between amygdala and perceptual RDMs across subjects. Direct comparisons of the correlations between the perceptual and amygdala RDMs (one-sample t tests with the null hypothesis of zero correlation) confirmed significant matches between the amygdala RDM and the valence RDMs at all time points: TR2 (t(9) = 7.44, p = 3.9 × 10−5), TR3 (t(9) = 14.52, p = 1.5 × 10−7), TR4 (t(9) = 16.95, p = 3.9 × 10−8), and TR5 (t(9) = 14.85, p = 1.2 × 10−7). In contrast, effects were not significant for the intensity RDMs at any time point (TR2, t(9) = 2.05, p = 0.07; TR3, t(9) = 0.21, p = 0.84; TR4, t(9) = −1.39, p = 0.20; and TR5, t(9) = −1.77, p = 0.11). Overall, results indicate that the human amygdala supports a continuous representation of the valence dimension ranging from unpleasant to pleasant.
Temporal differences in valence-specific ensemble coding in the amygdala
Earlier research has shown a bias, or asymmetry, in the dimensional representation of valence, such that unpleasant stimuli are perceptually more differentiated than pleasant stimuli (Rozin and Royzman, 2001; Kuppens et al., 2013). Moreover, asymmetry has been observed in the temporal dynamics of emotional responses, with earlier mobilization and regulation in the case of unpleasant stimuli (Taylor, 1991). It thus follows that amygdala representations for unpleasant (vs pleasant) stimuli might emerge earlier in the time course of stimulus delivery, resulting in greater pattern dissimilarity among unpleasant odors at early time points. To test this bias, we used two conceptual models: one in which ensemble pattern representations with unpleasant stimuli are more distinctive and another in which ensemble representations with pleasant stimuli are more distinctive (Fig. 6a). Results indicate that, early after stimulus onset, the observed amygdala RDM resembled the results predicted from the unpleasant differentiation model, whereas at later time points, the amygdala RDM more closely resembled the results predicted from the pleasant differentiation model (Fig. 6b,c), implying earlier differentiation of unpleasant stimuli.
To establish significance, for each TR, we conducted 10,000 random permutation tests to yield null distributions of β coefficients for the pleasant and unpleasant models, following the same strategy used for testing matches between the amygdala and perceptual RDMs. In line with the above observations, the unpleasant differentiation RDM significantly correlated with the amygdala pattern RDMs from TR2 to TR4 (β = 0.0117, p = 0.0029 for TR2; β = 0.0147, p = 0.0056 for TR3; β = 0.0063, p = 0.0452 for TR4; β = 0.0068, p = 0.0792 for TR5). In contrast, the pleasant differentiation RDM exhibited a significant match with the brain data one TR later, from TR3 to TR5 (β = 0.0044, p = 0.1555 for TR2; β = 0.0107, p = 0.0406 for TR3; β = 0.0068, p = 0.0323 for TR4; β = 0.0122, p = 0.0001 for TR5).
RDM analyses in additional regions
Unlike the amygdala RDM results, we found no significant correlation between the subjective valence RDM and the PC RDM for TR2 (r = 0.075, p = 0.307), TR3 (r = 0.291, p = 0.054), or TR4 (r = 0.026, p = 0.443), with only TR5 reaching significance (r = 0.306, p = 0.037). We also tested our two valence models on the PC. In contrast to our amygdala findings, the GLM coefficient β was only significant for the unpleasant model at TR5 (β = 0.0088, p = 0.02), but at no other time point for either the unpleasant or pleasant model. In OFC, the correlation between the subjective valence RDM and the OFC RDM was significant only at TR4 (∼6 s after stimulus onset, with r = 0.36, p = 0.02), coinciding with the peak of the BOLD response (Fig. 3), but at no other time points (TR2, r = −0.019, p = 0.538; TR3, r = 0.235, p = 0.091; TR5, r = 0.202, p = 0.115). When tested on the pleasant and unpleasant models, the positive model showed a significant match with the OFC RDM only at TR4, with β = 0.0075, p = 0.0082.
Discussion
How valence information is encoded in the human amygdala remains poorly understood. By combining olfactory fMRI with pattern-based multivariate techniques, we identified a robust correlation between ensemble pattern activity in the amygdala and subjective perception of valence. Complementary imaging analyses revealed that amygdala representations of valence evolve during stimulus presentation, prioritizing early sensitivity toward unpleasant stimuli. These findings help clarify the mechanisms of affective processing in the amygdala by underscoring the idea that unique spatial (pattern) and temporal features encode pleasant and unpleasant valence, and that the entire valence dimension from pleasant to unpleasant is represented in this brain region.
Prior human neuroimaging work has not consistently identified differentiation of valence in the amygdala (Lindquist et al., 2015). One reason why such findings have been elusive is that most of these studies have relied on univariate fMRI approaches, which are insensitive to distributed forms of valence content, as suggested by our univariate analysis in Figure 3. To the extent that pleasant and unpleasant stimuli are encoded as sparse and distributed representations in the amygdala (Paton et al., 2006; Belova et al., 2008; Herry et al., 2008), it follows that mean changes in the amygdala fMRI signal would be poorly suited to identify differences in valence. Using multivariate approaches, we were able to gain new insights into how subjective valence is encoded in the human amygdala. These effects were regionally specific, in that similar patterns were not observed in piriform cortex or OFC, although it is worth noting that the limited sample size and a potential gender imbalance in our sample could have affected the power to detect significant positive findings.
Interestingly, univariate fMRI responses in our study were also insensitive to the intensity of odor stimuli, findings that diverge from earlier findings showing that mean amygdala activity codes intensity (Anderson et al., 2003). Whereas this earlier study independently manipulated valence and intensity by using low and high intensity levels of one pleasant odor and one unpleasant odor, our study consisted of stimuli spanning the whole continuum of valence with only one intensity level per stimulus, which could explain why representations of odor intensity were not identified in the amygdala. Additionally, our prior work (Winston et al., 2005) indicated that perceived intensity does not exert a linear effect on the amygdala response but only emerges at the extremes of valence, when a neutral-valenced odor was also included in the stimulus set. As such, our experimental design was not optimized to replicate this particular finding.
By implementing multivariate fMRI techniques to examine similarities of neural representations (Edelman, 1998; Haxby et al., 2001; Hanson et al., 2004; Kayaert et al., 2005; Lehky and Sereno, 2007; Kay et al., 2008; Kriegeskorte et al., 2008a,b), we were able to show that spatial ensemble patterns in the amygdala capture systematic changes for different stimuli across the full valence spectrum. Thus, the amygdala showed stronger pattern correlations among odors of similar valence and weaker correlations among odors of dissimilar valence: for example, the amygdala pattern representations of vanilla and lemon smells were highly overlapping to each other but highly divergent to the pattern representation of rotten egg smell. It is important to note that, although our main goal focused on the match between the similarities of amygdala ensemble coding of subjective valence, the similarities of amygdala ensemble patterns were also correlated with objective valence, supporting the notion that there was strong intersubject consistency across valence ratings.
It is worth noting that our findings in the amygdala complement a recent neuroimaging study showing that ensemble patterns in the orbitofrontal cortex support a continuous dimension ranging from pleasant to unpleasant (Chikazoe et al., 2014). These orbitofrontal population codes were independent of sensory modality and could be used to classify experienced affect across participants. Surprisingly, this other study did not identify population coding of subjective valence in the amygdala, possibly because imaging parameters were not optimized for the medial temporal lobe, or because the amygdala may be less sensitive to visual and gustatory (as opposed to olfactory) stimuli, which were used in this other study. However, consistent with the Chikazoe data, our results did identify a multivariate valence code in the OFC at the peak TR, although it is worth noting that valence coding in the amygdala emerged earlier (from TR2) and was more prolonged (through TR5) than in the OFC. Additionally, the temporal differentiation from unpleasant to pleasant was demonstrated only in the amygdala. It will be important to examine in future studies whether the valence codes in amygdala and orbitofrontal cortex are redundant or whether these support unique functionality.
Our results showed a systematic linear differentiation of amygdala fMRI patterns from the pleasant end to the unpleasant end of the valence spectrum. This profile, in which the same set of amygdala voxels encodes pleasant and unpleasant odor content, but with different response topographies, provides positive support for a unidimensional bipolar model. However, our findings are unable to rule out the possibility of separate codes for pleasant (ranging from mid-neutral to very pleasant) and for unpleasant (ranging from mid-neutral to very unpleasant) in the amygdala, either within the same sets of voxels or across entirely different sets of voxels. Such representations might be more likely to emerge at the single-voxel level, or even at the neuronal (cell body) level beyond the resolution of our fMRI technique. Thus, although we believe that our multivariate pattern results are more compatible with a unidimensional representation ranging from pleasant to unpleasant, further investigation will be necessary to explore whether different codes are embedded in the amygdala response patterns at the neuronal or voxelwise level.
A unidimensional representation of valence in the amygdala is in line with similar valence structures observed in similarity judgments of emotional words across different cultures (Russell, 1983) and emotional expressions by adults and preschoolers (Russell and Bullock, 1986). The unidimensional representation implies that stimuli at opposite ends of the dimension are mutually exclusive, such that any degree of pleasantness is assumed to preclude unpleasantness and vice versa (Russell and Carroll, 1999), a proposal that has also received experimental support from several behavioral studies (Harris, 1929; Carroll et al., 1999; Zellner et al., 2003, 2006; Rashotte and Wedell, 2012). Our findings showing stronger correlations between ensemble patterns for perceptually similar odors, which decrease monotonically for perceptually dissimilar odors, imply that unpleasant and pleasant valence cannot coexist (be represented simultaneously) in the amygdala at the level of fMRI ensemble codes because pattern overlap of these two valence extremes would obscure their unique content. Such an organization would create greater flexibility in determining pleasantness or unpleasantness of stimuli. It would also provide a mechanism by which expectation, context, attention, and learning could influence decision boundaries for driving approach or avoidance responses via amygdala modulation (O'Doherty et al., 2002; Ochsner et al., 2002; Pessoa et al., 2002; Swanson, 2003; Etkin et al., 2006). The dense and bidirectional connectivity between amygdala and cortical and subcortical brain regions (Young et al., 1994; Barbas, 1995; Pessoa, 2008) reinforces the idea that the amygdala is an ideal substrate where second-order representations of value could be robustly modulated.
In conclusion, our study shows that valence representations in the amygdala are not static but change over time, prioritizing early differentiation of unpleasant stimuli. Amygdala representations for unpleasant (vs pleasant) stimuli were more distinctive as reflected by greater pattern dissimilarity among the unpleasant odors earlier during stimulus presentation. These findings are in line with behavioral studies that have shown that negative stimuli are generally construed as more elaborate and differentiated than the corresponding positive stimuli (Peeters, 1971; Peeters and Czapinski, 1990). A more diverse representation of negative stimuli may ensure a richer repertoire of behaviors that individuals can choose from more rapidly. As opposed to a generic approach behavior toward pleasant outcomes, adaptive emotion regulation in response to unpleasant stimuli involve more subtle differentiation earlier in time (Barrett et al., 2001). Finer, and earlier, discrimination of unpleasant stimuli in the amygdala is also in line with studies of behavioral and neural temporal dynamics of the emotional response (Huang and Luo, 2006; Yang et al., 2007, 2007) showing earlier mobilization and regulation of response for unpleasant compared with pleasant stimuli. Overall, whereas the amygdala represents the entire valence code ranging from pleasant to unpleasant, asymmetry in the spatial and temporal characteristics of this valence code ultimately allows for a greater advantage for unpleasant stimuli.
Footnotes
This work was supported by National Institutes of Health, National Institute on Deafness and Other Communication Disorders Grant R01DC010014 to J.A.G., supplemental National Institutes of Health, American Recovery and Reinvestment Act of 2009 3R01DC010014-01S1 to J.A.G. and A.M., and Stony Brook University to A.M. We thank James Howard, Nicholas Eaton, Christian Luhmann, and Todd Parrish for helpful suggestions; and Katie Phillips for assistance with data collection.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Aprajita Mohanty, Stony Brook University, Department of Psychology, Stony Brook, NY 11794-2500. Aprajita.Mohanty{at}stonybrook.edu