Abstract
There is a growing consensus in behavioral neuroscience that the brain makes simple choices by first assigning a value to the options under consideration and then comparing them. Two important open questions are whether the brain encodes absolute or relative value signals, and what role attention might play in these computations. We investigated these questions using a human fMRI experiment with a binary choice task in which the fixations to both stimuli were exogenously manipulated to control for the role of visual attention in the valuation computation. We found that the ventromedial prefrontal cortex and the ventral striatum encoded fixation-dependent relative value signals: activity in these areas correlated with the difference in value between the attended and the unattended items. These attention-modulated relative value signals might serve as the input of a comparator system that is used to make a choice.
Introduction
There is a growing consensus in decision neuroscience that the brain makes simple choices by first assigning a value to all of the options under consideration and then comparing them (Padoa-Schioppa and Assad, 2006; Wallis, 2007; Rangel et al., 2008; Kable and Glimcher, 2009; Rushworth et al., 2009; Rangel and Hare, 2010). This has motivated a growing interest in characterizing the exact computational properties of the processes responsible for the value comparison, and in understanding the extent to which they are able to generate reward-maximizing choices.
Although many popular models of value-based choice implicitly assume that the comparison process involves a trivial instantaneous maximization problem (Kahneman and Tversky, 1979; Mas-Colell et al., 1995), casual observation suggests that the underlying processes at work are more sophisticated and that visual fixations are likely to play a role. Consider, for example, a typical buyer at the grocery store choosing between two snacks: a bag of chips and a candy bar. Instead of approaching the counter and immediately selecting his or her preferred option, the individual's gaze shifts repeatedly between the items until one of them is eventually selected. This suggests that attention might play a role in the computation and comparison of values during the choice process.
We hypothesized that visual attention plays a critical role in the computation of value signals during the choice process. In particular, we hypothesized that the stimulus value signals known to be computed in ventromedial prefrontal cortex (vmPFC) (Padoa-Schioppa and Assad, 2006; Kable and Glimcher, 2007; Tom et al., 2007; Hare et al., 2008, 2009, 2010; Boorman et al., 2009; Croxson et al., 2009; FitzGerald et al., 2009; Basten et al., 2010; Smith et al., 2010) and ventral striatum (vStr) (Kable and Glimcher, 2007; Knutson et al., 2007; Tom et al., 2007; Rangel et al., 2008) at the time of choice would be modulated by visual attention so that at any given time they encode a relative value signal equal to the difference in value between the attended and the unattended items.
This hypothesis was motivated by two prior sets of behavioral findings. First, previous behavioral studies have shown that exogenous changes in fixation patterns during the decision process can affect choices (Shimojo et al., 2003; Armel and Rangel, 2008). This suggests that the brain might compute attention-dependent relative value signals. Second, a recent eye-tracking study showed that a version of the drift-diffusion model (DDM) of decision-making (Gold and Shadlen, 2002; Ratcliff and McKoon, 2008), in which the integration slope was determined by an input relative value signal modulated by attention, was able to account for basic psychometric and fixation patterns in the choice data with high quantitative accuracy (Krajbich et al., 2010). Again, consistent with our hypothesis, this suggests that the brain might compute attention-modulated relative value signals at some point in the decision-making process.
Materials and Methods
Subjects
Twenty right-handed subjects (20–35 years old; 12 males) participated in the experiment. All of them provided informed consent, as approved by the Institutional Review Board of the California Institute of Technology. Participants were in good health and had normal or corrected-to-normal vision. Furthermore, subjects had no history of allergies in response to the food items used in the experiment.
Task
Subjects were instructed not to eat for 4 h before each experimental session.
Participants completed three fMRI sessions (∼1.5 h/session) in a week (typically, on consequent days). In total, they performed one liking-rating task run (60 trials) and 14 binary choice task runs (30 trials for each run).
Subjects rated and made choices among 60 different foods (e.g., beef jerky, chocolate bar). The food stimuli were presented through an LCD projector at a resolution of 1024 × 768 pixels and a refresh rate of 60 Hz. The stimulus presentation and behavioral response collection were controlled by Presentation software (Neurobehavioral Systems). During the experiment, eye-position data were acquired with an Eye-Trac 6 system (Applied Science Laboratories).
Liking-rating task.
During the initial liking-rating trials, which took place in the first experimental session, each food image was shown once for a total of 60 rating trials. Food images were presented for 2 s at fixation, subtended ∼6.7° × 6.7° visual angle. During a subsequent 2 s response period, participants were asked to rate their preference for each food item with a five-point scale (strongly dislike, dislike, neutral, like, strongly like). To exclude motor-related responses of no interest, the response button mapping was counterbalanced (left to right; right to left) across participants. The stimuli were presented in random order. A white fixation cross (randomly jittered; 2–6 s) was presented between trials.
Binary choice task.
Every choice trial, subjects were presented simultaneously with two food images, one surrounded by a red frame and one surrounded by a green frame (for a sample screen, see Fig. 1A). The centers of the two frames were 11.2° apart. The pair and location of items were assigned randomly in each trial.
Critically, subjects were instructed to fixate their eyes on the food item marked with the target color frame (red or green) throughout the trial. The target color was counterbalanced across participants. Cued by the colored frames, subject's eye-fixation was alternatively directed to one of two food items. The initial position of the target frame was randomly selected (left or right) and then the target position was switched three times within a trial (L–R–L–R or R–L–R–L). The duration of each eye-fixation segment was randomly varied from 1 to 4 s in step of 1 s (for sample eye-fixation data, see Fig. 1A; for a sample trial, see Fig. 1B).
After stimulus presentation, participants indicated their choice for the trial using response button pads (left or right hand) during a 2 s response period. The start of the choice trials were separated by 18 s, and a white fixation cross was presented between trials.
At the end of each of the three fMRI sessions, subjects received the item that they chose in a randomly selected choice trial and were allowed to eat it immediately. Note that the participant's optimal strategy is to treat every trial as if it were the one that counted, since they did not know which one would be implemented, and that the optimal choice in each trial is independent of the decision made in other trials. A total of 420 choice trials were presented over the three fMRI sessions.
To familiarize subjects with experimental manipulations of alternating eye-fixation, 10 two-choice training trials were presented during the initial T1 scan.
MRI data acquisition
Anatomical and functional scans were acquired using a 3T TRIO scanner (Siemens Medical Systems) with an eight-channel phased-array head coil. Structural images were acquired first with a high-resolution MPRAGE anatomical sequence (TR = 1500 ms; TE = 3.05 ms; TI = 800 ms; 1 mm isotropic voxel; 256 mm field of view). Next, blood oxygenation level-dependent (BOLD) contrast functional images were acquired with gradient-echo echo-planar T2*-weighted imaging. To optimize functional sensitivity of signals in the orbitofrontal cortex, T2* images were acquired in an oblique orientation of 30° to the anterior commissure–posterior commissure line (Deichmann et al., 2003). Each functional volume consisted of 44 axial slices (TR = 2750 ms; TE = 30 ms; FA = 80°; FOV = 192 mm; 64 × 64 matrix; 3 mm isotropic voxel).
Eye-tracking data
Eye-position data were recorded at a sample rate of 120 Hz while subjects performed the task in the fMRI scanner. Of 60 fMRI sessions collected (20 subjects * 3 sessions), eye-position data for eight sessions was not available due to technical difficulties (recording, triggering, or calibration failures). Raw eye-position data were median-filtered over moving windows of 50 ms (6 samples) to remove artifacts and down-sampled at 10 Hz.
To quantify and check whether participants complied with our exogenous attentional manipulations, eye-position data were analyzed using an ROI (regions of interest) method. We created two ROIs that corresponded to the left and right color frames (9.9° × 9.9° visual angle). During binary choice trials, subjects were required to switch their eye fixations to the left or right items as directed by a colored frame (ROI). If the subjects' eye-fixation sample was positioned within the target ROI, it was counted as a correct fixation.
We applied the following rules to calculate eye-fixation accuracy of each trial. First, the trials that had >25% data loss (i.e., the eye tracker did not record eye-position data) were excluded as invalid trials. By this rule, seven sessions' eye-fixation data were excluded from further analysis. Thus, eye-fixation data from 45 of 60 sessions were used for an accuracy analysis. Second, to account the delay of eye-fixation transition, the first five samples (500 ms) after the onset of each stimulus display were not used for calculations. Third, accuracy was calculated by a ratio of the number of correct fixations divided by the total number of valid fixations. Fourth, the accuracy index for each subject was calculated by averaging the accuracies of all datasets.
Psychometric fits
The behavioral choice data were fitted using a logistic regression. The choice trials were divided into two types by relative eye-fixation duration (TL > TR or TL < TR) and the psychometric fits were performed separately. Each trial type was regressed against the value difference between the left and right options (VL − VR). In addition to the constant and slope parameters of a logit function, the point of subjective equality (PSE) was estimated for each individual. For group-level analyses, the parameters of psychometric curves were compared via one-sample and paired t tests.
fMRI preprocessing
Analysis of fMRI data were performed using SPM-8 (Wellcome Trust Centre for Neuroimaging, London, UK). The first four functional volumes of each run were removed to account for equilibration effects of magnetization. The following processing steps were applied: slice-time correction, motion correction, spatial resampling (2 × 2 × 2 mm) and normalization to the standard MNI (Montreal Neurological Institute) template, Gaussian spatial smoothing (FWHM: 8 mm), intensity normalization, and high-pass temporal filtering (using a filter width of 128 s).
ROIs selection
Although we performed and report results of whole-brain analysis, the main goal of this paper was to investigate the extent to which the stimulus value signals that are known to arise in the vmPFC and vStr at the time of decision-making (Kable and Glimcher, 2007; Knutson et al., 2007; Plassmann et al., 2007; Tom et al., 2007) are modulated by visual attention. As a result, we performed various ROI analyses in these two areas of interest.
The vmPFC and vStr ROIs were determined as follows. First, we used the data of the liking-rating run as a localizer. In particular, we conducted a whole-brain voxelwise analysis on only this dataset via a two-stage mixed-effects analysis (see GLM-1, below) to identify brain regions that encode stimulus values. Next, we constructed a vmPFC ROI that was given by all of the voxels in which activity was positively correlated with the ratings at p < 0.05 (corrected) within an anatomically defined mask of the medial OFC (bilateral medial-orbital and rectus gyrus masks of an SPM standard brain; MNI center: x = −6, y = 18, z = −10, 69 voxels). Finally, the ROI of vStr was determined using an analog method for the vmPFC (note that voxels superior to z = 4 were not included; MNI center: x = −11, y = 15, z = −5, 65 voxels). Because at the group level only the left vStr exhibited a significant correlation with the liking ratings at our omnibus threshold of p < 0.05 (corrected), the ROI analysis was not performed in the right vStr.
Generalized linear model analysis
We estimated several generalized linear models (GLMs) of the BOLD data with first order autoregression to test the hypotheses of interest. All the GLMs included motion parameters and run constants as regressors of no interest. A two-stage mixed-effects analysis was performed in which the regression coefficients for each condition of interest were tested across subjects via t tests or repeated-measures ANOVAs.
We performed multiple-comparison corrections at the cluster level by Monte Carlo simulations with AlphaSim program (http://afni.nimh.nih.gov). Statistical inferences from the whole-brain analyses were performed at a corrected threshold of p < 0.05 by imposing a p < 0.005 statistical threshold and a minimum cluster extent of 133 voxels. For the predetermined regions of interest, including the vmPFC, vStr and superior temporal gyrus (STG), we performed small-volume corrections at the cluster level (extent threshold: k = 40, 41, and 70 voxels, respectively). Activation coordinates are reported using MNI coordinates.
GLM-1.
GLM-1 was estimated on the liking-rating trials only to characterize the ROIs described above. The model included the following regressors: (1) an indicator function (1 for events, 0 otherwise) for the entire duration of the rating trials, and (2) a parametric modulator function constructed by multiplying an indicator function by each subject's stimulus values for the food items [i.e., ratings of each subject, ranging from strongly dislike (−2) to strongly like (+2)]. Both indicator and parametric functions were convolved with a canonical hemodynamic response function (HRF). Note that the parametric regressor was used to identify brain areas in which BOLD signals correlate with the magnitude of subjective stimulus values (i.e., for regions in which the beta coefficient for the second regressor is positive).
GLM-2.
GML-2 was estimated on all of the choice trials to investigate the effect of visual attention on the relative value computations. The model included the following regressors: (1) an indicator function of fixation left (fixL), (2) a parametric modulator function of fixation left by each subject's differences for the left (attended) and right (unattended) items (VL − VR), (3) an indicator function of fixation right (fixR), and (4) a parametric modulator function constructed of fixation right by each subject's differences of stimulus values between left (unattended) and right (attended) items (VL − VR). These regressors were modeled as boxcar functions (1 for events, 0 otherwise) matched with the length of eye fixation on that location (1–4 s). The subjective stimulus values used to construct the parametric modulators were taken from the responses of initial liking-rating trials. The two contrasts of interest in this GLM were the second regressor, fixL * (VL − VR); and the fourth regressor, fixR * (VL − VR).
GLM-3.
GLM-3 was estimated in a way similar to GLM-2 above, except trials were divided by choice type (left or right). This GLM was designed to investigate whether the relative value computations were independent of the eventual choice. The model included the following regressors: (1) an indicator function of fixation left of left choice trials (fixL|CL), (2) a parametric modulator function of fixation left of left choices by value difference between left and right items (VL − VR), (3) an indicator function of fixation right of left choice trials (fixR|CL), (4) a parametric modulator function of fixation right of left choices by value difference between left and right items (VL − VR), (5) an indicator function of fixation left of right choice trials (fixL|CR), (6) a parametric modulator function of fixation left of right choices by value difference between left and right items (VL − VR), (7) an indicator function of fixation right of right choice trials (fixR|CR), and (8) a parametric modulator function of fixation right of right choices by value difference between left and right items (VL − VR). The four contrasts of interest in this GLM were the second regressor, fixL * (VL − VR)|CL; the fourth regressor, fixR * (VL − VR)|CL; the sixth regressor, fixL * (VL − VR)|CR; and the eighth regressor, fixR * (VL − VR)|CR.
GLM-4.
This GLM was also estimated on all of the choice trials to investigate the effect of visual attention on the value computations, separately for attended and unattended items. The model included the following regressors: (1) an indicator function of fixation left (fixL), (2) a parametric modulator function of fixation left by each subject's stimulus values for the left (attended) items (VL), (3) a parametric modulator function of fixation left by each subject's stimulus values for the right (unattended) items (VR), (4) an indicator function of fixation right (fixR), (5) a parametric modulator function of fixation right by the right (attended) items (VR), and (6) a parametric modulator function of fixation right by the left (unattended) items (VL). To test our main hypotheses in the second-level analyses, we performed a two (fixation side: left vs right) by two (attention condition: attended vs unattended values) repeated-measures ANOVA with four parametric contrasts (fixL * VL, fixL * VR, fixR * VR, and fixR * VL) of the first-level GLM. We also investigated the simple contrast fixL versus fixR.
GLM-5.
GLM-5, estimated also in the choice trials, was designed to investigate the effect of attention on the computation of the value of individual stimuli. The attended and unattended item values were modeled separately regardless of fixation conditions. We estimated two GLMs. The full model, GLM-5a, included the following regressors: (1) an indicator function of fixation (left and right fixations were pooled together) (fixLR), (2) a parametric modulator function of fixation multiplied by the stimulus values of attended items (fixLR * Vatt) (the left and right fixation indicators were multiplied by their corresponding stimulus values and pooled together; fixL * VL + fixR * VR of GLM-4), (3) a parametric modulator function of fixation multiplied by the stimulus values of unattended items (fixLR * Vunatt; fixL * VR + fixR * VL of GLM-4). The nested (restricted) model, GLM-5b, was identical to GLM-5a except that it did not include a parametric modulator of unattended items value (fixLR * Vunatt). We then estimate the following two contrasts at the second-level using one-sample t tests. The first contrast was given by H1: fixLR * Vatt > 0. The second contrast was given by H1: fixLR * Vunatt < 0. Additionally, we carried a model comparison between the two nested models using an F test of model fit.
Psycho-physiological interaction analysis
We performed two exploratory psycho-physiological interaction (PPI) analyses to identify brain regions that exhibit attention-dependent functional connectivity with vmPFC.
In both cases, the analysis started by constructing a spatially averaged time series of the vmPFC that was extracted from a 4 mm radius sphere surrounded by individual peaks given by the main effect of attention condition in GLM-4 (fixL * VL − fixL * VR + fixR * VR − fixR * VL). The vmPFC ROI was selected as a seed region of PPI analyses. We expected that brain regions involved in attentional modulation might show increased task-dependent functional connectivity with this vmPFC area, influencing the computation of the value. Nuisance variances associated with motion parameters were removed from the time series, which was then deconvolved using a model of a canonical HRF function to construct a time series of neural activity in the vmPFC area of interest (Gitelman et al., 2003).
A previous study found significant functional connectivity between vmPFC and STG/middle temporal gyrus (STG/MTG) in a stimulus-valuation task (Hare et al., 2010). As a result, for these target regions of interest, we performed small volume corrections at the cluster level using anatomically defined bilateral mask of STG/MTG (as provided by SPM tools) to achieve a corrected threshold of p < 0.05. This was achieved by using a threshold of p < 0.005, uncorrected, with a minimum cluster extent of 70 voxels.
PPI-1.
The goal of the first PPI was to identify brain areas in which BOLD activity correlates with the vmPFC during the choice trials. To do this, we estimated a GLM with following regressors: (1) an interaction between the neural-level (deconvolved with a HRF) activities in the vmPFC and an indicator function for two-choice trials (PPI), (2) an indicator function for the choice trials (1 for task, 0 otherwise), and (3) the extracted time series in the vmPFC (seed). The first two regressors were convolved with a canonical HRF and the GLM also included six motion parameters and run constants as regressors of no interest. The second-level analysis was performed by calculating one-sample t tests on first-level contrasts of the interaction term (PPI).
PPI-2.
The goal of the second PPI was to explore whether the regions identified in PPI-1 exhibited differential functional connectivity with the vmPFC during the left and right fixations. To do this, we estimated a GLM with the following regressors: (1) an interaction between the neural-level (deconvolved with a HRF) activities in the vmPFC and an indicator function for fixation-left condition of choice trials (PPI fixL), (2) an interaction between the neural-level (deconvolved with a HRF) activities in the vmPFC and an indicator function for fixation-right condition of choice trials (PPI fixR), (3) an indicator function of fixation-left condition of choice trials (fixL: 1 for fixation-left, 0 otherwise), (4) an indicator function of fixation-right condition of choice trials (fixR: 1 for fixation-right, 0 otherwise), and (5) the extracted time series in the vmPFC (seed). Second-level analyses were performed by calculating one-sample t tests on the first-level contrasts of the interaction terms (PPI). Note that the first regressor identifies brain areas that show functional connectivity with the vmPFC during fixation-left condition, while the second regressor identifies brain areas that show functional connectivity during fixation-right condition.
Results
We used a human fMRI choice experiment to test the hypothesis that visual attention modulates the computation of relative value signals at the time of choice. Hungry subjects made choices between pairs of food stimuli while we exogenously manipulated the fixation process as described in Figure 1A. Subjects were instructed to maintain fixation during the evaluation process on the food item cued by a prespecified color frame (red or green). The duration of each eye fixation randomly varied from 1 to 4 s, and the target position was randomly switched across trials (L–R–L–R or R–L–R–L). After all of the fixations were completed, subjects were asked to indicate their choice with a left- or right-hand button press. To localize the areas of the vmPFC and vStr that correlate with stimulus values, in a separate task subjects were asked to provide subjective liking-ratings for all of the food stimuli (five-point scale, strongly dislike to strongly like). These ratings were used as measures of subject-specific stimulus values in the analysis of the choice data.
A, Trial structure. Yellow arrows illustrate target positions of eye fixation (not shown during the actual experiment). Participants alternated their eye fixation to the item cued by a target color frame before indicating their choice by pressing a button at the end of trial. The target frame color (red or green), the initial position of eye fixation (left or right), and the duration of each eye-fixation (1–4 s) were randomized. B, Sample eye-fixation data from representative individuals. The y-axis represents eye-fixation positions (13.5 units = 1° visual angle). The center coordinates of a screen were given [0,0]. The dotted lines represent the center coordinates of left and right frames. C, Logistic psychometric choice curves by relative eye-fixation duration in two-choice trials. The dashed horizontal line represents the half-maximum response threshold that determines the PSE. D, PSE comparison as a function of relative fixation time. Error bars denote SEs.
To demonstrate that subjects complied with our exogenous attention manipulations, we simultaneously measured eye-fixation data while subjects were in the fMRI scanner. Figure 1B depicts eye-fixation data from two representative subjects, indicating that the eye fixations followed the instructed pattern. An ROI (red and blue color frames) analysis using the eye-tracking data from all the subjects who participated in the fMRI study revealed that, on average and across the group, subjects fixated on the requested item 97.4% (SD = 2.7) of the time on each trial (one-sample t(19) = 92.8, p < 0.001).
Figure 1C depicts the psychometric choice curves as a function of relative value and relative fixation time. As expected, choices were highly sensitive to the relative values (VL − VR); the one-sample t tests of slope parameters of a logit fit were significant (against zero) in both cases (TL > TR: t(19) = 8.12, p < 0.001; TL < TR: t(19) = 8.78, p < 0.001). Consistent with previous studies, and despite the artificially long deliberation times that are necessary for the fMRI analyses, we found some evidence that choices were affected by relative fixation time. The paired t test of each individual's PSEs indicates that the psychometric curves were relatively shifted by experimental manipulations of fixation time (t(19) = −2.36, p < 0.05; Fig. 1D).
To test our main hypothesis, we performed an ROI analysis on the fMRI data in the areas of vmPFC and vStr that are known from previous studies to correlate with stimulus values at the time of choice. We estimated a GLM using the independent set of liking-rating trials to identify areas within these two anatomical regions in which BOLD activity correlated with the individual liking ratings. As shown in Figure 2, A and C, and Table 1, activity in the vmPFC and vStr was correlated with stimulus value measures of the food items shown in the trial (p < 0.05, corrected).
A, vmPFC activity correlated with stimulus ratings during the separate set of liking-rating trials (p < 0.05, corrected). All images and coordinates are in MNI space. B, ROI analyses of vmPFC activity during choice trials. C, vStr activity correlated with stimulus ratings during the initial liking-rating trials. D, ROI analyses of vStr activity during two-choice trials. Note that in both cases, activity correlated positively with the difference in value between the attended and unattended items. The y-axis of ROI plots represents average beta weights (estimated coefficients) for parametric regressors of the differences between left and right item values. Error bars denote SEs.
Regions correlated with stimulus values during liking-rating trials
We then estimated a GLM of the choice trials that allowed us to look for differences in the correlation with the relative value signal VL − VR during left and right fixations. Figure 2, B and D, and Table 2 show that the average correlation with VL − VR in the vmPFC and vStr is positive for left fixations (t(19) = 3.66, p < 0.005; t(19) = 4.65, p < 0.001) and negative for right fixations (t(19) = −2.21, p < 0.05; t(19) = −2.31, p < 0.05). Consistent with our hypothesis, these findings show that activity in the vmPFC and vStr correlates positively with the difference in value between the attended and the unattended items (i.e., VL − VR for left fixations and VR − VL for right fixations). Furthermore, a paired t test showed that the magnitude of the Vatt − Vunatt effect size did not depend on whether the left or right item was fixated in either the vmPFC (t(19) = 1.28, not significant) or the vStr (t(19) = 1.26, not significant).
Regions correlated with differences of stimulus values during two-choice trials
We explored the robustness of this finding in several ways. First, we estimated an additional GLM to verify that activity in vmPFC and vStr was modulated by attention. The model included four parametric regressors that represent (1) value of a left (attended) item during left fixations (fixL * VL), (2) value of a right (unattended) item during left fixations (fixL * VR), (3) value of a right (attended) item in right fixations (fixR * VR), and (4) value of a left (unattended) item in right fixations (fixR * VL). A two (fixation side: L vs R) by two (attention condition: attended vs unattended value) repeated-measures ANOVA showed a significant main effect of attention condition (F(1,19) = 10.34, p < 0.01; F(1,19) = 12.15, p < 0.01) in the vmPFC and vStr as well as in the anterior/posterior cingulate cortex, inferior frontal gyrus, superior/middle temporal gyrus, inferior parietal lobule/supramarginal gyrus, and superior/middle occipital gyrus (Figs. 3, 4; Tables 3, 4).
A, ROI analyses of the vmPFC during choice trials. B, ROI analyses of the vStr during choice trials. The y-axis represents average beta weights (estimated coefficients) for each parametric regressors. Error bars denote SEs. att, Attended stimuli; unatt, unattended stimuli.
Two (attention condition) by two (fixation side) ANOVA. A, The vStr showed a main effect of attention condition. B, Paired t test between fixation-left and fixation-right indicators showed activation in visual cortex ipsilateral to the location of overt fixations. p < 0.05 corrected.
Whole-brain two (attention condition) by two (fixation side) ANOVA results
Fix L versus Fix R contrast
Second, we performed an additional test to verify that the relative value activity in the areas that we have identified is independent of the eventual choice made in the trial, which is an essential property of a signal that serves as a precursor to the value comparison process. To do this, we estimated an additional GLM to make sure that the value signal in these areas took this form regardless of the item chosen in the trial (Fig. 5). A two (area: vmPFC vs vStr) by two (choice: L vs R) by two (fixation side: L vs R) ANOVA showed only significant main effect of fixation side (F(1,19) = 5.11, p < 0.05), while the main effects of area and choice and other interaction effects were not significant (all ps > 0.05). Subsequently, the paired t test results show that the relative value signal VL − VR was not different between the choice-left and choice-right trials in the vmPFC and vStr (all ps > 0.05), indicating that the sign and magnitude of the attention-driven relative value signals did not depend on the outcome of the choice process.
The relative value signals were not affected by the choice. A, The vmPFC activity of left-choice trials during choice trials. B, The vmPFC activity of right-choice trials during choice trials. C, The vStr activity of left-choice trials during choice trials. D, The vStr activity of right-choice trials during choice trials. The y-axis of ROI plots represents average beta weights (estimated coefficients) for parametric regressors of the differences between left and right item values. Error bars denote SEs. VC, Value of chosen items; VUC, value of unchosen items.
Third, we estimated two additional models to verify that activity in the vmPFC and vStr reacts to both the value of the attended items and the value of the unattended items. The first GLM (full model) included the separated parametric regressors for the attended and unattended item values regardless of fixation conditions, while the second GLM (restricted model) included only a parametric regressor for the attended item values. As shown in Figure 6, A and B, activity in the vmPFC and vStr ROIs were significantly positively correlated with the value of the attended items (t(19) = 3.85, p < 0.001; t(19) = 2.48, p < 0.05). In contrast, activity in the same ROIs was negatively correlated with the value of the unattended items (t(19) = −1.87, p < 0.05; t(19) = −2.19, p < 0.05). Since the models are nested, we could compare their relative fit for each individual using F tests. For the vmPFC ROI (Fig. 6C), 18 of the 20 participants showed a significantly better fit for the full model (Vatt − Vunatt). For the vStr ROI (Fig. 6D), the full model fit significantly better in 17 of 20 participants. This suggests that the value of unattended item was not ignored or suppressed but encoded (with a negative sign) in the vmPFC and vStr to compute the relative value of choice.
Value signals for attended and unattended items. A, The vmPFC activity for attended and unattended item values. B, The vStr activity for attended and unattended item values. The y-axis of ROI plots represents average beta weights (estimated coefficients) for parametric regressors. Error bars denote SEs. C, Frequency histogram of F statistics for the vmPFC ROI comparing nested models with and without the Vunatt regressor. D, Similar plot for vStr. The vertical dashed lines indicate the significance threshold at p < 0.05.
Finally, we performed two exploratory PPI analyses to identify brain regions that exhibit attention-dependent functional connectivity with vmPFC, and that thus might be responsible for modulating the activity in this area in response to changes in attention.
One area of particular a priori interest in this analysis was the STG. There were two reasons to expect that this area of the temporal lobe might play a role in modulating the attentional effects in vmPFC. First, a previous study by our group (Hare et al., 2010) found the vmPFC showed significant functional connectivity with the STG at the time of choice. Second, the STG is known to be involved in visuospatial attention (Karnath et al., 2001) and shifting attentional perspectives in social economic computations (Hampton et al., 2008).
The PPI analysis was performed in two steps. We first identified the region of vmPFC that was modulated by attention condition in the two-by-two ANOVA described above (Fig. 7B). We then performed a PPI analysis to identify areas that exhibited task-related functional connectivity with this area of vmPFC. We found that the areas of the STG/MTG, superior frontal gyrus, and anterior cingulate cortex exhibited increased connectivity with vmPFC during the evaluation period (Fig. 7A, Table 5). Finally, we performed a second PPI analysis to investigate how the functional connectivity of vmPFC changed during the fixation-left and fixation-right conditions. We found that the vmPFC exhibited significant functional connectivity with the left STG/MTG during the left fixation condition and with right STG/MTG during the right fixation condition (Fig. 7C,D, Table 5).
A, Bilateral STG/MTG showed task-related functional connectivity (task vs test) with the vmPFC seed. B, A main effect of attention condition of two-by-two ANOVA showed significant activations in the vmPFC. The vmPFC circled in purple was used as a seed region in PPI analyses. C, Left STG/MTG showed functional connectivity with the vmPFC seed during the fixation-left condition. D, Right STG/MTG showed functional connectivity with the vmPFC seed during the fixation-right condition. p < 0.05, corrected.
Areas exhibiting functional connectivity with vmPFC
Interestingly, the STG/MTG exhibited an ipsilateral pattern of activation that was also seen in visual cortex (Fig. 4B). This has a simple explanation. It is well known that the visual cortex processes information from the contralateral visual field. In our task, during the choice process, participants overtly switched their eye fixations to left and right. The attended item (located at the center of retina) was passed to both hemispheres, while the unattended item (located at the right visual field when fixated the left item; located at the left visual field when fixated the right item) was passed only to the contralateral hemisphere. Thus, the contrast of fixation left versus right (Fig. 4B) showed ipsilateral activation patterns in the visual cortex, similar to what we found in the STG/MTG in the PPI analyses.
Discussion
The results presented here show that the vmPFC and vStr encode a relative value signal at the time of decision making equal to the difference in value between the attended and the unattended items, and that the properties of these value signals do not depend on which item is eventually chosen.
A natural question to ask is why would the brain engage in such an attention-modulated computation of relative value. A growing body of data suggests that the brain makes simple decisions, such as the ones considered here, by assigning value to each of the options under consideration, which are compared with make a choice (Padoa-Schioppa and Assad, 2006; Wallis, 2007; Rangel et al., 2008; Kable and Glimcher, 2009; Rushworth et al., 2009). One intriguing possibility is that this type of attention-modulated relative value signal could be a useful component of the comparator process.
To understand why this might be the case, it is useful to make a distinction between the following three concepts. First, it has been shown that the brain computes value signals for each stimulus under consideration, which are often called stimulus value signals (Padoa-Schioppa and Assad, 2006; Kable and Glimcher, 2007, 2010; Knutson et al., 2007; Tom et al., 2007; Hare et al., 2008, 2009, 2010; Boorman et al., 2009; Croxson et al., 2009; FitzGerald et al., 2009). Second, multiple studies have shown that the psychometrics of simple choice (i.e., reaction times and choice accuracies) can be explained quantitatively with various versions of the drift-diffusion model (Ratcliff and Rouder, 1998; Smith and Ratcliff, 2004; Bogacz, 2007; Milosavljevic et al., 2010; Plassmann et al., 2010), including decision field theory (Busemeyer and Townsend, 1993; Busemeyer and Diederich, 2002). In this class of models, there are two distinct signals. One is the relative value signal (RVS), which at any point in time during the choice process reflects the current state of the integration process, and which eventually determines which option is chosen (when the RVS crosses a barrier). We refer to this signal as the comparator relative value signal (CRVS). The model assumes implicitly that there is also an instantaneous input relative value signal into the comparator process that reflects the underlying difference in value between the two options. This signal determines the slope with which the RVS is integrated. We refer to this input signal as the input relative value signal (IRVS). Note that the IRVS reflects an input to the comparison process, which is a precursor to choice, and thus it does not reflect the outcome of the choice. In contrast, the CRVS drives the choice, and thus it should reflect the output of the choice when appropriately measured. Third, a recent study by our group (Krajbich et al., 2010) has shown that a version of the DDM in which the slope of integration for the CRVS is modulated by attention was able to explain the psychometric and choice data with high quantitative accuracy.
Using these concepts, it is easy to see the conceptual link between the current findings and the processes that might be involved in comparing the value signals at the time of choice. Our findings show that activity in vmPFC and vStr correlate with the value of the attended item minus the value of the unattended item at every instant during the choice process. Furthermore, the results also show that the signals do not depend on the identity of the chosen item. These findings have a natural mapping to the conceptual framework described above. The attention-modulated relative value signals look like the IRVS to the comparison process. Results consistent with this hypothesis, although without an attention manipulation, have been found in a recent study of benefit and cost integration at the time of choice (Basten et al., 2010).
Note that under this interpretation, the IRVS signals play a critical role in the comparison process, since they already reflect a comparison between the two items. However, they are conceptually different from the CRVS of the DDM, and thus, to the extent that one thinks of the CRVS as the comparison signal that determines the choice when a barrier is crossed, the vmPFC and vStr might serve as inputs to the choice process, but are not comparator signals themselves.
An important caveat is that although this discussion allows us to make sense of a wide range of data, and to provide a computational rationale for why the brain might compute the type of attention-modulated relative value signals identified here, it cannot provide definite evidence in favor of the interpretation advanced here. In particular, the data are also consistent with the following alternative explanation. If the brain is able to quickly compute stimulus values of both items, then the Vatt − Vunatt signals that we find are linearly related to signals of the form Vatt − Mean(Vatt,Vunatt). This leads to an alternative interpretation inspired by many models in behavioral economics that assume that individuals assign values to stimuli relative to a reference point given by the expected level of consumption reward, which in this case is well approximated by Mean(Vatt,Vunatt) (Kõszegi and Rabin, 2006). Thus, the signal Vatt − Mean(Vatt,Vunatt)” could be interpreted as a reference-dependent value signal. Nevertheless, we favor our interpretation for two reasons. First, it is consistent not only with our data, but also with the body of work on the attention DDM discussed above. Second, it is not obvious how the brain would be able to compute the individual stimulus values quickly, which are necessary to compute the reference term Mean(Vatt,Vunatt). Finally, this alternative interpretation would require that the computations in the vmPFC and vSt change during the process of choice, by first computing absolute stimulus values and only later computing relative values, which seems a bit strange. This last part could be potentially tested with our experimental set-up, but unfortunately we do not have enough statistical power to do so.
Several previous studies have shown that individuals are able to make these types of dietary choices in 1 s or less (Hare et al., 2009; Litt et al., 2011, Milosavljevic et al., 2010). In contrast, in the current study, we found evidence for choice-independent relative value signals, which are precursors to choice, for much longer than that. This might seem surprising, but is consistent with optimal information processing. Since the subject could not shorten the trial by responding faster, as long as attention costs are sufficiently small, it is optimal to compute the relative value of the stimuli for as long as they are displayed.
A limitation of our study is that our attention manipulation has a significant but small impact of choices. The reason for this is that, to be able to investigate the impact of attention on value coding in this experiment using fMRI, we had to slow the choice process to a crawl, which significantly reduces the strength of effects. In fact, the computational model of how attention affects choices that we have developed and tested in previous work predicts that the impact of attention should become negligible when subjects are asked to compute and compare values for artificially long times before making a decision (Krajbich et al., 2010; Krajbich and Rangel, 2011). In addition, we looked for cross-subject correlations on the relationship between the impact of attention on the neural value signals and on behavior, but we did not find anything significant. There are several reasons why this might be the case. First, there is not enough variation across subjects on the size of the choice bias. Second, the magnitude of the choice bias also depends on computation time (i.e., the amount actually spent making a choice), which might be less than total trial time, and that by necessity we cannot measure with our design. This introduces another source of noise against our ability to find this type of cross-subject correlations.
It is also interesting to compare our findings with those of a recent study that found that activity in vmPFC correlated with the value of the chosen minus the value of the unchosen items (Boorman et al., 2009). Since such a signal reflects the output of the choice process, at first sight it seems at odds with our findings. However, a careful consideration of the evidence suggests a resolution of the apparent contradiction. In Krajbich et al. (2010), we found that, on average, chosen items were fixated on longer than unchosen items. This, together with the findings in the current study, suggests that the vmPFC signal would correlate more often with Vchosen − Vunchosen than with Vunchosen − Vchosen. As a result, a GLM that does not control for attention would find the signal in vmPFC correlated with Vchosen − Vunchosen, as it did in Boorman et al. (2009). We emphasize that we have no way of proving that this is the explanation for the findings in Boorman et al. (2009), but this interpretation is consistent with the body of data described here. Interestingly, we are unaware of any previous reports showing relative value coding in the vStr, which might reflect the fact that very few studies have looked at activity in this region during binary choices.
An important open question for future research is how do the computations identified in this study change when there are more than two items. In particular, which relative value signals are computed in the case of N > 2 items, and how are they modulated by attention? A recent eye-tracking study (Krajbich and Rangel, 2011) provides some clues. The study shows that a generalization of the attention DDM (Krajbich et al., 2010) is able to quantitatively explain the behavioral and fixation data for the case of trinary choice using the same parameters that fit the data in the binary case. This suggests that the brain might use similar processes to make binary and multi-item choices, at least when the number of options is small. Importantly, the Krajbich–Rangel model predicts that the relative value signals that need to be computed in the trinary case take the form of “value attended minus value of best non-attended item.”
Footnotes
This work was supported by the National Science Foundation, NIH, and the Betty and Gordon Moore Foundation.
- Correspondence should be addressed to Antonio Rangel, Division of Humanities and Social Sciences, MC 228-77, California Institute of Technology, Pasadena, CA 91125. rangel{at}hss.caltech.edu
