Abstract
Adaptively interacting with our environment requires extracting information that will allow us to successfully predict reward. This can be a challenge, particularly when there are many candidate cues, and when rewards are probabilistic. Recent work has demonstrated that visual attention is allocated to stimulus features that have been associated with reward on previous trials. The ventromedial frontal lobe (VMF) has been implicated in learning in dynamic environments of this kind, but the mechanism by which this region influences this process is not clear. Here, we hypothesized that the VMF plays a critical role in guiding attention to reward-predictive stimulus features based on feedback. We tested the effects of VMF damage in human subjects on a visual search task in which subjects were primed to attend to task-irrelevant colors associated with different levels of reward, incidental to the search task. Consistent with previous work, we found that distractors had a greater influence on reaction time when they appeared in colors associated with high reward in the previous trial compared with colors associated with low reward in healthy control subjects and patients with prefrontal damage sparing the VMF. However, this reward modulation of attentional priming was absent in patients with VMF damage. Thus, an intact VMF is necessary for directing attention based on experience with cue–reward associations. We suggest that this region plays a role in selecting reward-predictive cues to facilitate future learning.
SIGNIFICANCE STATEMENT There has been a swell of interest recently in the ventromedial frontal cortex (VMF), a brain region critical to associative learning. However, the underlying mechanism by which this region guides learning is not well understood. Here, we tested the effects of damage to this region in humans on a task in which rewards were linked incidentally to visual features, resulting in trial-by-trial attentional priming. Controls and subjects with prefrontal damage sparing the VMF showed normal reward priming, but VMF-damaged patients did not. This work sheds light on a potential mechanism through which this region influences behavior. We suggest that the VMF is necessary for directing attention to reward-predictive visual features based on feedback, facilitating future learning and decision-making.
Introduction
From cafeterias to speed-dating services, we are often called on to make decisions based on complex and noisy information. Selective attention allows us to filter the external world and focus on what matters, such as cues that predict rewards (Della Libera and Chelazzi, 2006, 2009; Kiss et al., 2009; Hickey et al., 2010a,b; Kristjánsson et al., 2010; Anderson et al., 2011a,b). This reward-related attentional tuning might reflect an adaptive process, highlighting features based on their predictive value, which will in turn guide future learning (Mackintosh, 1975; Navalpakkam et al., 2010; Gottlieb, 2012; Wilson and Niv, 2011). In this way, attention and associative learning are inextricably linked, with attention adjusting the gain on stimuli that potentially predict rewards.
Several lines of evidence support a role for the ventromedial frontal lobe (VMF) in value-based learning. Lesions to this region in patients and animal models impair learning about dynamic stimulus–reward associations (Berlin et al., 2004; Tsuchida et al., 2010; Walton et al., 2010). Imaging and monkey electrophysiology studies support these findings, showing that VMF activity encodes the relative value of options (Thorpe et al., 1983; Padoa-Schioppa and Assad, 2006; Kable and Glimcher, 2007; Boorman et al., 2009; Kennerley et al., 2011). However, the mechanism by which these stimulus value signals influence behavior remains unclear.
The VMF might play a role in learning the reward-predicting features of the environment: guiding attention to stimulus features associated previously with rewards. This region is connected robustly with subcortical regions involved in reward processing (Carmichael and Price, 1995a; Eblen and Graybiel, 1995; Cavada et al., 2000; Price, 2007) and has reciprocal connections with diverse sensory systems (Carmichael and Price, 1995b). Hence, the VMF is well situated for integrating value information with perceptual representations of stimuli (Barbas et al., 2011). Recent work has shown that functional connectivity between the VMF and higher-order sensory regions is modulated dynamically as a function of the behavioral relevance of information processed in these areas (Philiastides et al., 2010; Lim et al., 2013). Therefore, the VMF might play a role in prioritizing features that carry value information, contributing to the construction of an attentional set adapted to the current environment.
We hypothesized that the VMF plays a necessary role in using feedback to guide attention to reward-predictive stimulus features. We asked patients with prefrontal damage and healthy, demographically matched controls to complete a visual search task that induced trial-by-trial priming of a particular stimulus feature (color) based on its association with a high or low probability of a large reward, incidental to the instructed task. We expected that distractors that were primed by the high-reward color in the previous trial would capture attention more than the low-reward color in control subjects and patients with damage outside of the VMF. We predicted that these color–reward associations would have less influence on attention in VMF-damaged subjects.
Materials and Methods
Subjects.
Twenty-four patients with focal lesions involving the frontal lobes were recruited from the Cognitive Neuroscience Research Registry at McGill University, and nine patients were recruited from the Center for Cognitive Neuroscience at the University of Pennsylvania (Fellows et al., 2008). They were eligible if they had a fixed lesion primarily affecting the frontal lobes. One dorsomedial frontal lobe (DMF) patient, two lateral frontal lobe (LF) patients, and one VMF patient found the task too difficult and did not complete the experiment. One patient with VMF damage was removed from the study after it was found that she had extremely low accuracy for trials in which the target was on the right side of the screen. Removing this patient did not affect the main result: indeed, this patient showed a larger priming effect for the low-reward color than the high-reward color. Another patient was removed when it was found that the boundaries of her lesion could not be established accurately. The final sample included 27 patients with frontal lobe damage, 14 males and 13 females.
Patients were tested a minimum of 6 months after the injury (median, 6.4 years after; range, 8 months to 48.1 years). Damage to the DMF was caused by tumor resection in eight cases, aneurysm in one case, ischemic stroke in one case, and hemorrhagic stroke in one case. Damage to the LF was caused by tumor resection in three cases, ischemic stroke in three cases, and hemorrhagic stroke in one case. Damage to the VMF was caused by tumor resection in four cases, hemorrhagic stroke in three cases, and aneurysm in two cases. Eleven patients were taking one or more psychoactive medications, most commonly an anticonvulsant or antidepressant.
Patients were separated into groups a priori based on the location of their damage, assessed on their most recent MR or computed tomography imaging by a neurologist experienced with neuroimaging and blind to task performance. Patients with lesions primarily affecting the VMF were identified first, as the primary region of interest. The remaining patients were then subdivided further into DMF and LF groups. Patients' lesions were registered manually to a common brain space [Montreal Neurological Institute (MNI) brain] by neurologists at the research sites, blind to task performance, to allow overlap images to be generated and to support voxel-based lesion-symptom mapping. The overlap images for the three anatomically defined groups are shown in Figure 1.
Representative axial slices and midsagittal view of the MNI brain showing the extent of lesion overlap in the DMF (top row), LF (middle row), and VMF (bottom row) groups. Numbers above slices indicate z coordinates of axial slices in MNI space. Colors indicate extent of lesion overlap, as indicated by the color scale. R, Right; L, left.
Age- and education-matched healthy control subjects (n = 21) were recruited through local advertisement in Montreal, including seven males and 14 females. They were free of neurological or psychiatric disease and were not taking any psychoactive medication. They were excluded if they scored 25 or less on the Montreal Cognitive Assessment (Nasreddine et al., 2005). Mean ± SD performance on this test was 28.0 ± 1.5. All subjects provided written informed consent in accordance with the Declaration of Helsinki and were compensated with a nominal fee for their time. The study protocol was approved by the institutional review boards of both participating centers.
Apparatus.
All tests were programmed using E-Prime 1.2 (Psychology Software Tools). Seventeen patients and all 21 controls were tested at the MNI. They saw stimuli presented on a 19-inch monitor (Dell Computer Company) and responded using the up and down arrow keys on a standard PS/2 keyboard (Dell Computer Company). Ten patients tested at the University of Pennsylvania or in home visits in the greater Philadelphia and Montreal areas performed the experiment on a 13.5 inch laptop (Fujitsu) and used the up and down arrow keys of the laptop keyboard for their responses.
Procedure.
Subjects completed a visual search task in which they were asked to report the orientation of a T-shaped target character (pointing up or down) on each trial. This task was similar to those used in previous studies examining the effect of rewards on visual attention (Hickey et al., 2010a; Kristjánsson et al., 2010), with subjects primed to search for task-irrelevant features by associating those features with higher rewards incidental to the primary task.
The task is detailed in Figure 2. The task consisted of four blocks of 144 trials each. On each trial, subjects first saw a central fixation cross for 750 ms, followed by two square arrays made up of eight white T-shaped distractor characters (randomly pointing up or down) on the left and right of the screen for 500 ms. Subjects had to find a colored target T (randomly pink or orange on each trial) that was embedded in the center of either the left or right array and report its orientation via key press (pressing the up arrow key if the target was upright or the down arrow key if it was inverted). Opposite the target character was a salient distractor (a T turned 90°, facing left or right) that was also colored (pink if the target was orange or vice versa), which subjects were instructed to ignore. The search array was masked by scrambled images of the items in the search array. After 400 ms, the mask was removed and subjects saw the fixation cross for another 600 ms before the trial ended and feedback was displayed for 500 ms. Subjects could respond at any point in the 1500 ms period from the presentation of the search array to the presentation of feedback. A blank gray screen was shown for 700 ms between the termination of feedback presentation and the next fixation cross. Subjects were explicitly instructed that they could still respond after the search array had disappeared, until they saw feedback. Trials were balanced so that the target character appeared in both colors equally often in each block, as well as equally often on the left or right side of the screen, and oriented upright or inverted.
Task design. In each trial, subjects saw a search array composed of white T shapes (oriented randomly up or down) with an embedded colored target T that appeared randomly on the left or right side of the screen. Subjects reported the orientation of the target (up or down) through key press while ignoring a salient, perpendicular, colored, distractor on the opposite side of the screen. After 500 ms, the search array was replaced with a mask for 400 ms, followed by a 600 ms delay. Subjects could respond at any point in this 1500 ms period from the presentation of the search array to the termination of the delay screen. Subjects then saw a feedback screen for 500 ms, with the likelihood of high- or low-magnitude reward determined by the target color (counterbalanced across subjects). In the following trial (n + 1), the colors of the target and salient distractor would randomly either remain the same or switch.
On correct trials, the feedback screen showed the target and the number of points earned in the trial, as well as a running count of the total points earned in the block. Subjects earned points for correct responses within the allotted response window. The number of points earned depended probabilistically on the color of the target. For each subject, one color yielded a high reward (20 points) on 80% of trials and a low reward (1 point) the rest of the time, whereas the other color had the reverse reward association. Feedback was paired with a 500 ms high- or low-pitched “ding” sound to indicate the size of the reward. The colors associated with a greater or lesser chance of a high reward (referred to henceforth as the “high-reward” and “low-reward” colors) were counterbalanced across subjects in each group. Critically, the color of the target and salient distractor could randomly stay the same (“same color”) or switch (“switched color”) between trials. This manipulation was intended to elicit a “priming of pop-out” effect in which subjects take longer to detect the target on switch trials (Maljkovic and Nakayama, 1994). Ultimately, we were interested in whether this priming effect would be larger for trials in which the distractor appeared in the high-reward color compared with trials in which it appeared in the low-reward color.
On trials in which subjects responded too late or incorrectly, the feedback simply showed a red zero for the points earned and the total points earned in the block. This feedback screen was paired with a 500 ms “buzz” sound to alert subjects to their mistakes. Subjects were told that points would be awarded for responding quickly and accurately and that their points would be converted into a monetary bonus added to their base compensation for their time and inconvenience. They were not instructed about the color–reward associations.
Before the main experiment, subjects completed a practice version of the task. This practice task was identical to the main experiment, except that the target and salient distractor were both black rather than colored, and subjects did not earn points for correct responses. Instead, subjects simply saw the words “correct” or “error” as feedback on correct trials or errors and late responses. Feedback was not accompanied by any sounds during the practice. The practice block consisted of 72 trials. Most subjects only required one block of practice, although a few patients and controls completed an extra block to ensure they had adequately learned how to perform the task.
Data analysis.
We were interested primarily in testing whether VMF damage, and not other frontal damage, reduced priming of attention to highly rewarded colors. We thus focused on trials in which subjects made consecutive correct responses and therefore received high or low reward in the previous trial. We limited our analysis to trials in which the magnitude of reward received in the previous trial was congruent with the reward level associated with the color of the target. We also removed outlier trials by rejecting trials in which reaction time (RT) was >2 SDs higher than each subject's mean RT (mean ± SD, 4.6 ± 1.0% of trials per subject). After filtering the data, there were an average ± SD of 102.0 ± 14.6 trials per condition–subject available for the main analysis of reward priming effects. Unfortunately, there were not enough trials per condition to reliably analyze reward priming effects in trials in which subjects had received incongruent rewards on the previous trial (mean ± SD, 17.7 ± 4.2 trials per condition–subject). To reduce variance attributable to idiosyncratic differences between individuals in RT, we converted RTs to Z scores based on the mean and SD of each subject's RTs in the trials under analysis.
Statistical analysis.
Demographic variables for patients and controls were compared using uncorrected, unpaired t tests, or Mann–Whitney U tests when parametric tests were not appropriate. Neuropsychological screening scores were compared between groups using one-way ANOVAs or Kruskal–Wallis nonparametric tests. Wilcoxon's signed-rank tests were used to compare circle cancellation misses for the left and right side of the screen within each group to test for hemispatial neglect. Additional neglect screening was performed with a classic Posner spatial cueing task (Posner, 1980), which was tested with a two-way mixed-measures ANOVA, with a factor for group status and target location (left or right, or contralesional or ipsilesional).
Differences in task performance between groups on the visual search task (proportion of trials correct, incorrect, or missed) were evaluated using a χ2 test for independence. RT and arcsine transformed accuracy for ipsilesional and contralesional targets were compared within patients with unilateral damage using two-way mixed-measures ANOVAs, with separate factors for group status and target side (ipsilesional or contralesional).
Priming effects of reward-associated colors on normalized RT and arcsine-transformed accuracy were tested using a three-way mixed-measures ANOVA. Group status was treated as a between-subjects factor, and target and distractor color consistency (same or switch) and distractor color value association (high or low) were treated as the two within-subjects factors. Similarly, we also evaluated the effects of rewards on priming of position with another three-way mixed-measures ANOVA. Group status was treated as a between-subjects factor, with target and distractor position consistency (same or switch) and the reward level of the previous trial (high or low) as within-subjects factors. Reward priming, as measured by the interaction of color consistency and distractor color value, was computed in the first half and the last half of the experiment, and the interaction of experiment period and group on this priming effect was analyzed using a two-way ANOVA.
Behavior-based lesion analysis.
The Non-Parametric Mapping (version June 6, 2013) software (available freely at www.mccauslandcenter.sc.edu/mricro/npm/) was used for voxel-based lesion symptom mapping (VLSM) analysis. The interaction effect (difference of the priming effect for high- and low-reward colors) was used as a continuous measure to test when decreased reward priming was associated with lesion damage. Voxelwise comparisons between patients were performed using nonparametric Brunner–Munzel (BM) tests (Brunner and Munzel, 2000) in all voxels in which there were three or more patients with lesion damage. To control for multiple comparisons, a null distribution of BM Z scores was calculated from the same dataset using permutation tests (3000 permutations; Nichols and Holmes, 2002). This method provides an assumption-free means of controlling for multiple comparisons that is also more powerful than commonly used corrections, such as the Bonferroni's method (Kimberg et al., 2007). This test yielded a threshold of Z > 3.21 (for p < 0.05, corrected). Images of the results of this analysis were created using the software MRICron.
Results
Demographics and neuropsychological screening
Demographic and background information on controls and patient groups are provided in Table 1. There were no significant differences in age or education between controls and patient groups (unpaired t tests, t values ≤1.26, p values ≥0.2, uncorrected) or in lesion volume between different patient groups (Mann–Whitney U tests, z values ≤0.72, p values ≥0.5, uncorrected). Premorbid intelligence quotient (IQ) was estimated using the American Nelson Reading Test (AMNART; Grober and Sliwinski, 1991). AMNART IQ was significantly lower in the VMF group compared with controls (t(23) = 3.01, p = 0.006, uncorrected), although patients with LF or DMF damage were not different from controls (t values ≤1.62, p values ≥0.1, uncorrected), nor were patient groups different from each other (t values ≤1.02, p values ≥0.3, uncorrected). Scores on the Beck Depression Inventory II were also higher in all three patient groups relative to controls (t values ≥2.46, p values ≤0.02, uncorrected), although there were no differences between the patient groups (t values ≤0.48, p values ≥0.6, uncorrected).
Demographic information for controls and prefrontal patients
Subjects also underwent screening for visual neglect to test spatial attention to the left or right hemifield. There was no significant difference in the frequency of missed targets on the left or right side of the screen for any lesion group in a circle cancellation task (Wilcoxon's signed-rank tests, t values ≤1.34, p values ≥0.2). We also compared the difference in RT for detection of uncued and cued targets on the left and right side of the screen in a classic Posner spatial cueing task (Posner, 1980). We found no interaction between group status and target location (F(3,44) = 0.20, p = 0.9) or any difference between left and right targets (F(1,44) = 0.11, p = 0.7) or overall differences between groups (F(3,44) = 0.10, p = 0.9) for this cueing effect. We examined whether there were differences in this cueing effect for contralesional or ipsilesional targets in 19 patients with unilateral damage (seven DMF, seven LF, and five VMF patients). Once again, there was no significant interaction between target location and group (F(2,16) = 1.15, p = 0.3), target location (F(1,16) = 0.10, p = 0.7), or group (F(2,16) = 0.94, p = 0.4). Thus, patients showed no evidence of hemispatial neglect.
Subjects also underwent brief neuropsychological screening to test cognitive functions that were not under study but potentially affected by frontal lobe damage. There were no differences between patient groups in fluency tasks or backwards digit span (one-way ANOVAs, F(2,21) values ≤1.12, p values ≥0.3). Results from these screening tests are summarized in Table 2.
Performance on neuropsychological screening tests for controls and prefrontal patients
Visual search task performance
In the main experiment, subjects completed a visual search task in which they had to identify the orientation of a colored target that appeared opposite a salient, colored distractor on each trial. This task involved a priming of pop-out manipulation (Maljkovic and Nakayama, 1994), in which the color of the target and salient distractor would randomly remain the same, or switch, every trial. This manipulation causes an increase in RT for trials in which colors switch compared with when they stay the same, which is thought to arise from an experience-dependent priming of search for the features discriminating the target from distractors in previous trials (Kristjánsson and Campana, 2010; Becker et al., 2013). Critically, the color of the target in this task was predictive of a reward outcome on each trial but irrelevant to the task (reporting the orientation of the target). Previous work using similar paradigms have found a larger priming effect for colors paired with high rewards compared with priming by colors paired with low rewards (Hickey et al., 2010a,b, 2011; Kristjánsson et al., 2010; Anderson et al., 2011a,b).
We first assessed basic aspects of task performance, summarized in Table 3. We compared the percentage of correct responses, errors, and missed responses (i.e., failure to respond by the deadline) between groups. There was a significant effect of group on performance (χ(6)2 = 630.04, p < 0.001). In general, PFC groups performed worse than control subjects, with the worst performance in the DMF group on average. However, all subjects responded correctly in >70% of trials. We next compared raw RT for correct responses between groups. There was a significant effect of group on RT (F(3,44) = 3.02, p = 0.04), with post hoc tests showing that VMF patients were significantly slower than controls (Tukey–Kramer test, p < 0.05) but no other significant group differences (p values >0.05).
Basic task performance data for controls and prefrontal patient groups
We also tested for differences in accuracy and RT for targets presented to the contralesional or ipsilesional hemifield in patients with unilateral damage. There was no significant main effect of target side (F(1,16) values ≤0.17, p ≥ 0.7) or group (F(2,16) values ≤0.13, p ≥ 0.9) or any significant interactions between group and target side on accuracy and RT (F(2,16) values ≤1.94, p ≥ 0.2). These data are also summarized in Table 3. Thus, task performance appeared to be similar for contralesional and ipsilesional targets.
Reward priming of color
We anticipated that subjects' RTs would be longer after trials in which the colors of the target and salient distractor switched. We also expected that this effect would be larger for controls and patients with damage outside of the VMF when the salient distractor appeared in the high-reward color compared with trials in which it appeared in the low-reward color. In contrast, we expected that VMF patients would show no difference in this priming effect for distractors in high- or low-reward colors.
RTs were analyzed in a three-way mixed-measures ANOVA with factors for group status, color consistency (stay or switch), and distractor value (high or low; Fig. 3a). Across all groups, there was a robust main effect of color consistency (mixed-measures ANOVA, F(1,44) = 26.77, p < 0.0001), with higher RTs for switch trials than trials in which colors remained the same. There was also a significant interaction between color consistency and distractor value across groups (F(1,44) = 6.18, p = 0.02), with a larger color priming effect for high-reward distractors than low-reward distractors. Critically, there was also a significant three-way interaction between group, color consistency, and distractor value on RT (F(3,44) = 4.47, p = 0.008). Post hoc tests between groups on this interaction effect revealed that the reward priming effect for VMF patients was significantly lower than DMF and LF patients (Tukey–Kramer tests, p values <0.05), even trending in the opposite direction from the other groups (i.e., a larger priming effect for the low-reward color than the high-reward color; Fig. 3b). There were no significant main effects of distractor value (F(1,44) = 0.08, p = 0.8) or group (F(3,44) = 0.92 p = 0.4) on RT, nor were there any significant interactions between group and distractor value (F(3,44) = 0.22, p = 0.9) or between group and color consistency (F(3,44) = 1.92, p = 0.1). Thus, reward priming of attention in VMF patients was reduced relative to the other groups.
Effects of reward on color priming in each group. a, Normalized RTs for target discrimination when the salient distractor color was associated with high- or low-magnitude reward and when target and salient distractor colors remained the same or switched relative to the last trial. b, Difference in the priming effects for high- and low-reward colors on normalized RT (i.e., interaction effect from a). CTL, Control. c, Mean frequency of correct responses when salient distractor color was associated with high- or low-reward magnitude and when target and salient distractor colors remained the same or switched. d, Interaction effect of color consistency and distractor color value on normalized RT in the first half of the experiment (Blocks 1–2) and second half (Blocks 3–4). Error bars indicate SEM. *p < 0.05, Tukey–Kramer post hoc test.
The same pattern of results was seen when RT outliers were included. The three-way interaction between group, color consistency, and reward priming was somewhat weaker but still statistically significant (mixed-measures ANOVA, F(3,44) = 3.34, p = 0.03). Post hoc tests revealed significant differences in this interaction between the VMF group and controls, as well between VMF- and LF-damaged patients (Tukey–Kramer tests, p values <0.05) but no other significant differences (p values >0.05).
We also tested for any reward priming effects on accuracy and whether this effect differed between groups (Fig. 3c). There was a significant effect of group, with controls responding more accurately than PFC groups, as described previously (mixed-measures ANOVA, F(3,44) = 9.73, p < 0.0001). There was also a trend toward a main effect of color consistency (F(1,44) = 3.62, p = 0.06), with lower accuracy on trials in which the color switched compared with when it remained the same. There was no main effect of distractor value (F(1,44) = 0.08, p = 0.8), nor any interaction between distractor value and color consistency on accuracy (F(1,44) = 1.07, p = 0.3). Similarly, the three-way interaction between group, distractor value, and color consistency was not significant (F(3,44) = 1.34, p = 0.3). There was no interaction between color consistency and group on accuracy (F(3,44) = 0.85, p = 0.5), although there was a trend toward an interaction between distractor value and group (F(3,44) = 2.52, p = 0.07), with most groups showing a slight improvement in accuracy on high versus low distractor value trials, with the exception of the DMF group.
We next examined whether reward priming of attention changed over the course of the task by comparing the interaction effect of distractor value and color consistency in the first half and last half of the experiment (Fig. 3d). Specifically, we were interested in whether the VMF group simply learned reward associations at a slower rate and were therefore delayed in showing reward priming. There was a significant main effect of group (mixed-measures ANOVA, F(3,44) = 2.98, p = 0.04), with lower reward priming in the VMF group compared with other PFC groups and controls. However, we found no main effect of experiment period (F(1,44) = 0.04, p = 0.8) or interaction between the period of the experiment and group (F(3,44) = 0.46, p = 0.7). These data indicate that decreased reward priming in the VMF group was consistent over the course of the experiment.
Because the VMF group was also the slowest in responding during the task, we tested whether there was any relationship between overall RT and the reward priming effect. A simple Pearson's correlation between raw RT and the interaction effect of high- and low-reward color priming in the control group found no significant relationship (r(19) = 0.02, p = 0.9). Thus, differences in reward priming between groups were unlikely to be a consequence of response speed. We also examined whether the reward priming effect depended on the colors themselves (i.e., if reward priming was different when orange was associated with high reward versus pink). There was no significant difference in reward priming in control subjects between the two color conditions (unpaired t test, t(19) = 0.93, p = 0.4), indicating that both color–reward associations had equivalent effects on attention.
Priming of position
The design of our experiment also included a “priming of position” manipulation in that a target could randomly appear on the same, or opposite, side relative to the previous trial. Subjects are more efficient at detecting targets that appear in the same location compared with when the location changes and are less efficient when targets appear where a distractor appeared on the previous trial (Maljkovic and Nakayama, 1996; Kristjánsson et al., 2007). Previous work has found that reward expectations modify spatial attention, leading to prioritization of locations associated with greater likelihood of reward (Chelazzi et al., 2014; Hickey et al., 2014). Although reward was not contingent on target location in this experiment, testing whether priming of position effects were modified by rewards provides an interesting control condition to examine whether frontal groups or controls directed their attention based on spurious correlations between location and reward, in addition to the effects of consistent reward–color associations established over the course of the experiment.
We tested whether RT was affected by the change in target position from the last trial and whether this effect was influenced by the level of reward that subjects had received. There was a strong effect of change in target position (F(1,44) = 155.52, p < 0.0001), with subjects taking longer to respond after a change of position compared with when the target appeared in the same location (Fig. 4a). However, there was no significant interaction between change in target position and previous reward level, although there was a mild trend (F(1,44) = 2.81, p = 0.1). The three-way interaction between group and the effect of previous reward on change of target position was also not significant (F(3,44) = 1.11, p = 0.4; Fig. 4b). Thus, reward priming effects did not just arise through fleeting associations based on the previous trial but were strongest for the color feature that was informative about rewards through consistent (albeit probabilistic) associations across the whole experiment. There was also a significant main effect of previous reward magnitude (F(1,44) = 6.75, p = 0.01) and an interaction between previous reward magnitude and group (F(3,44) = 3.53, p = 0.02). All groups, with the exception of the VMF-damaged group, tended to be slower to respond after a high reward compared with a low reward (mean ± SD normalized RT difference high − low, collapsed across position consistency: control, 0.01 ± 0.09; DMF, 0.07 ± 0.08; LF, 0.08 ± 0.07; VMF, −0.03 ± 0.10). There was no main effect of group (F(3,44) = 0.01, p = 0.9).
Effects of reward on position priming in each group. a, Normalized RT for target discrimination when subjects received either high- or low-magnitude reward in the last trial and when target and salient distractor remained in the same position or switched. b, Difference in priming of position effect on normalized RT after high- and low-magnitude rewards. CTL, Control.
VLSM
The above results argue for the critical involvement of VMF, and not of other PFC regions, in reward priming of attention during visual search. The region-of-interest approach can obscure the effects of damage that crosses the regional boundaries imposed a priori. VLSM is an analytic approach that overcomes this limitation, systematically testing the effect of damage at the voxel level (Bates et al., 2003; Rorden et al., 2007), although with greater power for detecting effects in regions with greater lesion overlap (Kimberg et al., 2007). We applied this method in a secondary analysis, including all voxels in which at least three patients had damage (Coulthard et al., 2008; Haramati et al., 2008; Tsuchida et al., 2010).
To test the effects of lesion damage on reward priming, we used the interaction effect of distractor value and color map in the VLSM analysis (difference of priming by the high- and low-reward colors; Fig. 3b). The nonparametric BM test (Brunner and Munzel, 2000) was applied at all voxels with sufficient lesion overlap, and the threshold for statistical significance was determined using permutation testing to correct for multiple comparisons. Figure 5a shows the voxels in which there was sufficient power to detect effects of lesion damage at the permutation corrected threshold for p < 0.05 (Z > 3.21) in this group of patients, as assessed by Wilcoxon's rank-sum tests, as in the study by Gläscher et al. (2009). Colors indicate the maximum detectable Z score, representing the power for tests at each voxel. Voxels associated with a reduced, or reversed, priming effect for high- versus low-reward colors are shown in Figure 5b. Damage in the right orbitofrontal cortex was most strongly related to the behavioral effect. The strongest statistical effects (p < 0.05) were in two small clusters of voxels, one in the right gyrus rectus (MNI coordinates, 4, 49, −16) and another more posterocentrally (MNI coordinates, 16, 19, −22; Tzourio-Mazoyer et al., 2002). Testing the opposite effect (increased reward priming) with VLSM did not reveal any voxels that were above the permutation corrected criterion for statistical significance (Z = 3.17). However, there were two clusters of voxels above the uncorrected threshold, one in the left superior frontal gyrus (MNI coordinates, −15, 21, 57; Z = 2.98) and another in the left supplementary motor area (MNI coordinates, −14, 2, 65; Z = −2.06).
VLSM analysis. a, Map showing the voxels in which there was sufficient lesion overlap to detect an effect using VLSM methods, overlaid on the MNI brain in three-dimensional views (top) and in axial slices (bottom). Numbers above the axial slices correspond to z coordinates in MNI space. The color scale indicates the maximum Z score detectable at a given voxel, indicating the power to detect effects above the permutation corrected criterion for statistical significance. R, Right; L, left. b, VLSM statistical map for diminished, or reversed, priming by reward–color associations overlaid on the MNI brain on representative axial slices (top) in three-dimensional and midsagittal view (bottom) and on ventral surface (right). The color scale indicates BM Z scores. Statistical map is thresholded at p < 0.05, uncorrected. Voxels in yellow indicate when this effect was significant at p < 0.05, corrected with permutation tests.
Discussion
We examined the role of the VMF in priming attention to stimulus features associated with different levels of reward during a visual search task. We observed that controls and PFC patients with damage outside of the VMF showed greater priming by the color associated with higher overall reward compared with the color with lower overall reward, replicating previous work with similar tasks in healthy subjects (Hickey et al., 2010a; Kristjánsson et al., 2010). Reward priming was reduced significantly in VMF patients compared with other groups. These results confirm our hypothesis, implicating the VMF in guiding attention based on reward history.
Importantly, rewards in this experiment were incidental to the task itself, which simply required that subjects report the orientation of a target stimulus. This allowed us to test how attention is biased by reward independent of explicit task goals, shedding light on a potential mechanism underlying previous observations of impaired value-based learning and decision-making after VMF damage (Zald and Andreotti, 2010; Fellows, 2011). Although directing attention to reward-predictive cues was not adaptive in this setting, this behavior is critical in the far more common situation when goals and rewards are aligned.
An inability to form an attentional set for stimulus features based on feedback could explain the performance deficits in particular learning tasks observed after VMF damage: associative learning impairments have been observed after VMF or orbitofrontal damage in humans and nonhuman primates but only under specific conditions (Butter, 1968; Jones and Mishkin, 1972; Hornak et al., 2004; Noonan et al., 2010; Tsuchida et al., 2010; Walton et al., 2010; but see Jones et al., 2012; Rudebeck et al., 2013). Damage to this region does not affect learning about the reward value of stimuli in deterministic settings (Dias, Robbins, and Roberts, 1996; Fellows and Farah, 2003) or learning to associate probabilistic reward to actions (Rudebeck et al., 2008; Camille et al., 2011). Orbitofrontal lesions in monkeys do not affect updating the value of primary rewards themselves but rather the ability to associate this value to a paired stimulus (Rudebeck and Murray, 2011). Thus, damage to this region appears to disrupt the formation of stimulus–reward associations based on ambiguous information, while leaving sensitivity to the rewards themselves intact.
In the current study, there was no measure of subjects' ability to learn stimulus–reward associations beyond the reward priming effect that was the primary-dependent measure. Thus, we cannot definitively disambiguate whether VMF patients were impaired in allocating attention to reward primed cues or in learning the reward associations of those cues. Indeed, these processes are linked under most conditions, with reward modulation of the attentional set fundamental to successful learning in dynamic or ambiguous situations. Consistent with this claim, the reward priming effect remained low in the VMF group over the course of the experiment, arguing that these patients did not direct attention to reward-predictive cues even after extensive experience.
We believe that the current results provide new insights into the mechanisms underlying associative learning deficits after VMF damage. Within associative learning models, attention is generally thought to enhance learning about attended stimulus features (Le Pelley, 2010b). In this framework, attention is directed to features based on their predictive value, providing an additional layer of learning about the nature of the environment that adjusts the gain on prediction errors for individual stimulus features (Mackintosh, 1975; Pearce and Hall, 1980; Le Pelley, 2010a; Pearce and Mackintosh, 2010). We suggest that failure to direct attention to reward-predictive visual features may explain deficits in learning stimulus–value associations after VMF damage. As a result, the signal-to-noise ratio on new information would be lower for these patients, impairing learning. This ability would be particularly critical in complex environments in which options are defined by multiple features (Wilson and Niv, 2011; Niv et al., 2015).
This idea shares similarities with the hypothesis that the orbitofrontal cortex is involved in “model-based” learning, forming a cognitive map of the choice environment that facilitates decision-making (Daw and O'Doherty, 2013; Wilson et al., 2014; Stalnaker et al., 2015). This theory argues that this region is involved in inferring the value of stimuli based on feedback history. Inferred, or model-based, values could set attentional priority for reward-predictive features to filter information for future learning. In keeping with this theory, orbitofrontal lesions in rats reduce the sensitivity of dopaminergic neurons in the ventral tegmental area to prediction errors (Takahashi et al., 2011), which are considered a physiological correlate of the teaching signal described by animal learning theorists (Schultz et al., 1997).
Directing attention to reward-predictive cues might also have effects on decision-making more generally. Attentional bias to reward-associated stimuli is thought to reflect the activation of the behavioral approach system (Gray, 1991; Ikemoto and Panksepp, 1999) or a change in the incentive salience of previously neutral stimuli (Robinson and Berridge, 1993; Maunsell, 2004). Orienting to reward-predictive stimuli might shift the balance in the competition between various options during deliberation. This sort of biased competition has been suggested as a mechanism by which PFC could influence selective attention (Desimone and Duncan, 1995) and decision-making (Miller and Cohen, 2001; Cisek and Kalaska, 2010; Cisek, 2012).
Although we anticipated that VMF damage would result in a reduction in priming by the high-reward color, VMF-damaged patients trended toward enhanced priming for the low-reward color. Although this effect was not statistically significant, it is possible that some of these patients overemphasized rare, incongruent, feedback in which high rewards were paired with the (usually) low-reward target color. This sort of maladaptive priming could result from an impaired representation of the variance of rewards linked to the two colors, in line with electrophysiological evidence that orbitofrontal neurons encode this information (O'Neill and Schultz, 2010; Schultz et al., 2011). Testing the influence of incongruent rewards on attentional priming might give insight into this result, but there were not enough such trials in the current dataset to test this hypothesis reliably. This observation raises the possibility of multiple neural mechanisms for reward priming; additional work will be needed to address this.
The current study raises questions about how the VMF might influence visual attention based on feedback. One possibility is that the VMF directly influences attention to reward associated stimuli through its connections with higher-order sensory regions (Carmichael and Price, 1995b; Barbas et al., 2011). Magnetoencephalography data indicate that the VMF and ventral visual regions communicate within a timeframe that could allow the VMF to directly influence selective attention processes underlying visual search (Luck et al., 2000; Bar et al., 2006). Alternatively, the VMF might influence sensory regions through a mediator region or a more complicated network. Recent evidence points to a role for the lateral intraparietal area in representing the information value of cues for predicting future rewards (Peck et al., 2009; Foley et al., 2014). This region might play a role in assigning attentional priority to stimulus attributes based on value associations encoded by VMF, as suggested recently by Hunt et al. (2014). The amygdala and pulvinar nucleus of the thalamus could also mediate the influence of rewards on attention, similar to their proposed role in mediating attention to emotional stimuli (Pessoa and Adolphs, 2010).
Although our VLSM analysis showed that damage to the right VMF was associated with reduced reward priming, the apparent lateralization of this effect is likely a consequence of the limited coverage of the left VMF in the current sample. There was also limited coverage of the lateral orbitofrontal cortex, which is more directly connected with ventral visual areas (Price, 2007), possibly leading to false negatives in our analysis for that region. It is also worth noting that these lesions affect underlying white matter, which may influence brain regions distant from the site of injury. Although white-matter damage is represented in lesion overlap images and tested by VLSM, we cannot fully distinguish the effects of white-matter interruption on behavior from cortical damage based on these data alone. VLSM may also be biased toward the region of maximal overlap in a given sample, affecting the anatomical precision of structure–function claims (Kimberg et al., 2007; Mah et al., 2014). However, this does not limit the inferences that can be drawn from the primary region-of-interest analysis. Additional work using complementary methods (e.g., imaging, animal lesion models) will be important.
In summary, we showed that the VMF plays a necessary role in reward priming of attention during a visual search task. These findings indicate that the VMF facilitates the processing of cues that are predictive of rewards, even when not immediately task relevant. We suggest that the VMF may bias processing of cues that are informative about future rewards in sensory-perceptual regions, in turn bootstrapping learning stimulus–reward relationships when there are multiple potentially reward-predictive cues and influencing value-based decision-making.
Footnotes
This work was supported by Canadian Institutes of Health Research Operating Grant MOP 97821, a Fonds de Recherche en Santé du Québec Chercheur-Boursier award (L.K.F.), and a Desjardins Outstanding Student Award (A.R.V.). We thank Christine Déry, Arlene Berg, and Eileen Cardillo for their help with subject recruitment. We also thank Joseph Kable for facilitating access to patients at the University of Pennsylvania and Anjan Chatterjee for providing lesion tracings for these patients. Julio Martinez-Trujillo gave valuable advice for the design of this study.
The authors declare no competing financial interests.
- Correspondence should be addressed to Avinash R. Vaidya, Department of Neurology and Neurosurgery, Room 276, Montreal Neurological Institute, McGill University, 3801 University Street, Montreal, QC H3A 2B4, Canada. avinash.vaidya{at}mail.mcgill.ca
