Abstract
Reinforcing effects of reward on action are well established, but possible effects on sensory function are less well explored. Here, using functional magnetic resonance imaging, we assessed whether reward can influence somatosensory judgments and modulate activity in human somatosensory cortex. Participants discriminated electrical somatosensory stimuli on an index finger with correct performance rewarded financially at trial end, at one of four different anticipated levels. Higher rewards improved tactile performance and led to increased hemodynamic signals from ventral striatum on rewarded trials. Remarkably, primary somatosensory cortex contralateral to the judged hand was reactivated at the point of reward delivery, despite the absence of concurrent somatosensory input at that time point. This side-specific reactivation of primary somatosensory cortex increased monotonically with level of reward. Moreover, the level of reward received on a particular trial influenced somatosensory performance and neural activity on the subsequent trial, with better discrimination and enhanced hemodynamic response in contralateral primary somatosensory cortex for trials that followed higher rewards. These results indicate that rewards can influence not only classical reward-related regions, but also early somatosensory cortex when a decision is required for that modality.
- functional magnetic resonance imaging
- somatosensory cortex
- tactile judgment
- reward
- ventral striatum
- sensory perception
Introduction
The pronounced effects of reward on overt behavior have long been appreciated (Blake et al., 2002). Reward-related activity in dopaminergic neurons has now been well characterized in primates (Schultz, 2000) and related to computational formalization within models of reinforcement learning (Barto, 1994). Despite much research into the impact of reward on action and decision making (Platt and Glimcher, 1999; Glimcher, 2001; Gold and Shadlen, 2002; Sugrue et al., 2004; Small et al., 2005), less is known about whether and how reward may influence sensory processing. Although two recent studies reported reward-expectation effects on rat sensory cortices (Shuler and Bear, 2006; Pantoja et al., 2007), the functional consequences of any such influences have not been studied in humans hitherto.
Here we combined functional magnetic resonance imaging (fMRI) with psychophysics to determine whether reward, and level of reward, can affect somatosensory judgements and modulate hemodynamic responses in human primary somatosensory cortex. Our working hypothesis was that the well described effects of reward on action may also extend to the sensory domain. Subjects discriminated the frequency of two electrical stimuli applied sequentially to the index finger (Romo and Salinas, 2003; Pleger et al., 2006) and received financial rewards at different levels for each correct judgment at trial end. Importantly, our event-related fMRI protocol allowed separation of blood oxygen level-dependent (BOLD) signals elicited by somatosensory stimulation/discrimination from those attributable to subsequent critical reward-outcome signals given visually at trial end.
Materials and Methods
Event-related fMRI.
Twelve right-handed healthy subjects (four male, aged 19–32 years; mean, 27 ± 4.5 years) gave written informed consent in accord with local ethics. Functional and structural brain scans were acquired on a 3T head-scanner (Magnetom Allegra; Siemens). For functional scans, we used a BOLD-sensitive gradient echo T2*-weighted echo-planar imaging sequence [echo time, 30 ms; repetition time (TR), 2.21 s; flip angle, 90°]. In-plane resolution was 3 × 3 mm2, slice thickness 2 mm, and interslice distance 1 mm. One volume consisted of 34 oblique slices (transversal-coronal tilt, −10°) covering the whole cerebrum. During each fMRI session, we acquired 875 volumes continuously. After functional MRI, we recorded a high-resolution anatomical image using an isotropic three-dimensional spoiled gradient-recalled acquisition in a steady state sequence with 107 sagittal-orientated slices covering the whole brain. The anatomical images across subjects were used to calculate a mean group image. For initial spatial assignment of functional changes, parametric maps showing the group statistics were superimposed onto this mean structural image.
Somatosensory frequency-discrimination task.
Subjects performed a two-alternative forced-choice frequency discrimination task, which was a modified version of a previous study from our group (Pleger et al., 2006). These modifications enabled us to distinguish BOLD signals reflecting somatosensory evaluation from those reflecting specific aspects of reward feedback; namely, reward/nonreward crossed factorially with monetary level (Fig. 1 a). Subjects discriminated the relative frequency of two successive trains of electrical (square wave pulses, 200 μs duration) somatosensory stimuli (f1 and f2; f stands for frequency), applied sequentially to the index finger. We investigated four reward magnitudes (0, 20, 50, and 80 pennies per correct trial). Each reward magnitude was tested over 16 trials for each index finger in total. Four trials for one reward magnitude were grouped into miniblocks, randomly determined. Each miniblock also included one “null trial” to assess baseline activity. Its occurrence within each miniblock was selected randomly. These null trials had the same duration as “real” trials but did not include tactile stimulation or feedback.
The onset of each miniblock was signaled by a distinct visual cue, informing subjects about the level of possible reward for the upcoming miniblock, and whether the right or left finger had to be judged (Fig. 1 a, top row) when both were stimulated. Subjects fixated a small cross at the center of a screen. For electrical finger stimulation, we used a constant current neurostimulator (Digitimer DS7A), with disposable surface adhesive electrodes (Spes Medica) mounted on the radial side of both index fingers, cathode to the proximal and the anode to the distal phalanx. Stimulation intensity was adjusted to 2.5 times the sensory threshold, as determined by the methods of limits. We always stimulated both fingers simultaneously, but only the right or the left index finger was judged and rewarded in a given miniblock. Stimulation patterns were never identical for the two fingers on a given trial. One pattern consisted of the base frequency (20 Hz), and one had varied frequency (22, 24, 26, or 28 Hz). Overall, each frequency combination (20–22 Hz, 20–24 Hz, etc.) was presented twice at each reward level for each finger. One time 20 Hz was the first stimulus, whereas the other time it was the second frequency. For each subject, the stimuli sequences were chosen randomly within these constraints. Scores for sensory performance were pooled across frequencies of electrical stimulation (|f1 − f2| = 2, 4, 6, 8 Hz).
The mode of indicating the somatosensory judgment was counterbalanced across subjects. Half of the subjects had to indicate after the second stimulus whether the first or the second frequency on the relevant hand was higher, by pushing a pedal with both feet. The other half had to indicate whether the first or the second frequency was lower. In either case, participants were instructed to push the pedal once to indicate the first stimulus, or twice to indicate the second stimulus. Three to five seconds after the end of the second somatosensory stimulus (randomly jittered in steps of 1 s), and hence six to eight seconds after onset of the first somatosensory input on each trial, participants received a visual-reward feedback (actual coin images) or visual feedback indicating no reward (blank circles) (Fig. 1 a). The intertrial interval lasted between 3 and 5 s, and at the end of a given miniblock, 3–5 s likewise elapsed before the start of a new miniblock. The standard event-related fMRI approach (Josephs et al., 1997) of inserting delays of random duration helped us to separate stimulation/discrimination from later reward feedback events for fMRI analysis, as did the different possible outcomes (reward/nonreward) and different monetary levels. For correctly discriminated stimuli, visual feedback indicating reward delivery was as follows: for the “0 pence” magnitude, two smiley symbols; for the 20, 50, or 80 levels, 2, 5, or 8 coins of 10 pence each were shown visually (Fig. 1 a). On trials with incorrect discrimination responses, the corresponding amount of blank circles (in the same color as the smiley, or as the 10 pence coins) were shown. Subjects were reimbursed for their participation according to the summed reward across all trials when the experiment ended, so the financial rewards were real.
Subjects first practiced the frequency-discrimination task in a practice session, inside the scanner without scanning. This practice had the same length as the subsequent experiment, but we only presented 0 pence trials to avoid habituation to the reward magnitudes. Subjects were then removed from the scanner to relax for 20 min before commencing the main experiment during scanning.
Results
The accuracy of somatosensory judgments during scanning was not significantly different for right and left index fingers, with 67.4% correct responses overall for the right index finger, and 72% for the left. Figure 1 b plots accuracy against possible reward level for the corresponding miniblocks, indicating a clear monotonic effect of increased potential reward leading to increased accuracy (F (1,11) = 20.87; p = 0.001, when testing for a linear parametric effect of the four reward levels considered as successive steps; no interaction with side judged, p = 0.88). Thus, increased monetary incentive led to an improvement in somatosensory discrimination for the required judgment, for either hand.
a , Schematic of somatosensory-frequency discrimination task and reward paradigm. There were four possible reward magnitudes (0, 20, 50, and 80 pennies per correct trial), grouped into miniblocks of four trials. The onset of a miniblock was signaled by a distinct visual cue (4 different examples shown in top row), indicating via visual icons the potential reward for each of the next four trials and whether the right or left index finger should be judged (conveyed by an arrow below the icons). At bottom left, a schematic sequence of events is shown for one trial. Both index fingers were simultaneously stimulated electrically, twice in succession; subjects discriminated the frequency of the two successive stimuli (f1 and f2) for the finger indicated by the arrow shown with the preceding miniblock cue (see top row examples). They had to indicate after the second stimulus whether f1 or f2 was higher (or lower, counterbalanced across subjects), by pushing a pedal with both feet once for f1 or twice for f2. Three to five seconds after offset of the second electrical stimulus (randomly jittered in steps of 1 s), and thus 6–8 s after onset of the first stimulus, they received reward or no-reward feedback via visual icons (see examples in box at bottom right of a ). The jittered separation of reward delivery, via visual feedback at trial end, from the preceding somatosensory stimulation/discrimination allowed us (together with the rewarded or nonreward outcome, and the orthogonal different reward levels) to isolate hemodynamic responses that were specific to delivery of different rewards; see Materials and Methods and supplemental materials (available at www.jneurosci.org as supplemental material). b , Higher reward levels increase objective accuracy in the somatosensory discrimination task, shown separately for the right (left bars) and left (right bars) index fingers. Bars show the group mean, with its SE indicated by whiskers.
For analysis of the associated imaging data, we used regressors encoding different event types within each trial (e.g., the tactile events, or the visual feedback events indicating reward outcome) (see supplemental material, available at www.jneurosci.org) (Fig. 1 a). We further distinguished between trials on the basis of judged side (right or left index finger), reward magnitude (four monetary levels), and whether the trial was rewarded or not. An implicit baseline was included based on “null events” (Henson, 2003); see Materials and Methods.
To identify an overall brain network involved in the somatosensory task, we first compared tactile-discrimination events against the null baseline. In line with previous findings (Wang et al., 1995; Romo and Salinas, 2003; Golaszewski et al., 2006; Pleger et al., 2006), this contrast revealed activation in primary somatosensory cortex in postcentral gyrus [peak voxel at (in mm) x, y, z = −58, −24, 48 in left hemisphere and 58, −18, 40 in right hemisphere] and in secondary somatosensory cortices/parietal ventral cortex, as well as in prefrontal cortex, supplementary motor area, premotor cortex (PMC), posterior parietal cortex, insula, caudate nucleus, thalamus, and striatum in both hemispheres (see supplemental Table S1, available at www.jneurosci.org as supplemental material).
We next tested for BOLD differences between correct and incorrect trials, for the discrimination phase of trials. We found no differential effect for this categorical comparison in primary somatosensory cortex when time locked to that phase (see Fig. 3 b,c, time course plot in top left), nor for other somatosensory areas, in accord with other recent fMRI findings involving a similar paradigm (Pleger et al., 2006).
We then tested whether brain areas involved in the task [i.e., within the network identified above (supplemental Table S1, available at www.jneurosci.org as supplemental material), now elected by inclusive masking at p = 0.05 familywise error corrected] are susceptible to influences from reward outcome. Contrasting all visual feedback events that indicated reward delivery against the nonreward feedback revealed higher activity in ventral striatum (Delgado et al., 2000; Elliott et al., 2000) for rewarded trials (Fig. 2, left; see also supplemental Table S2a, available at www.jneurosci.org as supplemental material). Remarkably, despite the absence of concurrent somatosensory stimulation at this outcome point of the trial, which provided reward feedback only visually (Fig. 1 a), this contrast also revealed reactivation of primary somatosensory cortex (Fig. 2, right) (peak voxel at −42, −20, 44 in left hemisphere and at 58, −10, 48 in right hemisphere), and left PMC (supplemental Table S2a, available at www.jneurosci.org as supplemental material) (note that the right PMC at x, y, z = 56, 8, 46 showed a similar pattern below the statistical threshold: T = 3.55; p = 0.058).
Cortical regions showing increased BOLD signal for reward relative to nonreward feedback (given visually) at trial end. Activations are projected onto coronal and axial MRI slices (LH, left hemisphere; RH, right hemisphere), thresholded at p = 0.0005. Left, Activation of bilateral ventral striatum by reward minus nonreward feedback. Right, An analogous effect arising in bilateral primary somatosensory cortex (PSC), plus left PMC (right PMC was also affected below threshold; see Results). See supplemental Table S2a (available at www.jneurosci.org as supplemental material) for coordinates, p values, and T scores. This contrast is pooled across which hand was judged. See Figure 3 a for reward minus nonreward feedback effects shown separately for each judged hand.
We next assessed any reward effect separately for those trials in which either the left or right index finger had been judged. In either case, we found reactivation of primary somatosensory cortex contralateral to the judged hand, at the point of reward delivery, for rewarded versus nonreward feedback trials (Fig. 3 a; supplemental Table S2b, available at www.jneurosci.org as supplemental material). The group peaks of these activations were localized in the caudal bank of the central sulcus. Examination of individual peaks for all subjects also confirmed this localization (the means of those individually determined peaks were x, y, z = −48, −20, 48 for left primary somatosensory cortex; and x, y, z = 60, −10, 48 for right primary somatosensory cortex). Note that this “reactivation” was specific to primary somatosensory cortex contralateral to the rewarded hand. Thus, it was not found in left primary somatosensory cortex (−48, −20, 48: F (1,11) = 1.9; p = 0.19, NS) when the left/ipsilateral hand was rewarded, being seen only for the right/contralateral hand [Fig. 3 a (top), b], and analogously was absent in right primary somatosensory cortex (60, −10, 48: F (1,11) = 2.4; p = 0.14, NS) when the right/ipsilateral hand was rewarded, being seen only for the left/contralateral hand [Fig. 3 a (bottom), c].
a , Effect of reward versus nonreward feedback at trial end, now shown separately for trials in which the left index finger was judged, or the right finger instead. Activations are projected onto axial MRI slices (LH, left hemisphere; RH, right hemisphere), thresholded at a conventional p < 0.001. Note that activation in primary somatosensory cortex (PSC) is found contralateral to the corresponding index finger. For the right index finger, we additionally found activation in left PMC; see Results. See supplemental Table S2b (available at www.jneurosci.org as supplemental material) for corresponding coordinates, p values, and T scores. b , c , Time course plots of percentage signal changes in primary somatosensory cortex, averaged across all participants with SEs shown, contralateral to the right ( b ) or the left ( c ) judged index finger. Time courses are shown either time locked to somatosensory events (i.e., discrimination phase; top plots) or locked to reward-feedback events (i.e., feedback phase; bottom plots), extracted from the peak voxel in primary somatosensory cortex in postcentral gyrus as shown in Table S2b (available at www.jneurosci.org as supplemental material) and a (note that these two forms of time locking have to be shown on separate plots, because of the jittering of the delay between the somatosensory phase and the later reward phase). The plots are shown in multiples of the MRI TR, up to 11.05 s after the time-locked event. The left panels separate correct and incorrect trials regardless of reward level, whereas the right panels plot correct-minus-incorrect time courses for different reward levels. Note the expected somatosensory response in the top left of b and c , when time locked to stimulation, with a similar BOLD signal in response to the somatosensory stimuli regardless of whether the discrimination was correct or incorrect, and a hemodynamic delay as expected. Time course plots in the bottom (again showing BOLD signal group mean ± SEM) are derived from the same postcentral peaks, but now time locked instead to delivery of the later (jittered) visual feedback indicating reward or nonreward, at trial end. Note in the bottom left plots of b and c that primary somatosensory cortex is reactivated by reward delivery, but only on the rewarded trials, not the unrewarded (quite unlike the effect that is time locked to somatosensory stimulation, as shown in the time course plot above). Finally, note that when plotting correct-minus-incorrect time courses for specific monetary reward levels, a higher response is found for higher reward only when time locking to reward delivery (bottom right plots in b and c ).
In line with our use of the term “primary somatosensory cortex” in the conventional, generic sense of referring to all somatosensory Brodmann regions within the postcentral gyrus [for further detail on somatosensory areas, see Kaas (1983)], our critical reward-related effects all fell within Brodmann areas (BAs) 1, 2, 3a, and 3b, as confirmed by inclusive masking with cytoarchitectonic anatomy atlas of these areas (see supplemental Figs. S1, S4, available at www.jneurosci.org as supplemental material), developed for use in combination with functional SPM maps (Eickhoff et al., 2005) (see http://www.fz-juelich.de/inb/inb-3//spm_anatomy_toolbox). This atlas is accompanied by a toolbox for probabilistic assignment of activations to a specific Brodmann area. For the present reward-related activations in primary somatosensory cortex, the activation cluster peaking at −48, −20, 48 was assigned to BA 3b with 60% probability. The corresponding cluster in the other hemisphere (peak at 60, −10, 48) could not be differentially assigned to just a single Brodmann area with high confidence, bur clearly fell within primary somatosensory cortex. Supplemental Figure S1 (available at www.jneurosci.org as supplemental material) shows four illustrative individuals; the remaining eight also showed this pattern.
Figure 3, b and c (left), plots time courses for BOLD signals in primary somatosensory cortex contralateral to the currently judged finger (extracted from the regions shown in Fig. 3 a, listed in supplemental Table S2b, available at www.jneurosci.org as supplemental material). These time courses are time locked either to the electrical stimuli (top) or to the later jittered visual feedback (bottom), and show the hemodynamic response to either of these two phases across a time window of 0–11 s that should encompass any such hemodynamic response. Time locking to stimulation reveals the expected hemodynamic response attributable to somatosensory stimuli, but (as shown in the left top) equivalently for correct and incorrect trials that go on to be rewarded or not at trial end after the jittered delay. In contrast, time locking instead to the later reward delivery (see bottom left) confirmed the remarkable reactivation of primary somatosensory cortex by visual feedback for rewarded, but not for unrewarded, trials. To assess how many of our subjects showed this effect numerically, we inspected each within-subject contrast and found that all individual subjects showed the effect in this numerical sense. The consistency across subjects was further confirmed by one-sample t tests of reward versus no-reward feedback, computed separately for the left (p = 0.006) and the right (p = 0.002) hemispheres when the contralateral hand was rewarded, which can thereby be considered as independent replications of each other for the critical reward effect. A similar pattern was found for PMC activity when judging the contralateral finger (supplemental Fig. S2, available at www.jneurosci.org as supplemental material), consistent with a role for PMC in somatosensory judgments, as indicated also by some other studies (Stoeckel et al., 2003; Blankenburg et al., 2006; Pleger et al., 2006; Romo et al., 2002, 2004) that (unlike here) had not focused on reward. There were no reward effects in other areas implicated in the somatosensory task, such as secondary somatosensory cortex (see supplemental Fig. S3a,b, available at www.jneurosci.org as supplemental material).
The most striking and novel aspect of these results is the remarkable reactivation of primary somatosensory cortex at the time of (visual) reward-feedback delivery. We next tested whether this effect depended systematically on monetary level of reward [as suggested by the peak time courses in Fig. 3 b,c (bottom right)]. For this purpose, we extracted BOLD signals for the feedback phase of trials with different reward levels, from 5 mm spherical regions of interest (ROIs) (Aragri et al., 2006), centered at the peak of the regions in primary somatosensory cortex that had shown the categorical reward versus nonreward feedback effects after judgements of the contralateral hand (i.e., at −48, −20, 48 for the right and at 60, −10, 48 for the left index finger, as per Fig. 3 a and supplemental Table S2b, available at www.jneurosci.org as supplemental material). Note that these ROIs were thus unbiased, because they were derived from a categorical contrast orthogonal to (independent of) any parametric effect of monetary level of reward. At the second (or between-subject) level, we next computed the first eigenvariate (i.e., principal component) from all voxels within the sphere, considering each of the four reward levels separately (see also supplemental material, available at www.jneurosci.org). Finally, we tested for a linear parametric effect of reward level considered as successive steps (i.e., 0, 20, 50, then 80 pence) in repeated-measures ANOVA with a linear parametric term.
Figure 4 a plots the percentage signal changes at reward delivery, as extracted from the contralateral ROIs. In accord with the peak time course plots in Figure 3, b and c (bottom right), the ROI plot in Figure 4 a confirms that primary somatosensory cortex reactivation by reward feedback increased systematically with the monetary level of reward delivered for the contralateral finger. This parametric effect of reward level was significant as a linear trend (F (1,11) = 6.1; p = 0.03). No such effect of monetary level was found for nonreward trials (p > 0.3). Direct comparison of this negative outcome for no-reward trials, against the positive outcome for rewarded trials, revealed a significant difference in the “slope” against reward level for reward versus no-rewarded trials (F (1,22) = 5.19; p = 0.03), with a flattened function for no-reward trials. Moreover, in accord with the peak time course plots in Figure 3, b and c (top right), reward level did not affect primary somatosensory cortex for the stimulation/discrimination period, neither for ROIs centered at the peak response for reward delivery (p > 0.3) nor for ROIs centered at the peak postcentral response during stimulation (p > 0.7) (compare with supplemental Table S1, available at www.jneurosci.org as supplemental material). We found a substantial trend for a significantly steeper “slope” against reward level at the feedback phase than at the stimulation/discrimination period (with ROI centered at the peak response for reward delivery, F (1,22) = 3.9; p = 0.06; for ROI centered at the peak postcentral response during stimulation, F (1,22) = 3.6; p = 0.07). Note that these contrasts would become fully significant (p < 0.05) if one-tailed.
ROI plots of BOLD signal changes (group mean ± SEM) for rewarded trials, from 5 mm spheres located as described in Results (note that the ROI was always selected by an orthogonal contrast to the plotted effect). a , Primary somatosensory cortex contralateral to the currently judged finger (left somatosensory cortex displayed on left, right somatosensory cortex on right, to confirm that reward influences were replicated for each hemisphere) showed a significant parametric effect of the reward level of visual feedback (F (1,11) = 6.1; p = 0.03) that did not interact with hemisphere (p > 0.3). Importantly, there was no such effect of anticipated parametric reward level on primary somatosensory cortex during the earlier somatosensory phase (p > 0.3 or p > 0.7; see Results), only at the reward-delivery point on correct trials (see also Fig. 3). There was also no effect of monetary level on primary somatosensory cortex for nonreward feedback (p > 0.3; data not shown), only for reward feedback. Thus, primary somatosensory cortex showed significant effects of reward level for the visual feedback phase, but not for the earlier somatosensory phase of the trial. Interestingly, the reverse applied for ventral striatum: as displayed in b , BOLD signal changes from the ventral striatum ROI increased for higher potential reward levels during the initial somatosensory phase of the trial (F (1,11) = 8.1; p = 0.01), and thus in an anticipatory manner, whereas there was no effect of reward level (only of rewarded vs nonreward outcome) (Fig. 2, left) on ventral striatum at the outcome point of the trial, unlike primary somatosensory cortex; see Results. Thus, ventral striatum anticipates the monetary reward level, whereas primary somatosensory cortex is activated by this only when feedback is given to confirm receipt of the actual reward at that level after a correct judgment.
Unlike primary somatosensory cortex, the ventral striatum (ROI centered at x, y, z = 10, 14, −4) did not show an effect (p > 0.5) of reward level at the point of reward delivery. Instead, this reward-related region showed an anticipatory parametric effect of reward level at the earlier stimulation/discrimination point (F (1,11) = 8.1; p = 0.01), when no such effect had been found for primary somatosensory cortex (see above). Figure 4 b plots mean signal change for the four levels of financial reward from the ventral striatum ROI at the discrimination point. Note that this effect of financial level on ventral striatum was thus anticipatory/predictive, arising before a corresponding effect on primary somatosensory cortex seen at reward delivery. This effect may accord with a predictive reward signal originating in the striatum during the somatosensory processing phase, and later being passed on as a “teaching signal” to primary somatosensory cortex only at the end of the trial, when the rewards are actually administered after correct performance, via visual feedback (see also Discussion).
Our experimental design allowed us to separate reward-related signals at trial end from any attention-related modulation attributable simply to concentrating on one or other hand during stimulation/discrimination (see supplemental materials, available at www.jneurosci.org as supplemental material). Contrasting left-hand minus right-hand judgements (or vice versa) for the stimulation/discrimination phase revealed contralateral effects of attended side (at 46, −28, 66 for left minus right, T = 4.6; p < 0.0001; and a slightly weaker attention effect at −40, −30, 66, T = 2.36; p = 0.01, for right minus left). Extracting BOLD signal from these regions (using 5 mm spheres centered at the peaks coordinates listed here for attention effects) revealed no significant effect of reward level during the stimulation/discrimination phase (left hemisphere: F (1,11) = 3.3; p = 0.09; right hemisphere: F (1,11) = 0.12; p = 0.73), but also no effect in these attentionally modulated regions during the visual feedback phase at trial end (F (1,11) = 2.5, p = 0.13 and F (1,11) = 1.1, p = 0.3, for the two hemispheres, respectively). This indicates two things. First, although some contralateral attention effects did arise in sensory cortex during stimulation/discrimination, when attending to the left or right hand for the judgment, these effects were not modulated by anticipated reward level. Second, the somatosensory regions showing contralateral attention effects during stimulation/discrimination did not fully overlap with those showing the critical reward effect that depended on reward level during visual feedback at trial end. Supplemental Figure S4 (available at www.jneurosci.org as supplemental material) shows this, but also illustrates the very considerable overlap (orange) of the contralateral primary somatosensory regions activated during stimulation/discrimination (yellow) with the critical reward-related effects (red). In contrast, supplemental Figure S4 (available at www.jneurosci.org as supplemental material) shows that the attention-related effects (blue) of judged hand during stimulation discrimination do not overlap closely with the stimulation and reward effects. This further emphasizes that the present novel reward effects on somatosensory cortex cannot be reduced merely to previously established effects of tactile attention when judging one or the other hand (as is entailed by our design, in any case, because here we were able to separate reward-level-dependent effects at trial end, as signaled visually, from any standard attentional modulations during the earlier stimulation/discrimination phase).
Given that any reward here was received only subsequent to each correct discrimination, the question arises of how the enhanced reactivation of primary somatosensory cortex, elicited by delivery of a higher reward [Figs. 3 b,c (bottom), 4 a], could contribute to the observed behavioral improvements in somatosensory discrimination with higher rewards (Fig. 1 b). To address this, we next considered how receiving a particular level of reward on a given trial might affect somatosensory accuracy on the subsequent trial. We found (Fig. 5, left) that the conditional probability of the next trial being correct after receiving a reward on the preceding trial was significantly enhanced for higher rewards (F (1,11) = 6.1; p = 0.03). In contrast, this pattern of better performance with higher financial incentive was not found after an unrewarded trial (F (1,11) < 0.1; p > 0.79). Thus, only when a reward had actually been received on the previous trial did the level of that reward enhance performance on the next trial. This trial-to-trial effect was similar for the three possible points in each miniblock in which it could arise (from the first to the second trial, from the second to the third, or from the third to the fourth), because we found no significant differences (χ2, all p = NS) in the number of rewarded trials followed by correct performance (the trial-to-trial effect) at those various points.
Trial-to-trial effects of receiving a higher reward. Left, Conditional probability of being correct in the somatosensory discrimination on a trial, given that the previous trial was rewarded. Group means are shown, with whiskers indicating SEM. Note the significantly higher conditional probability after receipt of a higher reward on the previous trial; see also Results. Right, ROI analysis of primary somatosensory cortex (PSC; same ROIs as in Fig. 4 a) reveals increased BOLD signal (group mean ± SEM shown), now during the somatosensory phase of trials, only for trials preceded by actually receiving a higher reward at end of the prior trial, compared with being preceded by a nonreward trial under that monetary level.
Importantly, we were able to link this trial-to-trial reward effect to the level of BOLD response in primary somatosensory cortex during stimulation/discrimination for the next trial. Using the same unbiased postcentral ROIs as described previously, we found stronger BOLD responses to somatosensory stimuli in primary somatosensory cortex if a higher reward had actually been received on the previous trial, compared with a nonreward preceding trial. Thus, reactivation of primary somatosensory cortex by reward delivery on the preceding trial was associated with subsequent enhancement of the hemodynamic response during somatosensory discrimination on the next trial (Fig. 5, right), in the context that led to increased discrimination accuracy (Fig. 5, left). These trial-to-trial effects can thus provide a plausible, mechanistic explanation for why higher rewards led to better performance overall.
Discussion
It is established that reward can influence a wide range of overt human behaviors (Dehaene and Changeux, 2000; Glimcher and Rustichini, 2004; Pessiglione et al., 2006), with much recent interest in the neural basis of such reward influences (Platt and Glimcher, 1999; Schultz, 2000; Glimcher, 2001; Sugrue et al., 2004). Here, we extend the domain of reward-related influences to include early somatosensory processing, showing that reward can augment somatosensory performance (Fig. 1 b) and influence activity in human somatosensory cortex [Figs. 2 (right), 3 a–c, 4 a, 5 (right)]. Particularly striking is our observation that primary somatosensory cortex contralateral to the judged finger showed elevated hemodynamic response (or “reactivation” of stimulus-responsive regions) for reward delivery at trial end [Figs. 2 (right), 3], even though rewards for a correct somatosensory judgment were signaled only visually at this point. Moreover, this reactivation of primary somatosensory cortex increased monotonically with delivery of higher monetary reward [Figs. 3 b,c (bottom right), 4 a]. These differential effects of reward on primary somatosensory cortex were specific to the outcome point of the trial, thus arising in the absence of somatosensory stimulation, rather than being expressed during the earlier discrimination phase during such stimulation [Fig. 3, compare b, c (time course plots at top and bottom)], when any attentional effects might arise (see also supplemental Fig. S4, available at www.jneurosci.org as supplemental material). This suggests that in situations in which rewarding outcomes depend on veridical somatosensory discrimination, reward signals generated by receipt of a specific reward are propagated as feedback to early somatosensory systems, in a manner akin to that of a “teaching signal” that reflects the level of reward actually received.
Secondary somatosensory cortex showed clear somatosensory responses time locked to stimulation, as would be expected, but showed no impact of reward delivery nor reward level at any phase of the trial. It may be important to note that we always stimulated both fingers simultaneously here, whereas only the right or left index finger was judged and rewarded in a given miniblock. Unlike postcentral gyrus, secondary somatosensory cortex shows a response to both contralateral and ipsilateral finger stimulation (Ruben et al., 2001; Blankenburg et al., 2006). This may explain why the present influences of reward, which were specific to one hand versus the other, could be identified in postcentral gyrus more readily than in secondary somatosensory cortex, because both the currently relevant/rewarded hand and the currently irrelevant hand may be represented within secondary somatosensory cortex, unlike primary cortex.
It has previously been observed that ventral striatum responds not only at the time of reward presentation (Delgado et al., 2000; Elliott et al., 2000) but, after learning, can shift its response toward a predictive conditioned stimulus (O'Doherty et al., 2003). During our fMRI experiment, ventral striatum showed an anticipatory effect of monetary incentive level that arose early in the trial, before actual reward delivery (i.e., during the discrimination/stimulation phase, at which stage the potential reward level was known), hence anticipating the later effect on primary somatosensory cortex when a reward was actually received. These findings suggest that (presumably as a result of experience) when a higher reward level can be anticipated (recall that the financial level was known at the start of each miniblock), in a situation in which discriminations can potentially become akin to conditioned stimuli, then reward-level-related signals arise in ventral striatum before actual reward delivery (see also O'Doherty et al., 2003). The ventral striatum might thereby set the stage for reward signals to be propagated as confirmatory feedback to early somatosensory systems, in a parametric manner that reflects the magnitude of reward actually received, when rewarding outcomes are delivered that depend on veridical sensory discrimination, as here. Interplay between striatum and sensory cortex might thus allow incentive and veridical feedback to shape cortical responses to optimize discrimination performance (see also Wickens et al., 2007). Although an accurate teaching signal should not be propagated back to somatosensory cortex until the reward feedback confirms correct performance, as we found, the striatum can clearly anticipate the reward level at stake.
The reward-related effects on somatosensory cortex found here were expressed at a time when no somatosensory stimuli were present, at the point of reward delivery via purely visual feedback (unlike the ventral striatum effect of reward level, which were anticipatory). Moreover, our critical effects in primary somatosensory cortex did not just depend on correct versus incorrect performance, but instead depended on parametric level of financial reward. Thus, our conclusion that the postcentral response at trial end reflects a reward-related “teaching signal” is supported by the observations that this cortical signal scaled with actual reward magnitude [Figs. 3 b,c (bottom right), 4 a]; was specific to monetary level only for rewarded trials; and was clearly not found when time locking to the earlier stimulation/discrimination point, rather than to reward delivery [Fig. 3 b,c (top right)].
Moreover, we found that this “reactivation” of primary somatosensory cortex by reward delivery at the end of one trial also impacted performance and evoked responses for the next trial (Fig. 5, left and right). Receiving a reward on a preceding trial led to better performance on a subsequent trial, with an enhanced impact when a larger reward had been received (Fig. 5, left). In good agreement with the enhanced somatosensory performance, this receipt of reward impacted the magnitude of the BOLD signal response in primary somatosensory cortex to stimulation/discrimination for the next trial, with an enhanced cortical response contralateral to the judged stimuli when a larger reward had been received on the preceding trial (Fig. 5, right). These findings suggest that receiving a higher reward at the end of one trial sets up the somatosensory system more effectively for the next trial, possibly as a result of dopaminergic mediation (Schultz, 2000, 2006; Reynolds et al., 2001; Izhikevich, 2007). The latter possibility appears consistent with our demonstrated involvement of the ventral striatum in reward anticipation.
As in cases of pavlovian or instrumental conditioning, the occurrence of a delayed reward after a trigger stimulus, in our case after a correct somatosensory discrimination, raises the general issue of “credit assignment” (Redgrave and Gurney, 2006). In the present context, this refers to how the brain can assign reward feedback on each trial specifically to the preceding somatosensory response that had led to the reward, despite occurring several seconds earlier. As noted above, the activations we observed for ventral striatum seem consistent with dopaminergically mediated reward influences (Pessiglione et al., 2006; Schultz, 2006). The influence of dopamine has a time window that can extend for seconds, even for fine-grained phenomena such as spike-timing-dependent-plasticity (STDP) (Otmakhova and Lisman, 1996). We note a recent demonstration that dopamine modulation of STDP can arise even when reward is delayed by seconds (Izhikevich, 2007), as was the case here. Such time scales might therefore be reconciled with the present finding that reward effects can arise in primary somatosensory cortex even when reward is delayed by several seconds after somatosensory discrimination, and can also affect the subsequent discrimination on the next trial, several seconds later.
Our highly specific findings substantially extend prior work on brain activations at trial end that may reflect some form of “resetting” for the next trial (Reynolds et al., 2001; Shulman et al., 2002; Jack et al., 2006). Not only did we show specific effects for somatosensory cortex, in the context of a somatosensory decision, but we further show clear specificity to parametric reward level (rather than just trial end per se) and to rewarded rather than nonreward trials, with an associated effect on performance for the next trial. Moreover, our novel findings during visual reward feedback cannot merely be reduced to conventional attentional effects during somatosensory stimulation/discrimination (see also Results) (supplemental Fig. S4, available at www.jneurosci.org as supplemental material).
Previous invasive work in rats (Reynolds et al., 2001; Shuler and Bear, 2006; Pantoja et al., 2007) and monkeys (Newsome et al., 1989; Glimcher, 2001; Shadlen and Newsome, 2001; Sugrue et al., 2004) has begun to incorporate reward considerations into mechanistic accounts for motor choices, and increasingly also for other decisions. The present human study indicates that even basic sensory decisions and the function of primary sensory structures can be influenced by reward. Reward signals arising in structures well known to be reward related, such as ventral striatum, can evidently be propagated also to early somatosensory cortex, possibly via a cortico-striatal-thalamic network (Giménez-Amaya et al., 1995; Zhang et al., 2001), in a way that can shape basic sensory discrimination to optimize reward outcome in a sensory task. This raises the tantalizing new possibility that reward manipulations might be used to enhance pathologically deficient or lapsed sensory processes, analogously to how rewards can be used to shape or correct behavior. It also raises the new question of whether specific pharmacological manipulations (e.g., those affecting dopaminergic systems) might enhance or suppress reward-related influences on sensory processing, such as those identified here in humans for the first time.
Footnotes
-
This work was supported by Wellcome Trust Programme Grants (R.J.D., J.D.) and by the Medical Research Council United Kingdom (F.B., J.D., C.C.R.). J.D. holds a Royal Society–Leverhulme Trust Senior Research Fellowship. We thank Karl Friston, Stefan Klöppel, Stefan Kiebel, Mathias Pessiglione, and Klaas E. Stephan for fruitful discussions and Peter Aston and Eric Featherstone for technical support.
- Correspondence should be addressed to Burkhard Pleger, Department of Cognitive Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstrasse 1a, 04103 Leipzig, Germany. bpleger{at}cbs.mpg.de
