Abstract
Motivation enhances memory by increasing hippocampal engagement during encoding. However, whether such increased hippocampal activation reflects encoding of the value of highly rewarding events per se is less understood. Here, using a monetary incentive encoding task with a novel manipulation, we tested in humans whether the hippocampus represents abstract reward value, independent of perceptual content. During functional MRI scanning, men and women studied object pairs, each preceded by a monetary reward cue indicating the amount of money they would receive if they successfully remembered the object pair at test. Reward cues varied on both the level of reward (penny, dime, and dollar) and visual form (picture or word) across trials to dissociate hippocampal responses to reward value from those reflecting the perceptual properties of the cues. Behaviorally, participants remembered pairs associated with the high reward (dollar) more often than pairs associated with lower rewards. Neural pattern-similarity analysis revealed that hippocampal and parahippocampal cortex activation patterns discriminated between cues of different value regardless of their visual form, and that hippocampal discrimination of value was most pronounced in participants who showed the greatest behavioral sensitivity to reward. Strikingly, hippocampal patterns were most distinct for reward cues that differed in value but had similar visual appearance, consistent with theoretical proposals of hippocampal-pattern differentiation of competing representations. Our data illustrate how contextual representations within the hippocampus go beyond space and time to include information about the motivational salience of events, with hippocampal reward coding tracking the motivational impact on later memory.
SIGNIFICANCE STATEMENT Motivation, such as the promise of future rewards, enhances hippocampal engagement during encoding and promotes successful retention of events associated with valuable rewards. However, whether the hippocampus explicitly encodes reward value, dissociable from sensory information, is unclear. Here, we show that the hippocampus forms abstract representation of valuable rewards, encoding conceptual rather than perceptual information about the motivational context of individual events. Reward representation within the hippocampus is associated with preferential retention of high-value events in memory. Furthermore, we show that hippocampal-pattern differentiation serves to emphasize differences between visually similar events with distinct motivational salience. Collectively, these findings indicate that hippocampal contextual representations enable individuals to distinguish the motivational value of events, leading to prioritized encoding of significant memories.
Introduction
Memory is influenced by the motivational significance of events (Shohamy and Adcock, 2010). For instance, enhanced encoding of rewarding events results from increased activation in the hippocampus and parahippocampal cortex (PHC), driven by anticipatory responses in reward-related regions that project to the hippocampus (Lisman and Grace, 2005; Wittmann et al., 2005; Adcock et al., 2006; Shohamy and Adcock, 2010; Murty and Adcock, 2014). However, whether the hippocampus and PHC directly encode the value of events per se is not certain (Poucet and Hok, 2017). One electrophysiological study of rodents found no explicit coding of reward value in the hippocampus when reward codes could be dissociated from the locations in which they occurred (Tabuchi et al., 2003), while another found transient evidence for anticipatory reward coding in the hippocampus only during learning (Fyhn et al., 2002). In humans, hippocampal responses track differences in goal relevance (Hok et al., 2007; Dupret et al., 2010) and the rewards associated with events (Wolosin et al., 2013; Gruber et al., 2016), suggesting that hippocampal representations encode events in the context of their value. However, in these studies, value is typically confounded with the visual appearance of reward cues, leaving it an open question whether hippocampal and PHC activation pattern differences reflect the perceptual properties or the abstract value of such cues.
The current study had two goals. First, through tests using representational similarity analyses, we wanted to demonstrate abstract representation of potential reward in the hippocampus and PHC and the relationship between reward representation and individual differences in the reward modulation of memory. If the hippocampus and PHC encode reward value per se, events of the same value will evoke more similar representations than those with different value, regardless of the perceptual form of reward cues. In other words, events associated with perceptually similar cues that indicate distinct values should be represented differently, while events associated with perceptually distinct cues with the same value should be represented similarly. Furthermore, by relating individual differences in hippocampal and PHC abstract value coding to subsequent memory performance, we tested the hypothesis that reward-based memory enhancements stem from enhanced binding of event elements to their motivational context (Singer and Frank, 2009; Shohamy and Adcock, 2010).
Our second goal was to find out whether hippocampus-mediated pattern differentiation (Hulbert and Norman, 2015) augments the representation of the motivational context of individual events. Unlike hippocampal-pattern separation, which refers to an automatic orthogonalization of inputs (O'Reilly and Rudy, 2000), hippocampal-pattern differentiation is a process whereby perceptually similar events are actively made distinct to minimize their representational overlap and prevent interference (Gluck and Myers, 1993; Hulbert and Norman, 2015). As a result, hippocampal representations of similar events may become less similar than representations of two unrelated events (Favila et al., 2016; Chanales et al., 2017; Lohnas et al., 2018). Here, we tested whether the human hippocampus enhances value representation by actively differentiating perceptually similar cues indicating different reward values, making them especially distinct.
To achieve our two goals, we used a motivated encoding task with a novel manipulation that enabled us to dissociate perceptual and reward value effects on hippocampal and PHC representations. Participants underwent functional MRI (fMRI) while encoding pairs of objects, with each pair being preceded by a cue indicating the value of the monetary reward the participant would receive if they successfully remembered the pair. Reward cues represented one of three values (dollar, dime, penny), each presented in one of two visual forms (word, picture) across trials. Representational similarity analysis (Kriegeskorte et al., 2008a,b) was used to index hippocampal and PHC encoding patterns associated with distinct reward contexts and their relationship to later memory for the object pairs.
Materials and Methods
Participants
Thirty-four healthy, English-speaking individuals participated in the fMRI study. Before the experiment, participants gave informed consent in accordance with a protocol approved by the Institutional Review Board of the University of Texas at Austin. Participants received monetary compensation for their time and additional bonus money based on task performance up to $55.50 for perfect performance. Data from nine participants were excluded due to excess motion (four participants), scanning interruption (three participants), and missing data (two participants). Data from the remaining 25 participants (19 females; ages 18–31 years; mean age, 22 years) are reported in the article.
An additional 20 participants (five females; ages 18–24 years; mean age, 19.5 years) completed the same task but were not scanned. Their data are reported for comparison with the fMRI sample to show the behavioral consistencies and differences between samples.
Experimental design and statistical analysis
Stimuli
We used a modified version of the monetary incentive encoding task (Adcock et al., 2006). Stimuli consisted of 300 color photographs of common objects. Objects were presented with a label naming the object below. The stimuli were randomly organized into 150 unique object pairs, with different pairings for every participant. The pairs were randomly assigned to six possible reward conditions (25 object pairs per condition). Six unique reward cues varied on two dimensions: the value of the reward (penny, dime, and dollar) and the visual form of the cue (picture or word).
Monetary incentive encoding task
During an MRI scanning session, participants intentionally encoded object pairs across five event-related scanning runs. Each run lasted 9 min and consisted of 30 trials (five trials from each condition). On each trial, one of six possible reward cues was displayed for 1.5 s, indicating how much money a participant could earn for successfully recalling the upcoming association at test (Fig. 1A). Following the reward cue display, a fixation cross was displayed for 6.5 s. After the delay, the object pair was displayed for 3.5 s. Participants were instructed to create a story for each pair of objects (e.g., imagining the objects interacting with each other). Participants were asked to rate the cohesiveness and vividness of their stories on a 1–4 scale while the objects were on the screen (with a 0.5 s grace period for responses after the objects disappeared). These ratings were collected to ensure attention and were not included in the analyses. Presentation of the object pairs was followed by a 6.5 s intertrial interval, when a fixation cross was displayed. We selected fixed-duration trials based on our methodological work showing that fixed-duration trials are more advantageous when estimating trial-by-trial activation patterns for pattern-similarity analysis relative to jittered event onsets (Zeithamova et al., 2017).
After the conclusion of the scanning session, participants underwent a self-paced cued recall test for their memory of all 150 object pairs as well as a surprise source memory test for the reward cues. In the cued recall associative memory test, participants were shown the left object from each pair and asked to say out loud the name of the missing paired object (Fig. 1B). An experimenter immediately coded the accuracy of each response based on a preprinted answer sheet that had a unique order of trials for each participant based on their participant number. The vocal answers were also recorded for later verification if needed. Responses were accepted if they were synonyms (e.g., “traffic light” was accepted for “stop light”) or if the plurality was different (e.g., “traffic cones” was accepted for “traffic cone”). At the end of each associative memory recall trial, participants were asked to indicate which of the six reward cues preceded the object pair (Fig. 1C). Participants were instructed to guess when unsure.
MR data acquisition
MRI data were acquired on a 3 T Siemens Skyra MRI scanner at the Imaging Research Center at the University of Texas at Austin. Functional images were collected in 72 oblique axial slices, ∼20° from anterior commisure - posterior commisure line, using echo-planar imaging sequences (multiband acceleration factor, 3; generalized autocalibrating partial parallel acquisition factor, GRAPPA = 2; TR = 2000 ms; TE = 31 ms; flip angle, 73°; 128 × 128 × 72 matrix, resulting in 1.7 mm isotropic voxels). In addition to functional images collected during the motivated encoding task, two 6 min rest scans were acquired, one before and one after encoding, using the same fMRI parameters. The rest data are not reported in this article. The MRI session concluded with a collection of two anatomical images: a T1-weighted high-resolution MPRAGE image (256 × 256 × 192 matrix, 1 mm isotropic voxels) and a T2-weighted anatomical image acquired in an oblique coronal plane perpendicular to the hippocampal axis (TR = 13,150 ms, TE = 82 ms, 512 × 60 × 512 matrix, 0.4 × 0.4 mm in-plane resolution with 1.5 mm slices, no gap).
MRI data processing
Data were preprocessed and analyzed using FSL (Oxford Centre for Functional MRI of the Brain Software Library) version 5.0 (http://www.fmrib.ox.ac.uk/fsl), Advanced Normalization Tools (ANTs, http://picsl.upenn.edu/software/ants/), PyMVPA (MultiVariate Pattern Analysis in Python; www.pymvpa.org), and custom scripts. All analyses were performed in the native space of each participant. Anatomical images were skull-stripped using the Brain Extraction Tool (BET). The T1-weighted MPRAGE was used to create each individual participant's subcortical segmentation and cortical parcellation using Freesurfer (https://surfer.nmr.mgh.harvard.edu). Anatomical images were then coregistered to the functional data using ANTs and resampled to the functional space. Functional images were brain-extracted using BET, realigned within runs using the MCFLIRT (Motion Correction FMRIB Linear Image Registration Tool) from the FSL, and realigned across runs to the first functional image using affine transformation implemented in ANTs. Activation patterns related to each individual trial (separately for the reward cue and object pair trial phases) were estimated using the least squares single (LSS) method (Mumford et al., 2012). Events were modeled as stick functions of unit duration and convolved with the canonical hemodynamic response functions. Motion parameters were included as regressors of no interest. Single-trial activation patterns were concatenated into β-series images to serve as input patterns for the pattern-similarity analysis (Fig. 1D). The analyses focused on the preparatory (reward cue) period to determine how the reward context is represented in the brain and how that relates to the sensitivity to reward (Adcock et al., 2006).
Pattern-similarity analysis
Similarity of activation patterns between all pairs of trials was indexed using Spearman correlation coefficient (to reduce influence of extreme values that may drive Pearson's correlation) and Fisher z-transformed to conform to the assumptions of normality. Pairs of trials presented in the same run were excluded to avoid confounds driven by temporal autocorrelation within a run (Mumford et al., 2014). The similarity scores were then sorted into four bins (Fig. 1E) based on whether they reflect neural similarity between two trials with (1) identical cues (e.g., two pictures of a dime), (2) cues of the same value but different visual form (e.g., a picture of a dime and the word “DIME”), (3) cues with the same visual form but different value (e.g., a picture of a dime and a picture of a penny), or (4) two cues of both different value and form (e.g., a picture of a dime and the word “DOLLAR”).
Regions of interest.
Trial-by-trial activation patterns were extracted from anatomical regions of interests, derived from the Freesurfer segmentation of each participant's anatomy. The hippocampus served as our primary region of interest, where we predicted evidence for value coding (greater neural similarity between same-value than different-value cues) and pattern differentiation (disproportionate dissimilarity of trials of different value when reward cues are perceptually similar compared with when they are perceptually distinct) would be observed. We also measured neural-pattern similarity in the PHC, for which we predicted value coding, but not neural-pattern differentiation, would be observed. Finally, we included the lateral occipital cortex (LOC) as an object-sensitive control region. While the responses in the visual cortex are affected by value information (Anderson, 2017; Barbaro et al., 2017), its primary role is thought to be in object recognition (Grill-Spector et al., 2001), with stronger shape than value response (Persichetti et al., 2015). We thus expected pattern similarity in the LOC would be driven more strongly by the visual appearance of the reward cue than its value.
Assessing value representation.
To replicate prior findings and test whether patterns of activation within the hippocampus and PHC differentiate between trials of different value in our paradigm, we computed, for each participant, their average neural pattern-similarity score for all pairs of trials of the same value versus different value, regardless of the visual form. The scores were averaged across participants and compared between conditions (same value, different value) using a paired t test. To determine how the neural value representation relates to behavior, the value representation score (i.e., the difference between pattern similarity for same-value and different-value trials) was correlated with behavioral reward modulation scores across participants.
Dissociating the effect of the reward cue's visual form and value.
If the representation of reward value is abstract, it should be dissociable from the representation of the visual appearance of the cue. Cues indicating the same reward value should be represented similarly, even when they look different; cues indicating different values should be represented distinctly, even when they look similar. To estimate the effect that different aspects of the cue (form, value) have on the neural pattern-similarity scores, we compared pattern-similarity scores between pairs of trials that had the same cue versus pairs of trials in which cues differed. For each participant, normalized differences between pattern-similarity scores for same cue trials and different cue trials were computed separately for cues that differed in (1) visual form only, (2) value only, and (3) both visual form and value. For example, the score indexing the sensitivity to the visual form was computed as a normalized difference between pattern-similarity scores for pairs of trials with identical cues minus pattern-similarity scores for pairs of trials for which cues differed in their visual form but not in their value. A positive score indicates that neural pattern similarity decreases when cues differ, i.e., pattern-similarity scores are sensitive to the differences. If visual differences between reward cues drive discrimination of reward contexts, we should observe greater sensitivity to visual forms than to values. If reward value representation is entirely abstract, we should observe no sensitivity to the visual form when value is the same. Finally, observing disproportionally large sensitivity to value differences when cues have the same visual form (compared with pairs of trials in which both values and forms differ) would indicate a pattern differentiation coding strategy.
To test for these possible outcomes, the scores were submitted to the following analyses. First, the effect of cue difference (in form, value, or both) on neural-pattern similarity was tested using one-sample t tests against same-cue baseline (α = 0.017 to correct for multiple comparisons). Second, to test whether the neural pattern-similarity scores are sensitive to some aspects of the cue more than others, we performed a repeated-measures ANOVA [Greenhouse–Geisser corrected for nonsphericity when indicated (GG)], comparing the three conditions (form, value, both), followed by a pairwise comparison between conditions when appropriate. Finally, to test the degree to which behavioral sensitivity to reward tracks neural sensitivity to a visual cue's value and form, we conducted a multiple regression with value and form sensitivity as predictor variables and behavioral reward modulation as an outcome variable. We predicted that greater behavioral reward modulation would be associated with greater neural sensitivity to differences in value (for same-form cues) and/or relative insensitivity to differences in form (for same-value cues).
Results
Behavioral performance
fMRI sample
Two-way repeated-measures ANOVA with visual form (picture, word) and value (penny, dime, dollar) showed a significant effect of value (F(1.18,28.4) = 4.21, p = 0.044, GG) that was quadratic rather than linear (linear trend F(1,24) = 2.32, p = 0.141; quadratic trend F(1,24) = 10.13, p = 0.004). There was no effect of the visual form (F(1,24) = 0.16, p = 0.69) and no form * value interaction (F(2,48) = 1.24, p = 0.298). We thus collapsed across visual form for subsequent analyses. Figure 2A, which depicts the cued recall accuracy for each reward level averaged across participants, shows that the quadratic effect was driven by dollar trials being remembered best, unexpectedly followed by penny trials, with dime trials showing the lowest level of memory accuracy. To investigate the individual differences in memory sensitivity to the monetary reward, we computed a behavioral reward modulation score for each participant by subtracting memory scores for dollar and dime trials. Participants scoring high on this measure were those whose memory was most affected by the reward value (i.e., showed the biggest advantage for dollar trials). Other metrics of behavioral reward modulation (e.g., dollar minus penny memory accuracy) yielded very similar results. Across participants, behavioral reward modulation was not significantly correlated with overall memory performance (r = −0.26, p > 0.2; Fig. 2B). Because the behavioral reward modulation scores included outlying values (z > 3), we subsequently used rank order of the behavioral reward modulation scores when investigating its relationship to brain activation to avoid spurious correlations. The average source memory for the reward cues was at chance (mean, 0.171; SE, 0.009; t(24) = 1.04; p > 0.3 against the theoretical chance of 1/6 = 0.167). When we separately considered memory for the visual form alone (ignoring mistakes in value) and memory for the value alone (ignoring mistakes in the visual form), the source memory remained at chance for both source memory measures (mean form memory, 0.51; SE, 0.014; t(24) = chance = 0.50; mean value memory, 0.35; SE, 0.014; chance = 0.33; both t < 1.5, p > 0.15). We thus did not consider source memory in further fMRI analyses.
Behavioral sample
Two-way repeated-measures ANOVA with visual form (picture, word) and value (penny, dime, dollar) showed a significant effect of value (F(1.23,27.6) = 14.1, p = 0.001, GG) that was well characterized as both linear and quadratic (linear trend F(1,19) = 15.5, p = 0.001; quadratic trend F(1,19) = 10.8, p = 0.004). There was no effect of the visual form (F(1,24) = 0.21, p = 0.65) and no form * value interaction (F(1.45,27.6) = 0.70, p = 0.46, GG). The significant effect of value was driven by memory benefit for dollar trials (mean, 0.613; SE, 0.042), with respect to both dime (mean, 0.440; SE, 0.048; t(19) = 3.83; p = 0.001 vs dollar) and penny trials (mean, 0.437; SE, 0.043; t(19) = 3.94; p = 0.001 vs dollar). Thus, this sample replicated the memory advantage of dollar versus dime observed in the fMRI sample, but showed no difference between dime versus penny trials (t(19) = 0.17, p = 0.87). Unlike the fMRI sample, source memory was above chance (mean, 0.21; SE, 0.014; t(19) = 2.99; p = 0.008 against the theoretical chance of 1/6 = 0.167). This was driven by above-chance memory for value (mean value memory, 0.42; SE, 0.02; t(19) = 3.79; p = 0.001 against theoretical chance of 1/3 = 0.333) while the source memory for form remained at chance (mean form memory, 0.50; SE, 0.01; t(19) = 0.37; p > 0.7 against theoretical chance of 0.5). Thus, similar to the fMRI sample, participants in the behavioral sample showed behavioral reward modulation driven by differences between dollar and dime trials. In addition, they also showed above-chance memory for trial values, with no memory for the visual form and no effect of visual form on performance.
Neural value representation
A paired t test comparing neural pattern-similarity scores for pairs of trials of the same values versus different values (collapsed over visual form) revealed a significant effect of value in both the hippocampus (t(24) = 1.85, p = 0.039; Fig. 2C) and the PHC (t(24) = 2.20, p = 0.019; Fig. 2E). The degree of neural value representation (same-value minus different-value pattern similarity) was positively correlated with behavioral reward modulation across participants in both regions (hippocampus: r = 0.54, p = 0.005; PHC: r = 0.48, p = 0.016; Fig. 2D,F), such that participants who showed greater neural value representation also tended to show greater behavioral reward modulation. We also observed significant value representation in the LOC (t(24) = 2.94, p = 0.004; Fig. 2G), but its relationship to behavioral reward modulation did not reach significance (r = 0.34, p = 0.093; Fig. 2H).
Exploratory analysis: hippocampal representation of value distance
Given the novel finding of the overall representation of reward value in the hippocampus and its relationship to behavior, we asked whether the pattern similarity scales with the differences in value, indicating that hippocampal value representation is quantitative in nature. The analysis was exploratory as the current design—with only three value conditions unevenly spaced from one another—was not optimized to test this question. However, we reasoned that if hippocampal representations reflected the quantitative value of the reward, we should not only observe greater pattern similarity for the cues of the same values (i.e., zero levels apart) than cues of different values (Fig. 2), but further observe the neural-pattern similarity to be greater for pairs of trials with values one level apart (dollar vs dime, dime vs penny) than pairs of trials that differ by two value levels (dollar vs penny). To test for this pattern, we separated different-value pairs based on whether they differed by one-value or two-value levels. Pattern-similarity values were essentially the same for trial pairs that differed by one level (mean, 0.00483; SD, 0.00487) versus two levels (mean, 0.00477; SD, 0.00509; t(24) = 0.27; p > 0.7). Furthermore, for trial pairs that differed by one level, the pattern-similarity scores were similar for dollar–dime pairs (mean, 0.00474; SD, 0.00483) and dime–penny pairs (mean, 0.00492; SD, 0.00512; t(24) = 0.42; p > 0.6). Thus, while our central findings show that the hippocampus represents different-value trials more distinctly than same-value trials, the current data do not provide evidence that pattern similarity for different-value trials scales with differences in objective value.
Hippocampal sensitivity to the reward cue's form and value
To test how neural patterns are sensitive to each aspect of the cue (form, value), we computed, for each participant, the normalized difference in pattern-similarity scores for pairs of trials with the same reward cue and pattern-similarity scores for pairs of trials in which reward cues were different in form, value, or both. In the hippocampus (Fig. 3A), we found significant neural sensitivity to value differences when form was the same (one-sample t(24) = 2.44, p = 0.011), no sensitivity to visual-form differences when value was the same (one-sample t(24) = 0.24, p = 0.40), and no effect on pattern-similarity scores when both form and value differed (t(24) = 0.90, p = 0.19). Repeated-measures ANOVA showed that hippocampal sensitivity varied among conditions (F(1.59,38) = 4.06, p = 0.033, GG). Follow-up pairwise comparisons showed that sensitivity to differences in value was greater than sensitivity to differences in visual form (paired t(24) = 2.48, p = 0.020), indicating abstract representation of value that is not driven by the visual appearance of the reward cue.
Hippocampal activation patterns were also more distinct for two cues of different value when their visual form was the same compared with when the visual form differed (paired t(24) = 2.62, p = 0.015), which is consistent with the idea of hippocampal-pattern differentiation of visually similar cues. Pattern differentiation was not unique to one visual form. Instead, trial-by-trial pattern similarity for picture–picture pairs and word–word pairs were statistically equivalent. For the same-value pairs (t(24) = 0.11, p > 0.9), mean picture–picture pattern similarity was 0.00528 (SD, 0.0046), and mean word–word pattern similarity was 0.00533 (SD, 0.0048). For different-value pairs (t(24) = 0.01, p > 0.9), mean picture–picture pattern similarity was 0.00459 (SD, 0.0048), and mean word–word pattern similarity was 0.00458 (SD, 0.0054).
Multiple linear regression showed that a linear combination of value and form sensitivity in the hippocampus was significantly related to the behavioral reward modulation (F(2,24) = 4.86, p = 0.018). Both value sensitivity (β = 0.635, t(24) = 2.99, p = 0.007) and form sensitivity (β = −0.499, t(24) = −2.35, p = 0.028) were reliably related to behavioral reward modulation. Value sensitivity was positively related to behavioral reward modulation, indicating that participants most affected by the reward were those who represented same-form, different-value cues as distinct. Form sensitivity was negatively related to the behavioral reward modulation, indicating that participants most affected by reward were representing different-form, same-value cues as similar rather than distinct.
PHC sensitivity to the reward cue's form and value
In the PHC (Fig. 3B), sensitivity to cue differences did not reach the corrected threshold for any condition (form: t(24) = 0.37, p = 0.36; value: t(24) = 1.53, p = 0.070; both: t(24) = 2.15, p = 0.021). Additionally, differences between conditions did not reach significance (one-way ANOVA, F = 1.15, p > 0.3; linear trend F = 2.77, p = 0.11). Multiple regression showed that a linear combination of neural value and form sensitivity in the PHC was marginally related to the behavioral reward modulation (F(2,24) = 3.34, p = 0.054). From the two predictors (value, form), only value sensitivity was significantly related to behavioral reward modulation (β = 0.501, t(24) = 2.58, p = 0.017) such that participants who represented visually similar cues of different values distinctly tended to show greater behavioral reward modulation. Form sensitivity in the PHC was not reliably related to behavioral reward modulation (β = −0.168, t(24) = −0.85, p = 0.4).
LOC sensitivity to the reward cue's form and value
LOC (Fig. 3C) was sensitive to all aspects of the cue (all t(24) > 4, p < 0.001). Repeated-measures ANOVA showed that LOC sensitivity varied among conditions (F(1.11,27) = 74.8, p < 0.001, GG). As predicted, follow-up pairwise comparisons showed that neural-pattern differences for cues differing in value were smaller than those for cues differing in visual form (t(24) = 8.12, p < 0.001) or differing in both value and form (t(24) = 9.81, p < 0.001). For cues that differed in visual form, it did not matter whether they also differed in value or not (form vs both condition: t(24) = 0.73, p = 0.47). Multiple regression showed that a linear combination of value and form sensitivity was marginally related to behavioral reward modulation (F(2,24) = 3.26, p = 0.062). From the two predictors (value, form), neural sensitivity to value was significantly related to behavioral reward modulation (β = 0.447, t(24) = 2.32, p = 0.030) while form was not (β = −0.276, t = −1.43, p = 0.17).
Discussion
In this study, we show evidence for abstract coding of prospective reward in the hippocampus and PHC using a monetary incentive encoding task in which the visual appearance of reward cues was dissociable from their value. Hippocampus and PHC activation patterns differentiated the reward value, regardless of the visual appearance of reward cues. Moreover, hippocampal and PHC value representations were behaviorally relevant, tracking the memory advantage observed for high-value events across individuals. Strikingly, activation patterns in the hippocampus, but not the PHC, were most dissimilar for cues of different values when the cues had similar appearance, relative to when the cues differed in both value and form. These results indicate that hippocampal pattern-differentiation mechanisms mechanisms support formation of abstract reward codes, which emphasize differences in the motivational significance of events even when perceptual content is highly similar.
Electrophysiological recordings in rodents have shown that hippocampal activity is altered by the presentation of reward (Hölscher et al., 2003; Singer and Frank, 2009; Gauthier and Tank, 2018), differentiates between reward outcomes (Wikenheiser and Redish, 2011, 2015; McKenzie et al., 2014), and responds to changes in an animal's motivational state (Kennedy and Shapiro, 2009). In human fMRI studies, reward modulates hippocampal activation during encoding, rest-based consolidation, and memory-based decision-making (Wittmann et al., 2005; Adcock et al., 2006; Wolosin et al., 2012, 2013; Gruber et al., 2013, 2016; Murty and Adcock, 2014; Murty et al., 2016a,b; Patil et al., 2017). However, whether those prior findings reflect hippocampal representation of value per se in contrast to differences between the perceptual properties of motivational cues is difficult to discern (O'Doherty, 2014; Poucet and Hok, 2017). In rodents, for instance, Tabuchi and colleagues (2003) found that hippocampal place cell responses were unaffected by reward value when value and location were dissociable, while Gauthier and Tank (2018) found a population of hippocampal cells with reward responses regardless of location.
Here, we decoupled value from perceptual information and found that hippocampal activation patterns did not discriminate the visual appearance of reward cues, although visual properties of cues were reflected in an LOC control region. Instead, we found that hippocampal representations abstract reward value away from the cues' appearance, as reflected by both neural-pattern discrimination of reward-cue value and a lack of discrimination of the visual form of the cue. Importantly, the degree of abstract reward-value representation observed in the hippocampus tracked individuals' memory advantage for high-value events. These results provide a strong evidence for abstract coding of prospective reward within the hippocampus.
The abstract hippocampal representations of prospective reward revealed in the present study add to an emerging literature suggesting that hippocampal representations do not simply reflect the perceptual content of events, but rather their conceptual significance (Quiroga et al., 2008; Gauthier and Tank, 2018; Mack et al., 2018). Accordingly, a critical function of the hippocampus may be to organize events according to their conceptual similarities, regardless of their perceptual overlap (Mack et al., 2018; Morton et al., 2017). Consistent with this view, regions within the medial temporal lobe have view-invariant properties, responding similarly to different perceptual views of the same object (Barense et al., 2010) and to the same concept represented by strikingly different images or verbal labels (Quiroga et al., 2005). Furthermore, hippocampal responses track concept formation (Kumaran et al., 2009; Mack et al., 2016; Bowman and Zeithamova, 2018). The present findings extend prior work by showing hippocampal conceptual representations in the domain of reward value and further demonstrate that hippocampal-pattern differentiation shapes the organization of conceptual codes.
Pattern differentiation, an active process of driving hippocampal representations of similar inputs farther apart to reduce competition, has been long postulated (Gluck and Myers, 1993; McClelland et al., 1995; Norman et al., 2006). However, empirical evidence is only beginning to emerge (Hulbert and Norman, 2015; Favila et al., 2016; Lohnas et al., 2018). Here, we build upon this work to show that pattern differentiation contributes to the representation of motivational context. Specifically, we show that hippocampal representations for perceptually similar cues with distinct values are even more distinct than two cues with distinct values and distinct perceptual forms. By emphasizing the differences in value between perceptually similar events, hippocampal representations thus discriminate the behavioral significance during learning, leading to the prioritized encoding of high-value events.
Our data further highlight that pattern differentiation may be a representational strategy unique to the hippocampus. While PHC representations differentiated between abstract reward values (collapsed over visual form) in the present study, they did not show active differentiation of perceptually similar experiences. Consistent with our findings, prior studies (Hulbert and Norman, 2015; Favila et al., 2016) reported neural-pattern differentiation in the hippocampus only, while neural-pattern similarity in other regions were positively related to the degree of perceptual similarity among events (Favila et al., 2016). Together, these results lend support to theoretical accounts highlighting the unique hippocampal computation that supports discrimination of perceptually similar events (McClelland et al., 1995; O'Reilly and Rudy, 2000; O'Reilly and Norman, 2002).
While pattern differentiation was a coding strategy unique to the hippocampus, overall sensitivity to reward value was not. Instead, both the PHC and LOC showed, to some degree, differentiation of reward value. Furthermore, reward-value sensitivity in the PHC and LOC was correlated across subjects to reward modulation of memory, although this relationship was observed less consistently across different analyses compared with the hippocampus. The observation of reward-value representation in the PHC and LOC is consistent with a recent observation that reward signals may be more ubiquitous in the brain than previously thought, extending to sensory cortices (Pessoa, 2015; Pakan et al., 2018), and may play important roles in modulating neural processing across several domains of experience, including memory (Vickery et al., 2011).
An open question remains whether hippocampal representations of reward value are parametric in nature. In the present study, we found that hippocampal representations differentiate events based on their prospective value, but we did not find evidence for parametric scaling of representational distance with objective value distance. As our study was not optimized for a parametric analysis, this null finding does not preclude the possibility that scaled codes for value may exist in the hippocampus under different conditions. Notably, we selected exponentially increasing reward values in this study based on the idea that subjective utility is a concave function of objective value, with additional units of goods or money bringing diminishing marginal utility (Bernoulli, 1954). However, our behavioral results suggest that participants did not perceive much subjective utility difference between the penny and dime values. This potential lack of subjective utility may have prevented us from observing a parametric hippocampal code for reward value. Alternatively, responses to reward often depend on how the outcome differs from expectation. Outcomes whose value exceeds expectation are perceived as rewards, whereas outcomes that are less than an expected value are perceived as losses (Tversky and Kahneman, 1981; Seymour and McClure, 2008). Thus, the dime condition in the present study may have been perceived as a neutral condition, the dollar condition as a reward and penny as a loss. Both losses and rewards may be salient (Bartra et al., 2013), potentially explaining the U-shaped relationship between value and memory performance in the fMRI sample.
The role of the hippocampus in representing the spatial and temporal contexts that define where and when an event occurred is well documented (Smith and Mizumori, 2006; Staresina and Davachi, 2009; Kelemen and Fenton, 2010, 2016; Smith et al., 2012; Ezzyat and Davachi, 2014; Hsieh et al., 2014). The present findings extend this work to show that hippocampal representations of reward value differentiate the motivational context of individual events. Our findings converge in particular with a recent rodent study that revealed a population of hippocampal neurons that code the reward value of events independent of their perceptual features (Gauthier and Tank, 2018). These converging findings thus suggest a general organizational principle of the hippocampus to organize events according to the context (space, time, value) that differentiates them.
Behavioral relevance may determine which contextual features of the environment will be represented by the hippocampus. In our task, cue value was relevant while the visual appearance was not. Task demands have been shown to modulate hippocampal representation (Markus et al., 1995; Singer and Frank, 2009; Wikenheiser and Redish, 2015; Aly and Turk-Browne, 2016; Poucet and Hok, 2017). Here, reward value may be preferentially represented by the hippocampus over visual appearance due to its task relevance. However, even in the current data, the pattern differentiation findings do indicate sensitivity to visual form under certain circumstances. Although the visual form did not have an effect when representing same-value trials, the hippocampus represented visually similar different-value trials more distinctly that visually distinct different-value trials. This finding suggests that hippocampal representations can account for visual features as means to discriminate an event's context. In this way, the hippocampus may flexibly represent whichever contextual features of the environment are most relevant for determining future behavior.
In summary, the present study reveals that hippocampal representations of reward value are abstract and independent of the perceptual features of events. Moreover, our findings indicate that the motivational modulation of memory relates to hippocampal representation of prospective rewards. Participants who showed greater hippocampal differentiation of reward value during learning were also more likely to show enhanced subsequent memory for the events associated with the greatest rewards. Our findings further link theories of pattern differentiation with those related to motivational influences on memory. We show that hippocampal-pattern differentiation is a mechanism whereby events become organized according to their motivational significance. Collectively, our findings thus reveal how hippocampal representations of reward bias future memory decisions.
Footnotes
This work was supported by a National Science Foundation Faculty Early Career Development Award (A.R.P.), National Institutes of Health–National Institute of Mental Health (NIH-NIMH) Grant R01 MH100121 (A.R.P.), and NIH-NIMH National Research Service Award F32MH094085 (D.Z.).
The authors declare no competing financial interests.
- Correspondence should be addressed to either of the following: Alison R. Preston, 1 University Stop A8000, Austin, TX 78712, apreston{at}utexas.edu; or Dagmar Zeithamova, 1227 University of Oregon, Eugene, OR 97403, dasa{at}uoregon.edu