Abstract
Temporal discounting (TD) represents the mental devaluation of rewards that are available after a delay. Whether the hippocampus is critical for TD remains unclear, with marked discrepancies between animal and human studies: although animals with discrete hippocampal lesions display impaired TD, human participants with similar lesions show intact performance on classic intertemporal choice tasks. A candidate explanation for this discrepancy is that delays and rewards are experienced in the moment in animal studies but tend to be hypothetical in human studies. We tested this hypothesis by examining the performance of amnesic participants with hippocampal lesions (one female, six males) on a novel experiential intertemporal choice task that used interesting photographs occluded by thick lines as rewards (Patt et al., 2021). Using a logistic function to model indifference points data, we compared performance to that on a classic intertemporal choice task with hypothetical outcomes. Participants with hippocampal lesions displayed impaired patterns of choices in the experiential task but not in the hypothetical task. Specifically, hippocampal lesions were associated with decreased amplitude of the delay-reward trade-off, with persistent choice of the delayed option despite delay increase. These results help explain previous discrepancies across animal and human studies, indicating that the hippocampus plays a critical role in temporal discounting when the outcomes of decisions are experienced in the moment, but not necessarily when they are hypothetical.
SIGNIFICANCE STATEMENT Impaired temporal discounting (TD) has been related to maladaptive behaviors, including substance dependence and nonadherence to medical treatment. There is consensus that TD recruits the brain valuation network but whether the hippocampal memory system is additionally recruited remains unclear. This study examined TD in hippocampal amnesia, providing a unique opportunity to explore the role of the hippocampus in cognition. Whereas most human studies have used hypothetical outcomes, this study used a novel experiential task with real-time delays and rewards. Results demonstrated hippocampal involvement in the experiential task, but not in a classic hypothetical task administered for comparison. These findings elucidate previous discrepancies between animal and human TD studies. This reconciliation is critical as animals serve as models of human neurocognition.
Introduction
Temporal discounting (TD) represents the mental devaluation of rewards that are available after a delay rather than immediately (Ainslie, 1975). Animal studies suggest a critical role for the hippocampus in TD, with impaired intertemporal choice consistently demonstrated as a result of surgical or genetic manipulation of hippocampal function (Rawlins et al., 1985; Cheung and Cardinal, 2005; McHugh et al., 2008; Mariano et al., 2009; Abela and Chudasama, 2013; Bett et al., 2015; Masuda et al., 2020; Seib et al., 2021). By contrast, human patients with hippocampal lesions show intact performance on classic intertemporal choice tasks (Kwan et al., 2012, 2013; Palombo et al., 2015). Further, most functional brain imaging studies report no hippocampal activity during intertemporal choice tasks (Kable and Glimcher, 2007; Peters and Büchel, 2009; Pine et al., 2009; Cox and Kable, 2014; Hare et al., 2014; Massar et al., 2015; but see McClure et al., 2004; Sripada et al., 2011).
One key difference between classic human and animal intertemporal choice tasks is that animals experience the delays and rewards associated with each decision (e.g., waiting 10 s for two portions of food, or getting one portion of food immediately). The subjective value of options in experiential tasks must be computed on the fly, and may require the hippocampus through various mechanisms, including integrating delay and reward experiences into novel representations (Palombo et al., 2019) and revisiting these representations in subsequent decisions through episodic processes (Johnson and Redish, 2007). By contrast, most human research has used questionnaire-type tasks, consisting of a series of hypothetical choices between a smaller sooner and larger later reward (e.g., “Would you prefer $80 now or $100 in a week?,” “in two months?,” “in five years?”; Cohen et al., 2020). In such tasks, all choices are made before consequences can be experienced, and decisions can rely solely on preexisting subjective values, tapping into semantic knowledge that does not require hippocampal function (Moscovitch et al., 2006; Binder and Desai, 2011).
The present study sought to determine whether the hippocampus plays a critical role in intertemporal decisions when delays and rewards are experienced in the moment by examining the performance of amnesic patients with hippocampal lesions on an experiential intertemporal choice task. A main barrier to elucidating this question has been finding a suitable reward, with food or drink offering limited incentive for well-nourished human participants and heavily depending on personal preferences (Reuben et al., 2010). Here, we make use of a newly developed task that uses artistic photographs as rewards (Patt et al., 2021). The task capitalizes on the naturally rewarding effect of interesting perceptual information (Biederman and Vessel, 2006; Marvin and Shohamy, 2016) and constructs a continuum of reward by manipulating how much of the information is visible in the photographs. The task has been successfully validated in previous work, with demonstration of an effective reward-delay trade-off (Patt et al., 2021).
We predicted that the performance of patients with hippocampal lesions would be impaired on the experiential intertemporal choice task, but intact on a classic hypothetical task also administered for comparison. The expected nature of impairment in the experiential task, however, was less clear, as animal studies have related hippocampal dysfunction to both an increase in the choice of the sooner smaller reward (Rawlins et al., 1985; Cheung and Cardinal, 2005; McHugh et al., 2008; Mariano et al., 2009; Abela and Chudasama, 2013) and to a persistent willingness to wait for larger later rewards despite delay increase (Masuda et al., 2020). Another study that used delay adjustment rather than fixed option blocks, reported intact impulsivity or impatience, relating hippocampal TD impairment instead to difficulties learning reward-delay contingencies (Bett et al., 2015). Considering findings of both overdiscounting and underdiscounting in animal studies, we did not make a prediction in terms of the direction of impairment. Instead, we characterized the shape of indifference point curves using a logistic function model (Fig. 1) to identify differences in decision patterns.
Materials and Methods
Participants
Seven patients with amnesic syndrome secondary to medial temporal lobe pathology (one female, six males) participated in the study. Age ranged between 51 and 76 years (M = 62.3 years, SD = 9.1), education level ranged between 12 and 20 years (M = 14.9 years, SD = 3.0), and verbal IQ assessed using the Wechsler Adult Intelligence Scale-III (WAIS-III) ranged between 88 and 131 (M = 104.4, SD = 14.0). Etiology of amnesia included hypoxic-ischemic injury secondary to either cardiac or respiratory arrest (n = 4), encephalitis (n = 1), stroke (n = 1), and status epilepticus followed by left temporal lobectomy (n = 1). The neuropsychological profile for each patient confirmed severe cognitive impairment restricted to the domain of memory (for individual demographics and neuropsychological testing summary indices, see Table 1). Brain MRI or CT images of the lesions are presented in Figure 2 for all patients except P4, who could not be scanned because of medical contraindications. Medial temporal lobe pathology for this patient was inferred based on etiology (anoxia secondary to cardiac arrest) and neuropsychological profile. Of the patients with available scans, P3 had lesions that were restricted to the hippocampus, P7 had a lesion that included the hippocampus as well as the amygdala, P1 had a lesion that included the hippocampus and medial temporal cortices, P2 had a lesion that extended beyond the medial temporal lobe into the anterolateral temporal neocortex because of the temporal lobectomy, and P5 had lesion sites that included the medial temporal lobe and the basal ganglia. For P6, clinical MRI was acquired in the acute phase of herpes simplex encephalitis. At that time, there were no visible T1-weighted findings, but T2-flair images demonstrated bilateral hyperintensities in the hippocampus and MTL cortices and in the anterior insula. Across all patients with available brain imaging, the hippocampus was the only area of overlap.
Demographic and neuropsychological data for the participants in the patient group
Sixteen healthy control participants (2 females, 14 males) were matched to the patient group in age (M = 63.8 years, SD = 9.0, range = [50, 80]), education (M = 16.1 years, SD = 2.6 years, range = [12, 20]) and verbal IQ (M = 107.9, SD = 12.3, range = [88, 137]). Four other healthy participants were recruited for the study but were excluded because of missing data from one of the two study sessions (n = 3) or nonvalid performance as indicated by excessive incorrect responses to catch trials (n = 1). All participants provided informed consent in accordance with the Institutional Review Board at the Veterans Affairs Boston Healthcare System.
Our patient sample size was limited by the rare occurrence of hippocampal amnesia. A power analysis implemented with the R package 'pwr' (Champely, 2020) for a two-tailed two sample t test with unequal sample sizes n1 = 7 and n2 = 16, Type I error α = 0.05, and power level 1-β = 0.80 suggested that our design was equipped to detect effect sizes of Cohen's d ≥ 1.3. Using the Lenhard and Lenhard (2016) F tests and t tests to effect size converter, we estimated that Cohen's d effect sizes ranging between 1.0 and 2.1 were obtained in previous animal hippocampal lesion studies (Cheung and Cardinal, 2005; Abela and Chudasama, 2013; Bett et al., 2015) and in a human hippocampal lesion study using a classic TD task with added episodic demands (Palombo et al., 2015). Based on these considerations, we estimated that our study was adequately powered.
Paradigms
Two intertemporal choice tasks were administered, consisting of a series of decisions between a smaller sooner reward and a larger later reward: the novel experiential task with real time perceptual outcomes, and a classic task with hypothetical monetary outcomes. Both tasks were programmed and displayed using the MATLAB Psychtoolbox-3 (Kleiner et al., 2007), and were administered on a Sony VAIO S. Series 15.5′′ laptop computer with resolution manually reduced to 1600 × 900 pixels.
Experiential intertemporal choice task
In this task, participants made choices between viewing a partially occluded photograph immediately or a nonoccluded photograph after a delay. Delayed options consisted of delays of 1, 2, 4, 6, 8, 10, 12, 14, 16, 20, and 25 s. Immediate options comprised 10 possible occlusion levels and were constructed by varying line thickness and spacing to reveal 13%, 28%, 45%, 60%, 70%, 80%, 88%, 95%, 97% and 100% of the full photograph. The photographs were drawn randomly without repetition from a group of 150 photographs preselected from a bank of royalty-free pictures (https://www.pexels.com/) to include only pleasant content (e.g., artistic landscapes, wildlife, nature close-ups, etc.). At the time of decision, the content of the to-be-presented photograph was not revealed to ensure that novelty (rather than individual preferences in visual content) was a factor in the decision and constituted a large part of the rewarding experience.
For each decision, both options were represented visually on the screen with a small rectangular frame with static background (Fig. 3). For the immediate option, the rectangle was occluded by black lines with orientation, spacing, and thickness that were identical to the lines that would occlude the photograph at the time of reward. For the delayed option, the rectangle was not occluded but contained a loading bar depicting the delay before the full photograph would be visible. Immediate and delayed options were randomly presented on the left or right side of the screen. To discourage unconsidered responding, responses could only be provided 2 s after choice presentation: thickening of the rectangular frames indicated that a response could be made. Decisions were self-paced and were made by selecting a right or left key labeled on a computer keyboard. Pressing the key produced a highlight of the chosen option's rectangular frame. Choices then had to be validated by pressing a central key; until validation, participants could still change their mind. Following validation of a choice, if the immediate option was selected, the computer immediately displayed a full screen image of the photograph occluded by black lines. If the delayed option was selected, the corresponding loading bar started progressively filling in, with duration corresponding to the announced delay. The full view of the photograph was then displayed. Occluded and full view photographs were displayed for 5 s and were followed by a 0.5-s fixation point before next trial onset.
At the beginning of the task, detailed instructions were provided as well as a chance to practice. Participants were told that the goal of the study was to examine how people make decisions about viewing photographs and how seeing a partial view of a photograph may impact those decisions. To promote interest and highlight the quality of the photographs, participants were told that the pictures had been taken from a contest by amateur photographers. The concept of occluded photographs was first introduced through presentation of three versions of the same picture side by side, each depicted with a different occlusion level. This was repeated with another picture and three different occlusion levels. The examiner then walked participants through two example trials, demonstrating the effect of choosing the occluded photograph immediately and the effect of choosing the full view photograph after the specified delay. After that, participants completed 12 practice trials on their own. The delays and occlusion levels of the example and practice trials were predetermined but the photographs were randomly selected from the bank of pictures.
Each participant was then presented with an individualized series of choices, constructed using a semi-adaptive dichotomy algorithm, allowing efficient determination of indifference points at each preselected delay. In the present task, the indifference point for a preselected delay refers to the occlusion level for which viewing an occluded photograph now and viewing the full photograph after that delay have equivalent subjective value. For each preselected delay, the algorithm started with the presentation of an immediate option with occlusion level randomly selected to permit viewing 60%, 70%, or 80% of the full photograph. On subsequent trials involving that same preselected delay, the occlusion level of the immediate option was adjusted so as to cut in half the current uncertainty interval of the indifference point. For example, for decisions involving a delay of 10 s, the first decision might involve viewing 60% of the full photograph now or a full view of that photograph after a 10-s delay. If the participant chose to wait, it implied that the indifference point, or subjective value of a full photograph after a 10-s delay, was situated between 60% and 100% of the full photograph. The next decision at that delay thus proposed viewing 88% of the photograph now (i.e., the median level among the candidate occlusion levels 60%, 70%, 80%, 88%, 95%, 97%, and 100%) or the full photograph after a 10-s delay. The same process was then repeated again several times, each time cutting the uncertainty interval in half, until the indifference point converged to one specific occlusion level. Convergence for a preselected delay generally occurred after three to four trials. In order to limit decision monotony, trials with different delays were interspersed, with random selection of a delay for each new trial among those that had not yet converged to an indifference point. The semi-adaptive algorithm was run twice, with a short break between the two parts. This strategy permitted collecting two indifference points per delay for each participant and measuring response consistency between two consecutive series of decisions. Depending on the speed of convergence of the algorithm, the full task comprised ∼70–80 test trials per participant.
To verify adequate engagement and understanding of the decisions, 16 catch trials (eight per run of the semi-adaptive algorithm) were interspersed throughout the task, with no impact on the choice of the occlusion levels in the algorithm. Eight catch trials featured decisions between viewing an occluded photograph now or viewing a full photograph also now, the correct response here was the full photograph. The remaining eight catch trials featured decisions between viewing a full view of the photograph now or viewing that same full view photograph after a delay, the correct response was the immediate option. Performance was considered valid if participants made no more than 5/16 (31%) errors on the catch trials. Similar performance validity criteria have been used in another study of patients with brain dysfunction (Sturm et al., 2017) to allow for possible lapses in attention that are common in individuals with organic brain damage, while nonetheless ensuring adequate understanding of task demands. The task was designed with adaptive features to help compensate for possibly greater lapses of attention in patients compared with controls.
To enhance participant engagement, every five trials, a screen appeared asking participants to rate their subjective experience while viewing the most recently presented photograph on a Likert scale ranging from extremely unpleasant (−4) to extremely pleasant (+4). To equalize task duration across participants, a screen labeled “Processing Data” was then presented, featuring a screen-wide loading bar that progressively filled in. The duration of the loading bar was calculated to be 5 s plus the duration of all the nonchosen delayed options during the previous five trials. This procedure ensured that choosing the immediate option would not result in faster completion of the task or in the viewing of more pictures, an experimental confound that has been associated with increased discounting (Genty et al., 2012). Task duration adjustment was conducted every five trials rather than every single trial to avoid mental association between selection of the immediate option and subsequent waiting, which could be interpreted as punitive. The rationale for the “Processing Data” screen was not overtly provided, but during the instructions, participants were told that their decisions would not impact the duration of the task or the number of photographs that they would view.
Hypothetical intertemporal choice task
In this task, participants made choices between gaining varying amounts of money immediately or gaining $100 after a delay (Fig. 4). The possible delays were 1 d, 2 d, one week, two weeks, one month, three months, six months, one year, two years, five years, and 10 years. The possible monetary amounts for the immediate reward were: $1, $5, $10, $20, $30, $40, $50, $60, $70, $80, $90, $95, $99, and $100. The structure of the task was similar to the experiential task, with random right and left side alternation of the immediate and delayed options, a 2-s buffer-time before being able to provide a response, a two-step response selection and validation process, and a semi-adaptive algorithm for efficient determination of two indifference points per delay. Here, again, the algorithm was run twice with a short break in between, yielding two indifference points per delay, and permitting measurement of response consistency across two consecutive series of decisions. Unlike the experiential task, the consequences of the choices were not experienced, and responses were immediately followed by a 0.5-s fixation cross and by the onset of the next trial.
To verify adequate engagement and understanding of the decisions, 16 catch trials (eight per run of the semi-adaptive algorithm) were also interspersed throughout the duration of the task. Half of the catch trials featured choosing between $100 now or a smaller amount of money also now, and half of the catch trials featured choosing between $100 now or $100 after a delay. Here, again, performance was considered valid if the participant made no more than 5/16 (31%) errors.
Pleasantness rating task
To evaluate the subjective value of viewing occluded photographs, participants were presented with a series of photographs, one at a time, with varying occlusion levels (Fig. 5). The photographs were presented for 5 s and were always novel, randomly selected without repetition among the group of photographs that were not used during the experiential intertemporal choice task. Immediately after each photograph presentation, participants rated the pleasantness of their experience using a Likert scale ranging from extremely unpleasant (−4) to extremely pleasant (+4). Rating selection was followed by a 0.5-s fixation cross and onset of the next trial. The number of rating responses varied slightly across participants (M = 47.8, SD = 4.7), depending on the number of remaining unique photographs in the bank of preselected photographs. Occlusion patterns were randomly selected among the 10 occlusion levels used in the experiential intertemporal choice task and randomly assigned to the photographs. Each occlusion level was presented on average 4.8 times per participant (SD = 1.3), with a minimum of at least two presentations. Reward sensitivity was conceptualized as the magnitude of the relation between occlusion levels and pleasantness ratings (see also Patt et al., 2021).
Procedure
The two intertemporal choice tasks were administered in two separate testing sessions to limit decision fatigue, at least one week apart. Order of administration was counterbalanced. The pleasantness rating task was always administered immediately following the experiential task so that it would not affect intertemporal decisions. The session with the experiential task and pleasantness rating task lasted about 1 h and 15 min on average. The session with the hypothetical task took ∼30–45 min on average.
Data from three participants in the hippocampal lesion group (P2, P3, and P6) and one participant in the healthy control group initially produced data of questionable validity. P6 displayed extreme systematic choice of the now option in the hypothetical intertemporal choice (ITC) task (e.g., he endorsed preferring $1 now to $100 in 1 d) a behavior that was more than two standard deviations from the mean of the patient group and was inconsistent with his own behavior in previous temporal discounting studies in our lab (Palombo et al., 2015, 2016b); and P2, P3, and the control participant provided valid responses on fewer than 60% of the catch trials in the experiential ITC task. The three participants in the hippocampal lesion group were re-tested nine months to a year later, with the examiner further ensuring task comprehension, for example by reading out loud the options displayed on the screen to compensate for potential forgetting of task instructions. All other task procedures remained the same, with participants facing the computer screen and pressing the response keys themselves. These sessions yielded valid datasets with P6 producing varied responses, and P2 and P3 missing only two and three out of 16 catch trials, respectively. The control participant who had initially failed the validity check was not retested and was excluded. All other datasets passed validity criteria with the first task administration.
The tasks were generally well tolerated and timely completed by the participants. However, one participant in the hippocampal lesion group, P5, displayed extensive hesitation on each decision during both the experiential and hypothetical intertemporal choice tasks, leading to substantially longer task durations. Because of fatigue, administration of each task was discontinued at half task point (after ∼45 min). Enough responses were provided to permit convergence of the first run of the semi-adaptive algorithm and collection of one complete TD curve per task. As such, these data were included in the analyses.
Analytical approach
Indifference point TD curves
Several mathematical models have been proposed over the years across the research fields of psychology, cognitive neuroscience, and economics to model TD indifference point curves, i.e., curves that plot the subjective value of the fixed larger reward as a function of delay as it undergoes delay-dependent devaluation (Frederick et al., 2002; McKerchar et al., 2009; Peters et al., 2012; Doyle, 2013). Here, we use the logistic function model with logarithmic timescale validated in our previous work (Patt et al., 2021):
Illustration of the logistic function model to fit TD indifference point curves, using a linear time scale (top panel) or logarithmic time scale (bottom panel). The logistic function is characterized by parameters a, the slope at the inflection point; and b, the log-delay at that inflection point. The hyperbolic curve is a particular type of logistic function with parameters a = 1 and
Brain imaging of the medial temporal lobe (MTL) lesions of the patients with amnesic syndrome included in the present study. Imaging modality varied depending on availability. Available images include CT slices in the axial plane (P1), T1-weighted MRI images in the coronal and axial plane (P2, P3, P5, and P7), and T2-Flair MRI images in the axial plane (P6). Imaging could not be collected for P4 because of medical contraindication.
Illustration of one trial in the experiential intertemporal choice task, for a choice of the immediate reward (left panel) or choice of the delayed reward (right panel). The task entailed making decisions between viewing a partially occluded photograph immediately or a nonoccluded photograph after a delay. Outcomes unfolded in real-time following each decision. The content of the photograph was not known at decision time. Every five trials, a five-point Likert scale was displayed and participants were asked about the pleasantness of their experience when they were viewing the more or less occluded photograph. After they made their selection, they were then presented with a “Processing Data” screen, displaying a loading bar of variable duration, designed to equalize experimental time across participants.
Illustration of one trial in the classic hypothetical intertemporal choice task, for a choice of the immediate reward (left panel) or choice of the delayed reward (right panel). The task entailed choosing between getting $100 after a delay or a smaller amount immediately. The choices were hypothetical with no real time consequences.
Illustration of one trial of the pleasantness ratings task, where participants were presented with a series of novel photographs with varying occlusion levels and asked to rate the pleasantness of their experience. This task implicated no intertemporal decision and no mention or experience of delay.
Use of a logistic function model (Verhulst, 1845) is particularly advantageous because of the independence and straightforward interpretability of the model's parameters, and its well-known application and validation across research fields (cf. item response theory, logistic regression; Swaminathan and Rogers, 1990). Many of the most commonly adopted TD models are in fact special case logistic functions, including the hyperbolic model by Mazur (1987) and the hyperbolic model with power scaling of time by Rodriguez and Logue (1988) and Rachlin (2006), see Patt et al. (2021) for further details. The reason that these models appear seemingly different is that they tend to use a linear time scale. It is the use of a logarithmic time scale that reveals the S-shape that underlies TD curves (McKerchar et al., 2009; Koffarnus et al., 2017). The logarithmic timescale is particularly advantageous in TD research as it is inherent to the distribution of delays employed in classic intertemporal choice tasks (e.g., one week, one month, three months, six months, one year, five years). It intrinsically accounts for the Weber–Fechner law of perception (Fechner, 1912), which suggests diminished perception of change in a physical stimulus as the overall magnitude of that stimulus increases (e.g., 1 d may be perceived as sizable a week from now, but as negligible in one year). Akin to the increased flexibility of Stevens' power law over the Weber–Fechner law for modeling subjective perception (Stevens, 1957), letting parameter a vary in the logistic function model [also known as parameter “s” in the Rachlin (2006) power scaling model] enables accounting for individual variability in sensitivity to changes in the delay, with smaller a (shallower slope at inflection point) implying decreased sensitivity to delay change and larger a implying increased sensitivity to delay change.
To compare TD curves across groups and tasks, linear mixed modeling was conducted with participant as a random factor. This analysis enabled fitting the indifference data points of all participants at once, thus simultaneously taking into account within and across subject variance. The linear mixed model was constructed using algebraic reformulation of Equation 1 into a linear form (see also Patt et al., 2021):
Fit for the linear mixed models was computed using maximum likelihood as implemented in the lme4 package (Bates et al., 2015) of R (R Core Team, 2019). The fit of the models with fixed effects of interest was compared with the models without those fixed effects using the Akaike's Information Criterion (AIC; Akaike, 1974) and Bayesian Information Criterion (BIC; Schwarz, 1978). R2 marginal effect sizes, representing the amount of variance explained by the fixed effects, were estimated using the method developed by Nakagawa and Schielzeth (2013), implemented with the piecewiseSEM R package. Significance levels for the model comparisons were derived using a likelihood ratio test with χ2 distribution. Significance of all fixed effects was evaluated with a t test using Satterthwaite's method, as implemented using the R lme4 package (Bates et al., 2015).
Perceptual reward sensitivity
To explore group differences in the association between level of occlusion of the photographs and the subjective experience of the reward, linear mixed modeling was also conducted on the data from the pleasantness rating task, again using participant as random factor. A simplified version of the model can be expressed as follows:
Choice consistency
To further elucidate our findings, we explored possible group differences in response consistency by examining discrepancy in the indifference points obtained for each participant at each delay during part 1 versus part 2 of each intertemporal choice task. A measure of drift was derived by computing the signed difference between indifference points, allowing examination of a possible systematic upward or downward shift in willingness to wait; and a measure of response variability was derived by examining the absolute value of the differences. Linear mixed models were run on these measures using Participant as random factor and log T, Task, Group, and their interactions as fixed effect. We also examined possible changes in the shape (i.e., the logistic parameters) of the indifference point curves across part 1 and part 2 by adding the binary variable Part as fixed effect to the aforementioned temporal discounting model, as well as the interactions of Part with logT, Task, and Group. As before, model fit and effect significance were evaluated using AIC and BIC indices, likelihood ratio test, Satterthwaite's t test, and R2 marginal effect size. ΔR2 marginal refers to differences in effect sizes between two models. Lastly, consistency in attention level across groups was examined by comparing the total number of errors on catch trials using the nonparametric Wilcoxon rank sum test. The possibility of Group × Task interaction on catch trial errors was explored using the Friedman ranks and Koch ranks nonparametric interaction tests for repeated measures, implemented with the npIntFactRep R-package (Feys, 2015).
Results
Delay-reward trade-off
Results of the linear mixed modeling analyses conducted simultaneously on the indifference points data of the experiential and classic hypothetical intertemporal choice tasks showed that the model that included Group, Task, log T as well as their associated two-way and three-way interaction terms (AIC = 7296.3, BIC = 7369.8, R2 marginal = 0.301) provided a significantly better fit than the same model without Group (AIC = 7329.5, BIC = 7383.4, R2 marginal = 0.262, χ2(4) = 41.1, p < 0.001). Examination of the fixed effects revealed a significant three-way Group × Task × log T interaction (β = 5.69, SE = 0.98, t(921.5) = 5.80, p < 0.001), suggesting the presence of a group difference in delay-reward trade-off that varied depending on the experiential or hypothetical nature of the task.
In the hypothetical intertemporal choice task, follow-up linear mixed modeling analyses showed that the model comprising the fixed effects of Group, log T, and their interaction (AIC = 3297.8, BIC = 3331.5, R2 marginal = 0.316) did not provide a better fit than the same model without Group (AIC = 3295.3, BIC = 3320.5, R2 marginal = 0.308, χ2(2) = 1.44, p = 0.486), and the fixed effects of Group (β = −3.93, SE = 3.49, t(23.3) = −1.13, p = 0.271) and Group × log T (β = 0.748, SE = 0.717, t(23.2) = 1.04, p = 0.307) were not significant. The fixed effect of log T, however, was significant (β = 1.92, SE = 0.60, t(23.4) = 3.21, p = 0.004). These results suggest the presence of effective delay-reward trade-off during the hypothetical task, with a pattern that was similar across groups.
By contrast, in the experiential task, the model that comprised the fixed effects of Group, log T, and their interaction (AIC = 3753.7, BIC = 3787.4, R2 marginal = 0.282) provided a better fit than the model without Group (AIC = 3755.8, BIC = 3781.0, R2 marginal = 0.225, χ2(2) = 6.03, p = 0.049), and the Group × log T interaction was significant (β = 6.14, SE = 2.36, t(23.5) = 2.61, p = 0.016). In the healthy control group, follow-up analyses suggested that the model with fixed effect of log T (AIC = 2676.6, BIC = 2699.7) provided a significantly better fit than the model without the fixed effect (AIC = 2695.9 BIC = 2715.2, χ2(1) = 21.3, p < 0.001), and the fixed effect of log T was significant and large in effect size (β = 9.64, SE = 1.44, t(16.0) = 6.68, p < 0.001, R2 marginal = 0.297). In the hippocampal lesion group, the model with fixed effect of log T (AIC =1080.6, BIC = 1098.4) also provided a better fit than the model without the fixed effect (AIC = 1083.7, BIC = 1098.5, χ2(1) = 3.98, p = 0.024), but the fixed effect of log T, albeit significant, was substantially smaller in effect size (β = 3.43, SE = 1.28, t(6.9) = 2.68, p = 0.032, R2 marginal = 0.082) than that of the control group. These findings indicate a group difference in delay-reward trade-off during the experiential task, characterized by a shallower slope in the indifference point TD curves of the hippocampal lesion group compared with the control group.
To verify that re-administration of sessions for participants who failed performance validity did not bias the results, we also analyzed the original dataset from which no data were excluded. These analyses yielded very similar results, with a significant three-way Group × Task × log T interaction (β = 5.56, SE = 0.98, t(962.6) = 5.67, p < 0.001). In the hypothetical task, Group did not add significantly to the model (χ2(2) = 3.83, p = 0.147). In the experiential task, the model with Group provided a better fit (χ2(2) = 6.98, p = 0.031), and the Group × log T interaction was significant (β = 6.53, SE = 2.33, t(24.5) = 2.81, p = 0.0096). Follow-up analyses also revealed an effect of log T that was large in healthy participants (β = 9.52, SE = 1.36, t(17.0) = 6.99, p < 0.001, R2 marginal = 0.283) but smaller and not significant in patients with hippocampal lesions (β = 2.94, SE = 1.42, t(6.9) = 2.07, p = 0.078, R2 marginal = 0.053).
For illustration, individual TD curves are plotted in Figure 6 for each participant and each task, obtained by fitting individual sets of indifference point data with a two-parameter logistic function using least square fit. Flattening in the logistic function TD curves of the hippocampal lesion group in the experiential task but not the hypothetical task is apparent. Another way to characterize differences across groups in the experiential TD profiles may be to note that, in the control group, 13 out of 16 participants (81.3%) had curves that intersected the half-reward range line within the bounds of the experiment (i.e., delays ranging between 1 and 25 s, and amount of visible photograph ranging between 13% and 100%). By contrast, only two out of seven participants in the hippocampal lesion group (28.6%) had a profile that intersected that line (χ2 test: χ2(1) = 5.96, p = 0.015).
Individual TD curves for the experiential intertemporal choice task (left panel) and classic hypothetical intertemporal choice task (right panel) represented for each participant by fitting their indifference point data with a two-parameter logistic function. The profiles of the participants with hippocampal lesions are highlighted and labeled. The y-axes represent the subjective value (SV) of the larger reward, expressed in terms of proportion of photograph visible for the experiential task and in dollars for the hypothetical task. The x-axes represent the delay, displayed in a logarithmic scale.
Experience of the reward
To examine potential differences across groups in the experience of the reward in a context that did not involve delays, linear mixed modeling was also conducted on the data from the pleasantness rating task, analyzed simultaneously for all occlusion levels and all participants. First, a significant fixed effect of Occlusion was noted (β = −0.476, SE = 0.087, t(22.9) = −5.49, p < 0.001), indicating that the occlusion lines functioned effectively to produce a reward continuum, with greater portion of visible photograph associated with more subjective pleasantness. The model that comprised the fixed effects of Group, Occlusion, and their interaction (AIC = 4199.3, BIC = 4239.3, R2 marginal = 0.317) did not provide a better fit than the same model without Group (AIC = 4195.5, BIC = 4225.5, R2 marginal = 0.315), and the variance in pleasantness ratings explained by Group was nonsignificant (χ2(2) = 0.175, p = 0.916, ΔR2 marginal = 0.002). Specifically, there was no significant main fixed effect of Group (β = −0.392, SE = 0.958, t(23.1) = −0.41, p = 0.686), implying that the groups provided similar ratings of pleasantness overall; and there was no significant Group × Occlusion interaction (β = −0.039, SE = 0.104, t(23.0) = −0.375, p = 0.711), suggesting similar reward sensitivity across groups. For visual illustration of the data, the relation between level of occlusion of the photographs and pleasantness ratings is presented in Figure 7, using regression lines derived for each participant.
Relation between level of occlusion of the photographs and pleasantness ratings, presented for each participant with regression lines. The regression lines of the participants with hippocampal lesions are highlighted and labeled. Intercepts represent pleasantness ratings when viewing the full photographs, and slope steepness may be interpreted as a measure of perceptual reward sensitivity.
Choice consistency
Results of the linear mixed modeling analysis conducted on the drift measure provided no evidence for a systematic increase or decrease in indifference points between part 1 and part 2, with an intercept that was not significantly different from zero (β = −0.001, SE = 0.04, t(46.9) = −0.036, p = 0.972). There was also no significant effect of logT (β = −0.0008, SE = 0.005, t(462.0) = −0.17, p = 0.865), Task (β = −0.008, SE = 0.02, t(462.0) = −0.37, p = 0.712), or Group (β = −0.02, SE = 0.04, t(22.0) = −0.69, p = 0.498), and inclusion of the interaction term Group × Task did not provide a better model fit (χ2(1) = 0.06, p = 0.812).
Results of the linear mixed modeling analysis conducted on the response variability measure demonstrated response variability that was greater in the experiential compared with the hypothetical task (β = 0.072, SE = 0.017, t(462.0) = 4.2, p < 0.001) and slightly greater at longer delays (β = 0.0077, SE = 0.0038, t(462.0) = 2.0, p = 0.045). The effect of group was marginal, with the hippocampal lesion group demonstrating a trend for more response variability compared with controls (β = 0.042, SE = 0.024, t(22.0) = 1.8, p = 0.094). However, the model with Group did not provide a significantly better fit than the model without Group and demonstrated negligible increase in effect size (χ2(1) = 2.87, p = 0.090, ΔR2 marginal = 0.012). Furthermore, adding a Group × Task interaction did not improve the model fit (χ2(1) = 0.22, p = 0.635, ΔR2 marginal = 0.0004), suggesting that any difference in response variability in the hippocampal lesion group compared with controls was similar across tasks.
Systematic changes in the shape of the indifference point curves across part 1 and part 2 were examined by adding Part and its interactions with logT, Task, and Group to the logistic function-based linear mixed model of temporal discounting. Results indicated that the model with Part did not provide a better fit than the model without it (χ2(8) = 2.92, p = 0.940, ΔR2 marginal = 0.001), arguing against a systematic pattern of change between part 1 and part 2 in the log-delay and slope at inflection point of the indifference point curves.
Lastly, analysis of the number of errors on the catch trials indicated that participants with hippocampal lesions made more errors (experiential task: M = 2.57, SD = 1.40; hypothetical task: M = 0.86, SD = 1.21) than controls (experiential task: M = 1.00, SD = 1.32, Wilcoxon rank sum test: W = 91, p = 0.019; hypothetical task: M = 0.12, SD = 0.34; Wilcoxon rank sum test: W = 75, p = 0.086). The Group × Task interaction was not significant (Friedman Ranks test: F(1,21) = 3.63, p = 0.071; Koch Ranks test: F(1,21) = 2.02, p = 0.170), suggesting similar group differences across tasks. Therefore, although lapses in attention may have affected the performance of participants with hippocampal lesions, they cannot account for group differences in intertemporal choice behavior that were selective to the experiential task.
Discussion
The present study examined the role of the hippocampus in intertemporal decisions when delays and rewards are experienced in the moment by evaluating patterns of decision in amnesic participants with hippocampal lesions on an experiential intertemporal choice task with real time delays and rewards, as well as on a classic intertemporal choice task with hypothetical outcomes. Participants with hippocampal lesions displayed patterns of decision that differed from those of controls during the experiential but not the hypothetical task. These findings suggest a critical role for the hippocampus in intertemporal decisions when they involve consequences that unfold in the moment, but not when the consequences are hypothetical. These results are in line with the dissociation observed in the literature between animal studies that are experiential by nature and have reported impaired TD associated with surgical or genetic manipulation of hippocampal function (Mariano et al., 2009; Bett et al., 2015; Masuda et al., 2020; Seib et al., 2021), and human studies that have reported intact choice behavior in individuals with hippocampal lesion during classic hypothetical intertemporal choice tasks (Kwan et al., 2012, 2013; Palombo et al., 2015).
In both experiential and hypothetical intertemporal choice tasks, participants with hippocampal lesions missed more catch trials than controls and displayed a marginal increase in response variability between the first and second half of the task. Although these effect sizes were small and negligible, respectively, it is possible that attention lapses and response variability may have contributed in some way to the indifference point curves in the hippocampal group. However, these effects were generally equivalent across both tasks and thus are unlikely to account for the selective impairment that was observed in the experiential task.
The intact performance of participants with hippocampal lesions in classic intertemporal choice tasks is compatible with the notion that, when outcomes are hypothetical, decisions can rely on subjective values of options that are known ahead of time, tapping into semantic knowledge that is independent of hippocampal function (Moscovitch et al., 2006; Binder and Desai, 2011). By contrast, the subjective values of options in the experiential task need to be computed on the fly and may require encoding the aversive experience of the delay as well as the pleasant experience of the reward, integrating both into novel representations, and possibly revisiting these computations when making subsequent decisions. The hippocampus could be involved at several of these levels of processing (Johnson et al., 2007; Palombo et al., 2019).
The shape of the TD indifference point curve of participants with hippocampal lesions in the experiential task featured shallower slope at inflection point, indicating diminished amplitude in the delay-reward trade-off compared with controls. Specifically, the curves were characterized by persistent willingness to wait for the greater larger reward despite delay increase, consistent with a possible deficit in computing changes in the experience of the delay. In support of this mechanism, the TD curves of participants with hippocampal lesions were noted to be similar to those of rodents with genetic deletion of hippocampal NMDA receptors, who also demonstrated reduced delay-dependent neural activity in the hippocampal CA1 subfield (Masuda et al., 2020). Amnesic patients with hippocampal lesions have previously demonstrated intact estimates of duration for short time scales up to at least 90 s (Palombo et al., 2016a). Therefore, a possible deficit in encoding the experience of the delay during experiential intertemporal decisions would likely involve a level of information processing beyond simple time perception, perhaps requiring the integration of delay perception with reward representations.
Deficits in processing changes in reward magnitude have been proposed to underlie TD impairment in rodents with surgical hippocampal lesions (Bett et al., 2015). Here, we examined how participants with hippocampal lesions experienced perceptual rewards by asking them to provide subjective ratings of pleasantness after viewing photographs with varying occlusion levels. In that task, which was devoid of delay constraints, participants with hippocampal lesions reported similar patterns of subjective pleasantness compared with controls, both in terms of overall level of endorsed pleasantness, and in terms of sensitivity to changes in the perceptual stimuli. Therefore, it is unlikely that the impairment observed in experiential TD in the present study can be explained by diminished reward sensitivity. It remains possible, however, that the hippocampus is implicated in coding the experience of the reward in the intertemporal choice context, for example by integrating reward information with the experience of the delay into novel value representations. De novo computations of subjective values are likely to require the hippocampus (Palombo et al., 2019), relying on its unique properties in complex relational encoding (Cohen and Eichenbaum, 1993; Yonelinas, 2013). Such a mechanism is further supported by the identification of neurons in the hippocampal CA1 subfield that specialize in encoding reward-context conjunctions (Gauthier and Tank, 2018; Xiao et al., 2020).
Another mechanism that may play a role in the selective impairment observed in the experiential intertemporal choice task may relate to deficits in prospection in participants with hippocampal lesions, i.e., the ability to sample through past events and make predictive value representations to support decisions, a process that has been suggested to rely on the hippocampus (Johnson et al., 2007; Bornstein and Daw, 2013). Prospective processes are particularly relevant during the experiential task, as the subjective value of options can be updated and revisited during the task, based on the experience of the delay and experience of the reward. This mechanism has been suggested to be responsible for hippocampal involvement in animal intertemporal choice tasks (Johnson and Redish, 2007), and is supported by findings that CA1 neurons in the hippocampus are involved in trial-by-trial updating of option values (Jeong et al., 2018) and code for the expected values of potential choices as well as for the actual values of selected choices (Lee et al., 2012). It is also consistent with the longer investigatory period noted in animals with hippocampal lesions, arguably to compensate for difficulties in learning delay-reward contingencies (Bett et al., 2015). The idea that hippocampal lesions may impact experiential TD via prospective processes is also compatible with findings that the hippocampus plays a role in hypothetical intertemporal choices when the task engages future thinking via episodic cueing, that is, when participants are instructed to make mental predictions of what they might do with the reward in the future (Peters and Büchel, 2010; Benoit et al., 2011; Palombo et al., 2015; Wiehler et al., 2017).
Interestingly, Peters et al. (2012) re-examined the TD curves of their previous dataset (Peters and Büchel, 2010), using fit by the constant sensitivity discounting function of Ebert and Prelec (2007). Transforming delays to a logarithmic scale using simple algebra shows that the model of Ebert and Prelec (2007) can also be described by an S-shape, with parameters essentially describing the delay and slope at inflection point, much like the present logistic model, but with reversed a and b notations, and with the inflection point located at 1/e rather than at 1/2 of the subjective value range. Peters et al. (2012) found that episodic cueing left unchanged the parameter corresponding to the delay at inflection point, but led to a significant increase in the parameter corresponding to the slope at inflection point. This pattern directly maps onto the current findings of an association between hippocampal lesions and a decreased slope at inflection point in the experiential TD curves.
An additional consideration is whether our findings may reflect group differences in patience. By this view, participants with hippocampal lesions may have been more willing to wait for the larger later reward compared with healthy control participants, but the range of delays used in the current experiment may have been too short to reveal their TD curves' inflection points. In other words, it is possible that the use of longer delays in the experiment might have revealed a logistic curve in the hippocampal group with similar slope but delayed offset, thus indicating intact computation of the experience of the delay but greater patience. The possibility that hippocampal lesions lead to greater patience is unlikely, however, considering demonstrations in several animal lesion studies of a reduced willingness to wait for the larger later reward (Rawlins et al., 1985; Cheung and Cardinal, 2005; McHugh et al., 2008; Mariano et al., 2009; Abela and Chudasama, 2013). Considering the critical role of the hippocampus in encoding contingencies that are complex and multidimensional (Yonelinas, 2013), a common underlying impairment for computing complex delay-reward contingencies may be more likely, manifesting differently depending on task demands.
In most animal TD studies, contingencies are fixed within a block, involving two locations (left or right), two reward amounts (small or large), and two delay durations (now or fixed delay). Within this complex context, hippocampal impairment may lead to the dropping of one of the choice dimensions from the trade-off computations, perhaps the least salient depending on the circumstances. For example, failure to integrate differences in reward amounts would lead to simplifying the choices to “food on the left now” or “food on the right after a delay,” thus yielding increased choice of the immediate option with increased delay, a profile seen in several studies (Cheung and Cardinal, 2005; Abela and Chudasama, 2013). The experiments conducted by Bett et al. (2015) with a delay adjustment procedure also can be interpreted in this light. First, when the reward ratio was fixed (3:1), animals with hippocampal lesions required more exploratory trials before committing to their preferred option, but in the end demonstrated a comparable willingness to wait as sham animals, suggesting impaired learning of reward-delay contingencies rather than impatience. Second, when reward ratios were subsequently varied from session to session, the lesioned animals performed normally at a ratio of 1:1, but then continued to make decisions at ratios of 2:1 and 3:1 as if the amount of reward remained similar in both arms, thus leading to over-discounting in an experimental configuration where their performance was previously intact. Such a pattern suggests difficulty in taking into account the updated reward contingencies. Interestingly, the animals displayed intact willingness to wait again when the reward ratio became large enough at 5:1, further supporting a salience-based encoding impairment rather than a problem of impatience. In the present study, encoding and integrating the experience of the delays and experience of the rewards was complicated by the adaptive nature of the task, with options continuously changing in a pseudo-randomized order. Perhaps the most salient feature in our task was the continuum of rewards (as indicated by the bold lines drawn through the photographs). Accordingly, it was the subjective values of the changing delays that could not be integrated into the value computations of individuals with hippocampal lesions, leading to the shallower sloped TD profiles described above.
Interestingly, rather than constituting a limitation of the experiential ITC task, the use of a fixed range of delays with a 25-s upper boundary may have contributed to eliciting a different pattern of intertemporal choices in the patients. Indeed, experimental range adaptation is a process that is expected in intertemporal choice tasks in healthy individuals, with brain correlates demonstrated to track subjective values computed relative to the experimental temporal context, rather than in absolute terms (Cox and Kable, 2014). Here, it is notable that the experiential TD curves of participants with hippocampal lesions were also characterized by a marked decrease in range coverage of the possible subjective values compared with control participants. (Only two out of the seven participants with hippocampal lesions had TD curves that spanned beyond half of the experimental reward range, vs 13 out of 16 in the control group.) Thus, the current results could suggest a deficit in range adaptation mechanisms in participants with hippocampal lesions. These considerations may again be consistent with episodic processes, whereby subjective values are iteratively adjusted and calibrated based on prior experiences.
Whether the impairment in participants with hippocampal lesions reflects a failure to update subjective value representations will require further investigation. Because of the adaptive nature of the task, we were unable to track possible iterative adjustment or calibration of subjective value representations over time in the experiential task. Comparison of response patterns across the first and second half of the task revealed no systematic drift in subjective values and no change in the shape of the indifference point curves. However, this comparison may have been too coarse to reveal progressive updating of representations over the course of the task. Future studies including repeat choice trials immediately followed by pleasantness ratings will be necessary to assess value updating. Alternatively, changes in value updating could be experimentally induced, for example by manipulating the experience of the stated delays (e.g., by changing the delay duration unbeknownst to the participants) or by altering the range of delays across sessions.
Finally, although the hippocampus was the only area of lesion overlap in the patient group, brain injury varied and extended beyond the hippocampus in several participants. It is thus possible that other brain areas also contributed to the effects reported in the present study. Additional studies of patients with focal hippocampal lesions and examination of functional activation patterns during the experiential TD task in normal cognition will be important to further address the anatomic specificity of the observed effects.
In conclusion, the present study demonstrates that hippocampal lesions are associated with impaired intertemporal choice when the outcomes of decisions are experienced in the moment, but not when they are hypothetical. These results shed light on previous discrepancies across experiential animal studies and hypothetical human TD studies, where impaired and intact intertemporal choice performance has been reported, respectively. They suggest that different cognitive and neural mechanisms are involved in intertemporal decisions depending on the experiential versus hypothetical nature of the decisions, albeit that the nature of the impairment in experiential intertemporal choice remains to be further elucidated.
Footnotes
This work was supported by the United States Department of Veterans Affairs Clinical Science Research and Development Service Grant I01CX001748 and a Senior Research Career Scientist Award (M.V.). We thank Caroline Strang for research assistance.
The authors declare no competing financial interests.
- Correspondence should be addressed to Mieke Verfaellie at mieke.verfaellie{at}va.gov
