Abstract
Reward function appears to be modulated by the circadian system, but little is known about the neural basis of this interaction. Previous research suggests that the neural reward response may be different in the afternoon; however, the direction of this effect is contentious. Reward response may follow the diurnal rhythm in self-reported positive affect, peaking in the early afternoon. An alternative is that daily reward response represents a type of prediction error, with neural reward activation relatively high at times of day when rewards are unexpected (i.e., early and late in the day). The present study measured neural reward activation in the context of a validated reward task at 10.00 h, 14.00 h, and 19.00 h in healthy human males. A region of interest BOLD fMRI protocol was used to investigate the diurnal waveform of activation in reward-related brain regions. Multilevel modeling found, as expected, a highly significant quadratic time-of-day effect focusing on the left putamen (p < 0.001). Consistent with the “prediction error” hypothesis, activation was significantly higher at 10.00 h and 19.00 h compared with 14.00 h. It is provisionally concluded that the putamen may be particularly important in endogenous priming of reward motivation at different times of day, with the pattern of activation consistent with circadian-modulated reward expectancies in neural pathways (i.e., greater activation to reward stimuli at unexpected times of day). This study encourages further research into circadian modulation of reward and underscores the methodological importance of accounting for time of day in fMRI protocols.
SIGNIFICANCE STATEMENT This is one of the first studies to use a repeated-measures imaging procedure to explore the diurnal rhythm of reward activation. Although self-reported reward (most often operationalized as positive affect) peaks in the afternoon, the present findings indicate that neural activation is lowest at this time. We conclude that the diurnal neural activation pattern may reflect a prediction error of the brain, where rewards at unexpected times (10.00 h and 19.00 h) elicit higher activation in reward brain regions than at expected (14.00 h) times. These data also have methodological significance, suggesting that there may be a time of day influence, which should be accounted for in neural reward studies.
Introduction
Reward function in animals and humans appears to be adaptively modulated by the circadian system (Murray et al., 2002, 2009). Theoretically, the human reward system is primed to be more active during daytime hours when reward potential is high and risk relatively low, and less active overnight when this balance is reversed due to poor night vision (Watson, 2000). Existing research has found a circadian rhythm in self-reported positive affect (the subjective manifestation of reward activation), peaking in the early afternoon and paralleling the circadian rhythm of core body temperature under naturalistic conditions (e.g., Boivin et al., 1997; Murray et al., 2009). It has been hypothesized that this circadian reward rhythm should also be measurable in reward neurocircuitry (e.g., Murray et al., 2009), but research to date is limited.
A small number of studies have taken the preliminary step of testing for a diurnal rhythm (a waveform across the waking day that may or may not be of endogenous circadian origin) in various measures of neural activation in humans. Hasler et al. (2014) used a within-subjects fMRI procedure to observe greater striatal activity to monetary rewards in the afternoon compared with morning. A recent study (Masterson et al., 2016) found a significant time-of-day effect on neural activation in the right ventral striatum and left putamen in response to visual food stimuli. Activation was higher in these reward regions during the morning (06.30–08.30 h) compared with the evening scan (17.30–19.30 h) (Masterson et al., 2016). Together, these studies indicate the presence of a neural diurnal rhythm in response to rewards; however, the timing of this rhythm is ambiguous and has been limited to two measurement points thus far.
The reliable finding of a mid-afternoon peak in positive affect (above) does not necessarily suggest that a circadian rhythm in neural reward activation would have the same waveform. It has been argued (Schultz et al., 1992; Schultz, 2002, 2016) that dopaminergic neurons innervate the terminal striatal regions with greater intensity when an environmental reward deviates from previous reward expectancies. Additional work (McClure et al., 2003) has demonstrated higher levels of striatal activation (with strongest lateralized activation in the left putamen) in response to unexpected reward stimuli. Other human fMRI studies have found this same “prediction error” in response to reward with a positive correlation between unexpected reward and ventral striatum activity (O'Doherty et al., 2003; Hare et al., 2008; Valentin and O'Doherty, 2009) and more dorsal striatal regions (Valentin and O'Doherty, 2009). In sum, a circadian rhythm in neural reward activation may be measurable as an elevated response to rewards at unexpected times of day: earlier and later in the day in humans (Watson, 2000; Murray et al., 2009), and inverted relative to the waveform observed in positive affect under naturalistic conditions.
The present study used an fMRI procedure to examine the diurnal rhythm of activation in reward neurocircuitry in response to a reward stimulus. Activity in the mPFC, VTA, anterior cingulate cortex, caudate, NAc, and putamen was examined. To model the diurnal rhythm, we advanced existing research by scanning participants at three times of day (10.00, 14.00, and 19.00 h). Reward activation was hypothesized to vary with a quadratic waveform, consistent with an underlying circadian driver: that is, with 14.00 h having an altered neural reward level relative to 10.00 h and 19.00 h. Based on the “prediction error” assumption, higher levels of neural activation were predicted earlier in the day (10.00 h) and later in the day (19.00 h); however, the fitted sine curve was also modeled with a peak in the quadratic waveform at 14.00 h to align with Hasler et al. (2014) whose data indicate that neural reward activation may mirror the demonstrated rhythm in self-reported positive affect, peaking later in the day relative to earlier.
Materials and Methods
Participants.
Participants were 16 right-handed males (mean ± SD, 22.65 ± 2.87 years) screened to exclude previous and current mental illness, shift work, and transmeridian travel within 3 months of participation.
fMRI task.
The gambling reward procedure (Delgado et al., 2000) from the Human Connectome Project is a pseudo-reward task, which involves guessing the value of a card (1–9). The trial begins with a question mark displayed on the screen for 1500 ms with responses recorded on a response box. A white fixation cross is presented if a response is made before 1500 ms with feedback for 1000 ms. Cards are predetermined so that 40% of trials are rewarding ($1, green up arrow), 40% loss (−50 cents, red down arrow), and 20% neutral trials (−, double-headed gray arrow). Nonresponses are presented with a screen stating that response was too slow.
In the present study, eight trials were presented in four blocks, of which two were mostly reward (Reward Block = six reward trials, and two neutral or loss trials) and two mostly loss trials (Loss Block = six loss trials, and two neutral or reward trials). Following each block of eight trials, there was a 15 s fixation cross, which was used as the Baseline comparison. The data were acquired during four scanning runs (each consisting of four blocks taking 3 min: 12 s per run) at each time point with a short break between runs. Before the first run at each time point, a practice run acclimatized participants to the task. To increase task motivation across sessions, participants were informed that they would receive additional reimbursement for their best task performance across the three sessions. Because of the predetermined values, participants received $27 AUD.
MRI data acquisition.
Structural and functional images were acquired with a 3T Siemens TIM Trio MRI scanner at Swinburne University of Technology (Hawthorn, Victoria, Australia). During fMRI scanning, visual stimuli were presented on a rear projection screen viewed by participants through a mirror attached to the 32-channel head coil. All aspects of stimulus delivery and response logging were performed using E-Prime 2.0 (Psychology Software Tools 2002) for Windows.
Each scanning session began with the acquisition of a high-resolution T1-weighted scan using an MP-RAGE sequence (192 sagittal slices; 1 mm isotropic voxels; flip angle 9°; FOV = 256 × 192 mm; TR = 2200 ms; TE = 3.29 ms; matrix = 256 × 192). During each of the four fMRI task scanning runs in each session, 73 T2*-weighted images were acquired using a gradient echo EPI sequence (39 interleaved axial slices; 3 mm isotropic voxels; flip angle 90°; FOV = 205 mm, TR = 2000 ms TE = 25 ms; matrix = 64 × 64).
MRI data preprocessing.
All aspects of MRI image preprocessing and statistical analysis were conducted using SPM12 (Ashburner et al., 2014; Wellcome Trust Centre for Neuroimaging, University College London) and associated toolboxes. Initially, the high-resolution structural image and functional time-series were manually realigned to closely match the MNI template in SPM12. Subsequently, highly variant EPI slices were corrected using an interpolation algorithm in ArtRepair tools (version 5b) (Mazaika et al., 2009). These artifact-corrected images were slice time corrected using the middle slice acquired in time as a reference, then realigned to the first EPI acquired. The realigned images were then coregistered to the T1 image, which was then transformed (normalized) into MNI space. The parameters of this transformation were applied to coregistered EPIs, which were then smoothed with a 6 mm FWHM Gaussian filter and high-pass filtered (<128 s). Finally, Artifact Detection Tools (Whitfield-Gabrieli et al., 2011) were used to determine outlying images, defined as any image ±3 SDs from mean signal intensity of the time-series, or images exhibiting >1.5 mm of movement from the preceding image.
Statistical analysis of fMRI data.
Participant level modeling was performed using an epoch-based general linear model in SPM12. The BOLD signal for Reward and Loss blocks was modeled using a boxcar function defined by the onset and duration of each block convolved with the canonical hemodynamic response function supplied with SPM12. The periods of fixation cross presentation constituted the Baseline; but as is common practice in fMRI analyses, these blocks were not explicitly modeled as this leads to the model being overdetermined. Additionally, regressors of no interest were modeled, including one regressor for each motion realignment parameter (3 translational, 3 rotational) and one regressor for each outlying image determined using Artifact Detection Tools (https://www.nitrc.org/projects/artifact_detect/); <10% of images for each participant. After model estimation, the contrasts of Reward > Baseline, Reward > Loss, and Loss > Baseline were computed and then entered into a second-level, random-effects, repeated-measures ANOVA model with factor Time of Day (10.00, 14.00, and 19.00 h). We then examined the data for a main effect of Time of Day by using an uncorrected voxel level threshold of p < 0.001. Given the a priori reward regions of interest (ROIs), we used a small volume correction at the cluster level (Worsley et al., 1996).
Procedure.
Participants maintained sleep and daily activity diaries and wore an actigraph in the week before the study to test for sleep-related variables on the testing day. To account for repeated-measures confounds, participants' start times were counterbalanced, with all testing completed within 24 h. Scan time was 1 h for the first session (with structural scans completed) and 30 min for the second and third sessions. The task was run in E-Prime 2.0, with a BOLD signal fMRI time-series used to acquire images for the voxels within each ROI.
Results
A main effect was observed in the ventral portion of the left putamen (MNI coordinates of peak voxel: −28, 4, −2, peak F value: 13.97, cluster size: 23 voxels; for the Reward > Baseline contrasts, see Fig. 1). This effect was observed in a similar location in the Reward > Loss contrast (MNI coordinates of peak voxel: −28, 10, −8, peak F value: 11.51, cluster size: 4 voxels). In the bilateral caudate, a small cluster was observed; however, it did not survive cluster level correction (left caudate 1 voxel, F = 6.11, p = 0.61, MNI coordinates: −12, 14, 14; right caudate 4 voxels, F = 6.34, p = 0.49, MNI coordinates: 14, 16, 10). Even at a more liberal voxelwise threshold (p < 0.05, uncorrected) bilaterally, the mPFC, VTA, anterior cingulate cortex, NAc, and the right putamen were all nonsignificant to different activation at time of day. No clusters in the a priori reward regions showed a time-of-day effect for the Loss > Baseline contrasts.
BOLD contrast of Reward > Baseline with a repeated-measures Time of Day factor entered into the model. A, Activation of left putamen significant (p < 0.001) for a Time-of-day effect. B, Activation of left putamen significantly decreased at 14.00 h, compared with 10.00 h or 19.00 h. Error bars represent SEM.
A volume of interest (sphere, 2 mm radius) centered on the peak voxel of this cluster was constructed, and the first eigenvariate extracted from the modeled contrast images. These data were entered into multilevel modeling that tested the quadratic waveform fit of the repeated-measures Level 1 variable (Time of Day) with a nadir at 14.00 h. An intercept only model was conducted for the left putamen voxel cluster. The Level 1 Model (Left putamen activationij = β0j + β1j (time of day) + rij) tested the time-of-day effect, with group mean centering performed before model inclusion. β0j represents each participant's neural activation in the significant voxel, and rij represents the within-person variance. β1j represents the time of day slope of the fitted quadratic waveform with a nadir at 14.00 h for each participant, and no difference modeled between 10.00 and 19.00 h. The analyses dummy coded this as 1 (10.00 h), −2 (14.00 h), and 1 (19.00 h). The quadratic waveform provided a highly significant fit to the data (p < 0.001), with activation in the left putamen being significantly lower at 14.00 h than 10.00 h or 19.00 h.
For completeness, a whole-brain analysis was performed. Using an arbitrary clusterwise threshold of 10 contiguous voxels, a cluster in the left insula (in the posterior region; 18 voxels, MNI coordinates: −32, −30, 20) and the middle frontal gyrus (in the anterior region; 26 voxels, MNI coordinates: −32, 52, 6) was found for the reward > baseline analyses; these effects were all nonsignificant (p values >0.5). No other significant time-of-day threshold voxel clusters were observed, even when a liberal (p < 0.05, uncorrected) threshold was applied. There was no iteration effect of repeated-measures testing of the neural reward rhythm on putamen activation.
Discussion
This study investigated the rhythm of neural activation to rewards across the course of the waking day. The hypothesis that activation of reward circuitry would be lowest at 14.00 h compared with 10.00 h and 19.00 h gained preliminary support, with a significant waveform fit found in left putamen activation in the context of a validated reward task. Other reward ROIs did not show a significant time-of-day effect.
Left putamen exhibits diurnal changes
Left putamen activation exhibited a diurnal waveform with relatively decreased activation in the early afternoon. Existing literature suggests that the putamen is a core component of reward-related function in humans (O'Doherty et al., 2003), rodents (Gallardo et al., 2014), and monkeys (Muranishi et al., 2011). Muranishi et al. (2011) demonstrated that pharmacological inhibition of the left putamen in monkeys impaired reward-based decision making. Furthermore, Szczypka et al. (2001) found that sucrose preference in dopamine-deficient mice was restored with supplanted dopamine in the caudate-putamen or nucleus accumbens; however, dopamine replacement only in the caudate-putamen restored feeding behavior, suggesting that the putamen is core to the neural reward circuitry and has specified reward functions.
This study provides preliminary evidence of the importance of the putamen in understanding the putative interaction between circadian and reward function. In animal studies, the putamen has been innervated by the endogenous circadian system in reward functions, including the following: food anticipation (Gallardo et al., 2014), the circadian locomotor rhythm (Masubuchi et al., 2000), and in circadian gene expression following methamphetamine injection in rodents (Nikaido et al., 2001). Similarly, two earlier human diurnal imaging studies have found time-of-day variation in left putamen activation (Hasler et al., 2012b; Masterson et al., 2016). Given the interconnectedness of neural reward pathways and broader striatum region (Haber and Knutson, 2010), more work is now needed to investigate diurnal variation in other reward-related regions, including the mPFC, ventral striatum caudate, and anterior cingulate cortex. The present results suggest that these reward regions do not exhibit time-of-day effects, but larger samples and alternative imaging methods (discussed below) may detect additional signals of circadian modulation.
Putamen activation to reward is lowest in the early afternoon
Prior studies have found that positive affect, a subjective manifestation of reward activation, is highest in the mid-afternoon (Clark et al., 1989; Watson et al., 1999; Murray et al., 2009). This finding has been interpreted as indexing an adaptive preparedness to pursue rewards when environmental conditions are optimal (Wehr, 1990). In the present neuroimaging study, by contrast, neural reward circuitry was relatively low in the mid-afternoon: The diurnal waveform in the left putamen reward region had its nadir at 14.00 h.
As noted above, we propose that this pattern of findings can be understood as a type of prediction error. Specifically, we propose that rewards presented at 14.00 are expected (by circadian priming), and thus lack the novelty of rewards appearing at 10.00 h or 19.00 h. Consistent with this explanation, we note that a bigger hemodynamic response to unexpected reward has previously been observed in the left putamen compared with expected rewards (McClure et al., 2003; O'Doherty et al., 2003). When the brain expects rewards to be in abundance, then reward accrual elicits less neural excitation (Schultz et al., 1992; Schultz, 2016). Schultz et al. (1997) proposed that optimal reward functioning is contingent upon an organism's prior conditioning that predicts the timing and magnitude of rewarding events. Here, we extend this contention to the 24 h time frame by suggesting that the circadian system is the primary endogenous mechanism that conditions individuals to anticipate reward at different times of day.
An intriguing extension to these findings is that the neural reward response to reward stimuli may have an inverted diurnal waveform to self-reported ratings of positive affect. Multiple ambulatory (Clark et al., 1989; Watson et al., 1999; Stone et al., 2006; Miller et al., 2015) and circadian (Boivin et al., 1997; Murray et al., 2002, 2009) studies have found a peak in self-reported positive affect in the afternoon hours. Two explanations for the finding of lowered neural intensity in reward regions at times typically associated with higher self-reported positive affect warrant consideration. First, Masterson et al. (2016) found decreased activation in reward regions in response to food stimuli from 17.30 to 19.30 in the evening (as opposed to 0630–0830 in the morning), whereas self-reported interest in food and hunger was greater in the evening. The authors conclude that the dampened neural sensitivity to food stimuli in the evening may instigate a greater behavioral drive for food to obtain the same reward levels as observed in the morning. The type of reward stimuli is an important consideration, with Masterson et al. (2016) using food and our study and previous work in diurnal rhythms (e.g., Hasler et al., 2014) using monetary rewards. Neural reward activity may exhibit different daily activation patterns depending on stimuli used, and future work should attempt to investigate the role of diurnal rhythms across various reward stimuli.
Second, the observed decrease in neural activation of the putamen may be explained by a methodological limitation of BOLD fMRI imaging. The associated energy demands that the BOLD response is capturing are not sensitive to differences between excitation and inhibition of neuronal activity (Nair, 2005). Higher activation at unexpected times may indeed be capturing greater inhibition of reward regions to monetary incentives in the putamen. Future studies should monitor self-reported positive affect while collecting repeated-measures neural data to test this important proposition that self-report positive affect and neural activation in response to rewards may be inverted. Complex relationships between neural, subjective, and behavioral measures of reward function have been reliably documented (Berridge et al., 2009), and more research is required to understand this interplay in the diurnal/circadian context.
Although there are strong reasons to expect neural reward activation to be lowest in the afternoon hours, it is important to note that the present results conflict with the findings of Hasler et al. (2014). Several methodological differences should be noted between the two studies. First, whereas we found an effect in the dorsal striatum (left putamen), Hasler et al. (2014) found a time-of-day effect in the ventral portion of the striatum. Although both regions are associated with reward neurocircuitry, we do not have enough literature to assert whether reward regions exhibit similar diurnal neural rhythms. Second, the study by Hasler et al. (2014) had a single PM time point (15.06–18.38 h) sitting between our afternoon (14.00 h) and evening (19.00 h) scans. Third, unlike the present design, Hasler et al. (2014) experimentally controlled for individual sleep and wake times; a post hoc investigation revealed no significant effect of sleep variables on the diurnal neural waveform found here. Last, although the fMRI task was similar in both studies, the present design used more trials and runs and used a nonmotor Baseline, whereas Hasler et al. (2014) used a button-pressing control condition. More broadly, neuroimaging data analysis has been notoriously difficult to replicate particularly for less established or weaker findings (Poldrack and Poline, 2015; Poldrack et al., 2017). In sum, multiple methodological differences may partially explain the difference in findings here versus the sole related study, and we propose that this discrepancy should motivate more systematic and intensive research. Additionally, we agree with Hasler et al. (2014) that event-related designs, which allow for distinguishing between anticipation and consumption of rewards, may help to extend our understanding of the neural reward rhythm.
Limitations, clinical application, and future research
Although this study provides an important advance to our understanding of the diurnal rhythm of neural reward circuitry, several limitations should be noted. The primary limitation of this study was that, whereas the ultimate framework for this project is circadian, the data collected only speak to a diurnal rhythm, the endogeneity of which is unknown. To confirm the endogeneity of this rhythm, future research should examine whether the timing of this reward rhythm is generated from internal cues rather than external learned associations of the rewarding potential of a certain time of day. Although this would traditionally be done through constant routine or forced desynchrony circadian rhythm protocols, practical constraints around mobility of sleep laboratories and imaging equipment do not allow for this. Novel approaches are now required to consider how time-free environments could be created to test for a circadian relationship. Future work should consider using more testing sessions to examine more precise waveform characteristics over the waking day, similarly extending these findings to a larger sample, women, and a wider age range will help generalize the present results.
Understanding the role of the circadian system in modulating the neural reward response has potential clinical implications. Abnormalities of circadian reward functioning have been noted in individuals experiencing mood disturbance. For example, in studies of diurnal mood variation, individuals with depressive symptoms (Gordijn et al., 1994; Murray, 2007) exhibit lower variability in daily positive mood, and altered waveform patterns. Von Zerssen et al. (1987) found, for example, lower mood for clinically depressed individuals in the morning, whereas for healthy matched controls it was lowest in the subjective night when sleep was interrupted to take measurements. Beyond circadian phase, recent work suggests that individuals with depression and bipolar disorder may have a decreased circadian amplitude in positive mood variation (Murray, 2007; Grierson et al., 2016). Disruptions to both the phase and amplitude of circadian rhythms have long been hypothesized to contribute to the pathogenesis and maintenance of mood disorders (Czeisler et al., 1987).
These preliminary findings raise different avenues for future research. In the present study, task-based fMRI was used to examine neural reward activity in the context of rewarding stimuli. Alternative imaging methods, such as arterial spin labeling, may better capture hourly temporal changes as a more sensitive measure of the diurnal changes in regional brain function intensity (Hermes et al., 2007; Goel et al., 2013; Mikita et al., 2015). Future research should endeavor to investigate whether the known mid-afternoon circadian dip in alertness may be relevant to interpreting the neural signal in reward activation. Methodologically, the present findings speak to the necessity of controlling for time of day when performing neuroimaging studies. The diurnal rhythm in reward functioning observed here raises questions about findings from neuroimaging protocols that have neither controlled for nor reported time of day.
Within its limitations, this study is among the first to examine variation in human neural reward functioning in relation to time of day. This preliminary evidence suggests that there is a diurnal rhythm in left putamen activation, part of the neural circuitry involved in reward. It extends the small collection of studies that have looked at this relationship by using three time points to better capture the shape of the diurnal neural reward rhythm in the context of reward stimuli. This work is a first step in testing the chronobiological hypothesis of a circadian reward rhythm, adding to a burgeoning imaging literature interested in how the circadian system regulates reward circuitry (Forbes et al., 2012; Hasler et al., 2012a, 2014; Byrne and Murray, 2017). This novel finding also underscores the importance of future research between how vulnerability to, or experience of mental illnesses known to affect reward and circadian pathways may differ from healthy individuals, which can facilitate targeted clinical intervention. These insights provide a foundation for understanding that diurnal neural reward rhythms exist in healthy individuals.
Note added in Proof: The contributions of one of the authors, Sheri L. Johnson, was accidentally left out of the early release version, published August 21, 2017. The author contribution line has now been corrected.
Footnotes
This work was supported by the Barbara Dicker Brain Sciences Foundation
The authors declare no competing financial interests.
- Correspondence should be addressed to Professor Greg Murray, Centre for Mental Health, Faculty Health, Arts and Design, Swinburne University of Technology, John Street, Hawthorn, VIC 3122, Australia. gwm{at}swin.edu.au
