Abstract
Choice behavior is characterized by temporal discounting, i.e., preference for immediate rewards given a choice between immediate and delayed rewards. Agouti-related peptide (AgRP)-expressing neurons located in the arcuate nucleus of the hypothalamus (ARC) regulate food intake and energy homeostasis, yet whether AgRP neurons influence choice behavior and temporal discounting is unknown. Here, we demonstrate that motivational state potently modulates temporal discounting. Hungry mice (both male and female) strongly preferred immediate food rewards, yet sated mice were largely indifferent to reward delay. More importantly, selective optogenetic activation of AgRP-expressing neurons or their axon terminals within the posterior bed nucleus of stria terminalis (BNST) produced temporal discounting in sated mice. Furthermore, activation of neuropeptide Y (NPY) type 1 receptors (Y1Rs) within the BNST is sufficient to produce temporal discounting. These results demonstrate a profound influence of hypothalamic signaling on temporal discounting for food rewards and reveal a novel circuit that determine choice behavior.
SIGNIFICANCE STATEMENT Temporal discounting is a universal phenomenon found in many species, yet the underlying neurocircuit mechanisms are still poorly understood. Our results revealed a novel neural pathway from agouti-related peptide (AgRP) neurons in the hypothalamus to the bed nucleus of stria terminalis (BNST) that regulates temporal discounting in decision-making.
Introduction
In choice behavior, the preference for an immediate over delayed reward is known as temporal or delay discounting, which characterizes choice behavior in all species examined (Chung and Herrnstein, 1967; Green and Myerson, 2004; Kable and Glimcher, 2007), and is often significantly altered in psychiatric disorders (Story et al., 2015), addiction (Bickel and Marsch, 2001), and obesity (Volkow et al., 2011). The underlying mechanisms of temporal discounting, however, remain poorly understood.
The valuation of specific commodities, as reflected in choice behavior, reflects the current motivational state and the status of essential controlled variables. For example, studies have shown that obesity is associated with greater temporal discounting, suggesting that higher demand for food is associated with a tendency for immediate gratification (Weller et al., 2008). Yet the role of motivational state in regulating temporal discounting remains controversial, as previous studies produced conflicting results (Logue and Peña-Correal, 1985; Bradshaw and Szabadi, 1992; Wogar et al., 1992; Kirk and Logue, 1997; Oliveira et al., 2013; Skrynka and Vincent, 2019).
Recent work in mice has begun to elucidate the neural mechanisms underlying intrinsic motivational states like hunger and thirst (Betley et al., 2013; Chen and Knight, 2016; Zimmerman et al., 2017; Rossi et al., 2019). Can signals from these networks bias decision-making and modulate temporal discounting? And if so, which signals are critical? These important questions have been neglected, as the literature on choice behavior and decision-making has not been integrated with the physiology of motivation.
A neuronal population that plays a key role in energy homeostasis is found within the arcuate nucleus of the hypothalamus (ARC), and expresses agouti-related peptide (AgRP) and neuropeptide Y (NPY). AgRP neurons are activated by circulating hormones representing energy deficit, and inhibited by signals representing energy surfeit (Sternson, 2013). Moreover, AgRP neurons have been shown to be both necessary (Gropp et al., 2005; Luquet et al., 2005) and sufficient (Aponte et al., 2011; Krashes et al., 2011) for feeding in adult mice (Wu et al., 2009; Atasoy et al., 2012). While previous studies have shown that AgRP neurons play a key role in signaling hunger within the brain, whether and how their outputs ultimately influence decision-making remains unresolved (Rossi and Stuber, 2018).
Here, we showed for the first time that satiety abolished temporal discounting in mice, but selective activation of AgRP neurons can produce temporal discounting. AgRP neurons send projections to many subcortical brain regions including the bed nucleus of stria terminalis (BNST; Betley et al., 2013), a key component of the extended amygdala that influences stress and anxiety (Pleil et al., 2015; Chen et al., 2016), food intake (Betley et al., 2013; Jennings et al., 2013), and choice behavior (Davis and Whalen, 2001; Cullinan et al., 2008) and is highly interconnected with brain regions associated with decision-making and cognitive processing (Berthoud, 2007; O'Connell and Hofmann, 2012). We found that activation of AgRP axon terminals in the posterior BNST is sufficient to produce temporal discounting for both caloric and non-caloric food rewards. This effect can also be produced by activating NPY type 1 receptors (Y1Rs) within the BNST, suggesting NPY signaling within the AgRP→BNST pathway as a key regulator of choice behavior.
Materials and Methods
Subjects
For behavioral experiments, Ai32 mice (Madisen et al., 2012) were crossed with AgRP-IRES-Cre mice (Tong et al., 2008), generating mice with selective expression of ChR2 in AgRP-positive neurons (AgRP::ChR2; three to seven months). Controls were ChR2-negative AgRP-Cre littermates. For electrophysiological experiments, AgRP::ChR2 mice (n = 5, six to seven months, two males, three females) were used. For drug infusion experiments, mice were all wild-type (n = 6, three months, all males). For saccharin experiments, AgRP-IRES-Cre mice were injected with AAV-DIO-ChR2 into the arcuate nuclei, yielding mice with selective expression of ChR2 in AgRP positive neurons (n = 10, five males, five females, three to five months). Controls were injected with AAV-DIO-yfp (n = 5 females). All experiments were approved by the Duke University Institutional Animal Care and Use Committee and were conducted in accordance with the National Institutes of Health guidelines regarding the care and use of animals.
Bilateral fiber optic implant surgery
Fiber optic implants were constructed as previously described (Rossi et al., 2012). Mice were anesthetized with isoflurane. For AgRP cell body stimulation group, craniotomies were made at −1.8 mm AP and ±1.6 mm ML from bregma. Fibers were then lowered 5.3 mm from the brain's surface at an angle of 14° relative to the vertical plane and fixed in place with dental acrylic and skull screws. For AgRP-BNST terminal stimulation group, craniotomies were made at −0.3 mm AP and ±1.1 mm ML from bregma. Fibers were then lowered 4.1 mm from the brain's surface at an angle of 6°. Mice were allowed to recover for at least one week before testing began.
Chronic cannulae implants surgery for drug infusions
Mice were anesthetized with isoflurane, and craniotomies were made at −0.3 mm AP and ±1.5 mm ML from bregma. Cannulae (Plastic One, part number 8IC315GS4SPC) were then lowered 4 mm from the brain's surface at an angle of 10° and fixed in place with dental acrylic and skull screws. Finally, cannula dummies (Plastic One, part number 8IC315DC4SP) were fastened onto the cannulae to prevent clogging. Mice were allowed to recover for at least two weeks before testing began.
Temporal discounting behavior
Mice were group housed (two to five per cage), and water was always available in the home cage. All testing took place in operant chambers (Med Associates) designed for in vivo optogenetics as described previously (Rossi et al., 2013). The chambers had two retractable levers located in one wall and a food cup located between the levers.
For the one-pellet (same reward size both choices) temporal discounting task, the start of a trial was signaled by the extension of both levers. Mice could press either lever. Immediately following a lever press, both levers were retracted. For sessions in which a delay was imposed, one lever was designated the “delayed” lever, and the other, the “immediate” lever (lever designation was counterbalanced across mice). When mice pressed the immediate lever, a single food pellet (14-mg rodent purified diet containing sucrose, Bio Serv) was immediately delivered into the food cup. When mice pressed the delayed lever, a pellet was delivered into the food cup at a delay (4, 8, or 16). For the 0-s delay condition, pressing either lever yielded immediate delivery of one food pellet. The following trial began 20 s after either lever was pressed. Only one delay condition was present during a session.
For the four-pellet (larger reward for delay choice) task, pressing the immediate lever earned one pellet, but pressing the “delayed” lever earned four pellets instead of one pellet. During the 0-s delay condition, 1 lever was randomly designated the “delayed” lever. The procedures are otherwise identical to the one pellet condition. For the saccharin experiments, the same procedures were used, except the sucrose pellets were replaced by 20 mg, 0.5% rodent saccharin pellets (Bio Serv).
During initial lever press training, both levers yielded immediate delivery of one pellet. Following acquisition of lever pressing, temporal discounting sessions began. Sessions lasted 60 min and were self-paced. During testing, mice were hungry, sated, or prefed. When hungry, food was restricted such that they were maintained at ∼85–90% of free-feeding body weight. When sated, food was given ad libitum for 2–4 d before behavioral testing. When prefed (only in the first behavioral experiment), mice were allowed to consume 1.5 g of the same food pellets immediately before testing. Trained mice showed elevated rates of lever pressing and subsequent pellet consumption even when sated. We speculate that this is because of the animal's prior experience with food deprivation, the hedonic value of the pellets used, prior associations between the testing chamber and food delivery.
Mice were trained once a day for 5–8 d on each delay. The delay was progressively increased (from 0 to 16 s). To control for biases in lever preference, the delay and non-delay levers were switched for each delay condition. Stimulation or drug infusion experiments were performed once their performance had reached steady state on a given delay. The order of different stimulation parameters and drug concentrations was counterbalanced.
Optogenetic stimulation
All optogenetic stimulation was delivered bilaterally. Before testing, mice were connected to a 473 nm DPSS laser that delivered ∼10 mW (∼318 mW/mm2) into each hemisphere, as previously described (Rossi et al., 2012). For the entire session, laser pulse trains were delivered in 1-s bursts of square pulses (10-ms pulse width) at 10, 20, or 40 Hz followed by 3 s with no stimulation (Aponte et al., 2011). The stimulation parameters were chosen based on widely used protocols (Aponte et al., 2011; Atasoy et al., 2012; Betley et al., 2015) in which optogenetic activation of AgRP neurons at 10–20 Hz produces levels of food intake that are comparable to those observed when mice are fasted overnight. The order of stimulation was randomized. The lasers were controlled by a Blackrock data acquisition system.
Drug infusions
Drug infusions were administered bilaterally and before behavioral testing. Mice were first anesthetized with isoflurane. Then drug was delivered by a dual infusion pump (Harvard Apparatus, catalog #13612) at 100 nl/min infusion rate. To agonize Y1Rs, we used the Y1R selective agonist [Leu31,Pro34]-neuropeptide Y (Tocris catalog #1176; Fuhlendorff et al., 1990). For infusions, the drug was dissolved in 500 nl PBS was infused into each hemisphere (0.125, 0.25 mg/kg). For vehicle infusions, 500 nl PBS was infused into each hemisphere. After infusions, mice were left in their home cage for 20 min for recovery before testing.
In vitro electrophysiology
For whole-cell patch clamp recordings, coronal sections were sliced from AgRP::ChR2 mice (n = 5). Brains were isolated quickly and immediately drowned into ice-cold solution bubbled with 95% O2-5% CO2 containing the following: 194 mm sucrose, 30 mm NaCl, 2.5 mm KCl, 1 mm MgCl2, 26 mm NaHCO3, 1.2 mm NaH2PO4, and 10 mm D-glucose with pH adjusted to 7.4 with HCl and osmolarity set to ∼320 mOsm. After 5-min coronal slices were taken at 250 μm. During the recovery period (∼1 h) slices were left in 35°C artificial CSF (aCSF) solution bubbled with 95% O2-5% CO2 containing the following: 124 mm NaCl, 2.5 mm KCl, 2 mm CaCl2, 1 mm MgCl2, 26 mm NaHCO3, 1.2 mm NaH2PO4, and 10 mm D-glucose with pH adjusted to 7.4 with HCl and osmolarity set to ∼320 mOsm. Following recovery, recorded under continuous perfusion of aCSF at 29–30°C.
For voltage-clamp experiments, the internal solution contained the following: 120 mm cesium methane sulfonate, 5 mm NaCl, 10 mm tetraethylammonium chloride, 10 mm HEPES, 4 mm lidocaine N-ethyl bromide, 1.1 mm EGTA, 4 mm magnesium ATP, and 0.3 mm sodium GTP, pH adjusted to 7.2 with CsOH and osmolarity set to 298 mOsm with sucrose. For current-clamp experiments, the internal solution contained the following: 150 mm potassium gluconate, 2 mm MgCl2, 1.1 mm EGTA, 10 mm HEPES, 3 mm sodium ATP, and 0.2 mm sodium GTP, with pH adjusted to 7.2 with KOH and osmolarity set to ∼300 mOsm with sucrose. Pipettes impedances were between 3.5 and 5 MΩ.
Slices were stimulated with 470-nm light generated from an LED (Thor Labs) focused through a 40× objective (Olympus). During recordings, 10-ms flashes of light were delivered at 10–40 Hz to the entire 40× field with an LED current driver (Thor Labs). Power density was ∼5 mW/mm2.
All data were recorded by a MultiClamp 700B amplifier (Molecular Device) and filtered at 10 kHz and digitized at 20 kHz with a Digidata 1440A digitizer (Molecular Devices). In the whole-cell configuration, recording was only accepted when the series resistance is <20 MΩ. All data were analyzed using peak detection software in pCLAMP10 (Molecular Devices).
Anatomy and histology
Mice were perfused with 4% paraformaldehyde and brains postfixed for 24 h in 30% sucrose before vibratome sectioning at 60 mm coronally. Fiber and cannulae implantation sites were verified after sections were processed for the presence of cytochrome oxidase to visualize cytoarchitecture by rinsing in 0.1 m PB before incubating in a diaminobenzidine, cytochrome C, and sucrose solution for ∼2 h at room temperature. Mounted cytochrome oxidase sections were then dehydrated in 200 proof ethanol, defatted in xylene, and coverslipped with cytoseal. To confirm eYFP expression in AgRP cells in the ARC of AgRP-cre transgenic mice, select sections were rinsed in 0.1 m PBS for 20 min before being placed in a PBS-based blocking solution containing 5% goat serum and 0.25% Triton X-100 at room temperature for 1 h. Sections were then incubated with a primary antibody (polyclonal rabbit anti-AgRP; 1:200 dilution; ThermoFisher; catalog #PA5-78739) in blocking solution overnight at 4°C. Sections were then rinsed in PBS for 20 min before being placed in a secondary antibody used to visualize AgRP neurons in the ARC (goat anti-rabbit Alexa Fluor 594; 1:500 dilution; abcam; catalog #ab150080) for 1 h at room temperature. Visualization of NPY1R neurons in the BNST followed the same immunohistochemistry protocol with a different primary antibody (polyclonal rabbit anti-NPY1R; 1:200 dilution; Sigma-Aldrich; catalog #SAB2101626). Sections for fluorescent microscopy were mounted and immediately coverslipped with Fluoromount G with DAPI medium (Electron Microscopy Sciences; catalog #17984-24). Brightfield images for placement were acquired and stitched using an Axio Imager M1 upright microscope (Zeiss) and fluorescent images were acquired and stitched using a Z10 inverted microscope (Zeiss).
Experimental design and statistical analysis
Behavioral data were analyzed with GraphPad Prism. Choice on the delay lever was normalized to baseline lever preference with zero delay. Normalized choice preference was fit with a hyperbolic model (Doya, 2008):
where the value associated with the choice (Y) varies as a function of delay, d, and the discounting parameter, K, which determines the steepness of discounting.
For all behavioral data, the appropriate one-way or two-way ANOVA tests and post hoc comparisons were performed.
Results
Satiety abolishes temporal discounting in mice
Studies often measure temporal discounting as an approximation of self-control or impulsivity using a design in which one choice leads to an immediate reward, while an alternate choice leads to a larger, delayed reward (Bickel and Marsch, 2001). This design allows researchers to test whether an animal is willing to forgo a delayed but larger reward for immediate gratification (Green and Myerson, 2004). However, using larger rewards for the delayed option can introduce a confound by manipulating both delay and reward size simultaneously. In order to assess pure temporal or delay discounting, we used a design with the same reward size for both choices (Chung and Herrnstein, 1967). The pure discounting design avoids the ambiguity introduced by manipulating delay and reward size at the same time. In the experiments described below, we used both designs (same or larger delayed reward) to probe the role of motivational state in guiding choice behavior and to elucidate its underlying neurocircuit mechanisms.
Mice were presented with two levers on each trial (Fig. 1A). Pressing one lever always led to immediate food reward (14-mg pellet), whereas pressing the other lever led to delivery of the same food reward after a predetermined delay (0, 4, 8, or 16 s). We found that motivational state is a major determinant of temporal discounting. When mice were hungry, the discounting function was steep, and immediate rewards were robustly preferred over delayed rewards. However, when hungry mice were prefed immediately before testing or when mice were sated, they became indifferent to the delay (Fig. 1B). Discounting functions were fit with a hyperbolic model in which the discounting parameter, K, indicates the steepness of the curve and thus the degree to which immediate rewards are preferred (Doya, 2008). Temporal discounting was greater when mice were hungry compared with when they were sated or prefed (Fig. 1C; lever presses and body weights shown in Fig. 1D,E). A one-way ANOVA showed a main effect of satiety (F(2,31) = 9.36, p = 0.0007, Tukey's multiple comparisons test: sated vs hungry, p = 0.0012; sated vs prefed, p = 0.99; hungry vs prefed, p = 0.0066). A two-way ANOVA showed no interaction between motivational state and delay (F(6,121) = 1.63, p = 0.14) no significant main effect of delay (F(3,121) = 0.069, p = 0.98), and a significant main effect of motivational state (F(2,121) = 104.1, p < 0.0001). One-way ANOVA conducted on the average weights showed a significant main effect of motivational state (F(2,36) = 96.99, p < 0.0001). Post hoc comparison revealed that both hungry and prefed conditions had significantly lower weight compared with sated (p < 0.0001), but there was no significant difference between hungry and prefed conditions (p = 0.26).
Temporal discounting depends on motivational state. A, Schematic of temporal discounting task. During each session, mice experienced only 1 delay. Mice were trained for five to eight daily sessions on each delay. Temporal discounting was only measured after their performance had reached steady state. To control for biases in lever preference, the delay and non-delay levers were switched for each delay condition. B, Choice behavior varies as a function of motivational state. Satiety abolished discounting. When mice are hungry (n = 13), immediate rewards are preferred, but when sated (n = 8) or following prefeeding (n = 13), delayed rewards are discounted less. Dashed lines are hyperbolic fits using the equation: Y = 1/(1+K*d). Choice of the delay lever (Y) varies as a function of delay, d, and the discounting parameter, K, which determines the steepness of discounting. To obtain the normalized choice measure, we divided number of presses on the delay lever by the baseline number when there is no delay. Note that in the baseline condition, when both levers yield immediate reward, animals do not have a clear preference (no discounting). C, Discounting parameters corresponding to hyperbolic fits in B. The higher the value of K, the greater the preference for the immediate reward (steeper discounting function). Values are mean and SEM. D, Total number of presses/rewards per session. E, Body weights for all mice tested. **p < 0.01, ***p < 0.001.
AgRP activation produces temporal discounting in sated mice
AgRP neurons are necessary for voluntary feeding (Gropp et al., 2005; Luquet et al., 2005), respond to changing metabolic needs (Sternson, 2013), and can influence many motivated behaviors (Dietrich et al., 2015; Burnett et al., 2016; Padilla et al., 2016), but their role in decision-making is unknown. Thus, we aimed to test whether AgRP activation biases temporal discounting. To selectively activate AgRP neurons in vivo, Ai32 mice, which express a floxed-STOP codon and ChR2-eYFP under the ROSA26 locus (Madisen et al., 2012), were crossed with AgRP-Cre mice (Tong et al., 2008), yielding ChR2 expression restricted to AgRP neurons (AgRP::ChR2; Fig. 2A–D). To validate the efficacy of ChR2 within AgRP neurons, whole-cell patch clamp electrophysiology was performed in acute brain slices from adult AgRP::ChR2 mice. Blue light pulses result in inward currents in voltage clamp recordings and reliable spiking in current clamp recordings (Fig. 2E).
Genetic targeting of AgRP neurons with ChR2. A, Scheme for generating AgRP::ChR2 mice. B, Schematic representation of restricted ChR2 expression within the ARC. C, Representative expression pattern of AgRP-ChR2-eYFP within the ARC. Transgenic expression of ChR2 results in membrane-bound eYFP and nuclear exclusion. 3V = third ventricle. ARC = arcuate nucleus. D, Optic fiber placements above the ARC. E, Whole-cell patch clamp recordings from ChR2-expressing ARC neurons; 10-ms blue light pulses result in inward currents in voltage clamp recordings and spiking in current clamp recordings. Top row, 20-Hz stimulation; bottom row, 40-Hz stimulation.
We then tested the hypothesis that optogenetic activation of AgRP neurons is sufficient to produce temporal discounting in sated mice. We targeted the ARC of AgRP::ChR2 mice with optic fibers (AgRP::ChR2ARC; Fig. 3) to allow delivery of 1-s pulsed trains of 473-nm laser light (1 s pulsed at 10, 20, or 40 Hz followed by 3 s of no stimulation repeated for the duration of the experiment). Mice with chronically implanted optic fibers were trained on the temporal discounting task. Replicating our finding that motivational state biases decision-making, hungry mice strongly preferred immediate food rewards to delayed rewards, but such temporal discounting was abolished by satiety. In sated mice, however, AgRP::ChR2ARC stimulation produced discounting similar to that observed in the hungry state. A two-way ANOVA (stimulation frequency and opsin) on the discounting parameter (K) revealed no interaction (F(2,69) = 0.17, p = 0.84), no main effect of stimulation frequency, F(2,69) = 1.17, p = 0.32), and a main effect of opsin (F(1,69) = 6.99, p = 0.01). Likewise, for the number of pellets consumed there was no significant interaction (F(2,37) = 0.45, p = 0.69), no main effect of stimulation frequency (F(1.854,34.29) = 2.36, p = 0.11), and a main effect of opsin (F(1,29) = 33.28, p < 0.0001); ****p < 0.0001 (main effect of opsin).
AgRP stimulation produces delay discounting in sated mice (pure delay discounting). A, Bilateral optical fibers were implanted in AgRP::ChR2-eYFP mice targeting ARC; 1-s pulse trains were delivered every 4 s at 10, 20, or 40 Hz. B, Illustration of the pure delay discounting task, in which the delayed reward is the same size as the immediate reward (one pellet). C, Temporal discounting functions for control (AgRP-Cre) mice in the pure delay discounting condition. To obtain the normalized choice measure, we divided number of presses on the delay lever by the baseline number when there is no delay. Dashed lines represent hyperbolic fits; *p < 0.05 (main effect of opsin); n = 8 for all conditions. D, Temporal discounting functions for AgRP::ChR2 mice in the pure delay discounting condition. AgRP::ChR2 mice show state-dependent discounting of delayed rewards. Satiety largely abolishes temporal discounting. However, AgRP::ChR2 stimulation produces discounting in sated mice. Error bars indicate ± SEM. Hungry (n = 23), sated (n = 23), sated 10 Hz (n = 11), sated 20 Hz (n = 23), sated 40 Hz (n = 22). E, Summary of the discounting parameter (K), a measure of temporal discounting. F, Summary of the average number of pellets earned per session. Data from different delays are combined. ****p < 0.0001.
Laser stimulation failed to affect the choice behavior of AgRP-Cre littermates lacking functional ChR2 (ChR2-Cre control mice). AgRP::ChR2ARC stimulation increased the total number of lever presses (choices) and pellets earned per session, suggesting higher overall motivational drive acting independently of the choices made. This finding is consistent with previous studies demonstrating that AgRP activation increases willingness to work for food (Krashes et al., 2011; Atasoy et al., 2012). Taken together, these results indicate that, in addition to influencing food intake and competing innate behaviors, AgRP activation potently biases choice behavior toward impulsive decisions.
In addition, we also tested the same mice using the delayed/larger reward design often used as a test of impulsivity and self-control (Cardinal et al., 2001; Fig. 4). Instead of delivering one pellet following the choice of the delayed lever, four pellets were delivered. A similar hyperbolic discounting function was observed, but the larger reward slightly reduced the steepness of discounting when mice were hungry. Thus, with a larger delayed reward, AgRP::ChR2ARC stimulation also produces discounting. A two-way ANOVA (stimulation frequency and opsin) revealed no interaction (F(2,33) = 0.11, p = 0.98), no main effect of stimulation frequency (F(2,33) = 0.05, p = 0.96), and a significant main effect of opsin (F(1,33) = 7.43, p = 0.01). Similarly, for the number of pellets earned there was no interaction (F(2,33) = 0.11, p = 0.98), no main effect of stimulation frequency (F(2,33) = 0.05, p = 0.96), and a significant main effect of opsin (F(1,33) = 7.43, p = 0.01).
AgRP stimulation produces delay discounting in sated mice (larger delayed reward). A, Bilateral optical fibers were implanted in AgRP::ChR2-eYFP mice targeting ARC; 1-s pulse trains were delivered every 4 s at 10, 20, or 40 Hz. B, Illustration of the delay discounting task with larger reward (four pellets) associated with the delayed lever. C, Temporal discounting functions for control (AgRP-Cre) mice. Dashed lines represent hyperbolic fits. D, Temporal discounting functions for AgRP::ChR2 mice. Stimulation produces discounting in sated mice. Discounting can be increased to the level observed in the hungry condition by AgRP stimulation. Error bars indicate ±SEM. E, Summary of the discounting parameter (K), a measure of temporal discounting (higher K values indicate more discounting of delayed reward); *p < 0.05 (main effect of opsin). F, Summary of the average number of pellets earned per session. Data from different delays are combined; *p < 0.05 (main effect of opsin).
AgRP projections to posterior BNST mediate temporal discounting
The BNST is a target of AgRP projections (Betley et al., 2013) that regulates stress and anxiety (Pleil et al., 2015; Chen et al., 2016), feeding (Betley et al., 2013; Jennings et al., 2013), and high-level choice behavior (Davis and Whalen, 2001; Cullinan et al., 2008). In addition, the BNST is connected with brain regions associated with decision-making and cognitive processing, including the ventral tegmental area, hippocampus, and prefrontal cortex (Berthoud, 2007; O'Connell and Hofmann, 2012; Stamatakis et al., 2014). We stimulated the axon terminals of AgRP neurons in the BNST. As shown in Figure 5 for pure discounting, in AgRP-Cre control mice, discounting and lever pressing are unaffected by stimulation. AgRP::ChR2→BNST stimulation increased discounting and increased lever pressing in sated mice. Photo-stimulation increased discounting (as well as the number of lever presses and pellets earned). Stimulation considerably increased discounting. A two-way ANOVA revealed no interaction between stimulation frequency and opsin (F(2,33) = 1.02, p = 0.37), a main effect of stimulation frequency (F(2,33) = 0.97, p = 0.39), and a significant main effect of opsin (F(1,33) = 10.06, p = 0.003). For the number of pellets earned there was a significant interaction between stimulation frequency and opsin (F(2,22) = 8.31, p = 0.002). Post hoc comparisons showed that the ChR2 group is significantly higher than the control group for 10 Hz (p = 0.01), 20 Hz (p <0.0001), and 40 Hz (p < 0.0001).
Stimulation of AgRP::ChR2 terminals in BNST produces temporal discounting in sated mice (pure delay discounting). A, left, Schematic illustration of AgRP terminal stimulation in BNST. Bilateral optical fibers were implanted in the posterior BNST of AgRP::ChR2-eYFP mice. Right, Fiber placement summary. B, Illustration of the pure delay discounting task, in which the delayed reward is the same size as the immediate reward (one pellet). C, Temporal discounting functions for control (AgRP-Cre) mice in the pure delay discounting condition. There was no effect of stimulation on discounting. D, Temporal discounting functions for AgRP::ChR2→BNST mice. Stimulation produces discounting in sated mice. Discounting can be increased to the level observed in the Hungry condition by stimulation of AgRP terminals in the BNST. E, Summary of the discounting parameter (K), a measure of temporal discounting (higher K values indicate more discounting of delayed reward); **p < 0.01 (main effect of opsin). F, Summary of the average number of pellets earned per session. Data from different delays are combined; *p < 0.05, ****p < 0.0001.
Figure 6 shows results for larger delayed reward condition. Again, stimulation increased discounting. A two-way ANOVA revealed no interaction between stimulation frequency and opsin (F(2,30) = 1.80, p = 0.18), no main effect of stimulation frequency (F(2,30) = 2.29, p = 0.12), and a significant main effect of opsin (F(1,30) = 12.12, p = 0.0015). Stimulation also increased pellets earned. There was a significant interaction between stimulation frequency and opsin (F(2,22) = 3.74, p = 0.04). Post hoc comparisons showed that stimulation at 40 Hz resulted in more pellets compared with controls (p = 0.04).
Stimulation of AgRP::ChR2 terminals in BNST produces temporal discounting in sated mice (larger delayed reward). A, left, Schematic illustration of AgRP terminal stimulation in BNST. Bilateral optical fibers were implanted in the posterior BNST of AgRP::ChR2-eYFP mice. Right, Fiber placement summary. B, Illustration of the discounting task with larger reward (four pellets) associated with the delayed lever. C, Temporal discounting functions for control (AgRP-Cre) mice. There was no effect of stimulation on discounting. D, Temporal discounting functions for AgRP::ChR2→BNST mice. Stimulation produces discounting in sated mice. E, Summary of the discounting parameter (K), a measure of temporal discounting (higher K values indicate more discounting of delayed reward); **p < 0.01 (main effect of opsin). F, Summary of the average number of pellets earned per session. Data from different delays are combined; *p < 0.05.
NPY signaling in the posterior BNST is critical for temporal discounting
We next sought to probe the underlying pharmacological mechanism driving the observed changes in temporal discounting. AgRP neurons co-express the orexigenic neuropeptide, NPY (Hahn et al., 1998), and NPY receptor activation contributes to some AgRP-mediated behaviors (Atasoy et al., 2012; Dietrich et al., 2015). AgRP axons in the BNST are closely juxtaposed with neurons that express type 1 NPY receptors (Y1Rs), providing a potential mechanism for motivational signals to impact downstream structures mediating choice behavior (Fig. 7). To test whether BNSTY1R activation is sufficient to control temporal discounting, we employed a pharmacological approach. Mimicking the effects of Y1R activation, we injected the high-affinity Y1R agonist Leu-Pro NPY into the BNST region that receives AgRP projections (Fig. 7A,B). BNSTY1R activation is sufficient to produce temporal discounting in sated mice, in a dose-dependent manner (Fig. 7C,F). A one-way ANOVA showed a main effect of dose (F(2,15) = 32.9, p < 0.0001). Post hoc comparisons showed that, compared with vehicle control, 0.125 mg/kg condition (p = 0.0002) and 0.25 mg/kg (p < 0.0001) both earned more pellets. A one-way ANOVA showed a main effect of dose (F(2,15) = 3.82, p = 0.046). Post hoc comparisons showed that, compared with vehicle control, 0.125 mg/kg condition did not increase lever pressing (p = 0.40), but 0.25 mg/kg significantly increased lever pressing (p = 0.027). As shown in Figure 7G, BNSTY1R activation increased the discounting parameter. There was a significant main effect of drug (F(3,40) = 4.20, p = 0.013), a significant main effect of reward size (F(1,40) = 9.21, p = 0.004), but no significant interaction between dose and reward size (F(3,40) = 0.97, p = 0.41).
BNSTY1R activation is sufficient for temporal discounting. A, left, Axons of AgRP::ChR2-eYFP neurons are closely juxtaposed with postsynaptic BNST neurons expressing neuropeptide Y1 receptors (NPY1Rs). Ac = anterior commissure, BNST = bed nucleus of the stria terminalis, F = fornix, St = stria terminalis. Middle, Schematic illustration of bilateral cannulae chronically implanted into the BNST. Right, Coronal section showing representative cannula placement in the BNST. B, Summary of cannula placement in the BNST. C, Temporal discounting functions for Y1R agonist experiments using the same reward size for the delayed lever (one pellet). Dashed lines represent hyperbolic fits. A within-subject design was used: each mouse was tested once after receiving vehicle and each drug dose (n = 6 for each condition) for both one pellet and four pellets conditions. D, Total number of lever presses (choices) per 1-h session using the same reward size for the delayed lever (one pellet); ***p < 0.001, ****p < 0.0001. E, Temporal discounting functions using a larger reward size for the delayed lever (four pellets). Dashed lines represent hyperbolic fits. A within-subject design was used: the same mice were tested once after receiving vehicle and each drug dose (n = 6 for each condition). F, Total number of lever presses (choices) per 1-h session for the larger delayed reward condition; *p < 0.05. G, Discounting parameter summary for Y1R agonist (Leu-Pro NPY) experiments. Activation of Y1R generated delay discounting in sated mice. The delayed larger reward condition is associated with less discounting overall; *p < 0.05 (main effect of drug). ns, p = 0.40.
Discounting with a non-caloric saccharin reward
It is not clear whether the effect of AgRP stimulation is because of its ability to induce hunger, which in turn affects discounting, or whether it had a more general effect on temporal discounting per se. To test the latter hypothesis, we used saccharin, an artificial sweetener with no calories, as a reward. We chose to stimulate the axon terminals of the AgRP neurons in the BNST using the four-pellet delayed reward condition. This condition usually produces less discounting, so it is presumably more difficult to artificially generate discounting and provides a more challenging test for the hypothesis that activating the AgRP→BNST pathway can increase temporal discounting. We found that mice could learn to press a lever for saccharin rewards. However, compared with food rewards overall lever pressing is reduced. Most strikingly, although mice still earned many pellets, they often did not consume the saccharin pellets, leaving many pellets in the food magazine. We measured the number of pellets actually consumed and found that most earned saccharin pellets were not consumed (% consumed: hungry: 20.45 ± 1.6%, sated: 10.7 ± 1.2%, sated 10 Hz: 20.8 ± 1.9%, sated 20 Hz: 20.4 ± 1.8%, sated 40 Hz: 24.4 ± 1.65%). Nevertheless, mice still pressed the lever often and still exhibited temporal discounting in their choice behavior (Fig. 8). Most importantly, optogenetic stimulation produced similar effects as with sucrose pellet rewards. AgRP::ChR2→BNST mice increased discounting and increased lever pressing in sated mice with saccharin pellets. A two-way ANOVA on the discounting parameter revealed a significant interaction between stimulation frequency and opsin (F(2,39) = 3.53, p = 0.04). Post hoc comparisons showed that the ChR2 group is significantly higher than the control group at 20 Hz (p < 0.0003) and at 40 Hz (p < 0.0001), but not at 10 Hz (p = 0.12; Fig. 8E).
Stimulation of AgRP::ChR2 terminals in BNST produces temporal discounting in sated mice when the reward is non-caloric saccharin. A, left, Schematic of AgRP::ChR2→BNST stimulation strategy. Middle, Coronal section showing ChR2 expression and optic fiber placement. Summary of optic fiber locations within the BNST. B, Illustration of the discounting task with larger reward (four saccharine pellets) associated with the delayed lever. C, Temporal discounting functions for saccharin pellets. Left, Temporal discounting functions for control (AgRP-Cre) mice. There was no effect of stimulation on discounting. D, Temporal discounting functions for AgRP::ChR2→BNST mice. Stimulation produces discounting in sated mice. E, Summary of the discounting parameter (K); ***p < 0.001, ****p < 0.0001. F, Summary of the average number of pellets consume per session. Data from different delays are combined. Only a small proportion of earned saccharine pellets are consumed; ****p < 0.0001. ns, p = 0.12.
In addition, AgRP::ChR2→BNST stimulation also increased the number of pellets consumed (Fig. 8F). A two-way ANOVA of the stimulation data revealed a significant interaction between stimulation frequency and opsin (F(2,39) = 3.28, p = 0.05). Post hoc comparisons showed that the ChR2 group is significantly higher than the control group (p < 0.0001 for at all stimulation frequencies).
Discussion
To survive, organisms must maintain essential physiological variables within an acceptable range. A signal representing energy deficit can be used to drive food seeking such that feeding is initiated when there is a deficit and halted when the energy supply has been replenished. AgRP neurons are often thought to provide a homeostatic error signal that is reduced by satiety and increased by deprivation. When reward is defined as error reduction in such a control system, then there must be temporal discounting (Keramati and Gutkin, 2014). For example, in a system that controls for the rate of food intake, the error signal will be proportional to the degree of discounting. Larger errors will simply request a higher rate of intake, leading to a preference for immediate rewards.
We found that the AgRP→BNST projection strongly modulates choice behavior. Temporal discounting is largely abolished in sated mice, but is produced by activation of AgRP neurons in the ARC or their axon terminals in the posterior BNST, a major downstream target. As AgRP neurons are known to express NPY, we tested whether activation of postsynaptic NPY receptors (Y1Rs) on BNST neurons is sufficient to increase temporal discounting. Similar to optogenetic stimulation of AgRP cell bodies or axon terminals within the BNST, BNSTY1R activation dose dependently increased temporal discounting.
Motivational modulation of temporal discounting
Our behavioral results demonstrate that motivational state potently modulates temporal discounting. Hungry mice strongly preferred immediate food rewards, yet sated mice were largely indifferent to reward delay. This is true whether the delayed reward is the same size (pure delay discounting) or much larger than the immediate reward (self-control). Delay discounting is by definition discounting of the delayed reward (Chung and Herrnstein, 1967). The delay/reward size trade-off design was introduced primarily to study the question of “self-control,” which is concerned with whether one should choose a small reward now versus big reward later (Green and Myerson, 2004). We observed the same pattern of behavior when a larger reward was associated with the delayed choice. Increasing reward size for the delayed lever, as expected, merely reduced the steepness of discounting. Thus, in the pure temporal discounting design we manipulated only the delay to reward, but not the reward size. Not surprisingly, when the delayed reward is larger, there was less discounting, but AgRP cell body stimulation or AgRP→BNST terminal stimulation both increased discounting whether or not the delayed reward is larger.
Why should mice choose the delayed option at all when the delayed reward is the same size as the immediate reward? Because environmental contingencies are not always fixed in nature, some sampling of the currently less desirable option is an explorative strategy that allows the animal to update their representation of action-outcome contingencies. A major finding from this study is that such behavior is strongly modulated by motivational state. The preference for immediate gratification is directly proportional to hunger. Sated animals are indifferent to reward delay and show little or no discounting. This finding is in agreement with previous work showing that satiety also results in indifference to risk (variance in desired rewards): when mice are faced with the choice of certain reward and uncertain reward, they prefer certain rewards when hungry, yet become indifferent when sated (Leblond et al., 2011, 2014).
Previous studies have examined the effect of motivational state on choice behavior, but failed to reach any consensus (Logue and Peña-Correal, 1985; Bradshaw and Szabadi, 1992; Wogar et al., 1992; Kirk and Logue, 1997; Oliveira et al., 2013; Skrynka and Vincent, 2019) This could be because of differences in experimental design and species tested. Previous studies used the self-control design in which larger or more palatable rewards are associated with the delayed option (Rachlin and Green, 1972), which may introduce confounds. Moreover, previous work that failed to observe effects of motivational state on choice behavior used the same lever for the delayed choice (Logue and Peña-Correal, 1985). Consequently, animals can develop exclusive choice of the lever leading to immediate reward. They never learn about the actual delays associated with each lever, because they did not sample the other lever. In our experiments, we controlled for the lever or location bias. We switched the lever-delay assignment after changing to a different delay and trained the animals until their choice performance reached steady state. Thus, our design ensures that delay is the only factor determining choice behavior, not lever location or any other previously learned bias. Nevertheless, despite critical differences in experimental design and procedures, we cannot rule out the possibility that different conclusions reached by previous work in humans and pigeons could be because of differences in body size and energy reserves. It would be important for future studies on temporal discounting to adopt the experimental procedures used here to avoid key confounds like location bias.
The role of AgRP signaling in choice behavior
By optogenetically activating AgRP neurons in the ARC, we were able to produce temporal discounting in sated mice (Fig. 3). This is to our knowledge the first report of a causal role of a genetically defined neuronal population in temporal discounting. AgRP cell body stimulation produced temporal discounting in sated mice regardless of the reward size associated with the delayed lever.
Our results suggest a novel mechanism that allows AgRP signaling to influence choice behavior via BNSTY1R activation. Although we showed that AgRP projections to the BNST are responsible for the observed effects on discounting, this does not rule out the possibility that AgRP projections to other brain regions may also contribute to delay discounting. However, AgRP projections release NPY and activate Y1Rs in BNST neurons. Indeed, BNSTY1R activation, which is known to enhance inhibitory synaptic transmission (Pleil et al., 2015), is sufficient to produce delay discounting (Fig. 7). AgRP neurons are known to co-release GABA (Tong et al., 2008), and GABA release from AgRP neurons is critical for feeding (Tong et al., 2008; Wu et al., 2009). Activation of NPY can also increase GABAergic signaling (Pleil et al., 2015), supporting the hypothesis that AgRP projections release NPY to reduce the activity of a subset of postsynaptic BNST-Y1R-expressing neurons.
Because activation of AgRP neurons is often thought to generate hunger, it is unclear whether the effect on delay discounting is limited to food rewards, e.g., whether the same effect would be observed when a non-caloric reward is used. We therefore conducted an experiment using the non-nutritive sweetener saccharin as the reward (Fig. 8). We were able to train mice to press a lever for saccharin, although mice did not consume most of the earned saccharin pellets. Most strikingly, mice still exhibited delay discounting for non-caloric rewards. Mice still showed robust lever pressing and discounting although they rarely consumed all the pellets. Activation of the AgRP→BNST pathway was still capable of producing delay discounting for non-nutritive rewards. However, our results only show that this pathway regulates delay discounting for food-related rewards. Whether they can be extended to other types of rewards remains unclear.
The BNST has been extensively implicated in stress, anxiety, and other aversive motivational states (Sahuque et al., 2006; Marcinkiewcz et al., 2016). Interestingly, AgRP activation suppress anxiety-like behavior (Burnett et al., 2016) and territorial aggression via projections to the BNST (Padilla et al., 2016). Our results thus link a key node in the homeostatic feeding system with circuitry known to control aversive motivational states. This link is also supported by previous work showing that activation of the stress-related BNST circuit results in weight loss (Kocho-Schellenberg et al., 2014). We speculate that, by suppressing aversive states, NPY signaling in the AgRP→BNST pathway can bias decision strategies and promote immediate gratification. Future work will be needed to further elucidate such appetitive-aversive interactions.
Footnotes
This work was supported by National Institutes of Health Grants DA040701, MH112883, and NS094754 (to H.H.Y.) and by a National Science Foundation fellowship (M.A.R.). We thank Tatyana Sukharnikova, Erin Gaidis, and Guozhong Yu for their help with histology.
The authors declare no competing financial interests.
- Correspondence should be addressed to Henry H. Yin at hy43{at}duke.edu
