Abstract
Animal studies suggest the neurotransmitter dopamine (DA) plays an important role in decision-making. In rats, DA depletion decreases tolerance for effort and probability costs, while drugs enhancing DA increase tolerance for these costs. However, data regarding the effect of DA manipulations on effort and probability costs in humans remain scarce. The current study examined acute effects of d-amphetamine, an indirect DA agonist, on willingness of healthy human volunteers to exert effort for monetary rewards at varying levels of reward value and reward probability. Based on preclinical research, we predicted amphetamine would increase exertion of effort, particularly when reward probability was low. Over three sessions, 17 healthy normal adults received placebo, d-amphetamine 10 mg, and 20 mg under counterbalanced double-blind conditions and completed the Effort Expenditure for Rewards Task. Consistent with predictions, amphetamine enhanced willingness to exert effort, particularly when reward probability was lower. Amphetamine did not alter effects of reward magnitude on willingness to exert effort. Amphetamine sped task performance, but its psychomotor effects were not strongly related to its effects on decision-making. This is the first demonstration in humans that dopaminergic manipulations alter willingness to exert effort for rewards. These findings help elucidate neurochemical substrates of choice, with implications for neuropsychiatric diseases characterized by dopaminergic dysfunction and motivational deficits.
Introduction
Throughout the animal kingdom, individuals must decide how to allocate energy across opportunities for reward. Such opportunities vary not only in reward amount, but also in effort required (effort costs) and probability of reward receipt (probability costs). Consequently, accurate valuation of benefit, effort and probability is essential for appropriate decision-making. While recent studies have made progress in elucidating brain regions underpinning these decisional processes in humans (Hare et al., 2008; Kable and Glimcher, 2009; Venkatraman et al., 2009; Wunderlich et al., 2009; Samanez-Larkin et al., 2010) concomitant neurochemical mechanisms remain poorly understood.
Animal models implicate dopamine (DA), as a crucial neurochemical for valuation of effort. Over a series of studies, Salamone and colleagues (Salamone et al., 1991; Correa et al., 2002) have shown that rats choosing between less desirable but freely available foods (Low Cost/Low Reward; LC/LR) and more palatable foods accessed via lever pressing or barrier climbing (High Cost/High Reward; HC/HR), strongly prefer HC/HR options. Experimental manipulations that decrease DA in the nucleus accumbens (Salamone et al., 1991; Correa et al., 2002), shift animals' preferences toward LC/LR options (for review, see Salamone et al., 2007, 2009). In contrast, manipulations that enhance DA have opposite effects. Amphetamine, which raises extracellular DA, increases HC/HR choices in barrier-climbing (Bardgett et al., 2009) and (at moderate doses) in lever-pressing tasks (Floresco et al., 2008). Although amphetamine increases locomotor activity (Wise, 1988), effects of amphetamine on effort-based decision-making do not appear to be simply due to locomotor facilitation (Bardgett et al., 2009).
In addition to effort, DA appears involved in valuing probability costs. Paralleling findings with effort, DA antagonism reduces tolerance for probability costs, shifting preference from larger, uncertain rewards to guaranteed smaller rewards, while amphetamine increases preference for larger, riskier rewards (St. Onge and Floresco, 2009; St. Onge et al., 2010). These findings are consistent with theoretical models suggesting mesolimbic DA encodes various response costs in a similar manner (Phillips et al., 2007).
Despite extensive preclinical exploration, the role of DA in human valuation of effort and probability is unknown. Here, we tested acute effects of two moderate doses of d-amphetamine on the Effort Expenditure for Rewards Task (EEfRT; Treadway et al., 2009), in healthy volunteers. The EEfRT is modeled on the concurrent choice paradigm of Salamone et al. (1994). Participants are presented with a series of HC/HR versus LC/LR choices. Trials vary in both amount of reward for the HC/HR option, and reward probability. We predicted amphetamine would increase HC/HR choices, indicating increased willingness to expend effort. Additionally, we predicted amphetamine would induce greater willingness to expend effort particularly on trials with low reward probability.
Materials and Methods
Study design.
The study used a counterbalanced, double-blind, within-subject protocol with three separate testing sessions during which participants received capsules containing placebo, 10 or 20 mg of d-amphetamine (Mallinckrodt). At each session they completed the EEfRT. Sessions were separated by at least 3 d and a maximum of 113 d [mean (M) = 13.71 d, SD = 13.40].
Participants.
Healthy participants (n = 17; 6 male) ages 18–35 were recruited through flyers and online advertisements as part of a larger study (n = 36) on amphetamine and emotional responses. Participants completed a screening consisting of physical examination, electrocardiogram, modified Structured Clinical Interview for DSM-IV (SCID; First et al., 1996) and self-reported health and drug use history. Inclusion criteria were as follows: Body Mass Index between 18 and 35, no medical contraindications, not pregnant, nursing, or trying to become pregnant, no past year DSM-IV Axis I Disorders or lifetime drug dependence, mania or psychosis, some previous recreational drug use, no previous adverse amphetamine reactions, smoking <10 cigarettes per week, and high-school level education. Participants were primarily Caucasian (n = 14, 82%), young adults (M = 22.7, SD = 3.45), with some college education (M = 15.12 years, SD = 1.50), and light to moderate recreational drug use (Table 1). Three participants were missing EEfRT data from one session due to computer malfunction.
Demographic characteristics and drug use histories of participants (n = 17)
Participants were instructed to refrain from recreational and over-the-counter drugs 24 h before and 12 h after sessions. Compliance was verified using breath alcohol (Alcosensor III, Intoximeters Inc.) and urine tests for commonly used drugs (ToxCup, Branan Medical Corporation). Subjects were instructed to maintain normal caffeine intake 24 h before and 12 h after sessions, and to fast for 9 h before sessions. Female participants were urine tested for pregnancy before each session (AimStrip, Germaine Laboratories). Women not on hormonal birth control were scheduled during the follicular phase (White et al., 2002). Participants were informed that they might receive a stimulant, tranquilizer, marijuana-like drug, or placebo. All participants provided informed consent, and the University of Chicago Institutional Review Board approved all procedures.
Procedure.
Participants attended an orientation during which they were familiarized with procedures and practiced the EEfRT. They then completed three 4 h individual study sessions. Participants arrived at 9:00 A.M., completed breath and urine tests, then consumed a standard snack. At 9:30 they took two opaque size 00 gelatin capsules containing 10 or 20 mg of d-amphetamine with dextrose filler, or placebo (dextrose only). From 9:30 A.M. to 11:00 A.M. participants relaxed in the lab. At 11:10 A.M. participants completed emotional responsivity tasks presented previously (Wardle and de Wit, 2011). After these tasks, at 12:30 P.M. participants completed the EEfRT. At 1:30 P.M. participants completed an end of session questionnaire and left the laboratory.
EEfRT.
The EEfRT is a multitrial game in which participants are asked to choose on each trial between an HC/HR and LC/LR option to obtain varying monetary rewards (Fig. 1). A detailed description has been published previously (Treadway et al., 2009). Briefly, each trial presents the subject with a choice between a “hard task” (HC/HR option), requiring 100 button presses with the nondominant pinky finger within 21 s, and an “easy task” (LC/LR option), requiring 30 button presses with the dominant index finger within 7 s. For easy-task choices, subjects were eligible to win $1.00 for each successfully completed trial. For hard-task choices, subjects were eligible to win higher amounts that varied per trial within a range of $1.24–$4.30 (“reward magnitude”). Subjects were not guaranteed to win the reward if they completed the task; some trials were “win” trials, in which the subject received the reward amount, while others were “no win” trials, in which the subject received no money. To help subjects determine which trials were more likely to be win trials, subjects were provided with accurate probability cues during the choice period. Trials had three levels of probability: “high” 88% probability of a win trial, “medium” 50% and “low” 12%. Probability levels applied to both the hard and easy task, and there were equal proportions of each probability level across the experiment. Probability and reward information, task progress, and feedback displays (as depicted in Fig. 1) were presented on a computer screen. Button presses were completed on a standard keyboard.
Schematic diagram of a single EEfRT trial. A, A 1 s fixation cue. B, A 5 s choice period in which subjects are presented with reward magnitude of the hard task for that trial, and the probability of receiving reward for that trial. C, The 1 s “ready” screen. D, Subjects make rapid button presses to complete the chosen task for 7 s (easy task) or 21 s (hard task). E, Feedback completion of the task. F, Feedback on whether they received any money for that trial.
Statistical analysis.
Effects of d-amphetamine on EEfRT choice behavior were analyzed using Generalized Estimating Equation (GEE) models, a generalized regression technique able to model dichotomous outcome variables (e.g., HC/HR vs LC/LR choices) with correlated residuals (e.g., nested within a single subject; Liang et al., 1986; Zeger and Liang, 1986) using a link function. GEE can simultaneously model within-session parameters (e.g., trialwise changes in reward magnitude of the HC/HR option) and between-session parameters (e.g., drug condition). Because these models focus on the per-trial level as opposed to the per-subject level, they also increase statistical power for smaller sample sizes. GEE models were implemented in SPSS 17 using an unstructured working correlation matrix. The dependent measure was dichotomous HC/HR or LC/LR choice, modeled using a binary logistic distribution. Replicating our prior analytical approach (Treadway et al., 2009), all GEE models included reward magnitude of the HC/HR option (RM), probability (P) and Expected Value (EV), which was calculated as RM × P. Separate models were computed to test effects of drug condition on HC/HR choices, as well as interactions between drug condition and reinforcement variables (RM, P, and EV). All models included trial number as a covariate to control for possible fatigue over the task.
Results
Results of GEE models
We tested five GEE models with each model including all experimental task variables (RM, P, EV) and trial number. Results are reported in Table 2.
GEE models of d-amphetamine effects on EEfRT choice behavior (n = 17)
Model 1 tested main effects of EEfRT-task variables (RM, P, EV and trial number) across the drug conditions, and found that all EEfRT-task variables were significant predictors of task performance (all p values <0.01).
In model 2, we tested the main effect of d-amphetamine on choice behavior, and found it was a positive predictor of selection of HC/HR options (b = 0.138, p < 0.004). Follow-up analyses revealed the effect of d-amphetamine was significant when comparing placebo to 20 mg (b = 0.1.49, p < 0.001) and 10 to 20 mg (b = 0.206, p = 0.012), but only trend-level when comparing placebo to 10 mg (b = 0.121 p = 0.085; Fig. 2).
Bar graph of changes in mean proportions of HC/HR choices across the placebo, 10 mg of d-amphetamine, and 20 mg of d-amphetamine conditions with SEM error bars (n = 17, in within-subjects design). See Results and Table 2 for inferential statistical data. Post hoc tests conducted depicted as: *p < 0.05, significant pairwise effect of amphetamine doses; **p < 0.05, significant linear effect of amphetamine; ns, not significant. See Results for further information on post hoc tests.
In model 3 we tested the interaction between drug condition and trialwise RM, which was not significant (b = −0.056, p = 0.158), suggesting RM was an equally strong predictor of HC/HR choices across d-amphetamine conditions.
Model 4 assessed the interaction between drug condition and trialwise probability, which was significant (b = 0.093, p = 0.011). Follow-up analyses showed the effect of d-amphetamine was not significant for high probability trials, but was significant for medium (b = 0.178, p = 0.001) and low probability trials (b = 0.262, p < 0.001), indicating d-amphetamine increased the likelihood of choosing HC/HR options only for trials with lower probabilities.
In model 5, we tested the interaction between drug condition and EV, which was significant (b = −0.134, p < 0.001). Using a median split to divide trials into high and low EV, we found a significant effect of drug condition on low EV trials (b = 0.179 p = 0.001) such that participants were more likely to choose the HC/HR option during these trials after receiving amphetamine, but no effect of drug condition on high EV trials (b = 0.007, p = 0.844).
Effects of amphetamine-induced psychomotor speeding on EEfRT performance
We ran control analyses on button press speeds and task completion rates to rule out the possibility that amphetamine-induced improvements in psychomotor performance accounted for observed effects on choice behavior. Amphetamine significantly increased button-press rates, but button press speed for each trial was not a significant predictor of choice behavior (b = 0.000237, p = 0.348), suggesting increases in tapping speed were not a primary driver of choice behavior across the drug conditions. Additionally, there was no effect of drug condition on task completion rates (b = −0.036, p = 0.327).
Discussion
Consistent with predictions, d-amphetamine dose-dependently increased choice of the HC/HR option, indicating greater willingness to exert effort in pursuit of reward. This effect was most evident on low and medium probability trials, suggesting amphetamine increased tolerance for probability costs. However, amphetamine did not change the influence of reward magnitude on choices, suggesting it did not alter valuation of benefits. Importantly, although amphetamine increased psychomotor speed on the EEfRT, this speeding was not a significant predictor of exertion of effort. Thus, amphetamine's gross psychomotor effects do not appear to principally account for observed effects on effort-based decision-making.
Our findings are congruent with animal studies demonstrating a role for DA in effort- and probability-based decision-making (St. Onge and Floresco, 2009; Salamone et al., 2009; St. Onge et al., 2010). Specifically, our findings support the hypothesis that DA is crucial for overcoming costs when pursuing rewards, including effort, probability and time costs. In preclinical research, multiple studies have found that DA agonism and antagonism respectively increase and attenuate willingness to tolerate effort expenditure (Salamone et al., 2007; Bardgett et al., 2009), low probability of reward (Floresco and Whelan, 2009; St. Onge and Floresco, 2009; St. Onge et al., 2010), and temporal delays (Wade et al., 2000). Further demonstrating the specificity of DA to cost valuation, studies have found that when costs are low (e.g., an FR1 schedule), effects of DA antagonism on choice are minimal, but scale dramatically as effort demands increase (Salamone et al., 2001). In the present study, effects of amphetamine on probability costs show a similar pattern; amphetamine did not alter choices during high (88%) probability trials, where probability costs are relatively low. In contrast, the effect of amphetamine was significant for 50% trials, and almost doubled for low (12%) probability trials. Importantly, the lack of amphetamine effect on 88% trials is not a ceiling effect, as average proportion of HC/HR options at this probability ranged between 69 and 74%.
Theoretical models of cost/benefit decision-making have described a prefrontal/limbic network where initial appraisals of costs and benefits are made and then relayed via glutamatergic afferents to ventral striatal medium spiny neurons (Walton et al., 2006; Hauber and Sommer, 2009; Stuber et al., 2011). Extracellular DA may then modify postsynaptic effects of these incoming signals, shifting cost-benefit equilibrium in favor of reward attainment (Phillips et al., 2007). Consistent with this hypothesis, systemic DA agonism or antagonism have similar effects across various costs, while lesions to specific cortical areas are selective to cost type. For example, deactivation of anterior cingulate (ACC) reduces effortful responding without affecting delay-discounting, while lesions of orbitofrontal cortex impair delay-discounting while sparing effort-based decision-making (Rudebeck et al., 2006; Walton et al., 2006). Similar anatomical dissociation between effort and delay-based costs has been identified in humans (Prévost et al., 2010). While the present study did not directly investigate delay costs, past work has shown that amphetamine increases the willingness to tolerate delays in humans (de Wit et al., 2002). Together with our findings and the preclinical data, this supports the notion that DA transmission reduces sensitivity to multiple classes of costs during goal-directed behavior.
There are several potential alternative explanations to be considered. First, rather than altering valuations of costs, amphetamine may instead have changed valuation of benefits, increasing the appeal of reward. We believe this unlikely, as amphetamine did not significantly alter effects of reward magnitude on effort. The second alternative explanation is that amphetamine-induced psychomotor facilitation may have reduced the physical effort of the HC/HR option, and this may have accounted for changes in choice behavior rather than direct DAergic effects on decision-making. While amphetamine sped responses, the degree of speeding was not related to HC/HR choice, making this explanation unlikely. This is consistent with multiple preclinical studies demonstrating that DA's psychomotor effects are not wholly responsible for alterations in effort expenditure (Cousins et al., 1996; Denk et al., 2005; Salamone et al., 2007; Bardgett et al., 2009). Finally, amphetamine may have increased tolerance for delay rather than effort costs. In some animal studies, amphetamine only increased willingness to exert effort when delays to reward were longer for HC/HR than LC/LR options (although DA antagonism consistently reduced willingness to exert effort even when delays were equalized; Floresco et al., 2008; Wanat et al., 2010). In our study the HC/HR option took longer to complete, delaying receipt of feedback about wins and losses. However, actual reward receipt occurred after conclusion of all three sessions, so changes in valuation of delay costs seems unlikely to explain the results.
In addition to shedding light on normative cost/benefit decision-making in humans, the present findings may elucidate mechanisms relevant to neuropsychiatric disorders. DA-linked aberrations in cost/benefit decision-making have been identified across a wide range of neuropsychiatric illnesses (Volkow et al., 2004; Treadway and Zald, 2011). In Parkinson's patients, alterations in cortical valuation networks resulting from treatment with indirect DA agonists predict pathological gambling (van Eimeren et al., 2009). In studies of nicotine addiction, catecholamine depletion reduced willingness to expend effort to gain cigarettes (Venugopalan et al., 2011). Finally, abnormalities in effort-based decision-making have been linked to traits associated with aspects of depression (Treadway et al., 2009; Kurniawan, 2010; Treadway and Zald, 2011).
Limitations of this study include the limited range of DA manipulations, the “mixed” nature of the task decisions, and a lack of neurochemical and brain-regional specificity. First, in rodents, low to moderate doses of amphetamine have increased effort, while high doses suppress it (Floresco et al., 2008). Both doses here were in the moderate range, so a different relationship might appear at higher doses. Examining both higher doses and suppressing DA in humans will be important future directions. Second, the current task requires participants to weigh both effort and probability costs on each trial, and it is possible combining effort and probability decision-making might alter the effect of amphetamine on decision-making. In animals, larger doses of amphetamine have dissociative effects on effort and probability costs, increasing tolerance for probability cost while decreasing tolerance for effort costs (Floresco and Whelan, 2009). Third, although we have emphasized the role of dopamine, amphetamine also has noradrenergic and serotonergic effects, which could also mediate some of the observed effects on cost-benefit decision-making (Rogers, 2011). Finally, our study did not allow identification of specific brain regions mediating the effects of amphetamine. Future studies combining pharmacological manipulations with imaging may be particularly productive for examining a region-specific role for DA in human effort-based decision-making.
In sum, the present report significantly extends preclinical work on the role of dopamine in effort-based decision-making by demonstrating that a pharmacological manipulation of dopamine also alters effort-based decision-making in humans. Translational extensions of preclinical work on dopamine and effort such as this one may allow development and evaluation of more effective treatments for neuropsychiatric disorders involving DA dysfunction.
Footnotes
The National Institute on Drug Abuse supported this work through Grant R01 DA02812 to H.d.W. M.C.W. and L.M.M. are supported by a National Institute on Drug Abuse Training Grant, T32 DA007255. The contribution of D.H.Z. and M.T.T. to this work was supported by Grants R21 MH092751-01 and F31 MH1087015-02 from the National Institute of Mental Health. We thank Cassandra Esposito and Celina Joos for their work on this study.
The authors declare no competing financial interests.
- Correspondence should be addressed to Harriet de Wit, Department of Psychiatry and Behavioral Neuroscience, University of Chicago, 5841 S. Maryland Avenue, MC3077, Chicago, IL 60637. hdew{at}uchicago.edu
