Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE

User menu

  • Log in
  • My Cart

Search

  • Advanced search
Journal of Neuroscience
  • Log in
  • My Cart
Journal of Neuroscience

Advanced Search

Submit a Manuscript
  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Collections
    • Podcast
  • ALERTS
  • FOR AUTHORS
    • Information for Authors
    • Fees
    • Journal Clubs
    • eLetters
    • Submit
  • EDITORIAL BOARD
  • ABOUT
    • Overview
    • Advertise
    • For the Media
    • Rights and Permissions
    • Privacy Policy
    • Feedback
  • SUBSCRIBE
PreviousNext
Journal Club

Stress and Exploitative Decision-Making

Madeline B. Harms
Journal of Neuroscience 18 October 2017, 37 (42) 10035-10037; DOI: https://doi.org/10.1523/JNEUROSCI.2169-17.2017
Madeline B. Harms
Department of Psychology, University of Wisconsin–Madison, Madison, Wisconsin 53706
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • eLetters
  • PDF
Loading

Every day, we encounter situations in which we must decide whether to continue what we are doing or move on to a potentially better option (e.g., going to the same place as last year for vacation or traveling somewhere new; staying in a secure but unsatisfying job or embarking on a new career path). These types of decisions are examples of the explore/exploit problem. Exploration is defined as choosing an option about which we have less information, whereas exploitation is sticking with an option about which we know more. The decisions people ultimately make in these situations depend upon their past experiences, which create expectations about the type and likelihood of rewards and punishments that a given environment will yield. For example, if an individual perceives that the world has been harsh and unfair, she might expect that any choice she makes will lead to an undesirable outcome. This individual may therefore be more likely to exploit a familiar, but nonoptimal, option than to explore an alternative path. Lenow et al. (2017) argue that stress is one factor that leads to perceptions of an environment being harsh, and therefore hypothesized that stress would facilitate tendencies to exploit rather than explore.

To systematically examine the effects of stress on explore/exploit behavior, Lenow et al. (2017) used a virtual patch-foraging task in which participants spent time in each of four orchards, with the goal of harvesting as many apples as possible. On each trial, participants had the option to stay at the current tree/patch or move to a different tree. Each subsequent harvest of the same tree resulted in slightly fewer apples, so at some point it would be advantageous to move on to the next tree. In orchards representing rich environments, travel time to the next tree was short; and in orchards representing harsh environments, travel time was longer. An experimental group completely submerged their arm in cold water to induce acute stress before the foraging task, whereas a control group submerged their arm in warm water. Cortisol responses to the stressor were used as a continuous measure of acute stress. Participants also reported their perceived chronic stress.

Explore/exploit behavior was measured in terms of each participant's tree-level exit threshold (the average of the last two rewards before moving to the next tree). As expected, participants had higher exit thresholds in rich orchards than in low-quality orchards, showing that they used environment quality to guide their decisions and recognized that the opportunity cost of moving between trees was higher in the low-quality environment. In both environments, participants who showed higher cortisol responses and who reported higher chronic stress showed lower exit thresholds, indicating greater exploitation, than less stressed participants. To further explain results, exit thresholds were compared against the optimal threshold for leaving a tree that would maximize one's reward. Deviations from this optimal value were calculated for each subject and characterized in terms of underexploitation and overexploitation. Both acute and chronic stress were associated with more overexploitation.

Based on these results, the authors suggest that stress leads to overexploitation through biased perceptions of environmental quality (i.e., stressed individuals perceive the environment to be harsher than it actually is). There are other potential mechanisms that could influence behavior on foraging tasks, however. One possibility is that stress biases individuals toward inaction rather than action. A recent study found that stress specifically impaired learning to produce an action, regardless of whether participants needed to act to gain a reward or to avoid a punishment (de Berker et al., 2016). If moving between orchards represents an action, a general bias toward inaction could explain stressed participants' tendency to stay at the same patch rather than move to a new patch. Stress has also been shown to shift an organism's attention and cognitive focus to the present over the future (Frankenhuis et al., 2016). A present-oriented bias could decrease exploration during a foraging task if it interferes with one's ability to conceptualize the potential reward that could be gained in a future patch.

Another potential mechanism through which stress may influence explore/exploit behavior is by reducing cognitive flexibility, which is facilitated by the prefrontal cortex (Kim et al., 2011). Optimal exploration requires cognitive flexibility because individuals must update representations about their environment (in this case, depletion rate of the tree and travel time) when deciding when to move on to a new patch. In contrast, remaining at the same tree for longer than is optimal could be a form of perseveration, which does not require prefrontal function and is associated with stress (Schwabe and Wolf, 2009). The prefrontal cortex is highly sensitive to stress. Acute stress leads to the release of glucocorticoids, which appear to reduce the function of prefrontal cortex by disrupting intracellular signaling pathways (Arnsten, 2009). Chronic stress is also associated with reduced prefrontal function because chronically high levels of glucocorticoids appear to cause dendritic retraction and reduced spine number in this region (Joëls et al., 2007; Dias-Ferreira et al., 2009). At the same time, acute and chronic stressors appear to increase amygdala and striatal control over prefrontal cortex (Hermans et al., 2011; Fareri and Tottenham, 2016), facilitating habit-directed learning and perseveration (Schwabe and Wolf, 2009). In sum, both acute and chronic stressors appear to lead to impaired prefrontal function and increased reliance on striatal and limbic structures to guide decision-making. This altered brain function reduces cognitive flexibility and increases perseveration, potentially resulting in higher levels of exploitation.

The findings of Lenow et al. (2017) parallel those of a study that examined effects of early life stress on exploration and exploitation. Humphreys et al. (2015) compared adolescents who had been institutionalized as infants with adolescents who had no such history during a reward task in which each pump of a balloon could lead to either accumulating more points or losing all one's points. Previously institutionalized adolescents “cashed in” their earnings earlier than the comparison group, reflecting a tendency to exploit a safe option rather than explore the possibility of gaining more points. In addition, maternal separation in infancy has been associated with less physical exploration in adolescent rats (Spivey et al., 2008). These results are consistent with the notion that stress exposure reduces exploration.

The notion that stress interferes with an organism's tendency to explore different options has important implications for learning processes throughout the lifespan. Individuals learn in part through sampling information in their environment: in situations where the probabilities of various outcomes are unknown, individuals must explore different options to learn action-outcome associations (Sheth et al., 2011; Hertwig and Frey, 2017). If new information is sampled at a lower rate due to stress, then learning may be diminished. Indeed, recent studies show reduced associative learning ability in adolescents who were exposed to early childhood stress (Hanson et al., 2017; Harms et al., 2017). Results of Lenow et al. (2017) and Humphreys et al. (2015) suggest that this phenomenon could be partially explained by reduced exploration and information sampling due to stress.

In future research addressing the effects of stress on motivated behavior, it will be important to consider potential relationships between exploration/exploitation and reward processing, as well as their neural substrates. These processes rely on overlapping brain circuitry, including the striatum and prefrontal cortex (Daw et al., 2006). Stress exposure profoundly affects the structure and connectivity of prefrontal and striatal regions (Dias-Ferreira et al., 2009; Fareri and Tottenham, 2016), and severe early life stress has been linked to reduced striatal reward responsivity (Dillon et al., 2009; Goff et al., 2013). Furthermore, both striatum and prefrontal cortex are components of dopaminergic reward pathways. There is evidence that stress exposure alters dopamine function, although effects of stress on dopamine systems appear to be complex and may vary by type of stressor (Hollon et al., 2015). More research is needed to examine more systematically how different types of stressors influence dopamine systems, and how dopamine in turn regulates exploration and exploitation. This line of research could inform treatment for disorders in which dopamine systems may be disrupted, such as depression (Tye et al., 2013).

Although the negative aspects of stress are often emphasized, physical and behavioral responses to stressors evolved to promote survival of the organism. Stress tends to be associated with harsh environments, in which exploration is less likely to be associated with reward. Maladaptive effects of stress occur when there is a mismatch between the stress response and the current environment (e.g., an individual behaves as if he is in a harsh environment when he is in a rich environment). In Lenow et al. (2017), stress from the cold-water submersion and from daily life influenced decision-making during an unrelated foraging task, leading to overexploitation and reduced performance. More consequentially, neural and behavioral effects of early life stress appear to persist throughout the lifespan, potentially leading to alterations in decision-making that hinder learning. The findings of Lenow et al. (2017) reiterate the profound effects of stress on decision-making, but there is still much to learn about how specific types and timing of stress affect different aspects of motivated behavior, such as reward valuation, goal representations, and expectations about the future.

Footnotes

  • Editor's Note: These short reviews of recent JNeurosci articles, written exclusively by students or postdoctoral fellows, summarize the important findings of the paper and provide additional insight and commentary. If the authors of the highlighted article have written a response to the Journal Club, the response can be found by viewing the Journal Club at www.jneurosci.org. For more information on the format, review process, and purpose of Journal Club articles, please see http://jneurosci.org/content/preparing-manuscript#journalclub.

  • I thank Seth Pollak for comments on the manuscript. M.H. is funded by National Institute of Mental Health grant T32MH018931-28.

  • The author declares no competing financial interests.

  • Correspondence should be addressed to Dr. Madeline B. Harms, Department of Psychology, University of Wisconsin–Madison, 1202 West Johnson Street, Madison, WI 53706. mharms3{at}wisc.edu

References

  1. ↵
    1. Arnsten AF
    (2009) Stress signaling pathways that impair prefrontal cortex structure and function. Nat Rev Neurosci 10:410–422. doi:10.1038/nrn2648 pmid:19455173
    OpenUrlCrossRefPubMed
  2. ↵
    1. Daw ND,
    2. O'Doherty JP,
    3. Dayan P,
    4. Seymour B,
    5. Dolan RJ
    (2006) Cortical substrates for exploratory decisions in humans. Nature 44:876–879. doi:10.1038/nature04766 pmid:16778890
    OpenUrlCrossRefPubMed
  3. ↵
    1. de Berker AO,
    2. Tirole M,
    3. Rutledge RB,
    4. Cross GF,
    5. Dolan RJ,
    6. Bestmann S
    (2016) Acute stress selectively impairs learning to act. Sci Rep 6:29816. doi:10.1038/srep29816 pmid:27436299
    OpenUrlCrossRefPubMed
  4. ↵
    1. Dias-Ferreira E,
    2. Sousa JC,
    3. Melo I,
    4. Morgado P,
    5. Mesquita AR,
    6. Cerqueira JJ,
    7. Costa RM,
    8. Sousa N
    (2009) Chronic stress causes frontostriatal reorganization and affects decision-making. Science 325:621–625. doi:10.1126/science.1171203 pmid:19644122
    OpenUrlAbstract/FREE Full Text
  5. ↵
    1. Dillon DG,
    2. Holmes AJ,
    3. Birk JL,
    4. Brooks N,
    5. Lyons-Ruth K,
    6. Pizzagalli DA
    (2009) Childhood adversity is associated with left basal ganglia dysfunction during reward anticipation in adulthood. Biol Psychiatry 66:206–213. doi:10.1016/j.biopsych.2009.02.019 pmid:19358974
    OpenUrlCrossRefPubMed
  6. ↵
    1. Fareri DS,
    2. Tottenham N
    (2016) Effects of early life stress on amygdala and striatal development. Dev Cogn Neurosci 19:233–247. doi:10.1016/j.dcn.2016.04.005 pmid:27174149
    OpenUrlCrossRefPubMed
  7. ↵
    1. Frankenhuis WE,
    2. Panchanathan K,
    3. Nettle D
    (2016) Cognition in harsh and unpredictable environments. Curr Opin Psychol 7:76–80. doi:10.1016/j.copsyc.2015.08.011
    OpenUrlCrossRef
  8. ↵
    1. Goff B,
    2. Gee DG,
    3. Telzer EH,
    4. Humphreys KL,
    5. Gabard-Durnam L,
    6. Flannery J,
    7. Tottenham N
    (2013) Reduced nucleus accumbens reactivity and adolescent depression following early-life stress. Neuroscience 249:129–138. doi:10.1016/j.neuroscience.2012.12.010 pmid:23262241
    OpenUrlCrossRefPubMed
  9. ↵
    1. Hanson JL,
    2. van den Bos W,
    3. Roeber BJ,
    4. Rudolph KD,
    5. Davidson RJ,
    6. Pollak SD
    (2017) Early adversity and learning: implications for typical and atypical behavioral development. J Child Psychol Psychiatry 58:770–778. doi:10.1111/jcpp.12694 pmid:28158896
    OpenUrlCrossRefPubMed
  10. ↵
    1. Harms MB,
    2. Shannon-Bowen K,
    3. Hanson JD,
    4. Pollak SD
    (2017) Instrumental learning and cognitive flexibility processes are impaired in children exposed to early life stress. Dev Sci, in press.
  11. ↵
    1. Hermans EJ,
    2. van Marle HJ,
    3. Ossewaarde L,
    4. Henckens MJ,
    5. Qin S,
    6. van Kesteren MT,
    7. Schoots VC,
    8. Cousijn H,
    9. Rijpkema M,
    10. Oostenveld R,
    11. Fernández G
    (2011) Stress-related noradrenergic activity prompts large-scale neural network reconfiguration. Science 334:1151–1153. doi:10.1126/science.1209603 pmid:22116887
    OpenUrlAbstract/FREE Full Text
  12. ↵
    1. Hertwig R,
    2. Frey R
    (2017) How representations of knowledge shape actions. Knowledge Action 9:127–143. doi:10.1007/978-3-319-44588-5_8
    OpenUrlCrossRef
  13. ↵
    1. Hollon NG,
    2. Burgeno LM,
    3. Phillips PE
    (2015) Stress effects on the neural substrates of motivated behavior. Nat Neurosci 18:1405–1412. doi:10.1038/nn.4114 pmid:26404715
    OpenUrlCrossRefPubMed
  14. ↵
    1. Humphreys KL,
    2. Lee SS,
    3. Telzer EH,
    4. Gabard-Durnam LJ,
    5. Goff B,
    6. Flannery J,
    7. Tottenham N
    (2015) Exploration-exploitation strategy is dependent on early experience. Dev Psychobiol 57:313–321. doi:10.1002/dev.21293 pmid:25783033
    OpenUrlCrossRefPubMed
  15. ↵
    1. Joëls M,
    2. Karst H,
    3. Krugers HJ,
    4. Lucassen PJ
    (2007) Chronic stress: implications for neuronal morphology, function, and neurogenesis. Front Neuroendocrinol 28:72–96. doi:10.1016/j.yfrne.2007.04.001 pmid:17544065
    OpenUrlCrossRefPubMed
  16. ↵
    1. Kim C,
    2. Johnson NF,
    3. Cilles SE,
    4. Gold BT
    (2011) Common and distinct mechanisms of cognitive flexibility in prefrontal cortex. J Neurosci 31:4771–4779. doi:10.1523/JNEUROSCI.5923-10.2011 pmid:21451015
    OpenUrlAbstract/FREE Full Text
  17. ↵
    1. Lenow JK,
    2. Constantino SM,
    3. Daw ND,
    4. Phelps EA
    (2017) Chronic and acute stress promote overexploitation in serial decision making. J Neurosci 37:5681–5689. doi:10.1523/JNEUROSCI.3618-16.2017 pmid:28483979
    OpenUrlAbstract/FREE Full Text
  18. ↵
    1. Schwabe L,
    2. Wolf OT
    (2009) Stress prompts habit behavior in humans. J Neurosci 29:7191–7198. doi:10.1523/JNEUROSCI.0979-09.2009 pmid:19494141
    OpenUrlAbstract/FREE Full Text
  19. ↵
    1. Sheth SA,
    2. Abuelem T,
    3. Gale JT,
    4. Eskandar EN
    (2011) Basal ganglia neurons dynamically facilitate exploration during associative learning. J Neurosci 31:4878–4885. doi:10.1523/JNEUROSCI.3658-10.2011 pmid:21451026
    OpenUrlAbstract/FREE Full Text
  20. ↵
    1. Spivey J,
    2. Barrett D,
    3. Padilla E,
    4. Gonzalez-Lima F
    (2008) Mother-infant separation leads to hypoactive behavior in adolescent Holtman rats. Behav Processes 79:59–65. doi:10.1016/j.beproc.2008.05.002 pmid:18585869
    OpenUrlCrossRefPubMed
  21. ↵
    1. Tye KM,
    2. Mirzabekov JJ,
    3. Warden MR,
    4. Ferenczi EA,
    5. Tsai HC,
    6. Finkelstein J,
    7. Kim SY,
    8. Adhikari A,
    9. Thompson KR,
    10. Andalman AS,
    11. Gunaydin LA,
    12. Witten IB,
    13. Deisseroth K
    (2013) Dopamine neurons modulate neural encoding and expression of depression-related behavior. Nature 493:537–541. pmid:23235822
    OpenUrlCrossRefPubMed
Back to top

In this issue

The Journal of Neuroscience: 37 (42)
Journal of Neuroscience
Vol. 37, Issue 42
18 Oct 2017
  • Table of Contents
  • Table of Contents (PDF)
  • About the Cover
  • Index by author
  • Advertising (PDF)
  • Ed Board (PDF)
Email

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
Stress and Exploitative Decision-Making
(Your Name) has forwarded a page to you from Journal of Neuroscience
(Your Name) thought you would be interested in this article in Journal of Neuroscience.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
Stress and Exploitative Decision-Making
Madeline B. Harms
Journal of Neuroscience 18 October 2017, 37 (42) 10035-10037; DOI: 10.1523/JNEUROSCI.2169-17.2017

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Request Permissions
Share
Stress and Exploitative Decision-Making
Madeline B. Harms
Journal of Neuroscience 18 October 2017, 37 (42) 10035-10037; DOI: 10.1523/JNEUROSCI.2169-17.2017
Reddit logo Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Footnotes
    • References
  • Info & Metrics
  • eLetters
  • PDF

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

  • Unraveling the Mechanisms Underlying Disordered Sleep in Alzheimer's Disease
  • The ESCRT Pathway's Role in Prion Diseases and Beyond
  • What Role Does the Cerebellum Have in a Fatigue Network?
Show more Journal Club
  • Home
  • Alerts
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Issue Archive
  • Collections

Information

  • For Authors
  • For Advertisers
  • For the Media
  • For Subscribers

About

  • About the Journal
  • Editorial Board
  • Privacy Policy
  • Contact
(JNeurosci logo)
(SfN logo)

Copyright © 2023 by the Society for Neuroscience.
JNeurosci Online ISSN: 1529-2401

The ideas and opinions expressed in JNeurosci do not necessarily reflect those of SfN or the JNeurosci Editorial Board. Publication of an advertisement or other product mention in JNeurosci should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in JNeurosci.