Abstract
The surprising omission or reduction of vital resources (food, fluid, social partners) can induce an aversive emotion known as frustrative nonreward (FNR), which can influence subsequent behavior and physiology. FNR is an integral mediator of irritability/aggression, motivation (substance use disorders, depression), anxiety/fear/threat, learning/conditioning, and social behavior. Despite substantial progress in the study of FNR during the twentieth century, research lagged in the later part of the century and into the early twenty-first century until the National Institute of Mental Health's Research Domain Criteria initiative included FNR and loss as components of the negative valence domain. This led to a renaissance of new research and paradigms relevant to basic and clinical science alike. The COVID-19 pandemic's extensive individual and social restrictions were correlated with increased drug and alcohol use, social conflict, irritability, and suicide, all potential consequences of FNR. This article highlights animal models related to these psychiatric disorders and symptoms and presents recent advances in identifying the brain regions and neurotransmitters implicated.
Significance Statement
The surprising omission or reduction of vital resources (food, fluid, social partners) can induce an aversive emotion known as frustrative nonreward (FNR), which can influence subsequent behavior. FNR is integral for irritability/aggression, motivation (substance use disorders, depression), anxiety/fear/threat, learning/conditioning, and social behavior. This article highlights animal models related to these processes and neuropsychiatric disorders by presenting recent progress in identifying the brain regions and neurotransmitters implicated. After solid advancement in FNR research during the twentieth century, current progress was stimulated by the National Institute of Mental Health's Research Domain Criteria initiative, which included FNR as a component of the negative valence domain. The development of new research and paradigms on FNR is relevant to basic and clinical science alike.
Introduction
Frustrative nonreward (FNR) was first suggested by Amsel (Amsel and Ward, 1954; Amsel, 1958; M. R. Papini, 2008) as an emotional response with aversive motivational properties that included unconditioned and conditioned components, respectively, called primary and secondary frustration. Substantial behavioral research followed these early publications (Amsel, 1992). The key difference between plain nonreward and FNR is the presence of signals previously paired with reward that trigger an expectation. What made nonreward frustrating was the negative disparity between obtained and expected reward, whether in quantity or quality. Results consistent with FNR had been published decades earlier (Elliott, 1928; Tinklepaugh, 1928), but Amsel's empirical and theoretical works, and the challenges from competing data and hypotheses (Capaldi, 1966), contributed significantly to developing these ideas. The 1980s–1990s witnessed an expansion of research that identified a variety of behavioral consequences (M. R. Papini and Dudley, 1997), its emergence in infant rats (Amsel and Stanton, 1980), its psychopharmacology (Flaherty, 1996), and its apparent restriction to mammalian behavior across vertebrates (M. R. Papini, 2002, 2003).
During the initial years of the twenty-first century, the study of FNR was pursued by only a handful of researchers (M. R. Papini et al., 2015). This situation is changing following the Research Domain Criteria (RDoC) initiative launched by the National Institute of Mental Health in 2009 (https://www.nimh.nih.gov/rdoc). RDoC approaches mental health in terms of fundamental psychobiological systems, including arousal/regulation, positive valence, negative valence, sensorimotor skills, cognition, and social processes. The present article is concerned with negative valence, which includes FNR and loss. RDoC defines FNR as “Reactions elicited in response to withdrawal/prevention of reward, i.e., by the inability to obtain rewards following repeated or sustained efforts,” whereas it defines loss as “A state of deprivation of a motivationally significant conspecific, object, or situation.” Both definitions emphasize reward loss. These definitions are open to refinements that may come from the novel research unleashed by RDoC, as summarized in the following sections.
Operant Assays of FNR Useful for the Study of Substance Use Disorders (SUDs)
Given the strong evidence that dopamine ties together reward, reinforcement, motivation, and now FNR (Ishino et al., 2023), one might expect FNR could have strong implications for SUDs. Frustration affects motivation in many circumstances, and, since SUDs are often a function of unchecked motivation, FNR should be a component of SUDs. There is intriguing clinical data suggesting that FNR is indeed involved in SUDs. Individuals in SUD treatment programs had lower frustration tolerance than controls in Rosenzweig's test for frustration tolerance (Ramírez-Castillo et al., 2019), and the number of relapses is correlated with FNR sensitivity in a questionnaire of daily frustrations (Baars et al., 2013). Additionally, coping with frustration/anger is considered a major determinant of relapses (Ramo and Brown, 2008). One hypothesis that can be generated from these data is that in some individuals with SUDs, frustration fails to appropriately blunt motivation for drugs. These susceptible individuals will seek drugs in the face of any amount of frustration (such as problems acquiring drug, price hikes higher than one can pay, etc.), whereas others will terminate drug seeking. A novel therapeutic strategy for individuals for whom frustration fails to decrease motivation would be to recouple frustration with motivation, not by increasing frustration, but by enabling frustration to appropriately quell motivation.
Preclinically, operant tests of motivation in rats employ FNR to infer motivation (e.g., progressive ratio, extinction). Operant tests are characterized by assessing bar pressing during sessions in which the opportunity to respond is continuously available (free-operant procedures). Thus, anyone studying motivation in an operant context is usually studying FNR. However, until recently, it was difficult to measure frustration-related behavior during operant tasks.
One way to measure frustration-related behavior during operant tasks with rats is to record the duration of individual bar presses (Vasquez, et al., 2021a,b; Mármol Contreras et al., 2023). Rats downshifted from a four-sucrose-pellet reinforcer to a one-pellet reinforcer increased bar-press durations beyond those initially trained on one-pellet—termed operant successive negative contrast (oSNC). Importantly, bar-press durations are not isometric with other states, such as drug “wanting,” measured by the number of bar presses, or impulsive action, measured by premature bar presses.
Rats increase their average bar-press durations when raising fixed-ratio response requirements, decreasing intravenous drug self-administration dose, decreasing the size of reward, extinction, and progressive ratio training. Bar-press durations are a measure of the general frustration level rather than point-in-time frustration, that is, a single nonreinforced response will not produce a duration increase for the next bar press, but rather the rat will introduce several longer bar presses interspersed among the normal, short bar presses (response variability). As a result, data can be presented as rolling averages of 10 bar presses to get a snapshot of the frustration level (Fig. 1). Unpublished data show that manipulations affecting neuronal excitability in the nucleus accumbens shell, a region known to be important for motivation, potently control bar-press durations. Moreover, whereas duration increases with FNR are replicable, bar-press force shows no hint of changing with increases in fixed-ratio requirements, extinction, or progressive ratio training. Force only increases when FNR is used with already high-response requirements, such as 20 bar presses per reinforcement.
Representative rolling average plots during FNR. A, Within-session extinction showing the transition from fixed-ratio 1 to extinction (FR1, EXT). B, Progressive ratio. C, Relief from frustration showing the transition from extinction to FR1. Lines represent rolling averages of 10 bar presses.
Bar-press durations can be measured with any current lever-based operant task without changing the rodent's experience (i.e., no change in response requirement), enabling the study of the fundamental relationship between FNR and motivation as well as its neurobiology. The goal of this project is to place frustration among the other well-studied facets of SUDs, such as craving, impulsivity, and habit (Torres and Papini, 2016).
FNR Induced by Reward Downshift: A New Look at the cSNC Paradigm
Although animal models provide abundant information on the underlying mechanisms and adaptive value of FNR (M. R. Papini et al., 2022), important gaps remain, including (1) the potential for sex differences in the expression and resolution of FNR and (2) the underlying brain circuits (neuroconnectome) involved. These problems can be addressed using the consummatory successive negative contrast (cSNC) paradigm with rats. cSNC has been extensively validated as a source of FNR in behavioral, pharmacological, and neurobiological studies (Flaherty, 1996; M. R. Papini et al., 2015). In the cSNC paradigm, rats receive daily access to 32% sucrose during the ten 5 min sessions followed by an abrupt reduction to 2% sucrose over the four 5 min sessions (downshifted condition). Consumption in these animals is compared with that of controls that always received 2 or 32% sucrose (unshifted conditions). cSNC refers to a decrease in sucrose consumption in downshifted animals below the level of unshifted controls during postshift sessions (Flaherty, 1996). Consummatory behavior typically recovers to the level of 2% sucrose unshifted controls within 2–4 downshift sessions. Thus, cSNC allows for an assessment of both the initial response to reward loss and the recovery of behavior that follows (S. Papini et al., 2014). cSNC illustrates the modulatory effects of incentive relativity and reward expectancies on cognition, emotion, and action (Torres and Papini, 2017).
A comparison between the consummatory and spontaneous behaviors of male and female rats exposed to cSNC (Fig. 2) revealed only subtle sex differences. Regardless of the sucrose concentration (32 or 2% sucrose), female rats exhibited higher sucrose intake by body weight (mL/kg) than male rats, but the behavioral consequences of reward downshift were similar. Reward downshift significantly suppressed consummatory behavior, increased exploratory behavior (rearing, head-dipping, ambulation) and induced biting to the sipper tube in both females and males. However, when animals had simultaneous access to water and sucrose during training, female rats increased water consumption and decreased preference for sucrose across downshift sessions, compared with males.
Experimental schedule for cSNC. cSNC involves reward downshift (32 to 2% sucrose) compared with unshifted controls (32 to 32% and 2 to 2% sucrose). Consummatory behavior is suppressed on Session 11, but it recovers in subsequent postshift sessions. cSNC is used to map neural activation in a large set of brain regions in male and female rats.
The biological basis of these effects are being addressed by quantifying c-Fos expression, a protein resulting from the activation of the early expression gene c-fos and used as a marker of neuronal activity (Lara Aparicio et al., 2022). Standard immunohistochemistry methods were conducted to calculate the density of c-Fos–positive cells in several regions of interest selected based on a proposed neural circuit of cSNC (Ortega et al., 2017). Differences in c-Fos expression between downshifted and unshifted conditions were observed in the medial portion of the lateral habenula (but not in the lateral portion). A nonsignificant trend was also observed in the ventral portion of the medial habenula (unpublished data). There was no evidence that the hippocampus is involved in the cSNC effect, whether in terms of the dentate gyrus or in any of the CA fields (Hagen et al., 2024). Preliminary data also indicate higher c-Fos expression in downshifted than unshifted animals in the amygdala (central and basolateral nuclei), insular cortex, and nucleus accumbens (shell and core). Thus far, these results fit available data based on chemogenetic, lesion, and microdialysis manipulations of individual brain regions (Genn et al., 2004; Lin et al., 2009; Guarino et al., 2020, 2023).
A more comprehensive analysis of c-Fos expression patterns will be processed using graph theory to identify potential distinctions and similarities in neural networks derived from male versus female comparisons and from the initial response to reward downshift versus the recovery that follows. This will provide a new and more comprehensive understanding of the neurobiology underlying FNR in the extensively validated cSNC paradigm.
Dopamine Signaling for Persistent Reward Pursuit despite FNR
To obtain rewards, animals must often persist despite failures. Persistence is crucial in foraging and courtship, where outcomes are probabilistic and choices are limited. Failure to overcome FNR can reduce individual fitness and, in humans, lead to failure-induced depression or substance addiction (M. R. Papini et al., 2015; Torres and Papini, 2016). Experiments show that when there is reward uncertainty, animals often actively pursue another opportunity to obtain the reward, even when it is not immediately available (Ogawa et al., 2013). Behaviors associated with 50% reward are more resistant to extinction than those associated with 100% reward (Amsel, 1992). This partial reinforcement extinction effect is paradoxical because 50% reward should be valued <100% reward, and yet it generates more behavior in extinction. The ability to pursue the next reward after nonreward is thought to be based on associating expected FNR with occasionally obtaining reward. Such association leads to active coping with anticipated negative outcomes. However, the neural mechanisms underlying this coping ability remain poorly understood.
Studies showed that neurons in the lateral habenula (Matsumoto and Hikosaka, 2008), central amygdala (Calu et al., 2010), paraventricular nucleus of the thalamus (Do Monte et al., 2017), and anterior cingulate cortex (ACC; Kawai et al., 2015) are more activated in response to FNR than to reward. However, their roles in actively processing FNR and adjusting the pursuit of reward remain unclear. Midbrain dopamine neurons provide an error signal, termed reward prediction error (RPE), defined as the discrepancy between obtained and expected rewards, and are critical for value-based learning (Schultz, 1997). Typical RPE neurons (called “Type 1” dopamine neurons) decrease activity in response to unexpected nonreward, reducing reward value and supporting negative learning (Fiorillo et al., 2003; Lee et al., 2020). Although recent studies revealed that dopamine neurons are heterogeneous and signal more than RPE (de Jong et al., 2018; Engelhard et al., 2019; Jeong et al., 2022; Coddington et al., 2023), dopamine neurons for coping with the absence of expected reward were previously unknown.
To explore the role of dopamine neurons in FNR, Ishino et al. (2023) developed a task in which head-restrained rats were required to pursue a probabilistic reward by repeating a sequence of actions (Fig. 3). This task enabled a quantification of the ability to switch toward the new opportunity to obtain a probabilistic reward after a nonreward. Opto-tagging electrophysiological recording and single-cell calcium imaging showed that half of the dopamine neurons in the anterior part of the lateral ventral tegmental area increased responding after unexpected nonreward and decreased responding after unexpected reward (called “Type 2” dopamine neurons). The responses of these Type 2 neurons were slower (∼0.2 s in response to 50% nonreward) compared with the responses of RPE Type 1 dopamine neurons, suggesting that Type 2 neurons operate in conjunction with RPE neurons to counterbalance and facilitate the continued pursuit of rewards following FNR. Further examination of Type 2 dopamine responses in the nucleus accumbens during both a reward extinction task and a pavlovian task indicated that these responses signal an error, adjusting behavior to overcome FNR.
Head-restrained rats pursued a probabilistic reward (water) by repeating a sequence of actions (push/pull a lever) without any choice options. This task enabled a quantification of the ability to switch toward a new opportunity to obtain a probabilistic reward after a nonreward (from Ishino et al., 2023, reproduced with permission from Science Advances).
However, the function of Type 2 dopamine signaling remains unclear. To understand the significance of this activity, future studies will vary the extent of FNR and quantitatively evaluate behaviors aimed at overcoming challenges to obtain rewards, combined with computational modeling. Type 2 dopamine neurons could provide insights into the neural mechanisms underlying dopamine-dependent psychiatric disorders (Lüscher et al., 2020). For example, a fundamental feature of addiction is the continual pursuit of a particular target despite negative consequences (e.g., loss-chasing in pathological gambling; Chase and Clark, 2010). Abnormal hyperactivity in Type 2 dopamine neurons and/or an imbalance between RPE-type dopamine neurons and Type 2 dopamine neurons might contribute to this feature. This hypothesis presents an avenue for research that could be explored through translational studies involving rodents, nonhuman primates, and humans.
A New Paradigm for Studying FNR in Juvenile Mice
FNR can lead to anger and aggression. Irritability is an aberrant response to FNR (Tseng et al., 2019; Linke et al., 2023) and an impairing symptom especially prevalent in youth, where it is a common reason for psychiatric assessment and treatment (Brotman et al., 2017; Beauchaine and Tackett, 2020). Irritability is present in many psychiatric disorders, including disruptive mood dysregulation disorder (DMDD), oppositional defiant disorder, intermittent explosive disorder, attention-deficit/hyperactivity disorder, autism spectrum disorder, bipolar disorder, major depressive disorder, and anxiety disorders in both youth and adults (Beauchaine and Tackett, 2020).
Irritability can be operationalized as elevated proneness to anger relative to peers (Brotman et al., 2017). It is typically expressed as severe, recurrent temper outbursts that are disproportionate in intensity and/or duration and/or chronically grumpy mood. Temper outbursts are characterized by high motor activity and verbal and/or physical aggression. Such negative affect and behavior are impairing. Longitudinal studies in youth reported that chronic irritability in adolescence predicts depressive and anxiety disorders and suicidality in adults (Leibenluft and Stoddard, 2013). Despite its significant public health impact, little is known about the brain mechanism of irritability, and there are no FDA-approved treatments outside the context of autism.
To facilitate the search for neural mechanisms of frustration and novel treatments for irritability in youth, Naik et al. (2024) developed the alternate poking reward omission (APRO) paradigm and applied it to juvenile mice. Prior FNR paradigms for rodents require lengthy training periods not possible in juveniles (Burokas et al., 2012; Vasquez et al., 2021a). APRO induces FNR as the emotional and behavioral response to unexpected reward omissions (Amsel, 1958). Because aberrant responses to FNR are a central feature of irritability, FNR provides a framework for comparative studies of irritability in juveniles. Such studies can leverage molecular and circuit manipulations in model organisms to advance our understanding of the brain mechanisms mediating responses to frustration and irritability.
DMDD is a disorder diagnosed in 6–18-year-old children, which corresponds to 3–7 weeks of age in mice (Dutta and Sengupta, 2016). Thus, all experimental procedures including behavioral training and testing, as well as brain surgery for molecular and circuit manipulations, must be completed within 4 weeks. The APRO paradigm allows completion of training and FNR induction in 5 d (Fig. 4). During the training phase, mice learn to poke each end of a runway alternately to receive a water reward, making nearly 100% correct poking after three sessions of training. Mice then proceed to the FNR phase during which, for correct alternation, they experience reward on 50% of the trials on the first FNR session and 20% reward on the second FNR session. Control mice receive 100% reward in all sessions.
The APRO paradigm with mice. A, Apparatus. B, Schedule and behavioral testing following APRO. Upward arrow, elevated levels of activity (open field) and aggressive behavior (resident intruder test). Dash lines, no effects.
To test for the behavioral consequence of FNR, we exposed the mice to a battery of behavioral tests after APRO (Fig. 4). Both male and female mice in the FNR group selectively increase locomotion and aggression, which mirrors the behavior of frustrated humans. Anxiety-like behavior tested in the light/dark box and elevated zero maze, depression-like behavior tested with the forced swim and sucrose preference tests, and sociability, however, were unchanged.
To examine brain regions activated by frustration, we removed the brains after APRO and processed them for whole-brain clearing and c-Fos staining. Analysis of the c-Fos data at the network level shows that FNR shifts the brain network toward a more global processing mode (Naik et al., 2023), consistent with human fMRI studies in youth with severe irritability (Linke et al., 2023). Some brain regions associated with negative emotions, reward processing, and irritability, including the ACC and medial dorsal thalamus (MDT), showed elevated c-Fos activation in frustrated mice (Naik et al., 2023). ACC lesions in humans are associated with hostility, emotional unconcern, and irresponsibility (Repple et al., 2017; Bertsch et al., 2020). Recent rodent studies show that, in addition to their sensory-relay function, anterior and midline thalamic nuclei respond to aversive and rewarding stimuli (Alcaraz et al., 2016; Otis et al., 2019; Sweeney-Reed et al., 2021; Roy et al., 2022). Activation of ACC and MDT in frustrated mice suggests that these regions may be involved in the emotional state or behavioral expression of frustration. This possibility and the specific functions of ACC and MDT in frustration have yet to be investigated.
APRO is an effective paradigm to induce frustration in juvenile mice, producing behavioral and brain imaging findings resembling those reported in humans. To delineate the neural mechanism of frustration and irritability, APRO can be readily integrated with electrophysiological, imaging, circuit-specific manipulations, transgenic, and other techniques commonly used with mice. Although APRO is effective in wild-type C57BL/6 mice, its efficacy in other mouse strains with different irritability levels and in mouse models for psychiatric disorders with irritability symptoms awaits further testing. Using APRO in other mouse strains and disease models will provide insights into pathophysiological mechanisms and individual differences in irritability. Furthermore, to translate the mouse findings to humans and develop novel treatments for irritability, a cross-species approach is being taken to develop a human version of APRO so that parallel methods can be used to measure brain activity throughout the FNR process in mice and humans. Similar and distinct findings across species are expected; both are crucial to elucidate the mechanisms of frustration and irritability and to facilitate the search for more efficacious and specific treatments for irritability.
Conclusions
The current resurgence of interest in FNR triggered by the RDoC initiative is extending our understanding of well-established paradigms (cSNC) and resulting in the development of new procedures (oSNC, APRO). These new results promise to deepen our understanding of basic emotional, motivational, learning, and cognitive processes related to reward loss, as well as developing novel interventions for a variety of psychiatric disorders.
Footnotes
Grant support: M.R.P., Dean’s Opportunity Fund, College of Science and Engineering, Texas Christian University; T.A.G., National Institute on Drug Abuse R01DA060221 and R01DA047102; C.T., Ministerio de Ciencia e Innovación, Spain (PID2021-123338NB-100); M.O., Japan Society for the Promotion of Science KAKENHI (23K25753 and 24H02164) and JST FOREST Program (JPMJFR2040); Z.L., Intramural Research Program, National Institute of Mental Health (ZIAMH002881).
The authors declare no competing financial interests.
- Correspondence should be addressed to Mauricio R. Papini at m.papini{at}tcu.edu.