Abstract
Obsessive-compulsive disorder (OCD) is a debilitating illness consisting of obsessions and compulsions. OCD severity and treatment response are correlated with avoidant behaviors thought be performed to alleviate obsession-related anxiety. However, little is known about either the role of avoidance in the development of OCD or the interplay between anxiety states and avoidance behaviors. We have developed an instrumental negative reinforcement (i.e., active avoidance) paradigm in which mice must lever press to avoid upcoming footshocks. We show that mice (both sexes) can learn this task with high acquisition rates (75%) and that this behavior is largely stable when introducing uncertainty and modifying task structure. Furthermore, mice continue to perform avoidance responses on trials where lever pressing is not reinforced and increase response rates as they are maintained on this paradigm. With this paradigm, we did not find a relationship between negative reinforcement history and anxiety-related behaviors in well-established anxiety assays. Finally, we performed exploratory analyses to identify candidate regions involved in well-trained negative reinforcement using expression of the immediate early gene c-Fos. We detected correlated c-Fos expression in (1) corticostriatal regions which regulate active avoidance in other paradigms and (2) amygdala circuits known to regulate conditioned defensive behaviors.
Significance Statement
Studies in patients with OCD suggest that compulsions are performed to avoid perceived threats and modulate anxiety tied to obsessions and/or compulsions. The negative reinforcement of avoidance and alleviated anxiety could therefore be a key driver of compulsive behaviors. However, there are still outstanding questions concerning the relationship between these two behaviors and the neural circuits involved in mediating negative reinforcement. We have developed an operant negative reinforcement paradigm in mice with discrete avoid and escape behaviors that can be learned without prior reward training with high throughput (75% acquisition) with responding that persists during nonreinforced trials. However, no differences were observed between negative reinforcement versus unshocked and inescapably shocked controls, suggesting that continued negative reinforcement did not impact anxiety.
Introduction
Obsessive-compulsive disorder (OCD) is characterized by intrusive thoughts (i.e., obsessions) and persistent thoughts or actions performed to alleviate distress elicited by these obsessions (i.e., compulsions). These obsessions and compulsions can become disruptive to everyday functioning (Abramowitz et al., 2009), making OCD a debilitating illness. Nearly 60% of patients with OCD exhibit avoidant behaviors, which in turn correlate with greater obsession and compulsion severity (Starcevic et al., 2011; McGuire et al., 2012; Wheaton et al., 2018), suggesting a prominent role of avoidant behaviors in OCD. Furthermore, patients with OCD exhibit greater habitual avoidance of potential threats than control subjects and are more resistant to devaluation of avoidance responses to aversive stimuli (Gillan et al., 2014). Critically, these avoidant behaviors impact adherence to first-line therapies and reduce treatment efficacy (McGuire et al., 2012; Wheaton et al., 2018). Together, this suggests that avoidance regulation plays an important role in the development, progression, and successful treatment of OCD. However, the mechanisms underlying the acquisition and maintenance of avoidant behaviors in relation to compulsivity are unknown.
One hypothesis for the progression of OCD pathology is that aversive stimuli elevate anxiety, prompting responses to remove the aversive stimuli and avoid perceived threats. Performing these actions, and the resulting absence of harmful outcomes, negatively reinforces these avoidant behaviors (Abramowitz et al., 2009; McGuire et al., 2012; Milad and Rauch, 2012; Pauls et al., 2014). While comorbidity between OCD and anxiety disorders is found in both children and adults with OCD (Hofmeijer-Sevink et al., 2013; Vivan Ade et al., 2013), the relationship of anxiety to the development and progression of obsessions and compulsive behaviors is unknown.
Existing preclinical models for active avoidance include two-way shuttle avoidance, in which rodents learn to cross to the other side of a chamber to avoid impending footshocks, and platform-mediated avoidance, in which rodents can decide to mount a platform to avoid shocks, but at the cost of seeking and obtaining rewards. However, these models are limited by difficulties in precisely resolving the time of onset of avoidance or escape actions, as well as motion-related activity which can hamper interpretability of this behavior and its neural correlates. While operant responses (i.e., lever pressing or nose pokes) have been used to model active avoidance in mice, these tasks have historically produced low rates of acquisition in C57 strains (Kuribara and Tadokoro, 1986), a prominent mouse line for transgenic models that allows cell type- and circuit-specific dissection, or require prior shaping of operant responses with reward training (Kutlu et al., 2020). To minimize these confounds, we developed a negative reinforcement task in which mice are required to press a lever to avoid a mild footshock. Using this model, we investigated how avoidance learning impacts state anxiety in well-established anxiety-related paradigms. Previous studies suggest that avoidance performance positively correlates with elevated zero maze exploration, suggesting that low-anxiety phenotypes are associated with more active avoidance performance (Lopez-Aumatell et al., 2009; Vicens-Costa et al., 2011). Based on this literature, we hypothesized that active avoidance conditioning would reduce anxiety-like behavior but that uncertainty in avoidance outcomes during partial reinforcement of avoidance responses would increase anxiety-related behaviors. Furthermore, we assessed differences in brain-wide activity related to active avoidance history to identify regions of interest for regulating conditioned lever-press active avoidance, predicting that greater c-Fos-related activity would be observed in ventral and medial prefrontal regions of interest with known roles in regulating avoidance in other tasks (LeDoux et al., 2017; Diehl et al., 2020).
Materials and Methods
Animals
Male and female C57Bl6 mice (Jackson Laboratory; 8 weeks upon arrival) were housed in groups of 4–5 same sex mice in individually vented cages with ad libitum access to water and chow except where noted in Experiment 1. Mice were acclimated to a reverse light/dark cycle (12:12, lights on at 7 P.M.) a minimum of 12 d prior to operant training. All behavioral testing was performed in the dark cycle. All experimental procedures were approved by the Institutional Animal Care and Use Committee at the University of Pittsburgh, and all methods were carried out in compliance with the National Institutes of Health (NIH) guidelines for the care and use of laboratory animals.
Operant behavior
Experiment 1—impact of lever-press training with reward on negative reinforcement acquisition
Male and female C57Bl6 mice (n = 24) were divided into two groups: prior lever-press training for chocolate pellets (Group 1; n = 12, 6 male, 6 female) or no reward training prior to negative reinforcement training (Group 2; n = 12, 6 male, 6 female). In Group 1, mice were food restricted to 85–90% of their free-feeding bodyweight prior to operant training. Group 1 was initially trained for three sessions on a variable interval 60 s schedule (VI 60) where a chocolate-flavored grain pellet (Bio-Serv) was dispensed into a head entry port located in the center of one wall of the chamber. VI 60 was followed by 3 d of fixed ratio 1 (FR 1) schedule training where mice were given access to a single lever which they could press to obtain chocolate pellets, with a maximum of 40 rewards or 60 min per session. Prior to the final FR 1 session, Group 1 was taken off of food restriction to allow mice to recover to their free-feeding body weight. While Group 1 underwent reward training, Group 2 was habituated to operant chambers across six sessions prior to negative reinforcement training to balance exposure to operant chambers between Group 1 and Group 2. Group 2 had ad libitum access to chow during habituation. On the final FR 1 (Group 1) or habituation (Group 2) session, a black barrier was inserted into the chamber, giving mice access to 50% of the chamber (full chamber size, 8.19″ by 7.36″; restricted chamber size with barrier, 4.09″ by 7.36″). Barriers were used for all subsequent sessions to facilitate acquisition. Negative reinforcement training consisted of 15 daily sessions with five trials/session. Sessions started with 180 s of habituation to the operant chamber with a house light (covered in a red filter) illuminating the chamber and a fan on. Trials began with the extension of a retractable lever and simultaneous presentation of a white cue light above the lever. If mice lever pressed within 30 s of cue and lever onsets, the trial ended with the lever retracted and cue light extinguished, mice immediately entered a 20–40 s intertrial-interval (ITI), and the trial was coded as an avoid response. If mice failed to press within 30 s, the lever remained extended, but the cue light was extinguished concurrently with shock onset through the grid floor (0.3 mA, 2 s). Mice could press the lever at any time to escape the remaining trial shocks (up to 20 shocks/trial, 15 s intershock interval)—pressing the lever in this period initiated the ITI, and the trial was coded as an escape response. Failure to press the lever after 20 shocks were delivered ended the trial, initiated the ITI, and coded the trial as a failure (Fig. 1). Acquisition criterion was defined as receiving a maximum of 25 shocks in a session.
Outline of negative reinforcement session. A, Following 3 min of habituation, B, a trial starts with a 30 s avoid period signaled by a white cue light. If the mouse presses the lever, and C, the lever retracts and the intertrial interval (ITI) starts. D, Alternatively, the mouse fails to press to the cue and the escape period is started with the cue light extinguished, but the lever remains extended and can be pressed to escape footshock at any time to start the ITI. E, If the mouse fails to press the lever after a programmed number of shocks (training, 20 shocks; maintenance, 5 shocks), the trial is scored as a failure, the lever is retracted, and the ITI starts.
Experiment 2—impact of avoid cue versus avoid plus escape cue training on negative reinforcement acquisition
Male C57Bl6 mice were divided into two groups: training with a cue light presented during the 30 s avoidance period (Group 1, n = 12) or training with a cue light presented during the 30 s avoidance period and a 2.9 kHz pure tone presented during the escape period during which footshocks are administered (Group 2; n = 12).
Experiment 3—impact of differential reinforcement of avoidance responses on state anxiety
Male and female C57Bl6 mice (n = 64) were divided into three groups for initial training: the negative reinforcement task without exposure to shocks (no shock controls, n = 16: 8 male, 8 female); inescapable footshocks regardless of lever responses (shock controls, n = 16: 8 male, 8 female); and standard negative reinforcement training (n = 32: 16 male, 16 female). The no shock controls were presented with the lever and cue light the same as the negative reinforcement group but never received footshocks during the task. The shock controls were also presented with the lever and cue light, but presses did not retract the lever and prevent footshocks, allowing for multiple presses per trial. Shock controls only received five shocks/trial for a total of 25 shocks/session (i.e., the maximum shocks for acquisition criteria in the NR group). Following acquisition, eight mice (five females, three males) in the standard negative reinforcement training group failed to learn the task and did not progress to the next phase of the experiment. The remaining 25 mice from this group were further divided into two groups: (1) continued 100% reinforcement (n = 12, 7 male, 5 female) or (2) 50% reinforced trials (n = 12, 6 male, 6 female), which underwent daily negative reinforcement sessions with 20 total trials/session (potential to receive 5 shocks/trial) for 18 d (negative reinforcement maintenance sessions). The no shock controls ran through the 50% reinforced trial version of the modified negative reinforcement task, without receiving footshocks at any time. The shock controls went through the same task structure as the negative reinforcement groups (i.e., same exposure to the lever and cue light) but had a maximum of 10 total trials/session (50 total shocks; five shocks/trial) to receive the minimum number of shocks the 50% NR group received in each session.
Anxiety tests
Open-field test
Immediately after completing the seventh negative reinforcement (NR) maintenance session, mice were transferred to another room for an open-field test in low light conditions (∼60 lux). After acclimating to the room for a minimum of 30 min, mice were transferred into a 41× 41 cm Plexiglass open-field chamber for 30 min. Locomotor and time spent in the center of the arena activity were recorded by infrared beam breaks with data collected using Motor Monitor (Kinder Scientific) for each operant group.
Elevated plus maze
Immediately after completing the ninth NR maintenance session, mice were transferred to another room for elevated plus maze test with low light conditions (∼60 lux). After acclimating to the room for a minimum of 30 min, mice were placed in the center of an elevated plus maze (two open and two closed 30 × 5 cm arms) facing an open arm, with the two closed arms on either side of the mouse. Mouse activity was recorded with a top-down camera view (Imaging Source DMK 22AUC03) and automatically tracked using ANY-maze. The percentage of time spent in the open arms and number of open and closed arm entries was tracked for 5 min. For mice that had poor tracking, Noldus The Observer XT was used to manually score open arm time and entries, as no significant differences were found between manual and ANY-maze scored results (paired t test: t(2) = 1.547, p = 0.2620).
Light–dark test
Immediately after completing the 11th NR maintenance session, mice were transferred to another room for light–dark test with low light conditions (∼60 lux). Mice were acclimated to room conditions for a minimum of 30 min prior to testing. Light–dark chambers consisted of two chambers (10.80 × 21.59 cm) with one side enclosed and another exposed to bright light (∼300 lux). Mice were placed in the dark side of the chamber and allowed to freely explore both chambers for 20 min. Locomotor activity was scored by infrared beam breaks; data were collected using Motor Monitor (Kinder Scientific). The percentage of total time in light and dark chamber entries was tracked for each operant group.
Elevated zero maze
Immediately after completing the 13th NR maintenance session, mice were transferred to another room for elevated zero maze test with low light conditions (∼60 lux). Mice were acclimated to room conditions for a minimum of 30 min prior to testing. Elevated zero maze (60 cm diameter) consisted of a circular arena with two open sections and two closed sections. Mice were placed facing a closed portion of the maze and allowed to freely explore the maze for 5 min. Mouse activity was tracked using a top-down camera and automatically tracking using ANY-maze to determine percent of time spent in the open and closed arms and number of open and closed arm entries.
Tissue collection and immunohistochemistry
Mice were deeply anesthetized with ketamine 90 min after the start of the final NR maintenance session and transcardially perfused with 4% paraformaldehyde (PFA; Sigma-Aldrich) in 0.1 M phosphate-buffered saline (PBS). Brains were postfixed in PFA overnight and subsequently transferred to a 30% sucrose solution with 0.1% sodium azide (Sigma) for a minimum of 24 h. Brains were frozen on dry ice and sliced in 35 µm sections on a Leica cryostat. Free-floating sections were stored in PBS with 0.1% sodium azide at 4°C until used for immunohistochemistry. Tissue was washed three times with 0.1 M Tris-buffered saline (TBS; Sigma) prior to incubating in 1% hydrogen peroxide for 10 min. After additional TBS washing, sections were incubated in a blocking solution (3% normal goat serum in TBS) for 30 min before incubating in c-Fos primary antibody for 2 d (rabbit anti c-Fos, 1:1,000; Millipore Lot 3314747). Sections were then washed three times in TBS with 0.3% Triton X-100 (Sigma) for 10 min prior to incubating in goat anti-rabbit biotinylated secondary antibody (1:500; Vector) for 2 h. Sections were then washed in TBS plus Triton three times for 10 min before blocking in a tertiary ABC solution (1:100 in TBS plus Triton; Vector ABC Kit) for 1 h. Sections were washed in TBS three times for 10 min before being transferred to a 3,3′-diaminobenzidine chromogen solution (DAB 1:50 in buffer stock solution and H2O2, Sigma) for 5 min and subsequently washed three times in TBS. Tissue was mounted on charged slides and dried prior to ethanol washes (70, 80, 95, 100% ethanol; 3 min each) and washes in a histological clearing agent (xylene substitute; Sigma-Aldrich; 3 min) and coverslipped with DPX mounting agent (Sigma-Aldrich). Slides were imaged at 20× using a slide-scanning scope (Olympus, VS-200). c-Fos staining was automatically detected by pixel intensity thresholds (thresholds r:256, g:243, b:176 except striatal ROIs: r:256, g:193, b:188) and quantified using CellSense (Olympus). c-Fos density was calculated using the average c-Fos count within a 300 × 300 µm area in each hemisphere for 2–3 sections per animal (Calculation:
Statistical analyses
Two-way repeated-measures (RM) ANOVA or restricted maximum likelihood (REML) models were used to analyze differences in training between operant groups and to determine effects of training day on response types (i.e., avoid, escape, or failure) and effects on average latencies to first lever press (five session bins) with Bonferroni’s post hoc corrections used where stated if significant interactions were detected. Chi-squared (Fisher's) tests were used to detect differences in acquisition rates between operant groups and sex. Shock control press rates were calculated as the average number of presses per trial in each training session and analyzed using RM one-way ANOVA. Escape and avoid attempt press rates were calculated using average number of presses per trial during nonreinforced trials in the 50% negative reinforcement group divided by the total time for each phase of the trial (i.e., avoid period, 30 s; escape period, 70 s); these rates were compared across sessions using RM two-way ANOVA. The effects of operant training on anxiety test performance were analyzed using one-way ANOVAs with Bonferroni’s post hoc corrections used where significant effects were detected. c-Fos density across regions of interest and for each training group was analyzed using a two-way repeated-measures ANOVA. Pearson’s correlations were used for generating correlation matrices for c-Fos density across regions for each operant group with Benjamini, Krieger, and Yekutieli two-stage step-up false discovery detection at a threshold of 1% used for p value correction and discoveries. Two mice in the 100% negative reinforcement group were run through the negative reinforcement training protocol (five trials) prior to running the modified negative reinforcement protocol (20 trials) during the final operant session prior to tissue collection. As the number of avoid trials, escape trials, and shocks received for these mice fell within the range of the other subjects in the negative reinforcement group, their behavioral data (analyzed as 25 trials) and c-Fos results were included in the analyses. All statistical tests were performed in GraphPad Prism 10. Data is reported as mean ± SEM.
Results
Experiment 1: impact of lever-press pretraining on negative reinforcement acquisition
To determine if prior lever shaping was necessary for negative reinforcement acquisition, we divided an initial cohort of mice into two training groups: one group with no prior lever-press training before negative reinforcement sessions and one group with variable interval 60 s schedule training (VI 60; 3 d) followed by fixed ratio 1 (FR 1) lever training (3 d) for chocolate pellet rewards prior to negative reinforcement sessions (Fig. 2A). When examining performance in mice that acquired negative reinforcement behavior, the two training groups did not differ in shocks received across training (Fig. 2B: two-way RM ANOVA: main effect of reward training F(1,22) = 1.70, p = 0.21; main effect of session F(4.038,88.84) = 3.02, p = 0.022, ε = 0.29; interaction of reward training x session F(14,308) = 1.57, p = 0.088) or in response types across training (Fig. 2C: two-way RM ANOVA: main effect of training F(1,22) = 1.14, p = 0.30; main effect of session F(3.299,72.57) = 1.13, p = 0.35; interaction of training x session F(14,308) = 0.70, p = 0.78). The reward pretraining group had significantly more avoid response trials by the end of negative reinforcement training compared with escape or no response trials (Fig. 2D: two-way RM ANOVA: main effect of response type F(2,33) = 11.29, p = 0.0002; main effect of session F(5.21,171.8) = 0.000, p > 0.9999, ε = 0.37; interaction of session × response type F(28,462) = 8.61, p < 0.0001). Post hoc analysis detected significantly greater avoid responses in the reward pretraining group by day 12 compared with escape or no response trials; this significant difference was maintained through the final training session (Bonferroni’s post hoc days 12–15, p < 0.05 avoid vs failure and escape vs failure). Similarly, mice without reward pretraining increased avoid responses and performed fewer escape/fail responses across training (Fig. 2E: two-way RM ANOVA: main effect of response type: F(2,33) = 2.87, p = 0.071; main effect of session F(4.22,139.3) = 0.000, p > 0.9999, ε = 0.30; interaction of response type × session: F(28,462) = 11.86, p < 0.0001) with post hoc analyses finding significantly more avoid responses versus escapes and failures by day 14 (Bonferroni’s post hoc avoid vs escape and avoid vs failure days 14–15, p < 0.01). Thus, both groups demonstrated increased avoidance behavior throughout training, and the percentage of trials avoided was higher in the reward pretraining group only during the first session (Fig. 2F: two-way RM ANOVA: main effect of reward pretraining F(1,22) = 0.57, p = 0.46, main effect of session F(5.11,112.4) = 14.92, p < 0.0001, ε = 0.36, interaction of reward pretraining × session F(14,308) = 2.36, p = 0.0040; Bonferroni’s post hoc reward pretraining vs no reward pretraining day 1 p < 0.05). Acquisition rates did not differ between the groups, with one mouse failing to learn in the reward pretraining group and two mice failing to learn in the no reward pretraining group (Fig. 2G: chi-squared Fisher's exact test p > 0.9999). Given the similar acquisition rates and performance between the groups, subsequent cohorts (Experiments 2 and 3) did not undergo rewarded lever-press training before negative reinforcement training, to both streamline training and avoid the potential confound of associations between the lever and reward.
(Experiment 1) Reward pretraining does not improve negative reinforcement acquisition. A, Timeline of training for each group. Prereward (blue) began with three sessions of VI-60 head entry shaping (days 1–3), followed by three sessions of FR1 reward training prior to starting negative reinforcement training. The no reward group (green) was habituated to operant chambers over an equivalent amount of time that prereward groups spent in reward training prior to negative reinforcement training. B, Shocks received were significantly reduced across training in both prereward (n = 12) and no reward (n = 12) training groups. C, There were no differences in the number of lever responses (avoids and escapes) during negative reinforcement training between pre- and no reward groups. D, In the no reward group, avoids made up a significant percentage of responses compared with escapes and failures by day 14. E, Avoids made up a significant percentage of responses over escapes and failures by day 12 in the prereward group. F, Prereward groups made significantly more avoid responses only in the first training session versus the no reward group. G, There were no differences in acquisition rate (<25 shocks) between training groups (Fisher's test: prereward learned: 91.67%, no reward learned: 83.33%). Prereward (n = 12); no reward (n = 12). #, main effect of time; *, main effect of group; ‡, interaction of time × group. #, *, ‡, p < 0.05; ‡‡, p < 0.01; ####, p < 0.0001.
Experiment 2: impact of addition of escape cue on negative reinforcement acquisition
In addition to determining the impact of prior reward training on negative reinforcement acquisition, we tested whether including an additional conditioned stimulus during the escape period would improve learning. All mice were trained with an avoid cue consisting of a cue light illuminated over the active lever during the 30 s avoid period, which was terminated with onset of the first footshock. Half of the mice were also presented with an escape cue consisting of a 2.9 kHz pure tone delivered continuously from the first shock onset until the end of the escape period (Fig. 3A). The mice that experienced the single avoid cue (light) made significantly more avoid responses compared with escape responses/trial failures by session 14 (Fig. 3B: two-way RM ANOVA main effect of response type F(2,33) = 20.32, p < 0.0001; main effect of session F(4.32,142.7) = 1.084−3, p > 0.9999; interaction of response type × session F(28,462) = 9.68, p < 0.0001; Bonferroni’s post hoc avoid vs escape and avoid vs failure on days 14–15, p < 0.05). Providing cues at the onset of both avoid and escape periods produced selective avoidance responding by session 12 (Fig. 3C: REML model main effect of response type F(2,33) = 12.14, p = 0.0001, main effect of session F(4.62,151.4) = 1.45−3, ε = 0.33, p > 0.9999; interaction of response type × session F(28,459) = 13.60, p < 0.0001; Bonferroni’s post hoc avoid vs escape and avoid vs failure on days 12–15, p = 0.0001). However, similar decreases in footshocks were received across training between the one avoid cue and training with two cues (avoid, light; escape, sound), despite the slight decrease in the number of days required for task acquisition (Fig. 3D: REML main effect of session F(3.95,86.70) = 12.13, ε = 0.28, p < 0.0001; main effect of cue training F(1,22) = 0.24, p = 0.63; interaction of session × cue training F(14,307) = 0.52, p = 0.92). Furthermore, the percentage of trials avoided did not differ between single and dual cue training, with both training protocols producing similar increases in the percentage of trials avoided over time (Fig. 3E: REML main effect of training F(1,22) = 1.92, p = 0.18; main effect of session F(4.80,105.3) = 25.27, ε = 0.34, p < 0.0001; interaction of training × session F(14,307) = 1.15, p = 0.32). Given that mice successfully learned avoid-specific responses with a single warning cue to a similar degree as the dual light and tone cues with similar acquisition rates (chi-squared Fisher's exact test p > 0.9999), we conducted subsequent negative reinforcement sessions with only the single light cue during the avoid period.
(Experiment 2) Discretely cueing both avoid and escape periods does not improve acquisition over a single avoid period cue. A, Outline of negative reinforcement trials with a single avoid cue (left) or both avoid and escape cues (right). B, Both groups (single cue: green, n = 12; dual cue: orange, n = 12) received fewer shocks as training progressed at similar rates. C, Avoid responses increased across training with no differences between the single and dual cue groups. D, With only the avoid period cued, avoids were more frequent than escape or failure responses by day 14. E, Mice trained with both avoid and escape cues performed significantly more avoids than escapes and failures by day 12. Single cue (n = 12), dual cue (n = 12). #, main effect of time; ‡, interaction of time × group. +, ‡, p < 0.05; ###, ‡‡‡, p < 0.001; ####, p < 0.0001.
Experiment 3: negative reinforcement paradigm produces deliberate avoid responses
After optimization of the negative reinforcement paradigm, we confirmed that mice were deliberately lever pressing to avoid shocks by comparing performance to (1) a no shock control group exposed to the same cues and lever without shock delivery and (2) an inescapable shock control group exposed to the same cues and lever but administered a set number of shocks regardless of lever-press responses (Fig. 4B). When examining the proportion of trials without a lever-press response (i.e., no avoids or escapes), the negative reinforcement training group had a significantly lower percentage of no response trials compared with controls receiving either no shocks or inescapable shocks (Fig. 4C: two-way RM ANOVA main effect of training F(2,61) = 19.32, p < 0.0001; main effect of session F(6.23,380.1) = 1.44, ε = 0.45, p = 0.20; interaction of training × session F(28,854) = 1.83, p = 0.0056; Bonferroni’s post hoc negative reinforcement vs no shock and negative reinforcement vs shock controls days 1–4 and 6–15, p < 0.05). Furthermore, press latency decreased throughout training only in the negative reinforcement group as they learned to avoid and escape footshocks (Fig. 4D; REML: main effect of training F(2,61) = 24.20, p < 0.0001; main effect of session F(1.78,94.24) = 7.73, ε = 0.89, p = 0.0012; interaction of training × session F(4,106) = 5.43, p = 0.0005; Bonferroni’s post hoc negative reinforcement early vs middle and early vs late training, p < 0.0001; no shock controls and shock controls early vs middle, early vs late, and middle vs late training, ns). Consistent with the negative reinforcement group learning to attribute lever pressing to removing threats, post hoc analyses of the negative reinforcement group versus no shock controls detected decreases in press latencies in the negative reinforcement group in later training sessions (Bonferroni’s post hoc early p = 0.13; middle p = 0.0021; late p < 0.0001). While press latencies between negative reinforcement group and inescapable footshock controls were significantly different in early training, with slower press latencies in the negative reinforcement group (Bonferroni’s post hoc early p < 0.0001; middle p = 0.73; late p > 0.9999), this was explained by the differences in shocks administered per trial between the negative reinforcement group and the shock controls—i.e., shock controls received five (instead of 20) shocks per trial—reducing the total trial time available to shock controls. Furthermore, the inescapable shock group had comparable response times to the negative reinforcement group in later sessions, and the response rate was less than one press per trial, which did not significantly change across the acquisition period, suggesting spurious lever pressing across sessions (Fig. 4E; RM one-way ANOVA: F(3.57,53.60) = 0.67, p = 0.60). Consistent with our previous cohorts, responses transitioned from predominantly escape responses to avoid responses by the end of training in the negative reinforcement group (Fig. 4F: two-way RM ANOVA: main effect of response type F(1,62) = 0.45, p = 0.51; main effect of session F(6.16,382.1) = 0.76, ε = 0.44, p = 0.61; interaction of response type × session F(14,868) = 28.19, p < 0.0001; Bonferroni’s post hoc avoid vs escape days 1–3, p < 0.0001, day 4 p < 0.0003, days 14–15, p < 0.05). No sex differences were detected in acquisition of avoidance (Fig. 4G: two-way RM ANOVA: main effect of sex F(1,29) = 1.03, p = 0.32; main effect of session F(4.51,130.6) = 15.08, ε = 0.32, p < 0.0001; interaction of sex × session F(14,406) = 0.58, p = 0.88) or in acquisition rates (Fig. 4H: chi-squared Fisher's exact test p = 0.69). Together, these data suggest that our negative reinforcement paradigm produced deliberate avoidance responses at levels significantly greater than nonspecific pressing to neutral cues or unpaired footshocks and that there were no sex differences in avoidance acquisition.
(Experiment 3) Acquisition of deliberate avoid and escape responses with negative reinforcement training. A, Timeline for negative reinforcement, highlighting the 15 d training period. B, Table of operant groups. No shock controls (NS) were exposed to the same cued avoid period and had access to levers but were never administered shocks when they did not press the lever. Shock controls (SC) were exposed to the avoid cue and had access to the lever, but lever presses did not end the trial and all trial shocks (5 shocks/trial) were administered. The negative reinforcement group (NR) performed the task with cued avoid periods (30 s) and reinforced lever responses. C, The NR group had significantly more responses (avoid + escapes) throughout the training period than controls. D, Left, NR and SC groups initiated first trial presses more quickly than the NS group in middle and later sessions but did not differ in time to press between each other after early sessions. Black line at 100 s indicates maximum trial time for SCs, while green dashed line at 355 s indicates maximum trial time for NS and NR groups. Right, NR group average latency to press significantly decreased over training. E, Press rates in shock controls remained under one press/trial, suggesting no value was attributed to the lever despite relatively fast times to first press. F, NR group responses transitioned from escapes to avoids during learning. G, Avoid responses were similar in male and female mice over learning. H, There were no differences in acquisition rate between males and females. NS (n = 16), SC (n = 16), NR (n = 32; n = 16 males, n = 16 females). #, main effect of time; *, main effect of group; ‡, interaction of time × group. #,*, ‡, p < 0.05; **, ‡‡, p < 0.01; ***, ‡‡‡, p < 0.001; ####, ****, ‡‡‡‡, p < 0.0001.
Experiment 3: impact of uncertainty on avoidance behavior
Following training, mice that successfully acquired avoidance responses were further split into two groups: (1) maintenance of 100% reinforced lever pressing and (2) 50% reinforcement of lever pressing (Fig. 5A,B). In addition, the task was modified to increase trials for avoidance responses (i.e., the main behavior of interest) and to reduce opportunities for escape behaviors, by increasing to 20 total trials (from five) and reducing to five total shocks per trial (from 20), but keeping avoid duration, intershock interval, shock length, and intensity the same as training. Consistent with their performance during training, negative reinforcement groups continued to maintain high rates of lever-press responses compared with no shock and inescapable shock controls in this modified paradigm (Fig. 5C; two-way RM ANOVA: main effect of operant group F(3,52) = 37.80, p < 0.0001; main effect of session F(6.30,327.5) = 1.37, ε = 0.37, p = 0.22; interaction of operant group × session F(51,884) = 2.68, p < 0.0001). However, following the switch to the modified task structure, the 50% negative reinforcement group received more avoidable shocks than the 100% negative reinforcement group, suggesting more response variability in avoiding or escaping during reinforced trials (Fig. 5D: two-way RM ANOVA: main effect of operant group F(1,22) = 4.61, p = 0.043). As they were maintained on a 50% reinforcement rate, this group improved their performance and received fewer avoidable shocks in later sessions. The 100% negative reinforcement group also received fewer shocks over time, avoiding nearly all potential shocks by the final session (Fig. 5D: main effect of session F(3.37,74.13) = 5.16, ε = 0.20, p = 0.0019; interaction of operant group × session F(17,374) = 2.37, p = 0.0018). While the 50% reinforced group avoided less than the 100% reinforced group on reinforced trials, both the 50 and 100% groups increased avoidance responses across the 18 maintenance sessions (Fig. 5E: two-way RM ANOVA: main effect of operant group F(1,22) = 6.14, p = 0.021; main effect of session F(3.97,87.40) = 9.78, ε = 0.23, p < 0.0001, interaction of operant group × session: F(17,374) = 2.18, p = 0.0045). This suggests that changing task contingencies and task structure altered performance in early sessions by reducing avoid responses but that mice restabilized and continued to avoid after several sessions.
(Experiment 3) Continued negative reinforcement yields persistent responding. A, Timeline of the paradigm highlighting the 18 sessions of negative reinforcement maintenance, with trials increased to 20/session with five possible shocks/trial. B, Group breakdown of maintenance phase. Mice that acquired in the NR group (n = 24) were divided into continued 100% reinforcement (NR 100%) or pseudorandomly reinforced trials at a 50% reinforcement rate (NR 50%). C, NR groups continued to have nearly 100% response rates with lower press rates in the control groups. D, Probabilistic reinforcement reduces performance compared with 100% reinforcement with more avoidable shocks received in 50% groups. E, Differences in reinforced avoid responses between NR groups occur early in maintenance but stabilize over subsequent sessions. F, Mice continue to attempt avoid and escape responses on unreinforced trials with increasing rates. G, On average, press rates on unreinforced trials are stable within sessions. H, Press latency continues to decrease across sessions, irrespective of previous trial outcomes. NS(n = 16), SC (n = 16), 100% NR (n = 12), 50% NR (n = 12). #, main effect of time; *, main effect of group; ‡, interaction of time × group. #, *, #, *, p < 0.05; ‡‡, p < 0.01; ‡‡‡, p < 0.001; ####, p < 0.0001.
Experiment 3: persistence of lever pressing during unreinforced trials
To examine whether avoid and escape responses persisted when outcomes were uncertain, the 50% group was given nonreinforced trials for half of the session, during which mice could continue to press the lever with no consequence. This contingency change produced persistent lever pressing in the 50% reinforcement group, with increasing press rates during both the avoid and escape periods (Fig. 5F: two-way RM ANOVA: main effect of response type F(1,22) = 0.86, p = 0.36; main effect of session F(2.76,60.78) = 4.14, ε = 0.16, p = 0.012; interaction of response type × session F(17,374) = 0.97, p = 0.49). Avoid and escape attempts were stable within session, with no change in average response rates across nonreinforced trials (Fig. 5G: two-way RM ANOVA of average press rates per trial: main effect of response type F(1,22) = 1.74, p = 0.20; main effect of trial F(1.35,29.73) = 0.97, p = 0.36; interaction of response type × trial F(9,198) = 1.78, p = 0.074), suggesting that within-session information about nonreinforced responding did not impact their tendency to attempt avoid or escape presses. In addition, press latencies were not impacted by the previous trial outcome (i.e., reinforced vs not reinforced trials) but overall decreased across sessions (Fig. 5H: two-way RM ANOVA: main effect of previous trial F(1,22) = 0.055, p = 0.82; main effect of session F(5.08,111.7) = 6.33, ε = 0.30, p < 0.0001, interaction of previous trial × session F(17,374) = 0.70, p = 0.81), further suggesting that avoid responses were insensitive to trial outcomes and persisted despite introducing inescapable threats. Together, these data suggest that negative reinforcement performance restabilized following task modifications, with lever responses persisting despite no utility in nonreinforced trials.
Experiment 3: impact of negative reinforcement training on anxiety-like behavior
After 1 week of additional training at either 100 or 50% reinforcement, mice in all groups underwent a battery of anxiety tests following the negative reinforcement paradigm in the following order (anxiety-testing sessions every other day): open field, elevated plus maze (EPM), light–dark test, and elevated zero maze (EZM; Fig. 6A). No differences were found in the amount of time exploring the center of the open field (Fig. 6B: one-way ANOVA F(3,52) = 0.66, p = 0.58). Similarly, the percentage of time spent exploring brightly lit chambers in the light–dark test (Fig. 6C: one-way ANOVA: F(3,52) = 0.29, p = 0.83) or open arms of the EZM (Fig. 6D: one-way ANOVA: F(3,52) = 0.79, p = 0.50) did not differ across groups. However, mice with no shock history spent more time exploring the open arms of the EPM than inescapable shock controls (Fig. 6E: one-way ANOVA: F(3,52) = 3.26, p = 0.029; Bonferroni’s post hoc NS vs SC, t(52) = 3.05, p = 0.022), demonstrating an impact of the negative stimulus (inescapable shock) on anxiety-like behaviors. To determine if there was significant variability in anxiety-like behaviors between tests, anxiety test results were correlated across the OF, EPM, LD, EZM (Fig. 6F). EPM results were significantly correlated with LD and EZM results, whereas center exploration in the open field did not correlate with other anxiety task measures [Fig. 6F: Pearson's correlation with significant results following false discovery correction (p < 0.005) bolded]. These data indicate that experience with extensive negative reinforcement did not impact anxiety-like behavior and that this could not be explained by general variability in test-specific measures of anxiety.
(Experiment 3) Conditioned avoidance does not impact anxiety-related behavior in innate approach–avoidance tests. A, Highlighted time point in negative reinforcement maintenance where anxiety test sessions (green triangles—numbered schematics indicate order of anxiety tests: 1, OF; 2, EPM; 3, LD; and 4, EZM) followed operant sessions (gray). B, Percent of time spent in the center of an open field arena did not differ between control and negative reinforcement groups. C, Percent of time spent in the lighted portion of the light–dark chamber did not differ between groups. D, Percent of time in the open tracks of the elevated zero maze did not differ between groups. E, NS controls spent significantly more time exploring the open arms of the elevated plus maze compared with SC, but not compared with NR groups. F, EPM correlated with Light Dark and EZM, but these tests did not correlate with OF. NS (n = 16), SC (n = 16), 100% NR (n = 12), 50% NR (n = 12). *, p < 0.05, Pearson’s correlations: bold values for significance following false discovery correction p < 0.005.
Experiment 3: immediate early gene expression following negative reinforcement training
Finally, we quantified brain-wide c-Fos activation during a final negative reinforcement session in well-trained mice (Fig. 7A–C). c-Fos density generally differed across control and negative reinforcement groups and regions, but no interaction was found for c-Fos density in specific regions based on operant history (Fig. 7D: two-way ANOVA: main effect of operant history F(3,724) = 23.29, p < 0.0001; main effect of region F(16,724) = 47.30, p < 0.0001; interaction of operant history × region F(48,724) = 0.76, p = 0.89); post hoc analysis did show significant differences between control groups versus negative reinforcement groups in c-Fos density but did not detect differences between the NS and SC or between the 50 and 100% NR groups (Bonferroni’s post hoc NS vs 50%, NS vs 100%, SC vs 50%, SC vs 100% p < 0.0001; NS vs SC and 50 vs 100%, p > 0.999 and p > 0.97, respectively). To determine if there was c-Fos activity that was correlated between specific regions in negative reinforcement, we analyzed correlation matrices across all ROIs (Fig. 8; Extended Data Tables 8-1–8-4: corrected p values for no shock, shock controls, 100% reinforcement, 50% reinforcement, respectively; Extended Data Tables 8-5–8-8 correlations for no shock, shock controls, 100%, and 50% groups). After FDR correction, shock controls had significant c-Fos correlations between the basolateral amygdala and central amygdala (Extended Data Tables 8-2, 8-6), while the 100% reinforced avoidance group demonstrated c-Fos-related activity between (1) medial prefrontal cortices and nucleus accumbens subregions and (2) bed nucleus stria terminalis, central amygdala, and brainstem ROIs (Extended Data Tables 8-3, 8-7). No significant correlations were found in the 50% reinforcement group (Extended Data Tables 8-4, 8-8).
(Experiment 3) c-Fos activity differences following negative reinforcement. A, Timeline highlighting when tissue was collected for c-Fos quantification. B, Representative ROIs for c-Fos quantification: mOFC, lOFC, Cg, PL, IL, NAcC, NAcSh, DLS, DMS, dBNST, vBNST, CeA, BLA, PVN, VTA, dPAG, vPAG. C, Representative ROIs collected at 20× with c-Fos staining for each region. D, c-Fos density in ROIs differed by group and region, but no group × region interaction was detected. mOFC, medial orbitofrontal cortex; lOFC, lateral orbitofrontal cortex; Cg, anterior cingulate cortex; PL, prelimbic cortex; IL, infralimbic cortex; NAcC, nucleus accumbens core; NACs, nucleus accumbens shell; DLS, dorsolateral striatum; DMS, dorsomedial striatum; dBNST, dorsal bed nucleus of the stria terminalis; vBNST, ventral bed nucleus of the stria terminalis; CeA, central amygdala; BLA, basolateral amygdala; PVN, paraventricular nucleus; VTA, ventral tegmental area; vPAG, ventral periaqueductal gray; dPAG, dorsal periaqueductal gray. #, main effect of region; *, main effect of operant group. ####, ****, p < 0.0001. Sample sizes indicated in Extended Data Table 7-1
Table 7-1
Breakdown of number of subject samples collected for c-Fos analysis for each region. Data presented in Figure 7D. Subjects were removed for c-Fos analysis due to issues either with tissue collection or processing. Download Table 7-1, DOCX file.
(Experiment 3) Correlated c-Fos between cortical, striatal, amygdalar, and midbrain regions during negative reinforcement. A, Region correlations in no shock controls. B, Region correlations in shock controls. * indicates correlations meeting false discovery correction (p values <0.00054). C, Region correlations in 100% negative reinforcement. * indicates correlations meeting false discovery correction (p value <0.00062). D, Region correlations in 50% negative reinforcement. No correlations were significant following false discovery correction. Corrected p values are displayed for no shock, shock controls, 100% reinforced, and 50% reinforced groups in Extended Data Tables 8-1–8-4, respectively, and corresponding correlation values in Extended Data Tables 8-5–8-8.
Table 8-1
Corrected p-values correlation matrix of c-Fos across regions of interest in the No Shock Control Group. Values are corrected p-values corresponding to Figure 8A. from two-stage linear step-up Benjamini, Krieger and Yekutieli. Green bolding indicates correlations that were detected as significant following two-step false discovery correction (threshold: p-values less than 0.00054). Download Table 8-1, XLSX file.
Table 8-2
Corrected p-values correlation matrix of c-Fos across regions of interest in the Shock Control Group. Values are corrected p-values corresponding to Figure 8B from two-stage linear step-up Benjamini, Krieger and Yekutieli. Green bolding indicates correlations that were detected as significant following two-step false discovery correction (threshold: p-values less than 0.00054). Download Table 8-2, XLSX file.
Table 8-3
Corrected p-values correlation matrix of c-Fos across regions of interest with 100% Reinforcement of Avoidance. Values are corrected p-values corresponding to Figure 8C from two-stage linear step-up Benjamini, Krieger and Yekutieli. Green bolding indicates correlations that were detected as significant following two-step false discovery correction (threshold: p-values less than 0.00062). Download Table 8-3, XLSX file.
Table 8-4
Corrected p-values correlation matrix of c-Fos across regions of interest with 50% Reinforcement of Avoidance. Values are corrected p-values corresponding to Figure 8D from two-stage linear step-up Benjamini, Krieger and Yekutieli. No p-values were marked as discoveries. Download Table 8-4, XLSX file.
Table 8-5
Correlation matrix of c-Fos across regions of interest in the No Shock Control Group. Values are Pearson correlations corresponding to Figure 8A and Extended Table 8.1 p-values. Green bolding indicates correlations that were detected as significant following two-step false discovery correction. Download Table 8-5, XLSX file.
Table 8-6
Correlation matrix of c-Fos across regions of interest in the Shock Control Group. Values are Pearson correlations corresponding to Figure 8B and Extended Table 8.2 p-values. Green bolding indicates correlations that were detected as significant following two-step false discovery correction. Download Table 8-6, XLSX file.
Table 8-7
Correlation matrix of c-Fos across regions of interest with 100% Reinforcement of Avoidance. Values are Pearson correlations corresponding to Figure 8C and Extended Table 8.3 p-values. Green bolding indicates correlations that were detected as significant following two-step false discovery correction. Download Table 8-7, XLSX file.
Table 8-8
Correlation matrix of c-Fos across regions of interest with 50% Reinforcement of Avoidance. Values are Pearson correlations corresponding to Figure 8D and Extended Table 8.4 p-values. Download Table 8-8, XLSX file.
Discussion
We established an operant negative reinforcement training paradigm which produces deliberate avoidance responses within 100 trials with high throughput at an 80% acquisition rate (45/56 mice across all experiments) and stable performance over several weeks, consistent with timeframes for shuttle avoidance training (Peeler, 1987). This paradigm does not require prior lever-press shaping with rewards, removing any confounds of positive reinforcement history on the acquisition of this behavior, nor does it require prior escape response shaping (Oleson et al., 2012). Our paradigm produces greater acquisition than lever avoidance paradigms in previous studies using C57/Bl6 strains (Kuribara and Tadokoro, 1986) as well as trains mice to criteria in shorter timeframes than other instrumental negative reinforcement tasks which also include prior reward training (Kutlu et al., 2020). Our protocol focuses on signaling the avoid period and provides response feedback, unlike Sidman avoidance (Sidman, 1953) or protocols that only signal correct responses (Rescorla, 1968). Furthermore, this task employs avoids and escapes to remove threats within a trial, unlike Sidman tasks which are self-paced and delay, rather than remove, shock onsets (Sidman, 1953; Ayres et al., 1974). Lever responses also provide temporally specific resolution of avoid and escape response onsets, which can be difficult to delineate with shuttle avoidance since movement is the response metric (Oleson and Cheer, 2013). In addition, responses are deliberate in comparison with shock-naive and inescapable shocked controls exposed to the same cues and the lever, making this a useful model for monitoring regional and circuit activity specifically associated with active avoidance. Specifically, our data show a transition from escape to avoid response types and demonstrate that these responses only become conditioned in the experimental group. Presenting cues and levers with no pairing to aversive stimuli (no shock, NS), or presenting cues and levers with no tie to shock removal (shock control, SC), does not produce the same rates of response, suggesting that these responses are not due to general changes in stimuli presentation and feedback (Bolles, 1970). However, we did not specifically examine individual variability in acquisition and how differences in lever responses and acquisition resulted from failure to override other defensive responses (i.e., freezing or flight; Bolles, 1970). Future studies examining individual learning strategies will yield insight into how species-specific defensive behaviors impact acquisition.
This protocol produces continued avoidance-seeking in unreinforced trials, which could be a valuable phenomenon for understanding persistent avoidance, a phenotype with relevance to psychiatric disorders. The increase in avoidance attempts was specific to the 50% reinforcement group, suggesting a specific conditioned response generated by our protocol, and is in line with increased avoid responses during intermittent, unavoidable shocks in Sidman avoidance in rats (McIntire et al., 1968) and primates (Sidman et al., 1957). However, future experiments are necessary to determine if this paradigm produces continued avoidance under extinction conditions and whether this behavior can persist despite negative consequences, which are critical features of maladaptive avoidance (Ball and Gunaydin, 2022).
Two-factor theories implicate direct associations between avoidance actions and alterations to state anxiety (Mowrer, 1951). Indeed, active avoidance studies in human participants demonstrate an association between arousal/anxiety via skin conductance and the CS signaling avoidance (Lovibond et al., 2008). Additionally, common nodes for state and trait anxiety have been shown to reliably classify healthy versus OCD anxiety scores in functional connectivity studies of healthy participants versus patients with OCD (Takagi et al., 2018). However, our results suggest that anxiety-related measures of approach–avoidance in novel environments are not altered by active avoidance acquisition. This is consistent with findings suggesting that performance on anxiety tests is highly sensitive to assay selection. While some studies have shown increased anxiety-like behavior following fear conditioning using the EPM (Korte et al., 1999) and light–dark assay (Bruijnzeel et al., 2001), others show minimal or no effects on open-field center exploration following fear conditioning (Daviu et al., 2014). Furthermore, following fear memory retrieval in either predictable or unpredictable fear conditioning protocols, no group differences are observed in EPM exploratory behaviors several minutes after CS presentation (Seidenbecher et al., 2016). In addition, fear potentiation either has only transient effects on EPM exploration, producing anxiogenic phenotypes lasting up to 90 min following threat re-exposure (Korte et al., 1999) or has no impact on EPM performance (Hilton et al., 2023). Similarly, no differences in anxiety-like behavior measured via EPM exploration have been observed in naive rats versus rats that underwent two-way shuttle avoidance (Korte et al., 1999). Thus, as our tests of anxiety-like behavior were conducted several hours following operant tests, we may not have observed a transient impact of negative reinforcement or inescapable shock on anxiety-like behavior. Our results suggest that persistent avoidance exposure and chronic inescapable footshock stress do not produce long-lasting impacts on state anxiety behavior or produce overall anxiety differences. It will therefore be important to use alternative measures for anxiety-like responses within the context of negative reinforcement procedures or examine anxiety-related behaviors immediately following our negative reinforcement paradigm to further explore the potential relationship between these phenomena.
While changes in state anxiety are hypothesized to prompt avoidance behaviors, trait anxiety is also known to modulate avoidance responses and generalized fear (Haddad et al., 2012; Pittig et al., 2018; Klein et al., 2020). While our data do not suggest that state anxiety was differentially changed after learning conditioned avoidance, we did not directly investigate how trait anxiety may impact avoidance acquisition by performing baseline anxiety measures prior to the active avoidance task. However, the EPM, LD, and EZM had significant correlations between one another, suggesting stable anxiety-related behavior across several days, which did not correlate with shocks received in sessions prior to anxiety tests or to avoidance responses. This was surprising, given previous work in heterogeneous rat strains showing individual variability in anxiety phenotypes in the EZM that were associated with subsequent shuttle avoidance response (Lopez-Aumatell et al., 2009; Vicens-Costa et al., 2011). This discrepancy could be explained by the fact that our mice were well trained on the avoidance task prior to undergoing any anxiety tests, making individual variability in negative reinforcement performance unlikely to be leading to within-group differences in anxiety tests that could obscure between-group differences. In contrast, the studies in rats compared anxiety measures with performance from single session-training, likely resulting in higher performance variability. Additional experiments are therefore needed to determine whether baseline anxiety is associated with or predictive of active avoidance acquisition, if individual variability in avoidance performance is associated with state or trait anxiety, and whether anxiety phenotypes impact the persistence and extinction of conditioned avoidance.
To determine potential regions of interest specifically involved in regulating active avoidance behaviors, controls and negative reinforcement groups were perfused following the active avoidance paradigm. Broadly, we saw increased c-Fos only in the negative reinforcement group, consistent with studies demonstrating general increases in c-Fos activation in regions involved in HPA axis regulation, fear learning, habit, and defensive response selection in escapable footshock paradigms (Coco and Weiss, 2005; Jiao et al., 2015), but inconsistent with other studies which found higher c-Fos activation with inescapable shock compared with escapable shock, or no differences based on stressor controllability (Liu et al., 2009). While we found general increases in c-Fos in both the 50 and 100% negative reinforcement group, surprisingly, there was significantly less c-Fos with inescapable shock. Furthermore, the c-Fos density did not differ between the control groups, contrary to previous studies of escapable versus inescapable stress with no shock controls (Coco and Weiss, 2005; Liu et al., 2009; Worley et al., 2020). This discrepancy, particularly with regard to the lower than predicted c-Fos activation in our shock control group, could be from long-term exposure to repeated stress (in this case, repeated footshock), which has been shown to produce habituation of c-Fos activation in chronic restraint stress paradigms (Stamp and Herbert, 1999; Shoji and Mizoguchi, 2010) and with repeated footshock (Li and Sawchenko, 1998; Trentani et al., 2003). To identify potential circuits mediating negative reinforcement, we performed exploratory analyses of c-Fos-related activity between regions, which revealed several notable associations. BLA→CeA c-Fos associations were detected in inescapable shock controls, consistent with the role of BLA→CeA circuits in the development of conditioned fear (Hartley et al., 2019; Massi et al., 2023; Penzo and Moscarello, 2023) and of the opposing roles of these regions in mediating passive versus active fear responses (Choi et al., 2010; LeDoux et al., 2017). Surprisingly, no correlations were found with BLA in the negative reinforcement groups, despite the fact that it is a critical hub for active avoidance expression (Choi et al., 2010; Lazaro-Munoz et al., 2010; Bravo-Rivera et al., 2014; Bravo-Rivera et al., 2015). Specifically, in other avoidance paradigms, dmPFC→BLA circuits encode avoidance responses (Martinez-Rivera et al., 2019; Jercog et al., 2021; Kajs et al., 2022) and are necessary for avoidance (Diehl et al., 2020; Jercog et al., 2021), and NAcSh→BLA circuit disconnection impairs active avoidance expression (Ramirez et al., 2015). Our extensive task training prior to c-Fos quantification could explain the absence of significant BLA associations with other c-Fos regions, as BLA function seems to be involved in learning, whereas maintenance with overtraining is BLA independent (Lazaro-Munoz et al., 2010; LeDoux et al., 2017).
Notably, several significant regional c-Fos correlations were detected in 100% reinforced active avoidance, including mPFC, nucleus accumbens, BNST, central amygdala, and the periaqueductal gray, regions implicated in regulating fear expression and extinction (Sangha et al., 2014; Giustino and Maren, 2015; Piantadosi et al., 2018; Diehl et al., 2020), anxiety (Tovote et al., 2015; Lebow and Chen, 2016) and innate defensive responses (Lebow and Chen, 2016; Shackman and Fox, 2016; Murty et al., 2023). Infralimbic and nucleus accumbens shell circuits are involved in fear expression (Goode and Maren, 2019) and are individually necessary for avoidance expression (Piantadosi et al., 2018; Capuzzo and Floresco, 2020). The BNST and CeA are highly interconnected and thought to be active in response to distinct types of fear (i.e., distal vs proximal threats), acting as a threat surveillance system (Tovote et al., 2015; Lebow and Chen, 2016; Shackman and Fox, 2016; Shackman et al., 2017) with cell type-specific circuits that regulate contextual and cued fear memory (Zhu et al., 2023). In addition, BNST→vPAG circuits are modulated by fear cues (Kaouane et al., 2021). Surprisingly, c-Fos activity was highly correlated between dorsal bed nucleus and dorsomedial striatum specifically in 100% negative reinforcement. To our knowledge, how this circuit is involved in threat processing and negatively valenced behaviors has not been investigated and could be a novel target for understanding how active avoidance behaviors are learned and selected. Overall, the detected associations between these regions in negative reinforcement are consistent with prior work in fear conditioning, active avoidance learning, and anxiety. However, these data only provide a static snapshot of differential activity patterns between groups and may also be limited by ceiling effects in c-Fos activation to shocks, obscuring any specific regional differences in c-Fos activation between control and negative reinforcement groups. Future experiments using temporally precise imaging and manipulation techniques will be necessary to determine the relationship between activation of these regions, avoidance cues, and task responses to fully map the directionality, encoding, and necessity of these circuits for acquisition and performance of active avoidance.
Taken together, our studies demonstrate a high-throughput instrumental negative reinforcement task in mice. This task can be trained within a 15 d period and produces stable avoidance over a 1 month period. In addition, several candidate regions were identified for future experiments examining neural circuits mediating instrumental negative reinforcement learning and persistence. Future experiments will also be necessary to fully characterize how this behavior is or is not related to trait anxiety.
Footnotes
This work was supported by the Foundation for OCD Research. EAC was supported by T32 MH016804-42. Schematics were created using BioRender.com. We thank Stormy Green for assistance with tissue processing and the Ahmari lab for helpful discussions about the data included in this manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Susanne Ahmari at ahmarise{at}upmc.edu.
