Abstract
The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.
SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.
Introduction
Real-world decisions rarely involve choosing between unambiguously favorable versus unfavorable options. Often, options must be evaluated along multiple dimensions that incorporate an evaluation of the rewards themselves as well as the actions or efforts to procure them (Skvortsova et al., 2014). For example, we typically make decisions between options in comparison, where one outcome may be more costly (i.e., more effortful) yet more preferred than the other.
Outside of the striatum, which has been the major region of study in such effort-based decision-making studies (Salamone et al., 1991, 1994, 2003, 2007; Cousins et al., 1996; Nowend et al., 2001; Ghods-Sharifi and Floresco, 2010), the ACC is also involved in the evaluation of both physical and cognitive effort costs (Walton et al., 2003; Schweimer and Hauber, 2005; Floresco and Ghods-Sharifi, 2007; Hillman and Bilkey, 2010, 2012; Cowen et al., 2012; Hosking et al., 2014; Winstanley and Floresco, 2016). Neurons in both rat and primate ACC signal value during economic decision-making (Lapish et al., 2008; Azab and Hayden, 2017; Hunt and Hayden, 2017; Mashhoori et al., 2018). Neural responses in this region also track trial-by-trial outcomes of choices (Procyk et al., 2000; Shidara and Richmond, 2002; Seo and Lee, 2007; Akam et al., 2017), reward history (Bernacchia et al., 2011), reward prediction errors (Hayden et al., 2009; Kennerley et al., 2011; Hyman et al., 2017), and counterfactual options (“rewards not taken”) (Hayden et al., 2009; Mashhoori et al., 2018). Indeed, both anatomic and functional evidence supports the idea that ACC activity supports representations of reward value and action outcomes (Shenhav et al., 2013; Heilbronner and Hayden, 2016).
A paradigm that involves selecting between qualitatively different reinforcers may closely model human decisions where we encounter options that are more/less preferred, not more/less of the same reward identity (Salamone et al., 1991, 2007, 2017; Cousins and Salamone, 1994; Nowend et al., 2001; Randall et al., 2012, 2014a,b; Nunes et al., 2013; Yohn et al., 2016a,b,c). The majority of the seminal rodent studies probing ACC in effort-based choice (Walton et al., 2002, 2003; Floresco and Ghods-Sharifi, 2007; Hauber and Sommer, 2009; Winstanley and Floresco, 2016) have used traditional pharmacological, lesion, and electrophysiological approaches, so a fine-grained analysis involving cell type-specific, temporally restricted targeting of ACC in effort choice has not yet been reported.
Here, we tested the effects of inhibitory (hM4Di, or Gi) and excitatory (hM3Dq, or Gq) designer receptors exclusively activated by designer drugs (DREADDs) (Armbruster et al., 2007; Alexander et al., 2009; Roth, 2016) in ACC on the same effortful choice task that we previously probed following lesions (Hart et al., 2017) and pharmacological inactivations (Hart and Izquierdo, 2017). Briefly, our task required rats to choose between working for a preferred reward (sucrose) versus consuming a concurrently and freely available, but less preferred reward (standard chow). We assessed the role of ACC on (1) progressive ratio (PR) lever-pressing for sucrose pellets (i.e., general motivation), and (2) PR lever-pressing with choice (PRC) of a freely available alternative (i.e., effortful decision‐making: choosing between working for sucrose pellets vs concurrently available laboratory chow). We also tested the effects of ACC inhibition and excitation on the choice between sucrose pellets versus chow when these reinforcers were both freely available (i.e., “free choice”). Finally, in a separate cohort of animals, we looked for value coding cells in ACC during effortful behavior and reward consumption phases. For this, we used in vivo miniaturized fluorescence microscopy (University of California at Los Angeles “miniscopes”) (Ghosh et al., 2011; Aharoni et al., 2019) to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior in separate sessions where there was a choice (PRC) versus when there was no choice (PR).
Materials and Methods
Subjects
Subjects were N = 44 adult male Long-Evans rats (n = 12 Gi DREADD experiment, n = 12 Gq DREADD experiment, n = 10 GFP [null virus] control experiment, n = 4 calcium imaging experiment, n = 6 acute slice recording for validation of DREADDs). Rats were obtained from Charles River Laboratories, were postnatal day 60 at the time of arrival to the University of California at Los Angeles vivarium, and were singly housed for all phases of experiments, with the exception of the acclimation period and handling, during which they were pair-housed. All rats were handled for 10 min in pairs for 5 d after a brief acclimation period (3 d). Rats weighed an average of 309.1 g at the beginning of experiments. Three subjects were not included in the final data analyses: 1 from Gi experiment and 1 from Gq experiment due to unilateral (not bilateral) viral expression, and 1 from the GCaMP experiments due to poor stability of imaging over days.
The vivarium was maintained under a 12/12 h reverse light cycle at 22°C, and lab chow and water were available ad libitum before behavioral testing. Rats were food restricted 1 d before behavioral testing to ensure motivation to work for rewards. Given the sensitivity of the behavioral tests on motivation, special care was taken to maintain consistent food rations throughout the experiment. This was 12 g/d at the beginning of testing but then decreased to 8 g/d at the beginning of the choice phase (details below). Rats were monitored every other day for their body weight, and were never permitted to drop below their 85% free feeding baseline weight. Training and testing were conducted during the early portion of the dark cycle (∼0800-1200 h). Experiments were conducted 5-7 d per week, and rats were fed once daily on weekends (12 g) when testing was not conducted. All procedures were reviewed and approved by the Chancellor's Animal Research Committee at the University of California, Los Angeles.
Food restriction
One day before behavioral testing began, rats were singly housed, the amount of chow given to each rat was reduced to 12 g/d, and rats were given ∼10 sucrose pellets (45 mg dustless precision sucrose pellets; Bio-Serv) in their home cage to acclimate them to the food rewards. Rats were maintained on 12 g of food daily and were each fed within 30 min of completing the daily testing. Once rats progressed to the choice task, they were given 8 g of chow per d, in addition to the food they consumed during testing. At the time of death, rats weighed an average of 356.5 g.
Stereotaxic surgery
General surgical procedures were the same as those recently published (Hart and Izquierdo, 2017). Rats were anesthetized with isoflurane (5% induction, 2% maintenance in 2 L/min O2). Burr holes were drilled bilaterally on the skull for insertion of 26-gauge guide cannulae (PlasticsOne), after which 33-gauge internal cannulae (PlasticsOne) were inserted. Rats were infused with 0.5 μl of virus at a flow rate of 0.1 μl/minute, and injectors were subsequently left in place for 5 additional minutes to allow for diffusion of solution. In the Gi experiment, the virus used was AAV8-CaMKIIα-hM4D(Gi)-mCherry (Addgene, viral prep #50477-AAV8). In the Gq experiment, the virus used was AAV8-CaMKIIα-hM3D(Gq)-mCherry (Addgene, viral prep #50476-AAV8). In the GFP (null virus) control, the virus used was AAV8-CaMKIIα-eGFP (Addgene). In the imaging experiment, the virus used was AAV9-CaMKIIα-GCaMP6f (Addgene). The coordinates used for the guide cannulae targeting area 24 of ACC (van Heukelum et al., 2020) in the Gi, Gq, and GFP control experiments were as follows: AP 2.0 mm, ML ±0.7 mm, DV −1.9 mm from bregma. Four of 12 rats in the Gi experiment received infusions in more anterior ACC (area 32) to compare with other laboratory experiments on decision confidence (Stolyarova et al., 2019), at AP 3.7 mm, ML ±0.8 mm, DV −1.6 mm from bregma. Since no differences emerged from this differential targeting, we combined the ACC groups. In the imaging experiment, coordinates were as follows: AP 2.0 mm, ML ±0.7 mm, DV −1.4 mm (0.5 μl) from bregma, and a second 0.5 μl bolus of virus was injected at DV −0.9 mm. Injectors extended 1 mm beyond the tip of the cannula. Following the 5 min diffusion time, the cannulae and injectors were removed, incisions were stapled closed, and the rats were placed on a heating pad and kept in recovery until ambulatory before being returned to the vivarium.
Three days following viral infusions in the subset of rats receiving AAV9-CaMKIIα-GCaMP6f (for calcium imaging), rats were implanted with 1.8-mm-diameter 0.25 pitch GRIN lenses (Edmund optics part 64-519). Following similar surgical procedures, four anchor screws were secured to the skull, after which a 2.0 mm craniotomy was drilled 0.2 mm lateral to the center of the viral infusion hole. The dura was cleared and ∼0.5 mm of tissue was aspirated, after which the lens was placed 2.0 mm ventral from the surface of the skull and secured in place with cyanoacrylate glue and bone cement. The lens was protected with Kwik-Sil (World Precision Instruments).
Postoperative care for all rats consisted of five daily injections of carprofen (5 mg/kg, s.c.) and oral sulfamethoxazole/trimethoprim solution. Two to 3 weeks following lens implantation, a small aluminum baseplate was attached to the animal's head and secured with bone cement. The exposed lens was cleaned with 100% ethanol, and the baseplate was secured in a position where cells in the FOV and vasculature were in focus. A 3D printed cover was secured to the baseplate with an anchor screw at all times when recording was not occurring. Rats were allowed a 5 d free feeding recovery period following viral infusion (Gi, Gq, and GFP experiments) or GRIN lens implantation (imaging experiment) after which they were food-restricted and behavioral testing began.
Apparatus
All behavioral testing was conducted in chambers outfitted with a house light, internal stimulus lights, a food-delivery magazine, and 2 retractable levers positioned to the left and right of the chamber wall, opposite the magazine. All hardware was controlled by a PC running Med-PC IV (Med Associates).
Miniaturized microscope data collection
Microscopes were custom built according to plans available at www.Miniscope.org. Images were acquired with a CMOS imaging sensor (Labmaker) attached to custom data acquisition (DAQ) electronics via a 1.5 mm coaxial cable. Data were transferred to a PC running custom-written DAQ software over Super Speed USB. DAQ software was written in C++ and used Open Computer Vision (OpenCV) for image acquisition; 480 × 752 pixel images were acquired at 30 Hz and written to .avi files. DAQ software simultaneously recorded and time-stamped behavioral data and image data, allowing for offline alignment. All hardware design files and assembly instructions are available at www.Miniscope.org. Calcium signals were extracted using modified constrained non-negative matrix factorization scripts in MATLAB (The MathWorks, version 2016) (Pnevmatikakis et al., 2016; Zhou et al., 2018).
Lever press training
Rats were first given fixed-ratio −1 training where each lever press earned a single sucrose pellet (Bioserv). They were kept on this schedule until they earned at least 30 pellets within 30 min. Following this, rats where shifted to a PR schedule where the required number of presses for each pellet increased according to the following formula:
where 
is equal to the number of presses required on the ith ratio, rounded to the nearest whole number (Richardson and Roberts, 1996), after 5 successive schedule completions. No timeout was imposed. Rats were tested on the PR schedule until they earned at least 30 pellets on any given day (∼5 d) within 30 min. Upon meeting the PR performance criterion, a ceramic ramekin containing 18 g of lab chow was introduced (modified from Randall et al., 2012) during testing. Rats were free to choose between consuming freely available but less preferred chow or lever-pressing for preferred sucrose pellets. Rats (Gi, Gq, and GFP experiments) were given at least 5 choice testing sessions before clozapine-N-oxide (CNO) or vehicle (VEH) injections began (details below).
Drug treatment during different types of test sessions
Rats in the Gi, Gq, and GFP control experiments were given either VEH (95% saline, 5% DMSO, 1 ml/kg) or CNO (3.0 mg/kg i.p. in 95% saline, 5% DMSO, 1 ml/kg) (Tocris Bioscience) in counterbalanced order, 45 min before test sessions. Three types of test sessions were given: (1) a PRC session during which rats lever pressed on the PR schedule in the presence of the ceramic ramekin containing lab chow so that they were free to choose between lever-pressing for sucrose versus free feeding on chow; (2) PR only sessions during which we omitted the ceramic ramekin (so that there was no freely available lab chow) to assess whether manipulations decreased lever-pressing in the absence of choice; and (3) a free choice consumption test where there was free access to preweighed amounts of sucrose pellets and lab chow (18 g) in empty cages (different from their home cages). Following this, any remaining food was collected and weighed to determine rats' food preferences. All sessions were 30 min in duration. In a repeated-measures design, VEH or CNO was administered before a PRC testing session, a PR only testing session, and a free availability choice testing session, in that order, for each rat. The order of VEH versus CNO administration was counterbalanced for baseline choice performance. Rats were given at least 48 h between injections, and testing never occurred on consecutive days.
Death
Following behavioral testing, rats were killed by sodium pentobarbital overdose (Euthasol, 0.8 ml, 390 mg/ml pentobarbital, 50 mg/ml phenytoin; Virbac) and perfused transcardially with 0.9% saline followed by 10% buffered formalin acetate. Brains were postfixed in 10% buffered formalin acetate for 24 h followed by 30% sucrose cryoprotection for 5 d; 50 μm sections were coverslipped with DAPI mounting medium (Prolong Gold, Invitrogen), and visualized using a BZ-X710 microscope (Keyence).
DREADD quantification
Fifty micrometer sections taken from each animal were visualized at seven AP coordinates relative to bregma: 4.2, 3.7, 3.2, 2.7, 2.2, 1.7, and 1.6 mm. No fluorescence was observed beyond these coordinates. mCherry fluorescence was drawn by a blind experimenter on a GNU Image Manipulation Program document containing a schematic of each of these seven sections, drawn to scale. Spread was quantified as total pixel count across all seven sections.
Electrophysiological confirmation of DREADDs
Separate rats were prepared with ACC DREADDs using identical surgical procedures to the main experiments. Slice recordings did not begin until at least 4 weeks following surgery to allow sufficient hM receptor expression. Slice recording methods were similar to those previously published (Babiec et al., 2017). Six rats were deeply anesthetized with isoflurane and decapitated. The brain was rapidly removed and submerged in ice-cold, oxygenated (95% O2/5% CO2) ACSF containing (in mM) as follows: 124 NaCl, 4 KCl, 25 NaHCO3, 1 NaH2PO4, 2 CaCl2, 1.2 MgSO4, and 10 glucose (Sigma Millipore); 400-μm-thick slices containing the ACC were then cut using a Campden 7000SMZ-2 vibratome. Slices from the site of viral infusion were used for validation. Expression of mCherry was confirmed after recordings were performed, and ACC slices with no transfection were used as control slices. Slices were maintained (at 30°C) in interface-type chambers that were continuously perfused (2-3 ml/min) with ACSF and allowed to recover for at least 2 h before recordings. Following recovery, slices were perfused in a submerged-slice recording chamber (2-3 ml/min) with ACSF containing 100 μm picrotoxin to block GABAA receptor-mediated inhibitory synaptic currents. A glass microelectrode filled with ACSF (resistance = 5-10 mΩ) was placed in layer 2/3 ACC to record fEPSPs and postsynaptic responses elicited by layer 1 stimulation delivered using a bipolar, nichrome-wire stimulating electrode placed near the medial wall in ACC. Inhibitory validation in ACC with identical coordinates, reagents, and virus was previously performed by our laboratory, with the methods and data appearing elsewhere (Stolyarova et al., 2019), and so these experiments were not needlessly repeated. Briefly, we first recorded for 2 min without synaptic stimulation to measure spontaneous levels of activity. Presynaptic fiber stimulation (0.2 ms duration pulses delivered at 0.33 Hz) was then delivered, and the stimulation intensity was varied in 0.2 V increments to generate an input/output curve and identify the threshold for generation of postsynaptic responses. Stimulation strength was then set to the minimum level required to induce postsynaptic responses in ACC. Once stable responses (measured as the area of responses over a 4 s interval) were detected, baseline measures were taken for at least 10 min, followed by 20 min bath application of 10 μm CNO. In slices where CNO failed to elicit spontaneous activity, we generated a second input/output in the presence of CNO to test for CNO-induced changes in postsynaptic responses evoked by synaptic stimulation. Unless noted otherwise, all chemicals were obtained from Sigma Millipore.
Behavioral analyses
Behavioral data were analyzed using GraphPad Prism version 7 (GraphPad Software), SPSS version 25 (IBM), and MATLAB (The MathWorks, version R2017a). An α level for significance was set to 0.05. A mixed ANOVA with between-subject factor of virus (Gi, Gq, eGFP null) and within-subject factor of effort condition and injection (PRC, PR; VEH, CNO) was conducted on lever-pressing data as well as highest ratio achieved and number of pellets earned. Subsequently, paired-samples t tests (reported as means ± SEM) on data from the PRC and PR tests were used to test for effects of CNO versus VEH. Because the dependent measure was different in the free choice test (i.e., amount of food consumed, not lever presses), a separate mixed ANOVA with between-subject factor of virus (Gi, Gq, eGFP null) and within-subject factors of food type and injection (sucrose, chow; VEH, CNO) was conducted. Following these group comparisons, which included virus as a factor, paired t tests were used to test for effects of CNO on the total number of lever presses, highest ratio, and number of pellets earned in each of the groups. Two-way ANOVA was used to analyze the effects of CNO on temporal response patterns in each of the groups. Two-way ANOVA was used to test for effects of CNO on total consumption during free consumption testing in each of the groups. Mixed ANOVA was used to compare responding in the imaging and DREADDs animals, and repeated-measures ANOVA was used to compare responding in the different session types within the imaging group.
Calcium image analyses
Image analyses were performed using custom-written MATLAB (The MathWorks, version R2017a) scripts. First, images were motion-corrected using functions based on the Non-Rigid Motion Correction (NoRMCorre) package (Pnevmatikakis and Giovannucci, 2017), downsampled spatially by a factor of 2 (240 × 356 pixels) and temporally by a factor of 4 (to a frame rate of 7.5 fps). In order to remove background fluorescence, we performed a neural enhancement processing step as in the min1pipe processing framework (Lu et al., 2018). Briefly, from each frame, we constructed a background fluorescence estimate by performing a neuron-sized morphologic opening function, and subtracting this background frame from the original frame. This removes large fluorescence artifacts inherent in single-photon microscopy while preserving neural components. This motion-corrected, downsampled, and enhanced video was then processed using Constrained Non-negative Matrix Factorization for Endoscopic data (CNMF-E) (Pnevmatikakis et al., 2016; Zhou et al., 2018). This extracted individual neural segments, denoised their fluorescent signals, demixed cross-talk from nearby neighbors, and deconvolved the calcium transients to estimate temporally constrained instances of calcium activity for each neuron (Friedrich et al., 2017). These estimated calcium event timings were used to compare calcium activity time-locked to specific behavioral instances. Neurons were matched across recording sessions using CellReg (Sheintuch et al., 2017) by matching cells based on their contours and centroid locations.
Calcium response analysis
A cell's probability of generating calcium transients proximal to a trigger event (e.g., a lever press [LP] or magazine head entry [HE]) was compared against a baseline probability surrounding each event. LP bouts were defined by the first lever press within a ratio and HE bouts defined as the first time point where the infrared beam was broken via nosepoke, after completion of that ratio. LP and HE bouts where the last lever press and first head entry bout were separated by >30 s were excluded from analysis. We focused on HEs that occurred shortly after completion of LP bouts so that HE events during the chow consumption period before lever-pressing in PRC and CON sessions were excluded. This subsampling also excluded HE events with no sucrose delivery and LP events that were closely preceded by HE events (see Fig. 6J, bottom). This minimized contamination between LP and HE evoked calcium signals. For every cell recorded within a session, we created a 6.25 s (±3.125) perievent time histogram (PETH) divided into 47 equal time bins (bin size = 133 ms), with the 24th (middle) bin centered on the trigger events (which were either LP or HE bout starts). PETHs were constructed for LP and HE bouts from each sessions type, yielding 6 PETHs for each cell of the average transient rate surrounding each trigger event. Only half of the bout-centered PETH window (24 bins) was used as the ROI for statistical analysis.
To test whether a cell responded before LP events, we used a binomial test on each of the first 24 bins compared with the average transient rate in the 23 bins following LPs. The same procedure was applied to HE events, where a binomial test was applied to each of the last 24 bins compared with the average transient rate in the 23 bins preceding HEs. This pre-post event approach is consistent with what others have defined as task-evoked calcium responses (Jennings et al., 2015). These comparisons were used to identify cells modulated by each event while minimizing overlap between LP and HE responses (see Fig. 6J). A cell was classified as significantly responsive within the ROI if the transient rate within the ROI was greater than baseline, and one or more of three probability criteria were met: at least 1 of 24 ROI bins beat p < 0.00125, at least 2 of 24 ROI bins beat p < 0.0115, or at least 3 of 24 ROI bins beat p < 0.0285. These p thresholds were chosen to yield an equal probability of each criterion being met, and p = 0.01 for meeting one or more of the criteria by chance. Hence, the probability of erroneously classifying a cell as responsive was 1%. Examples of LP and HE responsive cells are show in Figure 7A and Figure 8A, respectively.
To compare responses of each individual cell with LP and HE events during different types of experimental sessions, we analyzed a subpopulation of cells that met two criteria: (1) the cell responded significantly to the trigger event (LP or HE) during at least one session; and (2) the cell fired (but did not necessarily respond to the trigger event) during at least one session of all three imaging session types (PR, PRC, and CON). For each cell meeting these two criteria for LP events, three PETHs were derived, one for each session type: PETHPR, PETHPRC, PETHCON (see Fig. 7B). For cells meeting these criteria for HE events, an additional PETH was computed for ramekin entries during PRC sessions (PETHR-PRC) to analyze similarity of responses between sucrose and chow consumption (see Fig. 8B). However, ramekin HEs within a session were few compared with sucrose HEs; thus, the PETHR-PRC is relatively undersampled. Each mean PETH was constructed by computing an unweighted average of session PETHs over all events from the same session type during which the cell was active. The averaged PETH was then smoothed by convolving it with a 5-bin Gaussian function of unit area. The smoothed PETH for each cell was normalized via division by a factor B, which was the value of the largest bin in any of the PETHs for that cell: B = max (PETHPR, PETHPRC, PETHCON). Finally, the frequency of responsive cells across session types for HE and LE events was compared against random number samples from a uniform distribution, analyzed using χ2 tests.
Satiety control condition
For a subset of calcium imaging sessions, we administered a satiety control condition, “CON.” For these sessions, we capitalized on the fact that rats typically consume chow early in the test session, presented them with the lever and the chow initially, allowed them to consume chow, but then removed the chow once the rat began to lever press. This control condition allowed rats to reach a comparable motivational (more sated) state relative to the PR-only condition, and thus controlled for satiety differences between PR and PRC sessions. Lever-pressing behavior in these sessions was similar to during PR-only sessions (see Fig. 6I). The ramekin was removed after lever-pressing behavior commenced, so we could not construct ramekin response PETHs (PETHR-CON) since no ramekin HEs occurred after the fifth lever press (defined by our baseline calcium transient rate calculation).
Results
Chemogenetic manipulations
Histology
Reconstructions of viral spread confirmed that most placements were centered on the targeted region of area 24, in rat ACC (van Heukelum et al., 2020). Results of histologic processing are shown in Figure 1. DREADD expression was driven by a CaMKIIα promoter, which is thought to selectively target projection neurons in cortex (Nathanson et al., 2009; Wang et al., 2013). Viral spread in the Gi and Gq groups was quantified by pixel count using GNU Image Manipulation Program software (see Materials and Methods). There was no significant difference between Gi and Gq groups (t(20) = 2.03, p = 0.056; Gi = 25,258 ± 3121 total mean pixels; Gq = 17,736 ± 2002 total mean pixels).
Expression of inhibitory and excitatory DREADDs and null virus eGFP in ACC. A, Representative photomicrograph showing hM4Di-mCherry DREADDs under CaMKIIα in ACC, labeled Gi. B, Schematic reconstruction of maximum (red) and minimum (pink) viral spread for all rats. Numerals indicate AP level relative to bregma. Scale bars, 1 mm. C, Representative photomicrograph showing eGFP (null virus) under CaMKIIα in ACC, labeled GFP. D, Schematic reconstruction of maximum (dark gray) and minimum (light gray) viral spread for all rats. Numerals indicate AP level relative to bregma. Scale bars, 1 mm. E, Representative photomicrograph showing hM3Dq-mCherry DREADDs expression in ACC, labeled Gq. F, Schematic reconstruction of maximum (light green) and minimum (dark green) viral spread for all rats. Numerals indicate AP level relative to bregma. Scale bars, 1 mm.
Ex vivo electrophysiological validation of DREADDs
We confirmed the efficacy of our DREADDs in slice recordings. A separate group of rats was prepared with Gq DREADDs in ACC using identical surgical procedures to the main experiments (Fig. 2A). As described in Materials and Methods, a bipolar stimulating electrode was placed near the medial wall in layer I of ACC, and a glass microelectrode filled with ACSF (resistance = 5–10 mΩ) was placed in layer 2/3 of ACC to record field potentials and multiunit responses elicited by layer 1 stimulation. Using similar methods, we have previously reported that application of CNO strongly suppressed evoked field potentials in Gi-transfected slices in ACC (Stolyarova et al., 2019). Here, we found that application of CNO induced spontaneous bursting in 4 of 6 Gq-transfected slices (spontaneous activity rate was increased from 0 to 0.06 ± 0.01 Hz, interburst intervals were 18.3 ± 2.6 s, mean ± SEM, range: 12.4–31.6 s, n = 4 slices from 3 rats) (Fig. 2B). In two other Gq-transfected slices where no spontaneous bursting was observed, CNO application reduced threshold for stimulation induced postsynaptic responses (n = 2 slices from 3 rats), with a sharp transition from no response to maximal response as a function of stimulation intensity (Fig. 2C). CNO had no effect on responses in nontransfected slices (n = 3 slices from 3 rats) (Fig. 2D).
Excitatory hM3Dq DREADDs validation in ACC. Electrophysiological effect of bath application of CNO in slice. A, Representative photomicrograph showing hM3Dq-mCherry DREADDs under CaMKIIα in a recording site of a 400 µm slice in ACC. B, Spontaneous ACC activity in a Gq DREADD-expressing slice before (top) and after 20 min bath application of 10 mm CNO (bottom). CNO-induced spontaneous bursting was seen in 4 of 6 Gq DREADD-expressing slices from 3 rats. C, In the two Gq DREADD-expressing slices that failed to show spontaneous bursting, there was a decrease in the threshold for stimulation-induced postsynaptic responses. Plot represents results from one of these slices before (baseline) and after CNO application. D, Summary of responses to CNO in Gq DREADD-expressing and control slices. CNO did not elicit either spontaneous bursting or decrease threshold for evoked responses in all three of the control (nontransfected) slices from 3 rats.
Group comparisons
Overall responding as measured by total lever presses was higher in PR than PRC sessions. A mixed ANOVA yielded a significant main effect of effort condition (F(1,29) = 173.31, p < 0.001). There was no main effect of virus type (Gi, Gq, eGFP null, F(2,29) = 0.96, p = 0.39). The interaction between effort condition (PRC, PR) and injection (VEH, CNO) approached the threshold for significance (F(1,29) = 3.53, p = 0.07). There was no significant group × effort condition × injection interaction (F(2,29) = 0.74, p = 0.48).
Identical analyses were performed for other measures of PR responding (highest ratio and number of pellets earned) with near-identical patterns of results. As expected, animals achieved higher ratios in PR than PRC sessions, as revealed by a significant main effect of effort condition (F(1,29) = 222.85, p < 0.001). Highest ratio achieved was not different among the three groups; there was no main effect of virus type (Gi, Gq, eGFP null, F(2,29) = 1.57, p = 0.23). The interaction between effort condition (PRC, PR) and injection (VEH, CNO) on highest ratio approached the threshold for significance (F(1,29) = 4.16, p = 0.05). There was no significant group × effort condition × injection interaction (F(2,29) = 0.63, p = 0.54). Complementary to this, animals earned more pellets in PR than PRC sessions, as revealed by a main effect of effort condition (F(1,29) = 238.33, p < 0.001). The number of pellets earned was not different among the three groups; there was no main effect of virus (Gi, Gq, eGFP null, F(2,29) = 0.92, p = 0.41). There was a significant interaction between effort condition (PRC, PR) and injection (VEH, CNO) on number of pellets earned (F(1,29) = 5.40, p = 0.027). There was no significant group × effort condition × injection interaction (F(2,29) = 0.55, p = 0.58). Thus, virus groups did not differ by any measure of PR responding, although the effect of CNO, as tested by all three measures, did depend on which task animals were tested in PR or PRC. To further clarify the effects of CNO, and test whether administration differentially affected PR or PRC testing, we conducted planned within-group comparisons, described in the next section.
Free choice consumption was not affected by CNO administration. There was no main effect of CNO (F(1,29) = 0.21, p = 0.65), nor was there a significant virus × injection (F(2,29) = 0.43, p = 0.65), virus × food type (F(2,29) = 0.19, p = 0.83), injection × food type (F(1,29) = 0.33, p = 0.57), or virus × injection × food type (F(2,29) = 2.03, p = 0.15) interaction. All animals consumed more sucrose than chow during free choice consumption, regardless of virus or injection (F(1,29) = 80.68, p < 0.001). To more fully explore the effect of inhibition and excitation of ACC (i.e., the effects of CNO vs VEH) on these measures, we conducted further analyses for each viral group, below.
Effects of DREADDs inhibition and excitation of ACC
We found that chemogenetic inhibition of ACC significantly decreased lever-pressing during PRC but not PR sessions. Planned comparisons revealed CNO significantly reduced the total mean number of lever presses during 30 min PRC sessions (t(10) = 3.047, p = 0.01; VEH = 216.1 ± 52.85 presses; CNO = 173.2 ± 40.27 presses) (Fig. 3A). The total amount of chow consumed during 30 min PRC sessions was not affected by CNO treatment (t(10) = 1.048, p = 0.32; VEH = 7.79 ± 0.33 g; CNO = 7.50 ± 0.36 g) (Fig. 3E). CNO had no effect on PR responding in the absence of freely available chow (t(10) = 0.26, p = 0.80; VEH = 905.1 ± 102.00 presses; CNO = 893.9 ± 128.3 presses) (Fig. 3D).
Effects on effortful choice behavior following DREADDs inhibition and excitation of ACC. A, Mean lever presses during PRC sessions, when rats were presented with both the possibility of lever-pressing under a PR schedule for sucrose pellets and freely available chow. Shown is within-subject, counterbalanced choice behavior under VEH and CNO. CNO significantly reduced the number of lever presses in the Gi and Gq conditions, but not in the GFP condition. B, Highest ratio achieved and (C) number of sucrose pellets earned during PRC sessions, when rats were presented with both options in the Gi, Gq, and GFP conditions, in rats receiving CNO compared with within-subject VEH. CNO significantly reduced these measures in the Gi and Gq conditions, but not in the GFP condition. D, There was no change in lever presses when chow was not available as an alternative option in Gi, Gq, and GFP conditions, in rats receiving CNO compared with within-subject VEH. E, Total chow consumed during choice testing was not different following VEH versus CNO in the Gi, Gq, or GFP condition. *p < 0.05, **p < 0.01.
Using identical procedures to the inhibition experiment, a separate group of rats was assessed on effortful choice following excitatory Gq DREADDs transfection in ACC. Similar to results for the Gi experiment, we found that chemogenetic excitation of ACC significantly decreased lever-pressing during PRC but not PR sessions. Planned comparisons revealed CNO significantly reduced the total mean number of lever presses during 30 min PRC sessions (t(10) = 2.31, p = 0.04; VEH = 210.7 ± 49.44 presses; CNO = 166.6 ± 40.4 presses) (Fig. 3A). The total amount of chow consumed during 30 min PRC sessions was not affected by CNO treatment (t(10) = 0.39, p = 0.71; VEH = 7.71 ± 0.36 g; CNO = 7.81 ± 0.42 g) (Fig. 3E). CNO had no effect on PR responding in the absence of freely available chow (t(10) = 1.15, p = 0.28; VEH = 1107.00 ± 99.66 presses; CNO = 1209.00 ± 156.3 presses) (Fig. 3D).
To test whether results for Gi and Gq DREADDs could be explained by virus exposure alone, a control virus carrying only GFP (but no DREADD receptors) was infused into ACC in a separate group of rats. A paired t test revealed no effect of CNO on the total mean number of lever presses during 30 min PRC sessions (t(9) = 0.48, p = 0.64; VEH = 314.9 ± 57.34 presses; CNO = 302.7 ± 59.66 presses) (Fig. 3A). The total amount of chow consumed during 30 min PRC sessions was not affected by CNO treatment (t(9) = 0.26, p = 0.80; VEH = 6.65 ± 0.99 g; CNO = 6.85 ± 0.34 g) (Fig. 3E). CNO also had no effect on PR responding in the absence of freely available chow (t(9) = 0.73, p = 0.48; VEH = 1001.00 ± 109.4 presses; CNO = 1043.00 ± 120.00 presses) (Fig. 3D).
The same pattern was observed for conventional measures of PR responding, highest ratio, and number of pellets earned, during PRC sessions (Fig. 3B,C). In the Gi group, CNO reduced the highest ratio achieved (t(10) = 3.48, p = 0.0059; VEH = 14.91 ± 2.63; CNO = 12.36 ± 2.04) and number of pellets earned (t(10) = 4.209, p = 0.0018; VEH = 30.18 ± 3.3 pellets; CNO = 27.45 ± 3.2 pellets). In the Gq group, CNO reduced the highest ratio achieved (t(10) = 2.43, p = 0.0355; VEH = 14.36 ± 2.68; CNO = 11.82; ± 2.05) and number of pellets earned (t(10) = 2.70, p = 0.022; VEH = 29.36 ± 3.71 pellets; CNO = 26.55 ± 3.50 pellets). CNO had no effect on the highest ratio achieved (t(9) = 0.33, p = 0.75; VEH = 19.9 ± 2.80; CNO = 20.5 ± 3.06) or the number of pellets earned (t(9) = 0.29, p = 0.78; VEH = 35.2 ± 3.69 pellets; CNO = 34.6 ± 2.54 pellets) in the GFP control group.
As with lever-pressing, these measures were not affected during PR sessions. In the Gi group, CNO had no effect on highest ratio achieved (t(10) = 0.71, p = 0.49; VEH = 43.00 ± 3.89; CNO = 44.36 ± 4.47) or number of pellets earned (t(10) = 0.07, p = 0.95; VEH = 54.55 ± 1.88 pellets; CNO = 54.45 ± 2.43 pellets). In the Gq group, CNO had no effect on highest ratio achieved (t(10) = 1.04, p = 0.32; VEH = 54.91 ± 4.47; CNO = 58.82 ± 5.71) or number of pellets earned (t(10) = 0.87, p = 0.41; VEH = 59.09 ± 1.77 pellets; CNO = 60.45 ± 2.46 pellets). CNO also had no effect on the highest ratio achieved (t(9) = 0.72, p = 0.49; VEH = 49.7 ± 4.56; CNO = 51.5 ± 5.12) or the number of pellets earned (t(9) = 0.60, p = 0.56; VEH = 57.2 ± 2.38 pellets; CNO = 57.8 ± 2.59 pellets) in the GFP control group.
Free choice consumption tests
In the Gi group, a two-way ANOVA (food type: sucrose, chow; injection: VEH, CNO) on the amount of sucrose and chow consumed during free choice consumption testing revealed a significant main effect of food type (F(1,10) = 12.02, p = 0.006; sucrose = 8.59 ± 0.83 g; chow = 4.19 ± 0.34 g), but no significant food type × injection interaction (F(1,10) = 1.415, p = 0.26), or main effect of injection (F(1,10) = 0.01, p = 0.91) was found. Hence, food preference was intact and not affected by CNO injection. In the Gq group, a two-way ANOVA (food type: sucrose, chow; injection: VEH, CNO) on the amount of sucrose and chow consumed during free choice consumption testing revealed a significant main effect of food type (F(1,10) = 47.63, p < 0.001; sucrose = 9.68 ± 0.47 g; chow = 5.10 ± 0.37 g), but no significant food type × injection interaction (F(1,10) = 2.27, p = 0.16) or main effect of injection (F(1,10) = 0.06, p = 0.81), indicating food preference was intact and unaffected by CNO. Similarly, in the GFP group, a two-way ANOVA (food type: sucrose, chow; injection: VEH, CNO) on the amount of sucrose and chow consumed during free choice consumption testing revealed a significant main effect of food type (F(1,9) = 80.63, p < 0.001; sucrose = 8.37 ± 0.60 g; chow = 3.19 ± 0.35 g), indicating food preference was intact. No significant food type × injection interaction (F(1,9) = 1.27, p = 0.29), or effect of injection (F(1,9) = 0.97, p = 0.35) was found (Fig. 4).
Free choice consumption in all three treatment groups following CNO administration. Mean consumption of sucrose and chow when rats were presented with both as freely available options. Shown is within-subject, counterbalanced choice behavior under VEH and CNO, indicating that food preference was intact: rats preferred the sucrose over chow. A, CNO had no effect on either sucrose or chow consumed in the Gi condition. B, CNO had no effect on either sucrose or chow consumed in the Gq condition. C, CNO had no effect on either sucrose or chow consumed in the null virus (GFP) condition. **p < 0.01, ***p < 0.001.
Time course of lever-pressing in PR and PRC
We found that the presence of an alternative option (chow) reduced lever-pressing in the PRC condition, and that both Gi and Gq DREADDs decreased lever-pressing in this PRC condition, but not in the PR condition where lever-pressing was more robust. Of course, reduced total lever-pressing over the course of 30 min does not necessarily indicate that lever-pressing was unaffected during PRC testing. It is possible that Gi- and Gq-transfected animals may have shown a different temporal pattern of lever-pressing behavior, despite showing a similar number of total presses. Likewise, it is possible that CNO did affect responding during PR testing, by altering the time course of presses rather than total number of presses. We therefore assessed the time course of lever-pressing in 5 min time bins, in PR and PRC session types, in Gi and Gq DREADDs, following CNO and VEH injections (Fig. 5).
Time course of lever-pressing in different session types. Mean lever-pressing across 5 min time bins. A, Lever-pressing during PRC sessions, when rats were presented with both sucrose pellet and chow options in the Gi condition, in rats receiving CNO compared with within-subject VEH. B, Lever-pressing during PR sessions, when rats were presented with a single option in the Gi condition, in rats receiving CNO compared with VEH. C, Lever-pressing in PRC sessions, when rats were presented with both options in the Gq condition, in rats receiving CNO compared with VEH. D, Lever-pressing during PR sessions, when rats were presented with a single option in the Gq condition, in rats receiving CNO compared with VEH. Error bars indicate SEM. *p < 0.05.
As above, we first conducted a mixed ANOVA to test for group differences in temporal pressing patterns between the Gi and Gq DREADDs groups. Two separate mixed ANOVAs with virus as a between-subject factor, and injection and time bin as within-subject factors, were conducted on PRC and PR data. We were unable to include the GFP group here because these data were lost due to hardware failure. Virus groups did not differ in the time course of lever-pressing during PRC testing. A mixed ANOVA did not yield a significant main effect of virus on PRC pressing (F(1,20) = 0.01, p = 0.93). Responding increased across PRC sessions as revealed by a main effect of time bin (F(5,100) = 3.17, p = 0.01), and CNO generally suppressed lever-pressing as revealed by a main effect of CNO (F(1,20) = 14.30, p = 0.001). There was no significant interaction between virus × injection (F(1,20) = 0.02, p = 0.88), virus × bin (F(5,100) = 0.98, p = 0.43), or virus × injection × bin (F(5,100) = 0.72, p = 0.61). Virus groups also did not differ in the time course of lever-pressing during PR testing. A mixed ANOVA did not yield a significant main effect of virus on PR pressing (F(1,20) = 2.11, p = 0.16). Responding changed over the course of PR sessions as revealed by a main effect of time bin (F(5,100) = 32.729, p < 0.0001). In contrast to PRC testing, CNO had no effect on the pattern of PR lever-pressing: there was no significant main effect of CNO (F(1,20) = 0.84, p = 0.37). There was no significant interaction between virus × injection (F(1,20) = 1.08, p = 0.31), virus × bin (F(5,100) = 0.51, p = 0.64), or virus × injection × bin (F(5,100) = 0.42, p = 0.83) during PR testing.
We separately tested the Gi and Gq DREADDs groups for effects of CNO. For the Gi DREADDs PRC condition, a two-way ANOVA revealed a significant effect of time bin (F(5,50) = 3.423, p < 0.01), a significant effect of injection type (F(1,10) = 9.44, p < 0.02; CNO = 33.06 ± 13.50 presses; VEH = 40.77 ± 16.65 presses, mean presses per time bin), but no significant time bin × injection type interaction (F(5,50) = 0.88, p = 0.50). For the Gi DREADDs PR condition, a two-way ANOVA resulted in a significant effect of time bin (F(5,50) = 21.27, p< 0.001), but no significant effect of injection type (F(1,10) = 0.02, p = 0.89) or time bin × injection type interaction (F(5,50) = 0.15, p = 0.98).
For the Gq DREADDs PRC condition, a two-way ANOVA revealed no significant effect of time bin (F(5,50) = 0.577, p = 0.71), a significant effect of injection type (F(1,10) = 5.93, p = 0.04; CNO = 31.62 ± 12.91 presses; VEH = 39.94 ± 16.31 presses, mean presses per time bin), but no significant interaction of time bin × injection type (F(5,50) = 0.766, p = 0.58). Finally, for the Gq DREADDs PR condition, a two-way ANOVA resulted in a significant effect of time bin (F(5,50) = 13.03, p < 0.001), but no significant effect of injection type (F(1,10) = 1.20, p = 0.30), or time bin × injection type interaction (F(5,50) = 0.725, p = 0.61). Thus, the reduction in total lever presses observed during PRC exhibited a similar pattern in Gi and Gq groups, and the pattern of responding was completely unaffected during PR testing. CNO only reduced responding during PRC testing, and it did so similarly in both DREADDs groups. We concluded that ACC interference disrupts high-effort lever-pressing behavior when a choice is involved, but not when it is the only food option available.
In vivo calcium imaging
To further investigate why DREADD manipulations in ACC exerted their effects only in the choice condition, but not when PR lever-pressing was the only food option, we performed in vivo calcium imaging experiments to track responses of individual ACC cells and compare their activity during PR versus PRC sessions. Imaging rats were also given “satiety control” (CON) sessions, where rats were prefed with chow in the operant chamber, and then were allowed to lever press on a PR schedule in the absence of chow. This was to control for the possibility that neural encoding could be influenced by satiety from chow consumption, regardless of whether chow was freely available during lever-pressing. Behavior during imaging sessions was similar to behavior in VEH groups from the chemogenetics experiments (Fig. 6H,I), with the highest levels of responding occurring during the early portion of PR sessions (tapering off later in PR sessions), and steady low rates of responding throughout PRC sessions. A mixed ANOVA with experiment group (GCaMP, DREADDs) as a between-subject factor and session type (PR, PRC) and time bin (5 min) as within-subject factors revealed a significant main effect of time bin (F(5,125) = 5.27, p = 0.0002). Not surprisingly, overall response rates were lower in the imaging group than in the VEH groups, as revealed by a significant main effect of experiment group (F(1,113) = 21.04, p < 0.0001) (Fig. 6H). This is due to rats wearing the miniscope and being tethered by the coaxial cable, which inhibited pressing behavior. But the effect of time on lever-pressing behavior across PR and PRC sessions did not depend on experiment group, there was no significant interaction between time bin × experiment group (F(5,113) = 1.10, p = 0.37). A one way repeated-measures ANOVA was used to test for differences in the total number of lever presses, averaged across all imaging sessions, in the 4 rats included in our calcium analyses, during PR, PRC, and CON sessions. As expected, session type did affect the total number of presses as revealed by a significant main effect of session type (F(2,6) = 9.13, p = 0.02). Post hoc comparisons using Tukey's correction for familywise error revealed that the total number of presses was lower during PRC than PR sessions (p = 0.04) as well as CON sessions (p = 0.017). There was no significant difference between PR and CON sessions (p = 0.79). Imaging rats showed reduced pressing during PRC sessions, much like VEH rats (Fig. 6I).
Calcium imaging during lever-pressing. A, Representative photomicrograph showing GCaMP6f expression and aspiration site for lens placement for ACC imaging. B, Schematic reconstruction of maximum (light green), minimum (dark green) viral spread, and maximum aspiration damage (gray). Black bars represent ACC recording sites. Numerals indicate AP level relative to bregma. Scale bar, 1 mm. C, Flow diagram for calcium imaging analysis pipeline. D, Example of PRC behavior session with the miniscope on the rat's head, and both sucrose and chow options available; reward port is on the right just out of view; chow is located in ramekin. E, Maximum projection image from the session in D after motion correction and neural enhancement. F, Same as in E, now with extracted cell contours overlaid. G, Contours matched across three different recording sessions, each taken 6 d apart. H, Number of presses during 5 min bins, averaged across all VEH sessions (gold) in the DREADDs groups and imaging group (greyscale). Pressing was reduced overall in the imaging group, but PR and PRC pressing exhibited similar patterns across time. I, Average number of total presses per session type during imaging sessions in the imaging group. Choice reduced lever-pressing in the imaging group, as in the DREADDs groups. J, On average, most HE events closely followed LP events (top). We excluded HE events that did not follow LP events and LP events that were very closely preceded by HE events, to avoid signal contamination (bottom). K, Proportions of cells that were active or responsive to stimuli. Cells that were active in at least one session of each type (PR, PRC, and CON) and had a significant response to either LP or HE events were included in further analysis. L, Proportions of cells that were responsive to LP and HE events. *p < 0.05. ns, not significant.
Calcium imaging in rat ACC
A group of 4 rats received infusions of AAV9-CaMKIIα-GCaMP6f and was subsequently implanted with 1.8-mm-diameter 0.25 pitch GRIN lenses in area 24 of ACC (Fig. 6). Rats were then trained on lever-pressing and tested during PR and PRC sessions (identical to those described above for DREADD experiments), as well as CON sessions described above. We recorded a total of 1151 neurons from ACC from the 4 animals (136 from Rat 1, 567 from Rat 2, 254 from Rat 3, 194 from Rat 4). Each neuron's calcium events were extracted by deconvolving its denoised calcium trace (sampling rate = 7.5 Hz; see Materials and Methods). To analyze neural responses to LP and HE events, PETHs were generated by combining data from all sessions of a given type (PR, PRC, or CON) during which the cell exhibited at least one calcium event. Calcium event probabilities were computed in 133.33 ms time bins within ±3 s of the trigger event (LP or HE). To analyze how neural activity varied with session type (PR, PRC, CON), we identified a subset of 227 neurons (20% of all recorded cells) that were as follows: (1) active during at least one session of each type (PR, PRC, and CON), and (2) significantly responsive to either LP or HE events (or both, see below for responsiveness criteria) in the session-averaged PETH for at least one of the three session types (Fig. 6G). Cells (n = 924) that did not meet these two criteria were excluded from further analyses.
Responses preceding LP bouts
We analyzed neural responses occurring before the onset of each bout of lever-pressing, under the assumption that this is a likely time window during which the decision to exert effort (i.e., press the lever) is made. The onset of an LP bout was defined as the first LP that occurred after a magazine head entry and was followed by a HE within 30 s after completion of the lever-press ratio trial. LP response PETHs were triggered only by these LP onset events, not by all LP events (Fig. 7A). A neuron was classified as LP-responsive if it showed a significantly higher probability of generating calcium events compared with the baseline rate within that session (see Materials and Methods). Approximately 60% (96 of 161) of LP-responsive neurons were also responsive to magazine head entries (Fig. 6L), and these were included in the analyses of LP-responsive cells presented below. These cells (significantly responsive to LPs and HEs) were typically due to significant responding on separate sessions.
Neural responses in ACC before lever press bouts. A, Top, Example of raw fluorescence from an LP-responsive cell during a PR session. Light gray bands represent 3 s window before lever press bout begins. Dark gray represents the first lever press. Light green bands represent 3 s window after the first head entry following the lever press bout. Dark green represents the first head entry timestamp. Red dots indicate times at which deconvolved calcium transients occurred. Middle, Rastergrams of calcium events show that this cell often fired within ∼3 s before lever presses during PR, PRC, and CON sessions. Bottom, Smoothed PETHs for this cell during each session type. B, Heat maps represent PETHs for all 161 cells that responded significantly before LP events. Color scale in each row is normalized to the maximum PETH bin value observed for the cell in that row across all three session types. C, Mean of PETHs for each session type in B. D, Mean normalized area under PETH curve for LP-responsive cells during the 3 s before LP events for PR, PRC, or CON sessions. E, Proportions of LP-responsive cells that were significantly responsive during all seven possible combinations of session types. Bars and shaded regions represent ±SEM. *p < 0.05. ns, Not significant after accounting for multiple comparisons.
For every LP-responsive cell, three LP-triggered PETHs were generated (one for each session type: PR, PRC, CON; Fig. 7B). Each PETH plotted the mean calcium event rate per time bin, averaged over all sessions of a given type during which the cell was active. To normalize the response of each cell across session types, the bins of all three PETHs for the cell were divided by the maximum value observed in any bin from all three PETHs (Fig. 7A, bottom). A population-averaged LP response curve was computed by taking the mean of the normalized PETHs for each session type (Fig. 7C). To statistically compare the pre-LP responses of neurons in the population, we computed the mean normalized response 3 s before lever press bouts for LP-responsive cells (Fig. 7D). By this measure, it was found that pre-LP responses differed significantly in magnitude by session type (Friedman's nonparametric ANOVA: p = 2.90e-5). Post hoc comparisons revealed that the pre-LP response was significantly larger during PR and CON sessions compared with PRC sessions (Sign rank test: PR > PRC, p = 1.35e-05; PRC > CON, p = 0.0017), whereas PR and CON pre-LP responses were not significantly different after correcting for multiple comparisons (Sign rank test: PR = CON, p = 0.033). Hence, at the population level, ACC neurons were significantly less responsive before the onset of lever press bouts when the rat was offered the choice of free chow from the ramekin as an alternative to lever-pressing during PRC sessions, and this effect could not be accounted for by satiety. We next evaluated how significantly responding cells were distributed across the different session types, and found that 60% (100 of 161 cells) were only responsive to LPs during one session type, which is not significantly different from what would be expected from a random allocation (expected: 102 of 161 cells; χ2 = 0.428, p = 0.52; Fig. 7E). However, we found that 21 cells were significantly responsive to LPs during all three session types, greater than would be expected from chance (expected: 8 of 161 cells; χ2 = 22.23, p < 0.0001).
Responses following HE events
We analyzed neural responses occurring immediately after magazine HE events, under the assumption that this is a likely time window during which signals encoding the value of the sucrose reward relative to the free alterative (chow) might be generated, as the reward is experienced (Fig. 8A). On average, most HE events followed LP bouts within several seconds (Fig. 6J, top). We focused our analyses on HE events that shortly followed LP bouts (Fig. 6J, bottom). A neuron was classified as HE-responsive if it exhibited a significantly higher probability of generating calcium events during the 3 s after HE compared with the baseline rate within that session (see Materials and Methods). Approximately 60% of HE-responsive neurons were also responsive to LP bout onset (Fig. 6I), and these were included in the analyses of HE-responsive cells presented below.
Neural responses in ACC following magazine head entry for sucrose reward. A, Top, Example of raw fluorescence from an HE-responsive cell during a PR session, plotted as in Figure 7A. Middle, Rastergram of calcium event times shows that this cell often fired just after HE events in PR, PRC, and CON sessions. Bottom, Smoothed PETHs for this cell during each session type. B, Heat maps represent PETHs for all 162 cells that responded significantly to HE events. For comparison, PETHs triggered by ramekin entries during PRC sessions are also shown (right). Color scale in each row is normalized to the maximum PETH bin value observed for the cell in that row across PR, PRC, and CON sessions. C, Mean of PETHs for each session type in B. Chow ramekin entries plotted in purple. D, Mean normalized area under PETH curve for HE-responsive cells during the 3 s following HE events for PR, PRC, or CON sessions. Head entry to chow ramekin plotted in purple (right). E, Proportions of HE-responsive cells that were significantly responsive during all seven possible combinations of session types. Bars and shaded regions represent ±SEM. *p < 0.05. ns, Not significant after accounting for multiple comparisons.
Three HE-triggered PETHs were generated for each cell (one for each session type: PR, PRC, CON), as well as an additional PETH triggered by head entries into the chow ramekin during PRC sessions (Fig. 8B). Head entries to the chow ramekin during CON sessions were not formally compared with sucrose HE. This was due to the fact that we sampled far fewer of these events compared with sucrose HE responses as shown in the individual data points in Figure 8D (right, plotted in purple). This also would not have been a fair comparison since these ramekin entries occurred before lever-pressing, during the time period that was excluded from analysis of HE responses. Nevertheless, on average, chow ramekin-evoked calcium responses were lower in magnitude than sucrose HE responses, during any of the session types (Fig. 8C,D, plotted in purple). Population-averaged responses to HE events were computed for each of the three session types (Fig. 8C), using the same methods described above for LP-triggered PETHs. It was found that post-HE responses differed significantly by session type (Friedman's nonparametric ANOVA: p = 0.046; Fig. 8D). Post hoc comparisons revealed that population responses during PR and CON sessions were not significantly different from one another (Sign rank test: PR = CON, p = 0.32), whereas PRC sessions showed a significantly lower post-HE response than either of the other two session types (Sign rank test: PR > PRC, p = 0.0021; CON > PRC, p = 1.28e-04). This pattern of results suggests that population-averaged responses of ACC neurons to rewarding outcomes were smaller in the presence of available alternative outcomes than when no alternative outcome was available, and this effect could not be explained by satiety. Additionally, we found that the distribution of significantly responding HE cells was different from for LP cells (Fig. 8E). More cells responded significantly to HEs in only a single session than would be expected from chance (80 of 162 cells, expected: 102 of 162; χ2 = 12.812, p = 0.0003), half of which (49%) were cells only responding during the CON sessions. Of the remaining population responding significantly in more than one session, 41 HE cells were responsive during all three session types, also greater than expected from chance (41 of 162 cells, expected: 9 of 162; χ2 = 120.47, p < 0.00001). Example movies of head entry (to reward port) and lever press activity during calcium imaging in freely moving rats are shown in Movie 1.
Comparison of Behavior, Imaging, and Calcium Transients in Anterior Cingulate Cortex during PR (No Choice) versus PRC (Choice) sessions. Example movies (2x speed) of head entry (to reward port) and lever press activity during 1-photon calcium imaging in freely moving rats. (Top) Behavioral recordings alongside spatial contour of activated cells in anterior cingulate cortex. (Middle) Behavioral rasters of lever press and head entry events. (Bottom) Calcium transients in anterior cingulate cortex during these recordings.
Discussion
We report that either chemogenetic silencing or stimulation of ACC excitatory neurons resulted in decreased effort for a qualitatively preferred option, and that this effect was only observed when a concurrently available, lower effort alternative was available, not when lever-pressing was the only response option. Chemogenetic manipulations had no effect on the ability to lever press for sucrose or on food preference. CNO administration also had no effect in rats lacking active DREADD (hM4D-Gi or hM3D-Gq) receptors. Slice electrophysiology confirmed robust inhibition (Stolyarova et al., 2019) and excitation in hM4D-Gi- and hM3D-Gq-transfected slices, respectively. Finally, using single-photon imaging, we found that ACC neurons showed differential task-evoked activity during lever-pressing and reward-retrieval behavior that depended on the availability of another food option. In the same way that interference only affected choice lever-pressing, we found that tracked ACC neurons exhibited different response profiles during PR and PRC sessions. Together, these findings support a role for ACC in the evaluation of effortful behavior, consistent with recent evidence from single-unit recordings (Porter et al., 2019) and human fMRI (Arulpragasam et al., 2018).
ACC chemogenetic silencing
The earliest studies probing rat ACC in effort-based choice made use of T-maze tasks where rats selected between the same food option but of different magnitudes (Walton et al., 2002, 2003; Schweimer and Hauber, 2006). Investigations where rats chose between qualitatively different options following ACC lesions have yielded mixed results with reports of both null effects (Schweimer and Hauber, 2005) and decreased effort in the context of choice (Hart et al., 2017).
Here, hM3Dq and hM4Di receptors were expressed under a CaMKIIα promoter, putatively targeting primarily excitatory pyramidal neurons (Nathanson et al., 2009; Wang et al., 2013), in contrast with prior studies using lesions or inactivations that silence all neural activity. ACC likely exerts its effects on choice behavior via projections to downstream targets, the densest of which are to dorsal striatum and mediodorsal thalamus (Vogt and Paxinos, 2014), although ACC also sends sparser efferents to ventral striatum and amygdala (Gabbott et al., 2005).
Though reliable, the magnitude of effect observed here with DREADDs was smaller than what we previously observed following lesions (Cohen's d = 1.39 lesions vs d = 0.28 Gi vs d = 0.29 Gq) (Hart et al., 2017), and also smaller than effects we have previously reported following pharmacological inactivation (Hart and Izquierdo, 2017) and drug exposure (Thompson et al., 2017; Hart et al., 2018). The smaller effect obtained with DREADDs could be due to different factors. First, DREADD receptors target only cells that recognize the CaMKIIα promoter (putative excitatory projection neurons). Second, although our slice experiments have shown that DREADD receptors modulate neural activity in ACC, these changes in neural activity may have weaker effects on behavior than complete pharmacological inactivation of ACC, or following chronic psychostimulant exposure (Hart et al., 2018). Nevertheless, the DREADD manipulations were enough to significantly bias behavior away from the preferred, effortful option during choice sessions.
ACC calcium imaging
ACC interference affected lever-pressing only during choice sessions. We used in vivo calcium imaging to observe how ACC neurons responded to lever-pressing and reward retrieval during such PR, PRC, and CON sessions. A total of 227 neurons from 4 rats were successfully recorded during at least one session of each type and had a significant response to lever-pressing or head entry. Over two-thirds of these neurons (161 of 227) were lever-press responsive in at least one of the three session types. Many of these cells (96 of 161 cells) were also significantly responsive to head entries in other sessions, indicating some cells maintain encoding properties while others alter response characteristics during different sessions. Calcium trace activity during prelever responses was not significantly different in CON compared with PR sessions. However, calcium activity during the prelever period was lower during PRC than both PR and CON sessions, suggesting that free chow availability, not satiety from chow, attenuated prelever calcium activity.
More than two-thirds of recorded ACC neurons (162 of 227) were reward-responsive. These sucrose responses were similar in magnitude during CON and PR sessions, indicating that satiety from prior chow consumption had little effect on ACC responses to sucrose. By contrast, sucrose-related calcium activity was significantly lower during PRC sessions than during PR and CON sessions, demonstrating that, independently of satiety state, free chow availability attenuated sucrose responses of ACC neurons below the levels seen when sucrose was earned in the absence of free chow.
We next evaluated whether single-cell responses reflected the overall population average but found very few cells that individually responded accordingly in their PETHs (i.e., PR > PRC, CON > PRC, PRC = CON). This implies that the population signal that disambiguates effortful contexts is an emergent property of many cells functioning independently, rather than as a homogeneous population. This finding is not surprising given ACC's role in diverse behaviors and in line with recent single-unit recordings within an effortful task demonstrating robust heterogeneity of response characteristics (Porter et al., 2019). We further looked to see whether individual neurons were evenly distributed among condition types (PR, PRC, and CON). For both LP- and HE-responsive cells, we found that many cells responded significantly in all three session types. Therefore, while individual average responses do not reflect the same population profile, individual cells are preferentially active across similar contexts and serve to disambiguate relative reward value through coordinated population activity. It has been theorized that such mixed selectivity in cortical regions is important in generating high dimensional representations for complex, adaptive behavior (Fusi et al., 2016). This type of heterogeneous population code would be more susceptible to perturbation by either bulk inhibition or excitation, as demonstrated in the results of our DREADD manipulations.
These findings converge on the idea that activity of ACC ensembles during sucrose consumption encodes the difference between the value of the sucrose versus the value of other available reward options. If no other options are available (as during PR and CON sessions), then the value of other options is zero, and thus nothing is subtracted from the value of the sucrose reward. But if a lower value reward option (e.g., lab chow during PRC sessions) is available while the rat is working for sucrose, then the nonzero value of the other option may be subtracted from the value of the sucrose reward, reducing the magnitude of ACC responses during sucrose delivery. If this relative value signal for sucrose in ACC is involved in driving motivated effort to work for sucrose, then it would be expected that the rat should exert less effort for sucrose under conditions where the ACC responses to sucrose are smaller. This is exactly what we observed: sucrose responses of ACC neurons were lower during PRC sessions (where lab chow was available as a competing reward option) than during PR or CON sessions (where sucrose was the only reward option).
In summary, neural activity associated with sucrose pellet collection in ACC is strongest when sucrose is the only available option, and weakened by the presence of the counterfactual choice (Blanchard and Hayden, 2014; Mashhoori et al., 2018), or the value of leaving a patch in pursuit of another option (Hayden et al., 2011). We add here the novel mechanism that ACC modulates this evaluation of options with a subpopulation of stable-coding neurons, which we harnessed the power of calcium imaging to reliably track. Overall, ACC responses to lever-pressing and reward-retrieval were lower during PRC sessions; therefore, a plausible explanation of our interference experiments would be that this lower, heterogeneous population activity is more susceptible to interference, thus explaining why CNO reduced lever-pressing selectively in PRC sessions.
ACC inhibition versus stimulation
If ACC activity encodes relative value signals that are involved in deriving an animal's motivation to exert effort, then it is natural to predict that disrupting ACC activity at the neural level would alter effort exertion at the behavioral level. We also expected that bidirectional manipulations of neural activity might yield bidirectional effects on motivated effort, such as manipulations of dopamine (Farrar et al., 2010; Nunes et al., 2013; Randall et al., 2014a; Yohn et al., 2015a,b). Enhancing dopamine transmission with major psychostimulants (Yohn et al., 2016c), dopamine transporter blockers (Randall et al., 2014b; Yohn et al., 2016a), adenosine A2A receptor antagonists (Randall et al., 2012), or 5-HT2C ligands (Bailey et al., 2016, 2018) can increase effort output in otherwise untreated rats.
We applied a similar rationale for our Gq and Gi DREADD experiments and tested whether ACC chemogenetic inhibition versus stimulation yielded bidirectional effects on behavioral responding. This was not the case: lever-pressing behavior during PRC sessions was similarly attenuated by both Gq and Gi DREADDs. Indeed, the contributions of ACC and other frontocortical regions to effort may be more complex: pharmacological stimulation of orbitofrontal cortex decreases PR responding (Munster and Hauber, 2017), and GABA antagonism of infralimbic cortex similarly decreases high-effort choice (Piantadosi et al., 2016). Therefore, in frontal cortex, there may be an optimal excitatory/inhibitory ratio for computing relative cost-benefit and, consequently, sending appropriate output to downstream targets. The results of ACC stimulation here are consistent with our manipulation introducing noise to otherwise normal neural computations (Mainen and Sejnowski, 1995; Stein et al., 2005), thus impairing behavior in a manner similar to when neural activity is inhibited. Lever-pressing rates and task-evoked activity levels in ACC were both lower during PRC sessions than PR or CON sessions. Consequently, disruptions in decision-making may occur via changes in signal-to-noise ratio in this region: either by decreases in the signal (i.e., Gi DREADDs) or by increases in background noise (i.e., Gq DREADDs).
In conclusion, our findings suggest that the role of ACC in effort-based choice may be to discriminate the utility of available choice options by providing a stable population code for the relative value of different reward options. A better understanding of ACC contributions to effort-based choice may yield insight into the mechanisms underlying motivational symptoms in depression (Nunes et al., 2013) and addictions (Robinson et al., 2013).
Footnotes
This work was supported by University of California at Los Angeles's Division of Life Sciences Recruitment and Retention fund to A.I., R01 DA047870 to A.I., National Science Foundation Neuronex 170748 to H.T.B., Training Program in Translational Neuroscience of Drug Abuse T32 DA024635 to Hart, Training Program in Neural Microcircuits T32 NS058280 to Hart, and University of California at Los Angeles Graduate Division for the Dissertation Year Fellowship to Hart. We thank members of the A.I. laboratory for helpful comments on a previous version of the manuscript; and Staglin Center for Brain and Behavioral Health for additional support related to fluorescence microscopy.
The authors declare no competing financial interests.
- Correspondence should be addressed to Alicia Izquierdo at aizquie{at}psych.ucla.edu
This is an open-access article distributed under the terms of the Creative Commons Attribution License Creative Commons Attribution 4.0 International, which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.
