Abstract
An adaptive transition from exploring the environment in search of vital resources to exploiting these resources once the search was successful is important to all animals. Here we study the neuronal circuitry that allows larval Drosophila melanogaster of either sex to negotiate this exploration-exploitation transition. We do so by combining Pavlovian conditioning with high-resolution behavioral tracking, optogenetic manipulation of individually identified neurons, and EM data-based analyses of synaptic organization. We find that optogenetic activation of the dopaminergic neuron DAN-i1 can both establish memory during training and acutely terminate learned search behavior in a subsequent recall test. Its activation leaves innate behavior unaffected, however. Specifically, DAN-i1 activation can establish associative memories of opposite valence after paired and unpaired training with odor, and its activation during the recall test can terminate the search behavior resulting from either of these memories. Our results further suggest that in its behavioral significance DAN-i1 activation resembles, but does not equal, sugar reward. Dendrogram analyses of all the synaptic connections between DAN-i1 and its two main targets, the Kenyon cells and the mushroom body output neuron MBON-i1, further suggest that the DAN-i1 signals during training and during the recall test could be delivered to the Kenyon cells and to MBON-i1, respectively, within previously unrecognized, locally confined branching structures. This would provide an elegant circuit motif to terminate search on its successful completion.
SIGNIFICANCE STATEMENT In the struggle for survival, animals have to explore their environment in search of food. Once food is found, however, it is adaptive to prioritize exploiting it over continuing a search that would now be as pointless as searching for the glasses you are wearing. This exploration-exploitation trade-off is important for animals and humans, as well as for technical search devices. We investigate which of the only 10,000 neurons of a fruit fly larva can tip the balance in this trade-off, and identify a single dopamine neuron called DAN-i1 that can do so. Given the similarities in dopamine neuron function across the animal kingdom, this may reflect a general principle of how search is terminated once it is successful.
Introduction
An animal exploring its environment may use past experience to guide its search for food. When food is found, however, it is adaptive to terminate search and instead exploit the food source. Here we study the circuits that allow larval Drosophila melanogaster to negotiate this exploration-exploitation transition. The numerical simplicity of the larval brain, combined with cell-specific transgene expression and knowledge of its synaptic connectivity (Pfeiffer et al., 2010; Li et al., 2014; Eichler et al., 2017; Saumweber et al., 2018; Eschbach et al., 2020), allows this to be investigated at the level of individually identified neurons.
D. melanogaster larvae readily associate odors with food reward (Scherer et al., 2003; Gerber and Hendel, 2006; Schipanski et al., 2008; Rohwedder et al., 2012; Apostolopoulou et al., 2014; Kudow et al., 2017, 2019). Critically, learned behavior based on these odor-food associations ceases if during a recall test the reward is present (Fig. 1A). Learned behavior can therefore be viewed as a search, which is adaptively terminated by the sought-for outcome (i.e., by the reward) (Schleyer et al., 2011, 2013, 2015a). Innate olfactory behavior is not likewise affected (Fig. 1B), suggesting that such innate behavior is organized in a reflex-like manner (Schleyer et al., 2011, 2013, 2015a,b). Thus, natural rewards can induce two effects: during the training, they provide a reward signal that can be associated with cues that subsequently guide reward search; and during the recall test, they can acutely terminate this learned search, preventing the animals from drifting away from a resource once it is found. This transition thus reflects a switch in the strategy for obtaining reward, not a change in motivation (i.e., not a loss of interest in the reward).
Sugar can confer reward and search termination signals. A, Larvae were trained such that an odor was presented either paired or unpaired with sugar. In a subsequent recall test, they were assayed for their odor preference either in the absence or in the presence of the sugar (sketch toward the top). Testing in the absence of sugar revealed associative olfactory memory, quantified as a positive PI. The behavioral expression of this memory was terminated in the presence of sugar. Thus, sugar can confer two kinds of signal: during the training, it provides a reward signal that can be associated with the odor to guide the animals later on in their reward search; and during the recall test, the sugar as the sought-for-reward provides a signal to terminate this learned search. Sample sizes: N = 29, N = 28. B, Larvae were tested for innate odor preference, either in the absence or in the presence of sugar (sketch toward the top). Sugar did not have an effect on innate olfactory behavior, suggesting its reflex-like organization. Sample sizes: N = 20 each. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). *Significance from zero (one-sample sign test, p < 0.05, corrected according to Bonferroni-Holm). ns, not significant. All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1. For the odor preference values underlying the PI scores of this experiment, see Figure 9A.
Figure 1-1
In vertebrates and invertebrates alike, dopaminergic neurons (DANs) provide reinforcing signals for associative learning (Waddell, 2013; Schultz, 2015; Kaun and Rothenfluh, 2017). In D. melanogaster, distinct sets of DANs convey appetitive and aversive reinforcement, respectively (larvae: Schroll et al., 2006; Rohwedder et al., 2016; Saumweber et al., 2018; Eschbach et al., 2020; adult flies: Schwaerzel et al., 2003; Claridge-Chang et al., 2009; Liu et al., 2012) (a similar scenario may be emerging in vertebrates, too: Lammel et al., 2012; Groessl et al., 2018; Menegas et al., 2018). Establishing a compartmental structure, these DANs intersect the parallel fibers of Kenyon cells (KCs), the intrinsic neurons of the highest-order olfactory center of the insects, called the mushroom body. Respecting this same compartmental structure, mushroom body output neurons (MBONs) collect information across the KCs and send it toward efferent circuitry (see Fig. 2A–D) (larvae: Selcho et al., 2009; Pauls et al., 2010; Eichler et al., 2017; Saumweber et al., 2018; Eschbach et al., 2020; adults: Sejourne et al., 2011; Placais et al., 2013; Aso et al., 2014a,b; Owald and Waddell, 2015; Takemura et al., 2017). Notably, a recent electron microscope reconstruction revealed that DANs have two main targets: the KCs and the MBONs (see Fig. 2B) (larvae: Eichler et al., 2017; adults: Takemura et al., 2017).
In larval D. melanogaster, activation of DANs from the pPAM cluster can exert a rewarding effect during training (Rohwedder et al., 2016). From the four DANs of this cluster, two can individually confer such a rewarding effect (DAN-h1 and DAN-i1) (Saumweber et al., 2018). Here, we chose to focus on DAN-i1 because it has previously been analyzed in more detail (Saumweber et al., 2018) and because it yields a more robust rewarding effect than DAN-h1 (Saumweber et al., 2018; unpublished data). We first provide a detailed account of the spatial arrangement of all synapses between DAN-i1 and both its major output partners, MBON-i1 and the KCs. Then, we ask whether optogenetic activation of DAN-i1 can mediate not only a reward signal during training, but also a signal to acutely terminate learned search for the reward during the recall test.
Materials and Methods
Experimental model and subject details
Third-instar feeding-stage larvae of both sexes (D. melanogaster), aged 5 d after egg laying, were used throughout. Flies were maintained on standard medium, in mass culture at 25°C, 60%-70% relative humidity and a 12/12 h light/dark cycle. We took a spoonful of food medium from a food vial, randomly selected the desired number of larvae, briefly rinsed them in tap water, and started the experiment.
We used transgenic larvae to express the ChR2-XXL light-gated ion channel in DANs. To this end, the effector strain UAS-ChR2-XXL (Dawydow et al., 2014) (kindly provided by R. Kittel, University Leipzig), was crossed to one of two driver strains: either 58E02-Gal4 (Pfeiffer et al., 2008; Liu et al., 2012; Rohwedder et al., 2016) (Bloomington Stock Center no. 41347) or the split-Gal4 driver strain SS00864 (Eichler et al., 2017; Saumweber et al., 2018) to obtain double-heterozygous offspring. As driver controls the driver strains were crossed to a local copy of w1118 (Bloomington Stock Center #3605, #5905, #6326). As effector controls a strain carrying the landing sites used for the Gal4 (attP2) or the split-Gal4 (attP40/attP2), yet without a Gal4 domain inserted (“empty”) (Pfeiffer et al., 2010), was crossed to UAS-ChR2-XXL. We confirmed the expression pattern of 58E02 and SS00864 by crossing them to pJFRC-10xUAS-IVS-mCD8::GFP (Pfeiffer et al., 2010) (Bloomington Stock Center #32185). Because ChR2-XXL is sensitive enough to be activated by daylight (not shown), flies were raised in vials constantly darkened by black cardboard wrapping. For the experiment presented in Figure 8E, UAS-ChR2 (Schroll et al., 2006) was crossed to 58E02-Gal4.
Experimental setup
For behavioral experiments, larvae were trained in Petri dishes of 9 cm inner diameter and tested in Petri dishes of either 9 or 15 cm inner diameter (both from Sarstedt) as mentioned in the figure legends, in all cases filled with 1% agarose (electrophoresis grade; Roth). As the odor, we used n-amyl acetate diluted 1:20 in paraffin oil (AM; CAS: 628-63-7; Merck), and in some cases additionally undiluted 1-octanol (OCT; CAS: 111-87-5; Merck).
Experiments were performed inside a 43 × 43 × 73 cm surrounding box equipped with a custom-made light table featuring a 24 × 12 LED array (470 nm; Solarox) and a 6-mm-thick diffusion plate of frosted Plexiglas on top to ensure uniform blue light for ChR2-XXL activation (120 µW/cm2). Petri dishes were placed directly on top of the diffusion plate. The Petri dishes were surrounded by a polyethylene diffusion ring; behind the diffusion ring 30 infrared LEDs (850 nm; Solarox) were mounted to provide illumination that was invisible to the larvae yet allowed the recording and tracking of their behavior for offline analysis. To this end, a camera (Basler acA204090umNIR; Basler) equipped with an infrared-pass filter was placed above the Petri dish. For additional details regarding the experimental setup, see Saumweber et al. (2018).
Associative odor-sugar learning
Learning experiments followed established protocols (Michels et al., 2017). Odor containers were prepared by adding 10 µl of odor substance to custom-made Teflon containers (5 mm inner diameter with a lid perforated with 7 holes of 0.5 mm diameter each). Petri dishes were covered with modified lids perforated in the center by 15 holes of 1 mm diameter each to improve aeration.
For odor-sugar training, ∼20 larvae were placed in the middle of a Petri dish filled with agarose that contained 2 mol/L D-fructose (CAS: 57-48-7; Roth) as reward (+), and equipped with two odor containers on opposite sides, both filled with AM (AM+). After 2.5 min, the larvae were displaced onto a fresh Petri dish with plain, tasteless agarose, equipped with two empty containers (EM), where they also spent 2.5 min. Three such “paired” training cycles were performed, in each case using fresh Petri dishes. In half of the cases, training started with reward-containing Petri dishes as indicated (AM+/EM), whereas in the other half of the cases the sequence was reversed (EM/AM+). For each cohort of larvae trained AM+/EM (or EM/AM+), a second cohort was trained reciprocally, that is, by “unpaired” presentations of odor and reward (AM/EM+, or EM+/AM, respectively).
Following training, the larvae were transferred to the middle of a test Petri dish and tested for their odor preference. Importantly, the recall test was conducted either on a plain Petri dish to assess memory-based search behavior or on a reward-containing Petri dish to determine whether the presence of reward terminated the memory-based search. In either case, one side of the test Petri dish was equipped with an AM container, and the other side with an EM container. The number of larvae on either side was determined from the camera image at 3 min after the test had started. From these numbers, an odor preference was calculated as follows:
Thus, odor preference values are constrained between 1 and −1, with positive values indicating a preference for and negative values indicating avoidance of AM.
From two reciprocally trained cohorts of animals we calculated an associative performance index (PI) as follows:
Thus, PI values can range from 1 to −1, with positive values indicating that the larvae preferred the odor more after paired than unpaired training. Positive PI values accordingly indicate an appetitive associative memory that is best interpreted as memory-based search behavior. Negative PI values, by contrast, indicate aversive associative memory.
Associative learning by optogenetic DAN activation
For optogenetic experiments, training and test were performed in an analogous way. However, all Petri dishes were filled with plain agarose (i.e., no rewarding tastants were used). Instead, for paired training, AM was paired with continuous blue light stimulation to activate either 58E02-DAN or 864-DAN for 2.5 min, followed by 2.5 min of darkness without an odor (AM+/EM). For unpaired training, odor presentation and light stimulation occurred separately (AM/EM+). For the experiment presented in Figure 8E, odor and light were presented for 5 min each; that experiment otherwise followed the procedures described by Rohwedder et al. (2016).
After three such training cycles, the recall test was conducted, and odor preference and PI were calculated as described. Critically, the recall test was conducted either without blue light activation to determine whether the DANs mediate a reward signal during training, or with blue light activation to determine whether in addition the DANs can mediate a signal to terminate search.
Variations of this protocol are indicated in the figure legends.
Innate olfactory behavior
Odor containers and Petri dishes were prepared as described above. Cohorts of ∼20 larvae were collected from the vial, briefly washed in tap water, and placed onto a Petri dish with an AM container on one side and an empty container on the other side. After 3 min, the innate odor preference was determined according to Equation 1. This preference test was conducted either in the absence or in the presence of sugar, or in the absence or in the presence of blue light activation, to determine whether these test conditions alter innate olfactory behavior.
Analysis of behavioral modulations after training with 864-DAN activation
Larval behavior was video-tracked and analyzed as described in detail by Paisios et al. (2017). In brief, two aspects of larval chemotaxis were analyzed. First, the modulation of head cast rate (HC per second) (HC/s) was calculated as follows:
This measure yields positive scores for attraction: that is, when larvae systematically perform more HCs while heading away from the odor(i.e., when odor concentration decreases) than while heading toward it (i.e., when odor concentration increases). Conversely, it yields negative scores for aversion.
Second, the modulation of HC direction was measured by the reorientation per HC as follows:
In this measure, the heading angle (HA) describes the orientation of the animal's head relative to the odor, with absolute heading angles (abs(HA)) of 0° or 180°, for example, indicating that the odor is to the front or to the rear of the larvae, respectively. This measure thus yields positive scores for attraction (i.e., when the HC directs the larvae toward rather than away from the odor target), whereas it yields negative scores for aversion.
Immunohistochemistry
Larval brains were dissected in Ca2+-free saline solution and fixed in Bouin's solution diluted 1:2 with Ca2+-free saline solution for 7 min. Three washing steps (each 10 min) in PBST (0.3% Triton-X 100 in PBS) were followed by incubation with the primary antibodies overnight at 4°C. After three washes (each 10 min) in PBS, the tissue was incubated with the secondary antibodies for 1 h at room temperature. After three final washing steps in PBS (each 10 min), samples were mounted in Vectashield (Vector Laboratories). Preparations were examined under a DM 6000 CS confocal microscope (Leica Microsystems).
As antibodies, we either used monoclonal anti-FASII mouse antibody (DSHB, diluted 1:50 in blocking solution containing 4% normal goat serum) and anti-GFP rabbit antibody (A11122, Invitrogen, diluted 1:1000 in blocking solution containing 4% normal goat serum) as primary antibodies and Cy3 anti-mouse antibody (715-165-150, Jackson ImmunoResearch Laboratories) and Alexa-488 anti-rabbit antibody (A11034, Invitrogen), both diluted 1:200 in PBS, as secondary antibodies; or we used monoclonal anti-ChR2 mouse antibody (ProGen Biotechnik) diluted 1:100 in 0.3% PBST as primary antibody, Cy3 anti-mouse antibody (715-165-150, Jackson ImmunoResearch Laboratories), diluted 1:300 in 0.3% PBST, as secondary antibody, and Alexa-488 anti-HRP (Jackson ImmunoResearch Laboratories), diluted 1:300 in 0.3% PBST, for counterstaining.
Dendrogram analysis
The neuron dendrograms are 2D sketches of the 3D neuron reconstructions from Eichler et al. (2017). As “linearized”, flattened versions of morphologically complex neurons, the dendrograms are compact visualizations that serve to display the spatial distribution of synapses. Details of the dendrogram drawing algorithm were described by Strauch et al. (2018). Briefly, the dendrograms are topologically correct neuron sketches with a dominant primary branch drawn as a straight line, and shorter side branches deflected at stereotypical angles and pointing upward and downward in an alternating fashion to avoid overlap. They are drawn such that cable length distances and synapse positions are approximately correct. Slight modifications to improve readability involve an automatic rearrangement of synapse symbols to resolve clutter and overlap (Strauch et al., 2018).
For the analysis of synaptic organization, we computed a (synapse × synapse) distance matrix containing geodesic distances (i.e., the distance as “cable length” along the neuron) between the locations of all synapses on the neuron. The geodesic distances were computed as Euclidean distances in the 3D coordinates of the reconstructed neuron by traversing along the neuron's branches. We clustered the synapse locations with divisive analysis clustering (DIANA) (Kaufman, 2005) operating on the geodesic distance matrix. Based on silhouette score analysis, we set the number of clusters per neuron to 10. Clusters that did not contain all four synapse classes of interest (KC→DAN-i1, DAN-i1→KC, DAN-i1→MBON-i1 left, DAN-i1→MBON-i1 right) were removed from the analysis. For each cluster, a central point was defined as the most central synapse of the cluster as determined by partitioning around medoids clustering (PAM with k = 1 cluster) (Kaufman, 2005) based on the geodesic synapse distances within the cluster. Finally, we determined the geodesic distances between all synapses and the cluster's central point.
Experimental design and statistical analysis
Two-tailed nonparametric tests were used, and statistical assumptions for these tests were met throughout. When multiple comparisons were performed within one analysis, a Bonferroni-Holm correction was applied to keep the experiment-wide error rate to <5% (Holm, 1979). All the performed statistical tests, and their results are reported in Extended Data Fig. 1-1 along with the source data.
For all behavioral experiments, values were compared across multiple groups with Kruskal–Wallis tests. In case of significance, subsequent pairwise comparisons used Mann–Whitney U tests. To test whether values of a given group differed from zero, one-sample sign tests were used.
The experiments in Figures 1, 6C–F, and 8 followed a two group-design, comparing two testing conditions for the experimental genotype. Both groups were compared with a Mann–Whitney U test, and their significance from zero was tested by one-sample sign tests.
The experiments in Figure 6A, B followed a six-group design, with three genotypes (experimental genotype expressing ChR2-XXL, driver, and an effector control) and two test conditions. First, a Kruskal–Wallis test was performed across all groups. In case of significance, we performed pairwise Mann–Whitney U tests between the experimental genotype and each of the genetic controls of the same test condition, as well as between the test conditions for the experimental genotype (total five Mann–Whitney U tests). Each group was tested for its significance from zero by one-sample sign tests.
The experiments in Figure 7 followed a three-group design with animals of the experimental genotype expressing ChR2-XXL. After an initial Kruskal–Wallis test across all groups, we performed pairwise Mann–Whitney U tests for the group without blue light exposure after training and both the other groups. Significance from zero was tested for each group (one-sample sign tests).
The experiments in Figures 9 and 10 followed a four-group design with animals of the experimental genotype expressing ChR2-XXL. After an initial Kruskal–Wallis test across all groups, we performed pairwise Mann–Whitney U tests between groups that had received paired and unpaired training within the same test condition, and between test conditions for a given kind of training (total four Mann–Whitney U tests). No one-sample sign tests were performed.
We present behavioral data as box plots with the median as the middle line and 25%/75% and 10%/90% as box boundaries and whiskers, respectively. Outliers are not displayed. Experimenters were blind to genotype and regarding the absence or presence of sugar reward during the recall test, if applicable. Sample sizes (biological replications) were chosen based on previous studies revealing moderate to mild effect sizes (Paisios et al., 2017; Saumweber et al., 2018) and are displayed in the figure legends. A sample size of n = 1 included ∼20 animals of both sexes for all innate preference experiments, and 2 × 20 animals (20 for each reciprocally trained group) of both sexes for all learning experiments.
To compare synapse numbers between the ipsilateral and contralateral side of the MBON-i1 neurons (see Fig. 3A), we tested for their distribution using χ2 tests, separately for KC→MBON-i1 synapses andDAN-i1→MBON-i1 synapses. To compare the distances of synapses to the center of their respective center-surround structure on the DAN-i1 neuron (see Fig. 4C,D), we performed Kruskal–Wallis tests across all synapse types, as well as pairwise Mann–Whitney U tests between all synapse types (total 6 Mann–Whitney U tests), separately for the left and the right DAN-i1 neuron.
Results
Spatial arrangement of synapses in a DAN-KC-MBON matrix
The recent electron microscope reconstruction of a complete larval mushroom body circuit has provided a connectome of the DANs, KCs, and MBONs based on the number of chemical synapses between them (Eichler et al., 2017). The axonal branches of individual DANs and the dendritic branches of individual MBONs are each confined to, and overlap within, specific compartments along the KC axon bundles (larvae: Pauls et al., 2010; Eichler et al., 2017; Saumweber et al., 2018; Eschbach et al., 2020; adults: Aso et al., 2014a; Takemura et al., 2017). DAN-i1 innervates the i-compartment of the mushroom body medial lobe (sometimes called the upper toe of the medial lobe). The reconstruction revealed that it not only signals to KCs, but actually has two main targets: the KCs and MBON-i1 (Fig. 2A,B) (Eichler et al., 2017). Correspondingly, MBON-i1 has two main upstream partners: the KCs and DAN-i1. A similar organization of DANs and MBONs is found across mushroom body compartments (larvae: Eichler et al., 2017; adults: Takemura et al., 2017). Specifically, for DAN-i1, ∼75% and 21% of its output synapses in the mushroom body connect to KCs and MBON-i1, respectively, and MBON-i1 receives ∼74% and 14% of its input synapses in the mushroom body from KCs and DAN-i1, respectively (Eichler et al., 2017; Saumweber et al., 2018). Although such analyses are informative in many ways, they do not consider the spatial arrangement of synapses along the neurons. To shed light on this arrangement in the i-compartment, we use dendrograms that allow us to display the complex, 3D Gestalt of a neuron as a 2D sketch that captures its branching structure, preserves branch lengths, and identifies synapse location along the branches (Strauch et al., 2018). Such detailed analyses surpass the resolution by which neurons can presently be manipulated in freely behaving animals (Aso and Rubin, 2020). Still, they offer a glimpse of the actual complexity of the neurons under study, and of the computational implications of the circuit motifs they are part of (see also Bilz et al., 2020). From our analyses, we highlight the following points (high-resolution images of the DAN-i1 and MBON-i1 neurons from both hemispheres as well as of all mature KCs are available in Extended Data Figs. 2-1, 2-2, and 2-3, respectively):
Both DAN-i1 and MBON-i1 harbor massive interhemispheric connections. This is remarkable given how little interhemispheric crosstalk is otherwise observed along ascending pathways, including the KCs (Fig. 2A) (Berck et al., 2016; Eichler et al., 2017; Saumweber et al., 2018; Thum and Gerber, 2019). Specifically, both DAN-i1 neurons are reciprocally connected with most mature KCs, in both hemispheres. Moreover, the DAN-i1 neurons provide output to both MBON-i1 neurons, both ipsilaterally and contralaterally (Fig. 2E; for high-resolution images of both DAN-i1 neurons, see Extended Data Fig. 2-1). This implies that each MBON-i1 neuron in turn receives input from both DAN-i1 neurons, both ipsilaterally and contralaterally (Fig. 2F; for high-resolution images of both MBON-i1 neurons, see Extended Data Fig. 2-2). Likewise, the MBON-i1 neurons receive input from most mature KCs, in both hemispheres (Eichler et al., 2017), and provide output outside the mushroom body, also in both hemispheres. The dendrograms more specifically reveal that the outputs from DAN-i1 onto the MBON-i1 neuron of the ipsilateral and contralateral hemisphere are intermingled, as are the inputs to MBON-i1 from the DAN-i1 neuron of the ipsilateral and the contralateral hemisphere (Fig. 2E,F). Notably, MBON-i1 is not hemispherically symmetrical in terms of the number of synapses it receives in the mushroom body: both MBON-i1 neurons receive more synapses in the hemisphere contralateral to their cell bodies (Figs. 2F, 3A,B); nevertheless, both MBON-i1 neurons receive information of the complete sensory input space in either hemisphere (Fig. 3C,D).
The branching patterns of both DAN-i1 and MBON-i1 are structured into few major regions. For DAN-i1, three such major regions can be identified. Starting from the cell body, in the first region, DAN-i1 receives input from its partners from outside the mushroom body (marked with 1 in Fig. 2C–E) (Eschbach et al., 2020), whereas the second and third region coincide with the i-compartment of the ipsilateral and contralateral mushroom body, respectively (marked with 2 and 3 in Fig. 2C–F). In the mushroom body, DAN-i1 is reciprocally connected to KCs and provides output toward MBON-i1. The primary neurite of MBON-i1 crosses the midline, and the neuron then innervates the i-compartment of the mushroom body contralateral to its cell body (region marked with 3 in Fig. 2C–F); here, it mainly receives input from KCs and DAN-i1. Then, MBON-i1 crosses back over the midline and likewise innervates the mushroom body ipsilateral to its cell body (region marked with 2 in Fig. 2C–F). In addition, MBON-i1 features several small branches in both hemispheres that connect to partners close to, but immediately outside, the mushroom body.
Within these regions, compartment-like divisions that are recognizable in the DAN-i1 and in the MBON-i1 neurons of both hemispheres cannot be identified (Fig. 2E,F; Extended Data Figs. 2-1, 2-2). However, for the DAN-i1 neurons, multiple, laterally “idiosyncratic” synapse-rich center-surround structures are apparent (Fig. 4). Toward their center, all the above-mentioned types of synapse of the DAN-KC-MBON matrix are found, whereas their surround features almost exclusively KC→DAN synapses (Fig. 4). Notably, any given KC usually establishes all its synapses in only one or two of those center-surround structures (Extended Data Fig. 1-1).
Regarding mature KCs, 86% and 88% of them establish synapses with the left and right DAN-i1 neuron, respectively, and 78% and 74% with the left and right MBON-i1 neuron, respectively. Individual KCs have medians of 4 KC→DAN-i1, 4 DAN-i1→KC, and 5 KC→MBON-i1 synapses (Eichler et al., 2017; for more details on DAN-i1, see Saumweber et al., 2018). According to the dendrograms of all the individual KCs, the KC→DAN-i1, DAN-i1→KC, and KC→MBON-i1 synapses appear to be intermingled with no apparent divisions across the array of KCs, as do the synapses with the left and the right DAN-i1 neuron, and the left and the right MBON-i1 neuron (Extended Data Fig. 2-3).
Circuitry of the DAN-KC-MBON matrix. A, The medial lobe chiasm. Odor information is processed to induce innate approach, as well as toward the calyx region of the mushroom body KCs (purple; displayed fully is one example KC on the right side). Taste reward information is likewise processed to guide innate behavior, as well as via mostly aminergic modulatory neurons (displayed is DAN-i1, turquoise/black) toward individual compartments of the mushroom body lobes (on the left side, compartments are indicated). Each of the four compartments of the medial lobe (h-k) receives input from a single DAN that also innervates the same compartment in the contralateral hemisphere. The MBONs of the medial lobe draw from single compartments, typically of both hemispheres (displayed is MBON-i1, magenta/yellow), and deliver their output, in configurations that differ between compartments, across the hemispheric divide. Collectively, the MBONs are responsible for memory-based search behavior. Data are based on Eichler et al. (2017) and Saumweber et al. (2018). B, The DAN-KC-MBON matrix in the i-compartment. DAN-i1 and most KCs establish mutual chemical synapses, and both DAN-i1 and KCs provide output to MBON-i1. For simplicity, only synapses with the left DAN-i1 and MBON-i1 neurons are shown in detail. The same connectivity is found in most, if not all, other compartments (Eichler et al., 2017; Saumweber et al., 2018). C, DAN-i1 (turquoise) receives input from outside the mushroom body (dark gray) ipsilateral to its cell body (region labeled 1), and then innervates the i-compartment in both hemispheres (labeled 2 and 3). MBON-i1 (magenta) crosses the midline twice; it receives input in the i-compartment of both hemispheres and provides output in regions adjacent to the mushroom body of both hemispheres. D, Same as in C, but in a slightly tilted view, showing skeleton reconstructions of the left DAN-i1 and MBON-i1 neurons and one example KC (black). Data are based on Eichler et al. (2017). E, Dendrogram of the left DAN-i1 neuron. Colored dots represent output synapses from DAN-i1. Triangles represent input synapses to DAN-i1. Colors represent the partner neuron of the respective synapse. All other synapses within the mushroom body are labeled “Other.” Not displayed are synapses that DAN-i1 forms with neurons outside the mushroom body, in particular in region 1 (Eschbach et al., 2020). Stippled lines indicate regions where the neuron crosses the midline of the brain. F, Same as in E, but for the left MBON-i1 neuron. Not displayed are synapses that MBON-i1 forms with neurons outside the mushroom body. For high-resolution versions of the DAN-i1 and MBON-i1 neurons of both hemispheres, as well as all mature KCs, see Extended Data Figures 2-1, 2-2, and 2-3, respectively.
Figure 2-1
Figure 2-2
Figure 2-3
Together, these analyses reveal the detailed spatial organization of the DAN-KC-MBON matrix of the i-compartment. Of note is that the synapses of DAN-i1 with its two main targets, the KCs and MBON-i1, are locally intermingled within the multiple synapse-rich structures along the axonal branches of DAN-i1, rather than being separated out between them (Figs. 2E, 4; Extended Data Fig. 2-1). Such architecture might imply that DAN-i1 delivers correlated, rather than independently modulated, signals to the KCs and MBON-i1 within these synapse-rich structures.
Organization of synaptic inputs to the MBON-i1 neuron. A, Number of KC→MBON-i1 and DAN-i1→MBON-i1 synapses to each MBON-i1 neuron, separated by hemisphere. Both MBON-i1 neurons have more synapses with those KCs and that DAN-i1 located in the hemisphere contralateral to their cell body. *Significant χ2 tests (p < 0.05, corrected according to Bonferroni-Holm). Data are from Eichler et al. (2017). B, Semischematic overview of the distribution of synapses from the ipsilateral and contralateral hemisphere to MBON-i1. KCs and both DAN-i1 neurons form more synapses with the contralateral part of MBON-i1 than with its ipsilateral part. C, Distribution of the number of mature KCs forming the indicated number of synapses with the left or right MBON-i1 neuron. KCs from the left hemisphere form fewer synapses with the left than the right MBON-i1 neuron (left). KCs from the right hemisphere, conversely, form more synapses with the left than the right MBON-i1 (right). Given that KCs do not cross the midline, all synapses that, e.g., the right MBON-i1 formed with KCs from the left hemisphere are located on the part of MBON-i1 contralateral to its cell body. D, Despite the fact that the MBON-i1 neurons receive less input from the KCs of the hemisphere ipsilateral to their cell bodies than from the contralateral hemisphere, both MBON-i1 neurons receive information about the full space of KC inputs, in either hemisphere. To determine the relationship between MBON-i1 and the sensory input projected onto the KCs, the total input that MBON-i1 receives from KCs that in turn receive their input from uniglomerular olfactory projection neurons (PNs) (white and dark gray), multiglomerular olfactory PNs (light gray), and nonolfactory PNs (black) were taken into account to calculate the matrix product of the respective PN→KC and KC→MBON-i1 synapses. This was done separately for MBON-i1 left and its connections to the right (i.e., contralateral) KCs (top left), and those to the left (i.e., ipsilateral) KCs (top right), and accordingly for MBON-i1 right (bottom). Data are based on Eichler et al. (2017). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
Organization of synapses on the DAN-i1 neuron. A, Cluster analysis revealed that synapses from and to the left DAN-i1 neuron in region 2 and 3 (see Fig. 2) are arranged in six clusters (labeled as 2′, 2′′, etc.). These clusters appear to have a center-surround organization: toward their center, DAN→KC, KC→DAN, and DAN→MBON synapses are found, whereas their surround features almost exclusively KC→DAN synapses. B, Same as in A, but for the right DAN-i1 neuron. Seven center-surround structures can be identified. C, To quantify this organization of synapses, we determined the distance (in cable length along the neuron) of different types of synapses to the center of their respective cluster, and found that KC→DAN synapses are further from the center than all other types of synapses. This confirms the organization of the clusters as center-surround structures. D, Same as in C, but for the right DAN-i1 neuron. E, Schematic sketch of the center-surround structure (top) and a magnification of the connections in the center (bottom). Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
Activation of the same neurons can establish memory and terminate its recall
We were intrigued by the fact that DAN-i1 has two main targets, namely KCs and MBON-i1; that its synapses with these two targets are so closely intermingled; and that natural rewards have two functions, namely, to establish a memory in training and to acutely terminate the behavioral expression of this memory during the recall test (Fig. 1A) (Gerber and Hendel, 2006; Schleyer et al., 2011, 2013, 2015a,b, 2018; Paisios et al., 2017). We therefore wondered whether optogenetic activation of DANs in training and during the test could confer these two functions. We started out using the 58E02-Gal4 driver strain that covers DAN-i1 plus two DANs innervating other compartments of the medial lobe (Fig. 5A,B) (Rohwedder et al., 2016), and UAS-ChR2-XXL as the effector strain (Dawydow et al., 2014). The 58E02-DANs were activated either paired or unpaired with odor in training, and the difference in odor preference in a subsequent recall test was used to quantify associative memory through the PI. We observed positive PI scores indicating appetitive associative memory in the experimental genotype, whereas genetic controls heterozygous for either only the transgenic effector or the driver construct did not show memory (Fig. 6A, top, black-filled box plots) (Rohwedder et al., 2016). Thus, 58E02-DAN activation during training can provide a reward signal for associative memory formation.
Transgene expression pattern of the 58E02-DAN and 864-DAN driver strains. A, B, A previous flip-out-based analysis of the 58E02-DAN driver strain identified the neurons DAN-h1, DAN-i1, and DAN-j1 (also called pPAM1, pPAM3, and pPAM4), as well as one additional neuron outside of the mushroom body (Rohwedder et al., 2016). This pattern was confirmed: the driver strain was crossed to either (A1-A5) pJFRC-10xUAS-IVS-mCD8::GFP or (B1-B4) UAS-ChR2-XXL. A1, B1, Signal of the α-GFP or α-ChR antibody, respectively. A2, B2, Signal of the background staining via the α-FASII or α-HRP antibody, respectively. A3, B3, Merge of the respective signals. A4, B4, A partial z projection reveals strong innervation of the medial lobe of the mushroom body. A5, No innervation of the lateral appendix is detected (arrow). C, D, The 864-DAN driver strain has been reported to express strongly in DAN-i1 (Saumweber et al., 2018). Here, we crossed it to either (C1-C5) pJFRC-10xUAS-IVS-mCD8::GFP or (D1-D4) UAS-ChR2-XXL. C1, D1, Signal of the α-GFP or α-ChR antibody, respectively. C2, D2, Signal of the background staining via the α-FASII or α-HRP antibody, respectively. C3, D3, Merge of the respective signals. C4, D4, A partial z projection reveals strong innervation of the i-compartment in both hemispheres (arrows), confirming expression in DAN-i1. C5, As previously reported (Saumweber et al., 2018), the driver strain occasionally expresses in additional neurons, but if so, typically in only one brain hemisphere, and in neurons differing in identity between preparations. Relatively most frequently, we observed the MBON-d1 neuron, innervating the lateral appendix in one hemisphere (arrow). Across the multiple animals used for our group-based assay, such stochastic additional expression is unlikely to cause systematic effects.
Activation of the same neurons can establish memory and terminate its recall. A, Top, Larvae were trained with odor presented either paired or unpaired with 58E02-DAN activation by blue light. In the recall test, positive PI values in the experimental genotype, but not in the genetic controls, indicate learned search behavior based on associative memory for the odor (leftmost box plot). Thus, during training, 58E02-DAN activation can mediate a reward signal. The memory-based search of the experimental genotype was tuned down when 58E02-DAN was activated by blue light during the recall test as well (leftmost blue-filled box plot). Thus, 58E02 activation can also confer a search termination signal. Sample sizes: 16, 15, 17, 18, 15, 18. Bottom, Larvae were tested for innate odor preference, either in the absence or the presence of blue light. All genotypes preferred the odor equally strongly under both conditions, suggesting that activation of 58E02-DAN had no effect on innate odor preference. Sample sizes: 24 each. B, Same as in A, but with 864-DAN activation by blue light. Top, A positive PI, and thus memory-based search, was observed only in the experimental genotype and only when the recall test was performed in darkness. Sample sizes: 19, 19, 19, 19, 19, 20. Bottom, 864-DAN activation had no effect on innate odor preference. Sample sizes: 16 each. C, In a replication of the experiment shown in A (top) for the experimental genotype, the larvae showed memory-based search in darkness. When 58E02-DAN was activated by blue light during the recall test, this search was tuned down. Sample sizes: 15, 16. D, In a replication of the experiment shown in B (top) for the experimental genotype, the larvae exhibited memory-based search in darkness. When 864-DAN was activated during the recall test, this search was terminated. Sample sizes: 28, 28. E, In a further replication of D, the test was performed on a Petri dish of 9 cm inner diameter instead of 15 cm. Memory-based search during the recall test was observed in darkness and, in this case, was partially tuned down through 864-DAN activation. Sample sizes: 22, 22. F, Larvae were trained in a differential, two-odor version of the experiment, using n-amyl acetate diluted 1:50, and undiluted 1-octanol. The recall test was performed on a Petri dish of 9 cm inner diameter. Memory-based search was observed in darkness, and was tuned down through 864-DAN activation. Sample sizes: 26, 26. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). *Significance from zero (one-sample sign test, p < 0.05, corrected according to Bonferroni-Holm). ns: Nonsignificance across all experimental conditions (Kruskal–Wallis-test, p > 0.05). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
Next, we repeated the experiment but activated 58E02-DAN during the recall test, too. Doing so reduced PI values (i.e., reduced memory-based search) to approximately half (Fig. 6A, top, blue-filled box plots; for a replication of this result, see Fig. 6C). Critically, just as is the case for natural rewards (Fig. 1B) (Schleyer et al., 2011, 2015a,b), 58E02-DAN activation did not affect innate olfactory behavior (Fig. 6A, bottom). Thus, 58E02-DAN activation can also provide an acute search termination signal during the recall test.
The previous experiments demonstrate that the three 58E02-DANs can confer the same two signals as a natural reward (i.e., the reward signal during the training and the search termination signal during the recall test). However, it remains unclear whether each of the three DANs confers both signals, or whether there is a division of labor, with different DANs being responsible for the reward signal and the search termination signal. We therefore chose the split-Gal4 driver strain 864, which supports strong and reliable transgene expression in only one of the DANs covered by 58E02-DAN, the DAN-i1 neuron, in both hemispheres (Fig. 5C,D) (Saumweber et al., 2018). Associative 864-DAN activation in training established positive performance indices (i.e., memory-based search; Fig. 6B, top, black-filled box plots) (Saumweber et al., 2018). Activating 864-DAN also during the recall test terminated this memory-based search (Fig. 6B, top, blue-filled box plots). Three independent datasets confirm the observation of search being tuned down (Fig. 6D–F). Innate olfactory behavior was not affected by 864-DAN activation (Fig. 6B, bottom).
Activity of DANs between training and the recall test (i.e., during the retention period) has previously been found to promote forgetting in adult D. melanogaster (Berry et al., 2012, 2015; Shuai et al., 2015; Aso and Rubin, 2016). We therefore wondered whether, in our paradigm, 864-DAN activity would promote forgetting, as opposed to acutely terminating memory-based search and leaving memory intact. To test for this possibility, larvae were trained as before, but before the recall test, we introduced a waiting period. During this period, the larvae were either kept in darkness or exposed to blue light for 3 min for 864-DAN activation (Fig. 7A). When the recall test was conducted in darkness, significant memory-based search was observed regardless of 864-DAN activation during the waiting phase (Fig. 7A, left and middle box plot). This suggests that 864-DAN activation during the waiting period does not promote forgetting.
Activation of 864-DAN between training and the recall test does not enhance forgetting or extinction. A, Groups of larvae were trained as in Figure 6. Then, the larvae were kept either in darkness or blue light for 3 min (Wait), and were subsequently tested. When that recall test was conducted in darkness, memory-based search was observed independent of the light condition, and thus independent of 864-DAN activity, during the waiting phase (left and middle box plot). We confirmed that also after such a waiting period 864-DAN activation during the recall test terminates memory-based search (right box plot). Sample sizes: 25, 24, 19. B, Same as in A, but this time with the odor present during the waiting period. Sample sizes: 28 each. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). *Significance from zero (one-sample sign test, p < 0.05, corrected according to Bonferroni-Holm). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
Indirect activation of DANs, via two-step cross-compartmental feedback from MBONs, has recently been discovered to promote extinction in adult flies (Felsenberg et al., 2018). Extinction refers to the learning that takes place when the odor, but no external reinforcement, is presented. To see whether 864-DAN activation would promote this process, we repeated the experiment, but this time presented the previously trained odor during the waiting period (without external reinforcement), either without or including 864-DAN activation. Again, we observed robust memory-based search in the recall test in both cases (Fig. 7B, left, and middle box plot). 864-DAN activation fully terminated learned search in both experiments (Fig. 7A,B).
Together, these results suggest that activation of 864-DAN, reliably covering only the DAN-i1 neuron, does not promote forgetting or extinction. Rather, 864-DAN activation can confer both a reward signal for memory formation during training and an acute search termination signal during the recall test.
Given that the 58E02- and 864-DAN neurons, just like natural rewards, can mediate both reward and search termination signals, we decided to investigate further the behavioral relation between the signals carried by these neurons and natural reward.
Relation of 58E02- and 864-DAN activation to sugar reward
To test for the relationship between the signals carried by 58E02- and 864-DANs and a natural sugar reward, we took advantage of the nature of memory-based behavior as a search. Regarding natural rewards, we have previously shown that memory-based search for a reward is terminated by a reward only if the sought-for reward matches the reward that is present (Schleyer et al., 2011, 2013, 2015a). Thus, if the activation of these neurons and sugar reward were plainly to “mean the same thing” to the larvae, then the search for one should be fully and mutually terminated by the presence of the other. However, in the case of 58E02-DAN, this effect is partial and not mutual; and in the case of 864-DAN, it is full yet not mutual, either (Fig. 8):
Memory-based search induced by 58E02-DAN activation during training was tuned down to about half in the presence of sugar during the test (Fig. 8A). Using a two-odor, differential conditioning paradigm, the same result was observed (Fig. 8E). In contrast, sugar did fully terminate memory-based search when the more specific 864-DAN activation was used for training (Fig. 8B; for two repetitions of this experiment, see Fig. 8F,G). These results can be explained by a representation of sugar reward that partially covers the 58E02-DANs, and fully covers the 864-DANs. Whether these DANs are necessary for the observed effects of sugar on search behavior is not known.
Second, we asked whether the memory-based search for sugar can in turn be terminated by activating the 58E02-DANs or 864-DANs, that is, whether activation of these DANs would be sufficient to terminate the search for sugar. This was not the case in the present type of assay (Fig. 8C,D). Thus, with respect to its behavioral significance, there are major aspects of the sugar reward representation that are covered neither by the 58E02-DANs nor the 864-DANs, with the result that the search for sugar continues despite the search termination signal conveyed by these neurons.
With respect to the behavioral relevance in the present type of assay, it is thus a plausible working hypothesis that the representation of sugar reward partially overlaps with 58E02-DAN, and fully encompasses 864-DAN (Fig. 8H). In order to lend further plausibility to this scenario, we investigated whether 864-DAN activation would share additional basic behavioral features with sugar reward. First, we asked whether 864-DAN activation, like sugar, can induce memories of opposite valence in a contingency-dependent manner.
Relation of 58E02- and 864-DAN activation to sugar reward. A, Larvae were trained with 58E02-DAN activation as in Figure 6. Memory-based search is expressed when the recall test is conducted in the absence of sugar but is tuned down in its presence. Sample sizes: 31, 21. B, Same as in A, but with 864-DAN activation. Memory-based search is observed when the recall test is conducted in the absence of sugar but is terminated in its presence. Sample sizes: 24 each. C, Larvae were trained such that an odor was presented either paired or unpaired with sugar. Memory-based search for sugar was observed regardless of whether or not 58E02-DAN was activated by blue light during the recall test. Sample sizes: 19, 17. D, Same as in C, but with 864-DAN activation. Memory-based search for sugar was observed regardless of whether or not 864-DAN was activated by blue light during the recall test. Sample sizes: 18, 18. E, The experiment shown in A, using 58E02-DAN, was replicated using a differential, two-odor version of the experiment with n-amyl acetate diluted 1:50 and undiluted 1-octanol. Each training trial lasted for 5 min, and training and test were performed on a Petri dish of 9 cm inner diameter. The larvae showed memory-based search in the absence of the sugar reward. When sugar was presented during the recall test, however, memory-based search was tuned down. Sample sizes: 14, 13. F, In a replication of the experiment shown in B, using 864-DAN, both training and test were performed on a Petri dish of 9 cm inner diameter. The larvae showed memory-based search in the absence but not in the presence of the sugar reward. Sample sizes: 10, 11. G, In a further replication of the experiment shown in B, the same result was observed. Sample sizes: 19, 18. H, The results from A–G suggest, as a working hypothesis, that the representation of sugar reward partially overlaps with 58E02-DAN, and fully encompasses 864-DAN. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). *Significance from zero (one-sample sign test, p < 0.05, corrected according to Bonferroni-Holm). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
864-DAN activation induces memories of opposite valence in a contingency-dependent manner
After paired training with odor and sugar, odor preferences during the test are increased relative to baseline, whereas after presenting odor and sugar in an unpaired manner (i.e., during separate trials), the odor preferences are decreased (Fig. 9A) (Saumweber et al., 2011; Schleyer et al., 2011, 2015b; Paisios et al., 2017; for review, see Schleyer et al., 2018). This conforms to widely applied learning theory (Rescorla and Wagner, 1972; Sutton and Barto, 1981; Malaka, 1999) and makes intuitive sense as well: after paired odor-sugar training, the odor predicts where sugar can be found; whereas after presentations of odor separately from sugar, the odor predicts precisely where sugar cannot be found. In both cases, the animals' behavior reflects a memory-based search for sugar, yet their memory guides them in opposite directions relative to the odor. Optogenetic activation of 864-DAN establishes both these types of memory (Fig. 9B), suggesting that, as for sugar, the reward signal carried by 864-DAN can induce memories of opposite valence depending on the contingency of its occurrence with the odor presentation. Strikingly, the resulting behavioral tendencies, memory-based odor approach and aversion, are both terminated by 864-DAN activation during the test (Fig. 9C). Thus, like sugar, activating 864-DAN provides a reward signal that can induce memories of opposite valence during training, and a signal to terminate the search behavior based on either kind of memory during the test. We next asked whether more specifically 864-DAN activation results in the same “microbehavioral” modulations of search behavior as sugar.
864-DAN activation induces memories of opposite valence in a contingency-dependent manner. A, After paired odor-sugar training, the larvae approached the odor when tested on plain agarose; after unpaired training, they avoided the odor. This difference was abolished in the presence of sugar, meaning that the larvae behaved independently of their odor-sugar associative memory. Therefore, the data from this condition can be combined as a baseline against which the effects of associative memory can be assessed (stippled line). This shows that paired and unpaired training with a natural reward establishes memories of opposite valence, leading to increased or decreased preference compared with baseline, respectively, during the recall test (for review, see Schleyer et al., 2018). Sample sizes: 29, 29, 28, 28. These data underlie the PI scores presented in Figure 1A. B, After paired training with 864-DAN activation, too, the odor preference is increased, whereas after unpaired 864-DAN training, it is decreased, compared with the baseline preference when the recall test is performed in the presence of sugar. This demonstrates that the reward signal carried by 864-DAN can establish memories of opposite valence, depending on the contingency with the odor. Sample sizes: 43, 42, 42, 43. These data are pooled from the experiments presented in Figure 8B, G. C, Just like sugar, activation of 864-DAN during the recall test terminates both the memory-based odor approach after paired training and the memory-based odor avoidance after unpaired training. Sample sizes: 47 each. These data are pooled from the data presented in Figure 6B, D. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). All statistical tests and their results are reported along with the source data in in Extended Data Figure 1-1.
864-DAN activation affects the same aspects of search behavior as sugar
Larval behavior in an odor gradient can be characterized by relatively straight runs, interrupted by lateral HCs that are followed by changes of direction (Fig. 10A,B). We find that search based on a 864-DAN memory, just like memories for sugar (Schleyer et al., 2015b; Paisios et al., 2017; Thane et al., 2019), can be characterized by modulations of both these two aspects of behavior, namely, the rate of HCs, and their direction (Fig. 10C,D) (Saumweber et al., 2018):
After paired training with odor and 864-DAN activation, larvae made more HCs while heading away from the odor than while heading toward it (Fig. 10C, leftmost box plot showing HC rate modulation > 0). Such a modulation of their HC rate brings the animals closer to the odor source. After unpaired training with odor and 864-DAN activation, the opposite was observed (Fig. 10C, second box plot from left showing HC rate modulation < 0).
In addition, after paired training, the animals directed their HCs more toward the odor source than after unpaired training (Fig. 10D, black-filled box plots).
We next wondered whether 864-DAN activation during the test can also terminate both these behavioral effects, as has been reported for sugar (Schleyer et al., 2015b; Paisios et al., 2017), and indeed found this to be the case (Fig. 10C,D, blue-filled box plots). Thus, the reward signal carried by 864-DAN causes the same modulations of memory-based search as sugar, and the search termination signal carried by 864-DAN, just like sugar, terminates both these behavioral modulations.
864-DAN activation affects the same aspects of search behavior as sugar. A, Sample track from a video recording of a larva with runs and HCs. An HC is detected whenever the angular speed of the head exceeds ±35°/s (for details, see Paisios et al., 2017). B, When the recall test is performed in darkness, larvae typically approached the odor after paired training with odor and 864-DAN activation, and avoided the odor after unpaired training (black-filled circles). When tested in light (i.e., while 864-DAN was activated), the animals' preference after paired and unpaired training was indistinguishable (blue-filled circles). Four example tracks for each condition are shown. Arrowheads indicate the start and the end of the tracks. C, After paired training with odor and 864-DAN activation, larvae modulated their HC rate such that they made relatively more HCs while heading away from the odor source than while heading toward it. After unpaired training, the opposite modulation was observed. When 864-DAN was activated during the recall test, larvae behaved independently of their previous training. D, After paired training with odor and 864-DAN activation, larvae directed their HCs more toward the odor source than after unpaired training. When 864-DAN was activated during the recall test, larvae behaved independently of their training. For this analysis, data from the experiments displayed in Figure 6B, D were used. Sample sizes from left to right: 46, 48, 48, 45. Different lettering above the box plots indicates pairwise significance (Mann–Whitney U test, p < 0.05, corrected according to Bonferroni-Holm). All statistical tests and their results are reported along with the source data in Extended Data Figure 1-1.
Discussion
The current study shows that the optogenetic activation of DANs can have two effects: it can confer a reward signal during training such that associated odors are learned and can later be used by the animal to direct its search for the reward. And during the recall test, the activation of the same DANs can confer a signal to acutely terminate that very search. Before we discuss which of the synaptic partners of the DANs might be receiving these signals, we compare the effects of DAN activation to those of sugar as a natural reward.
The relationship between natural rewards and optogenetic DAN activation
Optogenetic 864-DAN activation, just like sugar as a natural reward (Fig. 9A) (Saumweber et al., 2011; Schleyer et al., 2011, 2015b; Paisios et al., 2017; for review, see Schleyer et al., 2018), can establish memories of opposite behavioral valence in larvae, depending on the contingency with the odor (Fig. 9B). Thus, 864-DAN activation as such does not signal valence. Rather, as for natural rewards, the valence of 864-DAN activation arises only on convergence with olfactory processing. Extrapolating from what has been established in adult D. melanogaster, this might correspond at the physiological level to the depression of the KC→MBON synapse observed after paired odor-DAN training, and its potentiation after their unpaired presentation (Cohn et al., 2015; see also Yarali et al., 2012; Handler et al., 2019). Importantly, memories from both paired and unpaired 864-DAN training can be prevented from behavioral expression by optogenetically activating 864-DAN during the recall test (Fig. 9C), as is the case for sugar (Fig. 9A) (Schleyer et al., 2018). This shows that the search termination signal likewise pertains to memories established by 864-DAN, regardless of their behavioral valence.
Olfactory memories from 864-DAN activation of either valence further resemble sugar memories in the specific modulations of memory-based search behavior (Fig. 10) (Paisios et al., 2017; Saumweber et al., 2018). Moreover, both 864-DAN activation and sugar can also terminate the opposing modulations of behavior underlying learned search behavior after paired and unpaired training (Fig. 10).
Furthermore, the search termination signal from 864-DAN activation, like that conferred by sugar (Fig. 1) (Schleyer et al., 2011, 2015a,b), affects only memory-based olfactory search, but not innate odor preference (Fig. 6). This specific effect of 864-DAN activation on only learned behavior contrasts with the recently reported DAN-mediated enhancement of innate food exploitation behavior in C. elegans (Oranth et al., 2018).
Finally, the presence of sugar terminates memory-based search after 864-DAN training, suggesting that, after such training, larvae may be searching for a sugar-like reward (Fig. 8B).
However, the search termination signal conveyed by 864-DAN activation is sufficient to terminate search after 864-DAN training (Figs. 6, 7) but not after sugar training (Fig. 8D), consistent with 864-DANs being only a subset of the neurons representing sugar reward (Fig. 8H). Moreover, whether in an experiment the activation of the 864-DANs is necessary for sugar to terminate search is not known. Thus, our results suggest that in its behavioral relevance 864-DAN activation bears some resemblance to a sugar reward but certainly does not equal it.
Who receives the DAN signals?
Given the two effects of DAN activation (i.e., to confer a reward and a search termination signal) and given that DANs have two main targets (i.e., the KCs and the MBONs), the question arises what the effects of receiving a DAN signal are in KCs and MBONs, respectively.
Concerning the KCs, it is established that during training the coincidence of odor activation and dopaminergic input is detected by the type I adenylate cyclase and turned into presynaptic depression of the KC→MBON synapse (for larvae: for review, see Widmann et al., 2018; Thum and Gerber, 2019; adults: Cognigni et al., 2018; Tumkaya et al., 2018). For DANs carrying a reward signal, this would lead to less drive toward avoidance-promoting MBONs and hence to odor attraction based on the residual, intact drive from KCs to approach-promoting MBONs in other compartments (adults: Aso et al., 2014b; Hige et al., 2015; Owald et al., 2015; Felsenberg et al., 2018). Presenting the odor unpaired from DAN activation can lead to potentiation of the KC→MBON synapse (Cohn et al., 2015), a process that is not yet understood at the molecular level. Thus, during training, signaling from DANs can have opposite effects on the KC depending on the activity state of the KC (see also Aso et al., 2019; Handler et al., 2019). In the present case, such depression/potentiation of the KC→MBON synapse could support the odor approach/avoidance we observe after odor presentation paired/unpaired from 864-DAN activation, respectively (Fig. 9B). In summary, the reward signal generated by optogenetic 864-DAN activation is very likely delivered via the DAN→KC synapses.
Much less is known about the MBONs, almost exclusively from adult flies. However, the fact that they are required for the expression of learned behavior (larvae: Saumweber et al., 2018; adults: Sejourne et al., 2011; Placais et al., 2013; Bouzaiane et al., 2015; Owald et al., 2015; Shyu et al., 2017; Wu et al., 2017) makes them plausible candidates to receive a search termination signal from the DANs during the recall test. Driving adult DANs has a monosynaptic, excitatory effect on MBONs that is mediated by dopamine (Takemura et al., 2017), and adult MBONs express all four types of D. melanogaster dopamine receptors (Perisse et al., 2016; Crocker et al., 2016; Takemura et al., 2017; Aso et al., 2019). Whether any of these receptors mediates a search termination signal remains unknown, however. Indeed, DAN presynapses can harbor both dense-core and clear vesicles, suggesting that they can use an additional neurotransmitter (adults: Takemura et al., 2017; larvae: Eichler et al., 2017), and some DANs in adults can signal by nitric oxide (Aso et al., 2019). In any case, a plausible working hypothesis is that after paired training the activation of 864-DAN during the test would increase the reduced MBON activity back to baseline levels (Takemura et al., 2017), such that the balance between avoidance- and approach-promoting MBONs was restored and learned behavior was terminated. After unpaired training, according to such a scenario, the enhanced MBON activity would be decreased by DAN activation. In other words, very similar to what was discussed above for the effect of DAN activation on KCs, DAN activation might also have opposite effects on the MBONs, in this case depending on the activity state of the MBON. We would like to add that an alternative would be to use the DAN signal to short-circuit avoidance- and approach-promoting MBONs, such that the net output of the MBON network would be neutral. Indeed, adult MBONs express innexin genes (Aso et al., 2019), and there is a precedent for dopamine-dependent electrical coupling in cardiac motor neurons in crabs, Mauthner cells in fish, and the mammalian retina (Cachope and Pereda, 2012; Lane et al., 2018; Roy and Field, 2019). In summary, although it is a plausible working hypothesis that the search termination signal from the DANs operates through the MBONs, direct evidence is still lacking.
A general principle?
In Pavlovian terminology, the odor in our paradigm corresponds to a conditioned stimulus, sugar to an unconditioned stimulus (US), and optogenetic DAN activation to a US's reinforcing capacity. Furthermore, learned behavior toward the odor in our paradigm corresponds to the conditioned response, whereas behavior toward sugar would correspond to the unconditioned response. Terminating learned search in the presence of the US in Pavlovian terms would thus correspond to a “dominance” of the unconditioned response over the conditioned response. Although this seems to be generally adaptive because USs are of intrinsic value whereas conditioned stimuli are not, it is unknown how generally such a rule applies. Indeed, testing for the applicability of this rule requires testing for the conditioned response in the presence of the US, thereby defying what might be called Clause One of Pavlovian practice. In the present case, doing so has allowed us to reveal that in the brain of larval D. melanogaster the same DANs can mediate on the one hand a reward signal during training to establish associative memory and, on the other hand, a signal that can terminate its behavioral expression. Given the role of DANs in mediating reinforcement signals across animals and humans, we wonder whether this reflects a principle of DAN function.
Footnotes
This work was supported by Otto von Guericke Universität Magdeburg; Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz; Leibniz Institute for Neurobiology; HHMI Janelia Research Campus and the HHMI Janelia Visiting Scientist Program; Deutsche Forschungsgemeinschaft GE 1091/4-1 to B.G. and M. Schleyer, FOR 2705 Mushroom body to B.G., TH1584/3-1, TH1584/6-1 and TH1584/7-1 to A.S.T., and ME3737/3-1 to D.M.; Zukunftskolleg of the University of Konstanz to A.S.T.; and Elite Program of the Baden-Württemberg Stiftung to A.S.T. We thank T. Hübner, C. Jahn, J. Saumweber, N. Schieting, M. Thane, A. Voigt, and M. Wurth for experimental contributions; M. Dombach, M. Paisios, and F. Unterstab for technical assistance; C. Eschbach andJ.W. Truman for support with fly strains and reagents; Y. Aso, C. König, Y. Lozano Navarro, N. Mancini, G. Rubin, P. Tovote, N. Toshima, and A. Yarali for discussions; and R.D.V. Glasgow (Zaragoza, Spain) for language editing.
The authors declare no competing financial interests.
- Correspondence should be addressed to Michael Schleyer at michael.schleyer{at}lin-magdeburg.de or Bertram Gerber at bertram.gerber{at}lin-magdeburg.de
