Abstract
For a proper representation of the causal structure of the world, it is adaptive to consider both evidence for and evidence against causality. To take punishment as an example, the causality of a stimulus is unlikely if there is a temporal gap before punishment is received, but causality is credible if the stimulus immediately precedes punishment. In contrast, causality can be ruled out if the punishment occurred first. At the behavioral level, this is reflected in the associative principle of timing-dependent valence reversal: aversive memories are formed when a stimulus occurs before the punishment, whereas memories of appetitive valence are formed when a stimulus is presented upon the relieving termination of punishment. We map the temporal profile of memories induced by optogenetic activation of the PPL1-01 neuron in the fly Drosophila melanogaster (of either sex) and find that compromising tyrosine hydroxylase function, either acutely by pharmacological methods or by cell-specific RNAi, extends and blunts this profile. Specifically, it (1) enhances learning with a time gap between the stimulus and PPL1-01 punishment (better trace conditioning), (2) impairs learning when the stimulus immediately precedes PPL1-01 punishment (worse delay conditioning), and (3) prevents learning about a stimulus presented after PPL1-01 punishment has ceased (worse relief conditioning). Under conditions of low dopamine, we furthermore observe a role for serotonin that is pronounced in trace conditioning, weaker in delay conditioning, and absent in relief conditioning. We discuss the psychiatric implications if related alterations in the temporal profile of reinforcement were to occur in humans.
Significance Statement
Acting in conformity with the causal structure of the world is important for survival in animals and humans alike. To do so, it is crucial to consider both evidence for and evidence against causality. For example, the causality of a stimulus is a reasonable assumption if it precedes punishment, whereas causality can be ruled out if the punishment occurred first. This is reflected in the opposite memories that are established through the “bad” occurrence versus the “good” termination of punishment. We find in the fruit fly Drosophila melanogaster that compromising dopamine synthesis establishes a distortion of these processes and discuss the psychiatric implications if such distortions were to occur in humans.
Introduction
A proper representation of the causal structure of the world is important for adaptive behavior. Indeed, distortions in the assignment of causes to effects can have consequences ranging from the comical to the lethal. Following early insight into the importance of “constantly conjoined events” (Hume, 1978), causal learning is often studied in paradigms that vary the temporal relationship between cues and motivationally salient events (Shanks et al., 1989; Dickinson, 2001). This may concern evidence in favor of a causal relationship with a punishment, for example, or evidence against such causality. That is, a cue X that has preceded a punishment is evidence for punishment, whereas it is evidence against such causation if the punishment came first and cue X followed it. Accordingly, at the behavioral level and in associative learning experiments, aversive memory for cue X is the result when X occurs before punishment, whereas a characteristically weaker and opposing, appetitive memory is the result when cue X is presented only upon the termination of punishment, at the moment of “relief” (Solomon and Corbit, 1974). Such timing-dependent valence reversal reflects a cross-species principle of reinforcement processing with broad implications in biomedicine and computational science (Malaka, 1999; Gerber et al., 2014, 2019; Silver et al., 2016).
In the fruit fly Drosophila melanogaster, timing-dependent valence reversal is mostly studied for the association between odor cues and electric shock punishment. After odor → shock training, the flies show learned avoidance of the odor, whereas learned approach is observed after shock → odor training (Tanimoto et al., 2004). These memories are called punishment and relief memory, respectively (Gerber et al., 2014, 2019). Punishment learning in Drosophila involves the coincidence of olfactory processing and shock-evoked dopaminergic reinforcement in the mushroom body, the highest brain center of insects (Heisenberg, 2003; Cognigni et al., 2018; Boto et al., 2020; Li et al., 2020; Modi et al., 2020; Menzel, 2022; Davis, 2023). Pairings of odor presentation with the activation of the mushroom body input neuron PPL1-01 can establish aversive associative memory for the odor in a process that involves dopamine signaling from PPL1-01 to the mushroom body neurons (Tanaka et al., 2008; Aso et al., 2010, 2019; Hige et al., 2015; König et al., 2018; synonyms for PPL1-01 are PPL1-γ1pedc and MB-MP1; Fig. 1A). However, although relief memory is observed for odors presented upon the termination of PPL1-01 activation (Aso and Rubin, 2016; König et al., 2018), it is controversial whether this is mediated by dopamine, too, or involves cotransmitters of dopaminergic neurons such as nitric oxide (Aso et al., 2019). On the one hand, relief memory remained intact when we coexpressed in PPL1-01 both the optogenetic effector for activating it and an RNAi construct to knock down the transcript for the tyrosine hydroxylase enzyme (TH) required for dopamine biosynthesis [König et al. (2018), their Fig. 5]. On the other hand, relief memory through PPL1-01 termination was abolished in loss-of-function mutants for TH [Aso et al. (2019), their Fig. 5, Supplementary Fig. 3C; also see Handler et al. (2019)]. It is therefore imperative to clarify the contribution of dopamine in timing-dependent valence reversal by PPL1-01. A distinguishing feature of the present study is that we map out the full “fingerprint” of PPL1-01 reinforcement across multiple temporal intervals (Fig. 1B). Yet our results not only reconcile what appeared to be contradictory conclusions in König et al. (2018) and Aso et al. (2019). They further reveal a dissociation between two forms of punishment learning, namely, for procedures with versus procedures without a time gap between odor presentation and PPL1-01 activation (“trace” vs “delay” conditioning, respectively), moderated by both dopamine and serotonin. This unexpectedly complex modulation of reinforcement processing is discussed with respect to psychiatric implications that may pertain if related modulations were to occur in humans.
Pharmacologically inhibiting the TH enzyme extends and blunts the temporal profile of reinforcement by PPL1-01. A, Schematics of a fly, its brain, and the mushroom bodies (top) and a highly simplified working hypothesis of association formation during punishment learning (bottom; for references see body text). The intrinsic neurons of the mushroom bodies represent odors in a sparse and combinatorial manner (mushroom body neurons in black). The dopaminergic PPL1-01 neuron (blue), which can be activated by, e.g., electric shock punishment, intersects the axons of the mushroom body neurons in what is called the γ1pedc compartment. Associative coincidence of odor activation and signaling from PPL1-01 (red shade within the compartment) induces associative presynaptic depression (stars) at the synapses from the odor-activated mushroom body neurons toward an approach-promoting output neuron of the compartment (purple). As no such depression takes place in a neighboring compartment in relation to its avoidance-promoting output neuron, this shifts the balance across the mushroom body output neurons to net avoidance as the learned behavior. In total, the mushroom body has 15 compartments, only two of which are sketched. The compartment depicted at the top represents the two compartments known to receive input from punishing stimuli (γ1pedc and γ2); the compartment depicted at the bottom represents compartments known to receive reward input (γ4, γ5). B, Procedure for presenting the reference odor (open clouds), the paired odor (grey clouds), and optogenetic activation of PPL1-01 (blue light bulb). The interval between the onset of the paired odor and the onset of PPL1-01 activation is called the interstimulus interval (ISI). For more details, see Extended Data Figure 1-1. C, Schematic of dopamine biosynthesis and of the inhibition of tyrosine hydroxylase (TH) by 3-iodo-ʟ-tyrosine (3IY). The dopamine precursor 3,4-dihydroxy-ʟ-phenylalanine (ʟ-DOPA) should compensate for the effects of 3IY on dopamine levels. DDC, dopamine decarboxylase. Drug feeding was performed by the tissue paper method. D, Relative to controls, punishment memory after odor → PPL1-01 training (ISI −15 s) is decreased upon feeding of 3IY (N = 25, 25). Relief memory after PPL1-01 → odor training (ISI 120 s) is unaffected (N = 25, 23). E, The decrease in punishment memory by 3IY can be rescued by additionally feeding ʟ-DOPA (ISI −15 s; N = 16, 16, 16). Relief memory is unaffected by 3IY, and by combining 3IY and ʟ-DOPA (ISI 120 s) (N = 16, 16, 15). F, Mapping out the effect of 3IY on the temporal profile of PPL1-01 reinforcement (N = 20, 20; 20, 20; 19, 20; 20, 20; 20, 20; 20, 20; 20, 20). 3IY decreases punishment memory (ISI −15 s, delay conditioning) and leaves relief memory with an ISI of 120 s unaffected. For a longer relief ISI of 240 s a decrease in relief memory is revealed. For a training procedure with a −40 s time gap between odor and PPL1-01 (ISI −100 s, trace conditioning), an increase in memory scores by 3IY is observed. G, The effects of 3IY on memory scores after trace, delay, and relief conditioning (ISIs of −100 s, −15 s, and 240 s, respectively) can be largely rescued, or even overcompensated, by ʟ-DOPA (N = 30, 31, 31; 30, 31, 31; 30, 31, 30). Plotted in D–G are the memory scores according to Equation 2, reflecting associative memory for the odor paired with optogenetic activation of PPL1-01; positive and negative memory scores reflect appetitive and aversive memory, respectively. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Open box plots and circles refer to the control condition, and magenta and gray fill to groups fed with 3IY or with 3IY plus ʟ-DOPA, respectively. Flies were of the genotype PPL1-01 > ChR2-XXL. * and “ns” indicate significance and nonsignificance, respectively, in MW-U tests at an error rate of 5%, adjusted according to Bonferroni–Holm, except for E (ISI 120 s) where “ns” indicates nonsignificance in a KW test. In E (ISI −15 s), the exact p value is presented, which after Bonferroni–Holm correction is just about nonsignificant. Given that in all five other cases of comparison between these treatment groups, statistical significance is reached (Figs. 1D,F,G, 5B; Extended Data Fig. 1-2), our interpretation is that the narrow “miss” of significance in this case is a false negative. Data and statistical results are documented in Extended Data Tables and Table 1. Memory scores separated by sex are shown in Extended Data Figure 1-3. The anatomical image of the mushroom body in A is modified from Heisenberg and Gerber (2008).
Figure 1-1
Timing of odor presentation and PPL1-01 activation by blue light. At time 0:00 min, the flies were gently loaded into the experimental setup. From 3:00 min on, the reference odor (white box) was presented for 1 min. For the optogenetic activation of the PPL1-01 neuron, blue light (blue box) was applied at 8:35 min for 1 min with the help of 24 LEDs of 465 nm peak wavelength mounted on the inner surface of 2.5 cm-diameter and 4.5 cm-length hollow cylinders. These cylinders were fitted around transparent training tubes harboring the flies. Blue light was applied as 12 pulses, each 1.2-sec long and followed by the next pulse with a 5 sec onset-to-onset interval. The absolute irradiance in the middle of the training tube during blue light pulses was 200 µW/cm2 as measured with an STS-VIS Spectrometer (Ocean Optics). The paired odor (grey box) was presented for 1 min, too, at the onset-to-onset inter-stimulus-intervals (ISIs) from the blue light as indicated by the numbers within the grey boxes (s). Negative ISI values indicate that the presentation of the paired odor started before the blue light (ISIs -155 s, -100 s and -15 s); positive ISIs indicate the reverse order of events (ISIs 80 s, 120 s, 240 s and 300 s). Onset times for all ISIs are indicated above. At 17:00 min the flies were shifted to the choice point between the odors. At 21:00 min the test started, and the flies were released into the T-maze. After 2 min the arms of the maze were closed, and the flies on each side were counted, separated by sex. Download Figure 1-1, TIF file.
Figure 1-2
Repetition of the experiment in Figure 1D (ISI -15 s). (A and B) Repetition of the experiment shown in Figure 1D, for the ISI of -15 s. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. The grey clouds depict the paired odor, and the blue light bulb PPL1-01 activation, presented with the indicated temporal relationship. (A) Memory scores, determined according to equation 2, for the control condition (open box plot, N = 6) and the 3IY-fed case (magenta fill, N = 6). * indicates significance in a MW-U test. (B) Data from (A), separated by sex. Red fill of the box plots refers to data from females, blue fill to data from males. Open circles below the panels refer to the control conditions, magenta fill to the 3IY-fed cases. Other details as in the legend of Figure 1. Data are documented in the file Extended Data Tables. Download Figure 1-2, TIF file.
Figure 1-3
Memory scores from Figure 1 separated by sex. (A-D) Memory scores from Figure 1D-G, respectively, separated by sex. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Red fill of the box plots shows data from females; blue fill indicates data from males. The grey clouds depict the paired odor, and the blue light bulb PPL1-01 activation, presented with the indicated temporal relationship. Open circles below the panels refer to the control conditions, magenta fill to the 3IY-fed cases, grey fill to those additionally fed with L-DOPA. Other details as in the legend of Figure 1. Data are documented in the file Extended Data Tables. Download Figure 1-3, TIF file.
Extended Tables
Download Extended Tables, XLSX file.
Materials and Methods
Fly strains
D. melanogaster were reared in mass culture on standard food, at 60–70% relative humidity and 25°C and under a 12 h light/dark cycle, unless mentioned otherwise. For the behavioral assays, 1–3-d-old adult flies were collected, regardless of sex, and handled in mixed-sex cohorts of ∼60–100 flies with approximately equal numbers of females and males. Transgenic fly strains were used to express either the blue-light-gated cation channel ChR2-XXL or both ChR2-XXL and an RNAi construct against the TH enzyme in the dopaminergic mushroom body input neuron PPL1-01. Specifically, males of the driver strain MB320C-split-GAL4 (covering the PPL1-01 neuron; Bloomington stock center no. 68253; Aso et al., 2014) were crossed to females of the effector strains, which were either UAS-ChR2-XXL (Bloomington stock center no. 58374; Dawydow et al., 2014) or featured UAS-TH-RNAi in addition (Bloomington stock center no. 25796; Riemensperger et al., 2013). The flies from these crosses (henceforth PPL1-01 > ChR2-XXL and PPL1-01 > ChR2-XXL/TH-RNAi) were used for experiments and kept in light-shielded vials to avoid optogenetic activation by room light. Genetic controls carrying only the PPL1-01 driver or only the ChR2-XXL effector had previously been tested (König et al., 2018) and did not show memory upon pairing odor with blue light.
Pharmacological manipulations
Unless mentioned otherwise, we used 3-iodo-ʟ-tyrosine (3IY), an inhibitor of the TH enzyme which is rate limiting for the synthesis of dopamine (Fig. 1C), in a procedure that followed Thoener et al. (2021). Specifically, in different sets of newly hatched flies, either a plain 5% sucrose solution (CAS: 57-50-1, Hartenstein; in EVIAN water) was offered to the flies as their sole food, or it was offered in mixture with 5 mg/ml 3IY (CAS: 70-78-0, Sigma; stored at −20°C) or in mixture with 5 mg/ml 3IY plus 10 mg/ml 3,4-dihydroxy-ʟ-phenylalanine (ʟ-DOPA), a precursor of dopamine (CAS: 59-92-7, Sigma). Mixtures were prepared by a shaker at high speed for ∼60 min. Specifically, flies were transferred to small plastic vials (diameter, 25 mm; height, 60 mm; volume, 30 ml; K-TK) with tissue paper (Fripa) soaked with 1.8 ml of the solutions mentioned above, kept at 25°C, and used for experiments after 36–40 h. This procedure is henceforth called the tissue paper method.
We used para-chlorophenylalanine (PCPA, aka dʟ-4 chlorophenylalanine, fenclonine; CAS: 7424-00-2, Thermo Fisher Scientific, Acros Organics; stored at 4°C), an inhibitor of the enzyme tryptophan hydroxylase (TPH), which is rate limiting for serotonin synthesis, in a procedure that followed Pooryasin and Fiala (2015). Specifically, newly hatched flies were transferred to large plastic vials (diameter, 46 mm; height, 102 mm; volume, 170 ml; K-TK) with wet tissue paper and starved for 48 h at 18°C. After starvation, separate sets of flies were either transferred to small vials containing 1 ml of freshly prepared standard food medium mixed with 200 µl of 5% sucrose solution and 200 µl of water (EVIAN) and left with this as their sole food, or they were kept with this mixture plus in addition either 1.25 mg/ml of 3IY, or 1.25 mg/ml of 3IY plus 60 mg/ml of PCPA, or 60 mg/ml of PCPA. In all cases, the flies were kept at 25°C room temperature and used for experiments 4 d later. Mixtures were prepared by a shaker at high speed for ∼30 min. This procedure is henceforth called the food method.
The methods used can be expected to compromise TH function and thus to reduce dopamine levels, systemically or in the PPL1-01 neuron, but they are not expected to result in a total absence of dopamine. It is also likely that they leave intact the function of cotransmitters of dopamine neurons, such as nitric oxide (Aso et al., 2019). Thus, the present experiments make it possible to ascertain a role of TH and of dopamine in timing-dependent valence reversal but do not allow any remaining reinforcing effects to be assigned to residual dopamine versus any cotransmitter.
Behavioral experiments
Behavioral experiments for the association of odor with the optogenetic activation of PPL1-01 followed König et al. (2018), unless mentioned otherwise. In brief, these experiments took place in a custom-made set-up (CON-ELEKTRONIK; modified from Tully and Quinn, 1985) that allowed the simultaneous handling of four cohorts of flies, each with ∼60–100 flies. During training, dim red light was used to allow minimal vision for the experimenter while the ChR2-XXL channels remained mostly closed. Blue light for opening the ChR2-XXL channels and thus for neuronal activation was turned on only briefly and in the temporal relationship to odor presentation as described below and in the Results section. In all cases, blue light was presented in a pulsatile manner as 12 pulses, each 1.2 s long and followed by the next pulse with a 5 s onset-to-onset interval. Once the training had concluded, the testing was carried out in darkness.
As odorants, 50 µl of benzaldehyde (BA) and 250 µl of 3-octanol (OCT; CAS 100-52-7, 589-98-0; Fluka) were applied to 1-cm-deep Teflon containers of 5 or 14 mm diameter, respectively.
The key variable for the behavioral experiments is the relative timing (or interstimulus interval, ISI) of the pairing between an odor and the optogenetic activation of PPL1-01 by blue light (Fig. 1B, Extended Data Fig. 1-1). The ISI is defined as the time interval between the onset of the blue light and the onset of the paired odor presentation. In principle, either the paired odor was presented first followed by blue light stimulation (forward conditioning, defined as negative ISIs), or the blue light was presented first followed by presentation of the paired odor (backward conditioning, positive ISIs). Specifically, separate groups of flies were trained with one of seven ISIs (−155, −100, −15, 80, 120, 240, 300 s). Given the 60 s duration of blue light stimulation and the 60 s duration of the paired odor presentation, this resulted in gaps between the two events of −95 and −40 s for the two longest forward conditioning ISIs (−155, −100 s; “trace” conditioning), in a partial overlap for the −15 s ISI (“delay” conditioning), and in gaps of 20–240 s for the backward conditioning ISIs (“relief” conditioning). In all cases, a second odor was presented as a reference, unpaired from blue light during training. The use of BA and OCT as the paired and the reference odor was balanced across repetitions of the experiment.
After one cycle of training, which lasted for a total of 15 min, the flies were given a 4 min accommodation period and were then shifted to the choice point of a T-maze apparatus, with the paired and the reference odor on either side. After 2 min, the arms of the T-maze were closed, and the numbers of flies (#, as the sum of male and female flies) in each arm were counted by an assistant blind to the experimental conditions to calculate the preference for BA as follows:
To quantify the effect of compromising TH function on memory scores, the difference between the memory scores of the TH-compromised condition and the memory scores of the control condition was determined.
Measurement of biogenic amine levels
After drug feeding using the tissue paper method, we performed a high-performance liquid chromatography (HPLC) analysis of brain-wide biogenic amine levels. For each of the respective treatments, six male and six female brains were dissected in Ca2+-free saline and, separated by sex, immediately frozen in −80°C liquid nitrogen. The samples were analyzed using HPLC with electrochemical detection to measure dopamine and serotonin levels. The column was an ET 125/2, Nucleosil 120-5, C-18 reversed phase column (Macherey & Nagel). The mobile phase consisted of 75 mM NaH2PO4, 4 mM KCl, 20 μM EDTA, 1.5 mM sodium dodecyl sulfate, 100 μl/L diethylamine, 12% alcohol, and 12% acetonitrile, adjusted to pH 6.0 using phosphoric acid (Carl Roth). The electrochemical detector (Intro; Antec) was set at 500 mV versus an ISAAC reference electrode (Antec) at 30°C. This setup allows the simultaneous measurement of dopamine and serotonin (Amato et al., 2020; Kalinichenko et al., 2021).
Experimental design and statistical analyses
We used non-parametric statistical tests throughout (Statistica 11.0; StatSoft, and R 2.15.1; www.r-project.org). For comparisons across more than two groups, the Kruskal–Wallis test (KW) was applied. For subsequent pairwise comparisons between groups, Mann–Whitney U tests (MW-U) were performed. To test whether values of a given group differed from chance levels, i.e., from zero, one-sample sign tests (OSS) were used. When multiple tests of the same kind were performed within one experiment, significance levels were adjusted by a Bonferroni–Holm correction to keep the experiment-wide type 1 error limited to 0.05 (Holm, 1979). In no case were data compared within-subjects. Data are presented as box-dot plots which represent the median as the middle line and the 25%/75% and 10%/90% quantiles as box boundaries and whiskers, respectively; single data points are displayed as dots. For the behavioral experiments, each such sample N = 1 is based on cohorts of approximately n = 120–200 individual flies, with approximately equal numbers of females and males. For the HPLC measurements, each N = 1 is based on six female or six male flies. Sample sizes of the respective experiments are stated in the figure legends. All plotted data are documented in Extended Data Tables. Per-experiment details of experimental design and statistical results can be found in Tables 1⇓⇓⇓–5.
Summary experimental design and statistics Figure 1
Summary experimental design and statistics Figure 2
Summary experimental design and statistics Figure 3
Summary experimental design and statistics Figure 4
Summary experimental design and statistics Figure 5
Results
Pharmacological inhibition of TH extends and blunts the temporal profile of PPL1-01 reinforcement
Flies expressing the blue-light-gated ion channel ChR2-XXL for optogenetic activation of the PPL1-01 neuron showed punishment memory after odor → PPL1-01 training (Fig. 1D; ISI −15 s). Acute feeding of 3IY, an inhibitor of the TH enzyme required for dopamine synthesis, impaired such punishment memory (Fig. 1D; ISI −15 s; for a repetition, see Extended Data Fig. 1-2). Of note is that 3IY feeding leaves task-relevant sensory-motor faculties intact (Thoener et al., 2021). In a further repetition of the experiment, the effect of 3IY feeding on punishment memory could be rescued by an additional feeding of ʟ-DOPA (Fig. 1E; ISI −15 s). In contrast, relief memory after PPL1-01 → odor training was unaffected by feeding of 3IY (Fig. 1D,E; ISI 120 s).
We next mapped out the effect of 3IY feeding on the temporal profile of PPL1-01 reinforcement more systematically, that is on the association of odor and PPL1-01 activation across multiple intervals between these events. This revealed that this temporal profile is both, extended and blunted by 3IY feeding. For both the intervals used before, this once more replicated the finding that 3IY feeding leads to a decrease in punishment memory, and that there is no such detrimental effect on relief memory (Fig. 1F; ISIs of −15 and 120 s, respectively). Strikingly, however, 3IY feeding increased punishment memory when there was a −40 s gap between the offset of the odor and the start of PPL1-01 activation (Fig. 1F; ISI −100 s) and decreased relief memory for relatively long intervals between PPL1-01 activation and odor presentation (Fig. 1F; ISI 240 s, corresponding to a 180 s gap). In a follow-up experiment, we confirmed these three kinds of effect exerted by 3IY feeding and showed that they can be rescued by additionally feeding ʟ-DOPA (Fig. 1G; see Extended Data Fig. 1-3 for the memory scores separated by sex). HPLC measurements of whole-brain homogenates upon 3IY feeding reveal a selective decrease in dopamine but not in serotonin levels, which was likewise rescued by additionally feeding ʟ-DOPA (Fig. 2; see Extended Data Fig. 2-1 for data separated by sex).
Pharmacological inhibition of the TH enzyme reduces brain levels of dopamine. Whole-brain levels of dopamine and serotonin after feeding 3-iodo-ʟ-tyrosine (3IY), an inhibitor of the tyrosine hydroxylase (TH) enzyme required for dopamine biosynthesis. Feeding of 3IY reduced dopamine levels, an effect that was restored by feeding the dopamine precursor 3,4-dihydroxy-ʟ-phenylalanine (ʟ-DOPA) in addition (N = 20, 20, 20). Drug feeding, performed by the tissue paper method, was without effect on serotonin levels (N = 20, 20, 20). Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Open box plots and circles refer to the control condition, and magenta and gray fill to groups fed with 3IY or with 3IY plus ʟ-DOPA, respectively. Flies were of the genotype PPL1-01 > ChR2-XXL. * indicates significance in MW-U tests at an error rate of 5%, adjusted according to Bonferroni–Holm. “ns” indicates nonsignificance in such a MW-U test (dopamine) or in a KW test (serotonin). Data and statistical results are documented in Extended Data Tables and Table 2. Data separated by sex are shown in Extended Data Figure 2-1.
Figure 2-1
Levels of biogenic amines from Figure 2 separated by sex. Brain-wide levels of dopamine (left) and serotonin (right) from Figure 2, separated for females and males as indicated. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Open box plots and circles refer to the control condition, magenta and grey fill to groups fed with 3IY or with 3IY plus L-DOPA, respectively. “ns” indicates non-significance in KW-tests, except for the case of dopamine measurements in males, where * and “ns” refer to significance and non-significance, respectively, in MW-U tests at an error rate of 5%, adjusted according to Bonferroni-Holm. Other details as in the legend of Figure 2. Data are documented in the file Extended Data Tables. Download Figure 2-1, TIF file.
These results suggest that optogenetic activation of PPL1-01 establishes both punishment memory and relief memory through a 3IY-sensitive, TH-dependent, dopaminergic process. To our surprise, we found that compromising TH function had opposite effects upon training with a −40 s gap between odor and PPL1-01 activation (ISI −100 s, punishment memory after trace conditioning) as compared with training without such a gap (ISI −15 s, punishment memory after delay conditioning; Fig. 1F,G).
Local knockdown of TH likewise extends and blunts the temporal profile of PPL1-01 reinforcement
We next asked whether the three observed effects of compromising TH function on memory scores, namely, on punishment memory after (1) trace and (2) delay conditioning, as well as on (3) relief memory, require dopaminergic signaling from PPL1-01 itself. We therefore coexpressed in PPL1-01 both the optogenetic effector and an RNAi construct for the knockdown of the TH enzyme (Riemensperger et al., 2013) and mapped out the temporal profile of PPL1-01 reinforcement. This again revealed an increase in punishment memory after trace conditioning (ISI −100 s), a decrease in punishment memory after delay conditioning (ISI −15 s), as well as a decrease in relief memory (ISI 300 s; Fig. 3A,B; see Extended Data Fig. 3-1 for the memory scores separated by sex).
Local knockdown of the TH enzyme extends and blunts the temporal profile of reinforcement by PPL1-01. A, Schematic of dopamine biosynthesis and of the inhibition of tyrosine hydroxylase (TH) by RNA interference (RNAi). B, Mapping out the effect of TH-RNAi in the PPL1-01 neuron on the temporal profile of PPL1-01 reinforcement (N = 32, 32; 34, 34; 40, 40; 33, 34; 42, 42). Relative to controls, TH-RNAi promotes punishment memory upon trace conditioning (ISI −100 s) and decreases punishment memory upon delay conditioning (ISI −15 s). Relief memory is decreased (ISI 300 s). Control flies were of the genotype PPL1-01 > ChR2-XXL (open box plots and circles); flies for TH knockdown in the PPL1-01 neuron additionally carried the TH-RNAi construct (PPL1-01 > ChR2-XXL/TH-RNAi; box plots and circles with magenta fill). Other details are as in the legend of Figure 1. * indicates significance in MW-U tests at an error rate of 5%, adjusted according to Bonferroni–Holm, “ns” indicates nonsignificance in such tests. Data and statistical results are documented in Extended Data Tables and Table 3. Memory scores separated by sex are shown in Extended Data Figure 3-1.
Figure 3-1
Memory scores from Figure 3 separated by sex. Memory scores from Figure 3B, separated by sex. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Box plots with red fill show data from females; blue fill indicates data from males. The grey cloud depicts the paired odor, and the blue light bulb PPL1-01 activation, presented with the indicated temporal relationship. Open circles below the panels refer to the control conditions, magenta fill to the cases with RNAi against TH. Other details as in the legend of Figure 1. Data are documented in the file Extended Data Tables. Download Figure 3-1, TIF file.
We conclude that compromising TH function extends and blunts the temporal profile of reinforcement by PPL1-01 (Fig. 4A): trace conditioning (ISI −100 s) is improved, delay conditioning is impaired, and relief conditioning is abolished for longer intervals (ISIs 240 and 300 s).
Compromising TH function extends and blunts the temporal profile of reinforcement by PPL1-01. A, Summary of the effects of compromising TH function on the temporal profile of PPL1-01 reinforcement, combined for 3IY and TH-RNAi, and across the present study. Shown are the memory scores of the respective control (open box plots) and TH-compromised cases (box plots and circles with magenta fill; N = 20, 20, 120, 122, 159, 165, 20, 20, 101, 99, 108, 113, 62, 62). B, Data for only the control cases shown in A, separated by sex. Neither for trace conditioning (ISI −100 s), nor for delay conditioning (ISI −15 s), nor for relief conditioning (ISIs 240 and 300 s) were sex-dependent differences observed (for a justification of why relief conditioning with ISIs of 80 and 120 s is not included in this grouping, see below; N = 120, 120, 159, 157, 169, 169). C, For the data in A, the difference in memory scores of the TH-compromised cases minus the scores in the controls is plotted, separately for female and male flies, to quantify how strongly compromising TH function affects memory scores, in either sex. For trace conditioning (ISI −100 s), the effect of compromising TH function was less pronounced in females than in males, whereas no such difference was observed for delay (ISI −15 s) and relief conditioning (N = 118, 119, 159, 157, 168, 167). For relief conditioning, data were considered only for those ISIs for which compromising TH function had an effect to begin with (A, 240 and 300 s). Other details are as in the legend of Figure 1. * indicates significance in MW-U tests at an error rate of 5%, adjusted according to Bonferroni–Holm, “ns” indicates nonsignificance in such tests. # indicates significance in OSS-tests at an error rate of 5%, adjusted according to Bonferroni–Holm. Data and statistical results are documented in Extended Data Tables and Table 4.
Separating data by sex
Our lab routinely acquires behavioral data separated by sex, but in aversive short-term memory we have so far never observed reliable differences between female and male flies. Indeed, for the control conditions summarized throughout the present study, memory scores do not differ between the sexes for trace, delay, or relief conditioning (Fig. 4B). On examining the difference in memory scores of the TH-compromised cases minus the controls, however, we were surprised to find that specifically for trace conditioning (ISI −100 s) the effect of compromising TH function, which was significant in both sexes, was less pronounced in females than that in males (Fig. 4C). Two observations suggest that this sex difference is not a statistical artifact. Firstly, this sex difference can be discerned for both 3IY feeding as an acute, systemic intervention (Extended Data Fig. 1-3C,D) and for TH-RNAi as a constitutive, cell-specific intervention (Extended Data Fig. 3-1). Secondly, when the brain-wide HPLC measurements of biogenic amines after 3IY feeding were separated by sex, this revealed only a nonsignificant tendency toward a decrease in dopamine levels in the females, whereas a significant decrease in dopamine levels was observed in the males (Extended Data Fig. 2-1). This suggests that decreases in dopamine levels that in females remain below the significance threshold in brain-wide HPLC measurements (Extended Data Fig. 2-1) can nevertheless have behavioral effects (Fig. 4C) and that these behavioral effects are weaker in females than those produced by the more pronounced decreases in dopamine levels in males (Extended Data Fig. 2-1, Fig. 4C).
An inhibitor of the TPH enzyme can reverse the effects of 3IY on trace conditioning
PPL1-01 is one of a total of 12–15 dopaminergic neurons in the PPL1 cluster, and specifically in the subset of six of these that innervate the mushroom body (Mao and Davis, 2009; Aso et al., 2014; Li et al., 2020). It has been reported that within the PPL1 cluster there is at least one neuron that is not only immunoreactive against TH but also against serotonin and that constitutively compromising TH function by genetic means can both increase the number of anti-serotonin immunoreactive neurons in the PPL1 cluster and alter the pattern of anti-serotonin immunoreactivity in the mushroom body, specifically at the tips of the α and α’ lobes, which receive both dopaminergic and serotonergic input (Niens et al., 2017). Preliminary results suggested that our method for acutely lowering dopamine levels by 3IY left the number of anti-serotonin immunoreactive neurons in the PPL1 cluster unchanged but altered patterns of serotonin-immunoreactivity in a way similar to what was previously reported (Niens et al., 2017). This encouraged us to test whether downregulating serotonin synthesis would alter the effects we observed by feeding 3IY. To test for this possibility, we used PCPA, an inhibitor of the TPH enzyme (Fig. 5A). For trace conditioning (ISI −100 s), the increase in punishment memory caused by 3IY was fully reversed by an additional feeding of PCPA (Fig. 5B). For delay conditioning (ISI −15 s), the 3IY-induced decrease in punishment memory was only partially reversed by PCPA. For relief conditioning (ISI 240 s), however, the decrease in relief memory through 3IY was not moderated by PCPA. Under low-dopamine conditions, the additional feeding of PCPA thus had a graded effect on memory scores, in the sense that it was strong for trace conditioning (ISI −100 s) and tapered off as the ISIs were increased to −15 and 240 s. Of note is that feeding PCPA alone, that is, feeding PCPA under conditions of normal dopamine levels, had no effect on either form of conditioning (Fig. 5C; see Extended Data Fig. 5-1 for the memory scores separated by sex).
Temporal profile of reinforcement by PPL1-01 upon pharmacologically inhibiting the TPH and the TH enzyme. A, Schematic of serotonin and dopamine biosynthesis, of the inhibition of tryptophan hydroxylase (TPH) by para-chlorophenylalanine (PCPA), and of the inhibition of tyrosine hydroxylase (TH) by 3-iodo-ʟ-tyrosine (3IY), respectively. 5-HTP, 5-hydroxytryptophan; ʟ-DOPA, 3,4-dihydroxy-ʟ-phenylalanine; DDC, dopamine decarboxylase. Drug feeding was performed by the food method. B, Relative to controls, punishment memory after odor → PPL1-01 trace conditioning (ISI −100 s) is increased upon feeding of 3IY, an effect that is fully reversed by an additional feeding of PCPA (N = 38, 39, 45). For delay conditioning (ISI −15 s), punishment memory is reduced by 3IY, an effect that is partially reversed by PCPA (N = 29, 33, 32). The reduction of relief memory (ISI 240 s) by 3IY was not reversed by PCPA (N = 25, 28, 29). C, PCPA alone has no effect on punishment memory after trace conditioning (ISI −100 s) (N = 39, 39) or delay conditioning (ISI −15 s) (N = 45, 48) and leaves relief memory intact, too (ISI 240 s; N = 44, 41). Open box plots and circles refer to the control condition, magenta and green fill to groups fed with 3IY or with 3IY plus PCPA, respectively. Flies were of the genotype PPL1-01 > ChR2-XXL. Other details are as in the legend of Figure 1. * indicates significance and “ns” nonsignificance in MW-U tests at an error rate of 5%, adjusted according to Bonferroni–Holm. Data and statistical results are documented in Extended Data Tables and Table 5. Memory scores separated by sex are shown in Extended Data Figure 5-1.
Figure 5-1
Memory scores from Figure 5 separated by sex. (A and B) Memory scores from Figure 5B and C, respectively, separated by sex. Box plots represent the median as the middle line, 25%/75% quantiles as box boundaries, and 10%/90% quantiles as whiskers. Red fill of the box plots shows data from females; blue fill indicates data from males. The grey clouds depict the paired odor, and the blue light bulb PPL1-01 activation, presented with the indicated temporal relationship. Open circles below the panels refer to the control conditions, magenta fill refers to feeding with 3IY, and green fill to feeding with PCPA in addition (A) or PCPA alone (B). Other details as in the legend of Figure 1. Data are documented in the file Extended Data Tables. Download Figure 5-1, TIF file.
Discussion
We report that compromising TH function, either acutely by pharmacological means (Figs. 1, 5; Extended Data Fig. 1-2) or by cell-specific RNAi (Fig. 3), extends and blunts the temporal profile of PPL1-01 reinforcement. Specifically, it improves trace conditioning (ISI −100 s), impairs delay conditioning (ISI −15 s), and abolishes relief conditioning for longer intervals (ISIs 240 and 300 s; Fig. 4A).
Relief conditioning: short versus long ISIs and the relation to “frustration” conditioning
The present results on relief conditioning with PPL1-01 reinforcement can reconcile earlier reports by König et al. (2018, their Fig. 5) and Aso et al. (2019, their Fig. 5, Supplementary Fig. 3C). Those two studies differ in a number of ways, making it difficult to quantitatively relate the ISIs that were used. Differences include the use of a two-arm T-maze versus a horizontal, four-field arena setup, and the use of ChR2-XXL versus CsChrimson-Venus as the optogenetic effector, respectively. Nonetheless, by mapping out multiple ISIs, the present results confirm the TH independence reported by König et al. (2018) for short relief ISIs, whereas for longer relief ISIs they are consistent with the TH dependence reported by Aso et al. (2019).
The PPL1-01 neuron innervates the elongated axons of the mushroom body neurons as they pass through the γ1pedc compartment (Tanaka et al., 2008; Fig. 1A), one of the two known punishment compartments in the mushroom body. Upon delay conditioning with PPL1-01, a memory trace is established as a compartmentally local presynaptic depression of output synapses of those mushroom body neurons that are activated by odor, reducing drive to the approach-promoting compartmental output neuron (Hige et al., 2015). However, it is not clear whether upon relief conditioning with PPL1-01 the memory trace is likewise localized, and whether it would manifest, conversely, as synaptic potentiation. Using a single, short ISI, indeed, the results of Hige et al. (2015) did not suggest so. This prompted speculation that signaling from PPL1-01 to dopaminergic neurons outside the γ1pedc compartment may be involved in relief conditioning (König et al., 2018), possibly via multiple synaptic steps including the PAM-07 DANs (Aso et al., 2014; Li et al., 2020). Such heterotopy would contrast with the homotopy suggested by the results of Handler et al. (2019) for the other known punishment compartment (γ2) as well as for the two known reward compartments (γ4 and γ5; the γ1pedc and γ3 compartments were not studied). That is, in these cases Handler et al. (2019) found that timing-dependent valence reversal manifests as depression/potentiation within the same compartment.
Trace versus delay conditioning
Our results show a dissociation between trace and delay conditioning with PPL1-01 reinforcement. Trace conditioning, surprisingly, is improved by compromising TH function, whereas as expected (König et al., 2018; Aso et al., 2019) delay conditioning is impaired (Fig. 4A; ISIs −100 vs −15 s). This adds to earlier evidence that trace and delay procedures engage partially distinct downstream mechanisms in odor–shock associative learning. Mutants in the rutabaga gene, coding for the type I adenylate cyclase which acts as a molecular coincidence detector for this association, are unaffected in trace conditioning but are impaired in delay conditioning (Shuai et al., 2011). In turn, expression of a dominant-negative form of the Rac protein improves trace conditioning but leaves delay conditioning unaffected (Shuai et al., 2011). In a visual learning paradigm, Grover et al. (2022) showed a selective role for the dopamine receptor Dop2R (CG33517) for trace but not delay conditioning.
Trace and delay conditioning certainly also have features in common. For odor–shock associations these commonalities include impairment in mutants lacking Synapsin (Niewalda et al., 2015), the asymptotic memory strength upon repeated training trials (albeit reached at a slower rate for trace conditioning), the rate of memory decay, the profile of generalization (Galili et al., 2011), the likely site of the memory trace in the mushroom body, and their requirement of the dopamine receptor Dop1R1 (CG9652; Shuai et al., 2011; also see Grover et al., 2022).
A role for serotonin in trace conditioning—under low-dopamine conditions
We show that pharmacologically compromising TH function leads to a decrease in brain-wide levels of dopamine rather than serotonin (Fig. 2). Further, both this decrease in dopamine levels and the abovementioned effects on reinforcement learning were rescued by ʟ-DOPA (Figs. 1G, 2). Interestingly, under low-dopamine conditions, our results uncover a role for serotonin that is particularly strong for trace conditioning (ISI −100 s) and that tapers off with increasing ISIs (Fig. 5B). Under conditions of normal dopamine levels, however, our data do not provide evidence for a role of serotonin (Fig. 5C); it is unclear whether this is at variance with the experiments reported by Zeng et al. (2023), as these lacked critical genetic and procedural controls.
Implications of compromised TH function
At a cognitive level, one may view long-gap trace-conditioned cues as providing no evidence, and delay- and relief-conditioned cues as providing, respectively, evidence for and evidence against the causation of punishment. In a human subject at least, the temporal profile of reinforcement upon compromising TH function as found in the present study on flies (Fig. 4A) would thus imply a state in which causality is attributed to cues that do not merit it, whereas credible evidence in favor of as well as credible evidence against the causation of punishment is not properly appreciated. Such a state may promote delusional beliefs about causal event structure, which are a hallmark symptom of schizophrenia (McCutcheon et al., 2019; also see Garety et al., 2005; Moritz and Woodward, 2005; Uhlhaas and Silverstein, 2005; Dudley et al., 2016). Regarding flies, we remain expressly agnostic as to whether their behavior might be based on causal beliefs and whether schizophrenia-like states might be the result when the temporal profile of reinforcement is distorted as reported here.
In practical terms, the present study shows that mapping out the full “fingerprint” of reinforcement across multiple temporal intervals may be required to understand how a given manipulation affects reinforcement learning. Indeed, our results (Fig. 4A) provide a case of an experimental manipulation where focusing solely on any one single interval between odor and reinforcement will lead to drastically different conclusions, namely, that memory is improved, impaired, or unaffected, depending on which interval is chosen. It therefore seems possible that apparent discrepancies in the literature can be resolved through mapping the full temporal profile of reinforcement.
Footnotes
This study received institutional support by the Otto von Guericke Universität Magdeburg (OVGU), the Wissenschaftsgemeinschaft Gottfried Wilhelm Leibniz, the Leibniz Institute for Neurobiology (LIN), as well as grant support from the Deutsche Forschungsgemeinschaft (FOR 2705 Mushroom body to B.G., MU 2789/20-1 to C.P.M., RI 2419/3-1 to T.D.R.) and the Interdisziplinäres Zentrum für Klinische Forschung Erlangen (J93 to L.S.K.). T.D.R. was supported by the AXA Research Fund as a team member of the AXA Chair From Genome to Structure and Function awarded to Kei Ito, Universität zu Köln. Expert help with fly brain dissections by Anna Ciuraszkiewicz and Juliane Thöner (LIN) and discussions with P. Stevenson (U Leipzig), M. Fendt (OVGU Magdeburg), and Juliane Thöner (LIN) are gratefully acknowledged. We thank R.D.V. Glasgow (Zaragoza, Spain) for language editing.
↵*F.A. and C.K. contributed equally to this work.
The authors declare no competing financial interests.
S.K.’s present address: Division of Psychological and Social Medicine and Developmental Neurosciences, Faculty of Medicine, Technical University of Dresden, Dresden 01307, Germany.
- Correspondence should be addressed to Christian P. Müller at christian.mueller{at}uk-erlangen.de or Bertram Gerber at bertram.gerber{at}lin-magdeburg.de.
