Neural correlates of learning and memory formation have been reported at different stages of the olfactory pathway in both vertebrates and invertebrates. However, the contribution of different neurons to the formation of a memory trace is little understood. Mushroom bodies (MBs) in the insect brain are higher-order structures involved in integration of olfactory, visual, and mechanosensory information and in memory formation. Here we focus on the ensemble spiking activity of single MB output neurons (ENs) when honeybees learned to associate an odor with reward. A large group of ENs (∼50%) changed their odor response spectra by losing or gaining sensitivity for specific odors. This response switching was dominated by the rewarded stimulus (CS+), which evoked exclusively recruitment. The remaining ENs did not change their qualitative odor spectrum but modulated their tuning strength, again dominated by increased responses to the CS+. While the bees showed a conditioned response (proboscis extension) after a few acquisition trials, no short-term effects were observed in the neuronal activity. In both EN types, associative plastic changes occurred only during retention 3 h after conditioning. Thus, long-term but not short-term memory was reflected by increased EN activity to the CS+. During retention, the EN ensemble separated the CS+ most differently from the CS− and control odors ∼140 ms after stimulus onset. The learned behavioral response appeared ∼330 ms later. It is concluded that after memory consolidation, the ensemble activity of the MB output neurons predicts the meaning of the stimulus (reward) and may provide the prerequisite for the expression of the learned behavior.
Learning leads to changes of neural activity at the single-neuron and population levels [e.g., hippocampus: Sutherland and McNaughton (2000) and Suzuki (2008); prefrontal cortex: Goldman-Rakic (1995) and Histed et al. (2009); orbitofrontal cortex: Rolls et al. (1996); reward system in the ventral tegmentum: Schultz (1998)]. These changes reflect adaptations of the connectivity, giving the network the power to store previous experience and to retrieve it later for behavioral control (McGaugh et al., 1990; Milner et al., 1998; Malenka and Nicoll, 1999; Suzuki, 1999; Carroll and Malenka, 2000). The processes involved in learning-related plasticity have been intensively studied in the olfactory system of both vertebrates (e.g., Keverne, 1995; Ravel et al., 2003; Wilson and Mainen, 2006) and invertebrates, particularly in insects (Faber et al., 1999; Daly et al., 2004; Yu et al., 2004; Okada et al., 2007). However, little is known about the relationship between single-neuron and population changes, or whether all neurons of a defined morphological class undergo the same type of plastic changes. Here we address these questions by focusing on a defined morphological class of neurons in the honeybee brain, the mushroom body (MB) extrinsic neurons of the α lobe (ENs). In honeybees, these neurons can be studied during olfactory reward conditioning of the proboscis extension response (PER) (Mauelshagen, 1993; Okada et al., 2007). They provide one of several outputs of the MB, a neural structure known to be devoted to higher-order processing and formation of associative olfactory memory in the insect brain [Drosophila: Heisenberg (2003); honeybee: Menzel and Müller (1996)]. In Drosophila, short-term aversive odor memory appears to occur in a specific subregion of the MB, the γ-lobe, and the respective long-term memory occurs in a different subregion, α/α′and β/β′-lobes (Pascual and Préat, 2001; Isabel et al., 2004; Yu et al., 2006), indicating that memory consolidation may lead to a rearrangement of “engram” in the MB. In honeybees, associative plasticity of MB intrinsic neurons, the Kenyon cells (KCs), has been documented at their postsynaptic sites in the lip region of the MB calyx (Szyszka et al., 2008), but little is known about stimulus representation after being processed in the MB as a whole and how this representation changes after learning.
Here, we report how ENs change their olfactory response profile after odor–reward conditioning of the PER. In naive animals, the majority of units respond to a large number of odors. After associating one odor with a reward, two groups of ENs appear in relatively equal proportion. (1) “Switching” ENs change their odor response spectra by newly establishing and/or losing odor sensitivity. (2) “Modulated” ENs increase and/or decrease their odor response to the learned odors but do not change their odor response spectra. These neural changes do not appear during acquisition but only during retention. Three hours after conditioning, the population response of both groups of units together codes the CS+ differently from CS− and control odors ∼140 ms after stimulus onset.
Materials and Methods
Forager honeybees (Apis mellifera) were caught at the entrance of the hive in the afternoon 1 day before the experiment and harnessed as described by Bitterman et al. (1983). They were fed with a 30% sucrose solution to satiation and kept overnight. Before the experiment started, heads of the animals were glued with wax to the tube and the scapi of both antennae were fixed with low-melting-point wax onto the head capsule. A small window (1.5 × 1.5 mm) was cut between one compound eye and the midline between the antennae. Glands and tracheal sacs above the α-lobe were removed carefully before the electrode was inserted at the ventral border of the α lobe and positioned in a depth between 70 and 250 μm. The brain was covered with a droplet of silicon (1:1 mixture of the two components of KWIK-SIL) to prevent it from drying, to fix the electrode relative to the brain, and to stabilize the brain. Recordings typically lasted for several hours and can be maintained over several days (Strube-Bloss, 2008).
A 12-channel olfactometer was adapted from Galizia et al. (1997). Each channel was equipped with a 5 ml syringe, and a constant air stream (1.5 m/s speed) was delivered through a Teflon tube (6 mm in diameter). The needles of the syringes were inserted into the air stream. Odors were diluted in paraffin oil to a 0.1 volume concentration. Filter papers (2 cm2) were soaked with 10 μl of odor solution and placed in the syringes. During the 3 s odor stimulation, only half (2.5 ml) of the syringe air volume was injected into the constant air stream. The data acquisition as well as the stimulation was computer controlled.
To estimate the delay between valve switch and odor perception at the bee's antennae, we measured the difference between stimulus trigger time and the switching of the magnetic valves (3.6 ms) and added the duration of airflow from the odor valve to the bee's antennae (33 ms). The latter we calculated based on the air speed and the distance between the odor injection point and the bee. In total, we estimated a delay of 37 ms.
In a first set of experiments, we tested odor representation at the level of the MB output in naive bees. During these initial test series, 10 different odors (1,9ol = 1-nonanol; 2,8ol = 2-octanol; 1,7al = 1-heptanal; lin = linalool; lim = limonene; eug = eugenol; cin = cineole; 6al = hexanal; 8al = octanal; 2,9on = 2-nonanon; air = paraffin oil only; Sigma-Aldrich Chemie) and air were presented in an alternating sequence with intertrial interval (ITI) of 1 min. Each odor was repeated 10 times (compare Fig. 1D). In total, we recorded 26 units out of 10 bees and characterized their initial response properties.
In a second series of experiments, we tested whether learning can change the initially observed response properties. We applied a differential conditioning experiment designed in the following way: During the preacquisition test (pretest) a subset of five different odors (1,6ol = 1-hexanol; lin = linalool; lim = limonene; 1,7al = 1-heptanal, 8ol = octanol) were tested 10 times each in a pseudorandom order (ITI = 1 min). Fifteen minutes later, we randomly chose two out of the set of five odors as conditioned stimuli (CSs). One of them, the CS+, was forward paired with the sucrose reward (US, 30% sucrose solution, for 3 s), and the other was presented unrewarded (CS−). CS+ and US overlapped during the last second of the odor stimulus. CS+ and CS− were presented in pseudorandom order, 10 times each (ITI = 1 min). In a postacquisition test (posttest), 3 h after conditioning all five odors were tested again as described for the pretest (compare Fig. 1D). As well as CS, US was always presented to both antennae.
Three wires (polyurethane-coated copper wire, 14 μm in diameter, Electrisola) were waxed together as described by Okada et al. (1999, 2007). Wires were connected to an AC preamplifier (Headstage-27 Amplifier Neuralynx). Two silver wires with a diameter of 25 μm (Nilaco) were used, one to be the reference electrode and the other to record a myogram of the muscle M17 involved in extending the proboscis.
Visualization of the recording position.
After recording neuronal activity, one of the electrode wires was used as active electrode during electrocoagulation. The reference electrode served as passive electrode. Voltage pulses of 10 V and 10 ms duration were applied using a frequency of 10 Hz for 10 min through the active electrode. The electrocoagulation produced a small spot of autofluorescence (Fig. 1A), and was imaged with a confocal laser scanning microscope (Leica TCS SP2) using a Leica HC PL Apo ×10/0.4 dry lens objective. Coagulated spots and neuropil background fluorescence induced by the fixative were excited by using the 488 nm line of an ArKr laser or the 543 nm line of an HeNe laser.
The electrodes were inserted into the ventral α-lobe. The dominating neuron types at this recording site belong to the MB ENs and can be related to the A1, A2, A4, A5, and A7 soma clusters (Rybak and Menzel, 1993). EN neurites in that area are very thick (∼10 μm) and induce very prominent spike shapes in our recording electrodes, whose single electrode wires were 14 μm in diameter. In contrast, the neurites of KCs have very small diameters (<0.5 μm). Recorded with the same extracellular electrode type, KC neurites induce a much smaller and especially shorter waveform (nearly half the size of EN units), which we never observed in our recordings at the ventral α-lobe (for details, see supplemental Fig. 7, available at www.jneurosci.org as supplemental material).
Neural activity was measured differentially from all pairwise combinations of the three wires (Fig. 1) using the Patch Panel ERP-27 (Neuralynx). After being preamplified with a head stage (HS-27 Neuralynx), recorded electrical signals were amplified by a Lynx-8 Amplifier (Neuralynx), bandpass filtered at 1 Hz to 9 kHz, and sampled at 20 kHz. For spike detection, data were high-pass filtered (600 Hz).
We applied the semiautomatic spike-sorting technique (template-matching) provided with the Spike2 software (Cambridge Electronic Design). Mean activity and SD of the high-pass filtered channels were calculated. Thresholds for detecting events were always set above 3 × SD. Threshold-crossing events were used to compute templates of spike waveforms, which were subsequently used to assign individual spikes. Sorting results were shown in supplemental Figures 3–6 (available at www.jneurosci.org as supplemental material). We compared the unit separation between the preconditioning and postconditioning phases by visualizing the detected waveforms using principal component analysis (PCA) using the tools that Spike2 provided (cf. supplemental material, section 2, available at www.jneurosci.org). However, if the unit separation in the early (preconditioning) and late (postconditioning) experimental phases showed different single-unit waveforms, which were not separated by the first three components of the PCA, we excluded these units from our further analyses of the conditioning effect. In total, we excluded six units from three bees (compare Fig. 1D). Since we used three differential recording combinations in each bee, we checked for double detection of units on separate channels and used only the channel that showed the highest signal-to-noise ratio (SNR) for isolating the respective unit. In addition, we checked the interspike interval (ISI) distribution of the separated units during 3 min of spontaneous activity, recorded before odor stimulation. Examples of eight representative ISI distributions are shown in the supplemental material, section 2 (available at www.jneurosci.org). So far the refractory period of this neuronal type in honey bees is not reported, and it is not known whether all ENs show similar refractory times. ISI distributions for PE1 as well as for non-PE1 neurons show intervals smaller than 4 ms (Okada et al., 2007), and intracellular recordings near the integrating segment show peak-to-peak intervals of double spikes around 2 ms [Okada et al. (2007), their Fig. 5]. Thus we assume that the refractory period of ENs is possibly in the range between 2 and 4 ms, which potentially explains the small ISIs we found in many of the recorded ENs. We can also not rule out the contamination of small ISIs by false-positive spike detections. Spike sorting is always subject to potential misclassification (Harris et al., 2000; Joshua et al., 2007), and thus single-unit activity is per se not equivalent to single-neuron activity as obtained by means of intracellular recordings. In our recordings from the ventral α-lobe, we obtained comparatively high spike signal amplitudes (Fig. 1B). To ensure optimal and robust results of the spike-sorting procedure, we evaluated the first three PCs of the related spike wave forms before and after conditioning as illustrated in supplemental material, section 2.1 (available at www.jneurosci.org), rather than using the ISI distribution as a criterion of the sorting quality.
Throughout the text, we use the term “extrinsic neuron” (EN) as a synonym for the term “unit” to improve readability and to stress the fact that we recorded and sorted spiking activity from mushroom body extrinsic neurons and not from any other type of neuron.
To detect whether a specific stimulus caused a neuronal response, we used a dual approach. First, for each odor we pooled spike trains from all 10 trials and constructed the peristimulus time histogram (PSTH) using a bin width of 50 ms. We then estimated the mean m and SD across all bins during the 3 s preceding the stimulus and then tested for a rate bin that crossed a threshold of m + 3 × SD in a response window from 50 ms to 350 ms after stimulus onset. Inhibitory response were detected by first subtracting the baseline m and then applying the negative threshold criterion (−3 × SD). Second, we pooled the ISIs from all trials during the 3 s preceding the stimulus and in a response window from 100 ms to 600 ms. We then tested for a significant (p < 0.01) difference in the median of the two ISI distributions using a Wilcoxon rank-sum test. The former approach can detect short rate responses with considerable amplitude; the latter approach is sensitive to subtle excitatory or inhibitory rate changes that extend over several hundred milliseconds. We counted a response if either of the two methods indicated a success.
Time-resolved firing rate estimation.
Firing rates were estimated by means of kernel convolution (Nawrot et al., 1999; Meier et al., 2008). We used an asymmetric kernel with the shape of the α function k(t) = t × exp(−t/τ) with time resolution τ. We aligned the resulting rate function at time w, which indicates the center of mass of the kernel. In some cases, we used a causal kernel of exponential shape k(t) = exp(−t/τ) with a time resolution of τ = 25 ms.
For each single unit, we computed the lifetime sparseness SL = 1 − (Σ|rj|/N)2/Σ(rj2/N), where N = 5 denotes the number of tested odors. rj is the unit's response rate to stimulus j computed as the trial-averaged mean firing rate within the response interval (50 ms, 350 ms) after stimulus onset minus the trial-averaged mean baseline rate within the 3 s interval before stimulus onset. This measure was introduced to quantify the sparseness of neurons that show clear responses only for a small subset of all stimuli, while the majority of the stimuli evoke no or only very weak responses (Vinje and Gallant, 2000; Willmore and Tolhurst, 2001). SL has been repeatedly used to quantify the response sparseness of neurons in the insect olfactory pathway and thus allows for a direct comparison of the tuning characteristic of our EN recordings to that of olfactory receptor neurons and antennal lobe (AL) projection neurons (e.g., Wilson et al., 2004) and MB KCs (e.g., Perez-Orive et al., 2002). SL is less suitable in cases where most or all stimuli evoke a response of similar strength.
To further quantify the tuning strength of single units, we computed the signal-to-noise ratio (Mehring et al., 2003): SNR = σs2/σnoise2, based on the spike count measured in the response interval (50 ms, 350 ms). σs2 denotes the variance of the trial-averaged spike count computed across the five individual stimulus responses. This constitutes the signal. The higher σs2, the stronger the modulation of the spike count across stimuli is. σnoise2 denotes the variance across all 50 single trial spike counts after subtraction of the stimulus-specific trial-averaged spike count and thus quantifies the trial-by-trial variability of the unit's response. This constitutes the noise. We then compute the SNRempiric by dividing σs2 by σnoise2. Due to the stochastic nature of the spiking activity and for a finite number of trials (in our case 10 trials per odor), σs2 will be different from zero even if a unit does not show any stimulus tuning. To estimate and remove this bias, we apply a bootstrap method as follows. For each stimulus, we randomly draw 10 trials, allowing for repeated drawings. From this surrogate dataset, we again compute the SNR. This procedure is repeated m times (we used m = 200), and by taking the average, we obtain SNRbootstrap. Finally we compute the estimate of the unbiased SNRunbiased = SNRempiric − |SNRbootstrap − SNRempiric|. The SNR takes into account the trial-by-trial variability of responses and has been shown to correlate well with single-trial decoding performance of response signals (Mehring et al., 2004), and it is well suited to characterize broad stimulus response spectra.
For an ensemble of n neurons and a given stimulus a, we constructed the n-dimensional rate vector va at each point in time. From a pair of rate vectors, we then computed the time-resolved Euclidean distance (L2-Norm) as d(t) = (Σ[via(t) − vib(t)]2)1/2.
Data analyses were performed with Matlab. Custom-written Matlab functions, e.g., for computing time-resolved rate estimates, SL, and SNR, are provided in the FIND open source toolbox (Meier et al., 2008; http://find.bccn.uni-freiburg.de/).
Odor representation at the output of the mushroom body
To characterize odor representation at the level of the mushroom body output neurons, we recorded extracellular the activity of α-lobe ENs. Differential recordings from pairs of electrode wires allowed us to extract EN activity of 70 single units out of 27 animals (Fig. 1D; cf. Material and Methods). In a first series of experiments, we characterized the odor representation at the MB output regarding odor sensitivity and odor specificity. We presented 10 different odors and paraffin oil, which we used to dilute the odors. Figure 2C shows a single EN unit being excited by all of the 10 tested odors. In a second experimental series, we addressed how learning changed the initial response properties. We tested a set of five different odors before and after learning. The responses during the initial test phase before conditioning were included in our analysis of initial response properties. Interestingly, not all units were odor sensitive under naive conditions. Three-quarters (75%, N = 44) (Fig. 2A) of the EN units showed a trial-averaged response to at least one of the presented odors and were therefore called “odor sensitive”. The remaining 25% were initially insensitive and did not respond to any of the tested odors. In unconditioned bees, nearly 70% of the responding units (N = 52) were sensitive to five or more odors, typically showing excitation (Fig. 2B). We rarely observed inhibitory responses (Fig. 2B; green). If a unit showed inhibition, it was observed in all of its responses. Typically we did not find units being excited by one odor and inhibited by another. Only two examples (unit #3 and unit #8 in Fig. 4) showed a combination of inhibitory and excitatory responses, but only after conditioning.
Two types of learning-induced changes in EN units
To investigate changes in neural response due to learning, we devised a series of conditioning experiments. Five odors were tested before and after differential odor conditioning. Two of the five odors were used to perform differential conditioning during 10 acquisition trials. The recorded bees showed normal acquisition behavior and expressed a typical learning performance. Within an average of 468 ± 34 ms, the bees decoded the odor stimulus, made a behavioral decision and finally extended their proboscis (cf. supplemental material, section 3, available at www.jneurosci.org).
We determined odor response profiles of the recorded units by comparing their significant odor responses before and after conditioning. This allowed us to broadly classify two groups of EN units with respect to their learning-related change in activity: (1) A first group of 19 out of 35 responding units developed (“recruited”) and/or lost (“dropped”) odor responses in the course of PER conditioning. Two examples are shown in Figure 3B. Unit 1 recorded in bee 67 was recruited to express a response to the CS+ after conditioning. The second unit (unit 1 in bee 87) dropped its clear initial responses to the control odors B and C as a result of conditioning. Units showing dropped and/or recruited responses are called further “switched” units. (2) A second group of 16 units had stable and broad odor response spectra, which did not change during conditioning. However, they expressed plasticity by specifically increasing or decreasing their response rates. Figure 3C shows two examples. Unit 1 recorded in bee 83 increased its response strength to the CS+, whereas unit 1 in bee 73 decreased its response strength most clearly to the CS−. These modulating effects were also observed as odor-specific combinations. Units showing increased and/or decreased response strengths after learning are called further “modulated” EN units. Both groups of neurons showed different peak response rates during odor stimulation, which were significantly smaller in switched than in modulated units with average 14.8 and 36.1 spikes per second, respectively (cf. supplemental material, section 1.1, available at www.jneurosci.org).
The spontaneous firing rates in both unit groups were similar with an average of 17.7 in switched and 20.2 spikes per second in modulated units and the overall spontaneous firing rate distributions pooled for switched and modulated units (N = 35) before and after conditioning remains stable (p = 0.58, Wilcoxon rank sum test) (cf. supplemental material, section 1, available at www.jneurosci.org). However, in eight single units, we observed significant changes of the spontaneous firing rate during the 20 conditioning trials (10 trials CS+ and 10 trials CS−) tested with Spearman's rank-order correlation coefficient (p < 0.05, two-sided test). Four out of the nineteen switched units decreased their spontaneous rate (example unit 10 shown in supplemental Fig. 1C, available at www.jneurosci.org as supplemental material), and two switched units increased their rate. Out of the 16 modulated units, we found 2 units changing their spontaneous firing rate, one decreased (example unit 33 shown in supplemental Fig. 1D, available at www.jneurosci.org as supplemental material) and the other increased. Additional analyses of spontaneous EN activity with respect to serial interval correlation and classification of ISI distributions is provided in Farkhooi et al.(2009).
“Switched” EN units become recruited to encode odor–reward association
We found associative changes by “switching” in 19 out of the 35 odor-sensitive units; two of those were initially odor insensitive and established a response to the CS+ and a second odor after conditioning (Fig. 4A, units 1 and 2). The number of units responding to the CS+ after conditioning increased significantly by 50% to ∼80% (binomial test: p < 0.001). In addition, significantly more units responded to CS+ compared to CS− (binomial test: p < 0.001) and control odors (binomial test: p < 0.001) after conditioning (post). In the posttest, the number of units responding to the CS− and control odors decreased slightly but not significantly (binomial test: CS− and control, p > 0.2). Thus, the number of units responding to CS− and control odors remains rather stable at ∼40–50% (Fig. 4B). Switches in response to CS+ were always recruitments. By contrast, CS− and the control odors caused both types of switches (dropped and recruited) at which odor sensitivity was lost more frequently than gained (Fig. 4C). Thus the change in response profiles of switching units in the course of learning is dominated by an exclusive recruitment of responses to the reward-associated odor.
“Modulated” EN units changed their response strength after learning
We found modulatory changes in a similar large fraction of units (N = 16). These units showed broad and stable odor response spectra combined with odor-specific changes of their response strength as a consequence of learning (Fig. 5A). In some units, the CS+ evoked an increase, whereas CS− and control odors caused a decrease in response strength after conditioning (Fig. 5B, unit #28). In other units, only the conditioned odors (CS+ and CS−) showed an effect. EN unit #29 in Figure 5B provides an example where the response strength to CS+ and CS− increased, whereas the response strength to the control odors did not change. In rare cases, we detected small reductions in response strength to CS+ in single units after conditioning (Fig. 5B, cf. #23). Overall, learning resulted in increased response strength for the rewarded odor. This is illustrated by the median of the rate changes (Fig. 5C), which was significantly larger than zero for CS+ (p < 0.05; Wilcoxon signed rank test) but not for CS− (p = 0.07) and control (p = 0.35) odors.
Switching and modulation at the MB output reflects long-term memory
To verify the time course of the reported changes to the CS+ presentation, we analyzed whether the changes we found were already established during the acquisition phase, and might be related to short-term effects, or whether switching and modulation depend on a longer timescale and might involve consolidation mechanisms. Figure 6A includes all 30 CS+ trials (10 trials before, 10 trials during, and 10 trials after conditioning) of two switched units (units 10 and 13) both recruited to the CS+, and of one modulated unit (unit 35), which increased its response to the CS+. In all three examples, the reward association first occurred during the posttest phase 3 h after acquisition, while we could not observe learning-related changes during acquisition. All example units in Figure 6A increased their response rate due to US presentation during the conditioning, indicating an internal representation of the sucrose reward. The US response also occurred in unit 37, which was generally insensitive to all tested odors. This US sensitivity is also reflected in the response rate difference between the acquisition phase and the pretest phase averaged across all switched and modulated units (Fig. 6B).
Overall, in both switched and modulated units, the changes to the rewarded odor occurred first during the posttest. This is illustrated by the observed response rate difference between the posttest and the pretest. No such increase of the CS+ response can be observed when computing the difference between the acquisition phase and the pretest (Fig. 6B). In addition, the peak response rates for all 30 trials averaged across the population of switched and modulated units (Fig. 6C) show that the increased CS+ response occurred only during the first posttest trial and maintained an increased level during all 10 retention tests. We therefore conclude that the recruitment of a new response to the CS+ as well as the increase of the response strength to the CS+ requires a longer timescale and possibly involves mechanisms of consolidation (see Discussion).
Different tuning characteristics of “switched” and “modulated” units
When comparing Figures 4A and 5A, it becomes apparent that switched ENs expressed a more specific odor response spectrum, while modulated ENs typically responded to all of the five odors. To quantify the odor tuning properties of both EN types, we used two different measures. A widely used tuning measure is the lifetime sparseness, SL. It is specifically designed to quantify the kurtosis of the single-neuron response distribution across the set of tested stimuli for cases where the response distribution is narrow, i.e., when only few stimuli result in a neuronal response (sparse responses; see Material and Methods). Consequently, SL yields high values for units that show specific and strong responses to a small subset of the presented stimuli and weak or no responses to the remaining stimuli. For switched units, SL assumes an average initial value of 0.23 ± 0.13 (Fig. 7A). This is increased to 0.31 ± 0.16 after conditioning, and the median difference is significantly larger than zero (p < 0.05; Wilcoxon signed rank test). As to be expected, modulated units show comparably small values of SL with an average of 0.08 ± 0.11. The learning-induced increase to 0.12 ± 0.13 is again significant (p < 0.05).
The fact that EN units, and in particular the group of modulated EN units, show comparably broad response spectra associated with low values of SL (Fig. 7A) indicates that this measure is not optimal to quantify their tuning strength. We therefore applied a second tuning measure, the SNR (see Material and Methods). It quantifies the variation of the single-unit rate responses across different odors (signal) and normalizes by the trial-by-trial variation of the response amplitudes (noise). For the empiric SNR, the relation between switched and modulated units is reversed. While switched units show comparably small values of SNR due to the overall smaller peak response rates (cf. supplemental material, section 1.1, available at www.jneurosci.org), the modulated units exhibit higher values of SNR, indicating a relatively strong odor tuning of their firing rates (Fig. 7B). Switched units increased the SNR as a consequence of conditioning from average 0.03 ± 0.12 to 0.47 ± 1.0, the median of the difference being significantly larger than zero (p < 0.05; Wilcoxon signed rank test). Modulated units significantly increased the SNR from 0.56 ± 1.21 to 1.24 ± 1.71 (p < 0.01). Single units could reach high values of SNR (maximum: 5.9), and thus their response rates are particularly informative about the stimulus.
We are now in the position to compare our results for the EN units to the previous reports on lifetime sparseness at different stages of the insect olfactory pathway. SL is high in first-order olfactory neurons, the olfactory receptor neurons (ORNs), as reported in Drosophila (SL ≈ 0.69-0.98) (Wilson et al., 2004). Projection neurons (PNs) are the second-order olfactory neurons and carry the odor code at the output of the antennal lobe. Uniglomerular PNs show odor responses that are considerably less specific than ORNs in bees (Krofczik et al., 2008) in Drosophila (SL ≈ 0.16–0.72) (Wilson et al., 2004) and in locust (SL(mean) ≈ 0.4) (Perez-Orive et al., 2002). Third-order olfactory neurons, the MB Kenyon cells, have been shown to be highly sparse with respect to their odor response profile in the locust (Perez-Orive et al., 2002), associated with high values of SL(mean) ≈ 0.63. Our results demonstrate that the mushroom body output neurons show comparatively low odor specificity. The neurons' odor specificity at different stages of the olfactory network develops in parallel to their relative numbers. In the honeybee, ∼60,000 odor-specific ORNs provide the input to the AL. From the AL, ∼950 projection neurons convey olfactory information to the mushroom body, each PN being responsive to about half of the olfactory stimuli (Krofczik et al., 2008). Projections to a large fraction of the ∼160,000 KCs in the mushroom body basal ring and lip region (Mobbs, 1982; Kirschner et al., 2006) mean an expansion of the neuronal space by a factor of ∼100, resulting in the observed high odor specificity in KCs. At the MB output, ∼400 ENs sample the KC output (Rybak and Menzel, 1993), where each EN on average receives input from ∼2000 KCs, which explains the low stimulus specificity of EN responses.
Rapid reward representation at the MB output
Odor representation at the MB output is drastically changed after learning particularly with respect to the CS+. Switching units developed a response preference to the CS+ (Fig. 4C), and modulated units increased their odor response rates after conditioning to the rewarded odor (Fig. 5C). Next, we assess the temporal evolution of stimulus representation in the ensemble of ENs (Fig. 8). We computed the time-resolved Euclidean distances in the neuronal ensemble space including “switched” and “modulated” units for all pairs of odor stimuli (see Material and Methods). At the MB output the ensemble response of EN units to the CS+ follows a different processing dynamics as compared to the CS− and three control odors after learning. The average Euclidean distance between the CS+ and the three controls assumed considerably higher values than the average distance between CS− and controls, and controls among each other, as shown in Figure 8B. The Euclidean distance between CS+ and CS− were nearly the same as for CS+ and control odors. Different processing of the CS+ lasted throughout the 3 s of stimulation. A significant odor driven ensemble response was generated around 60 ms after odor onset as observed before and after conditioning, and independent of the odor identity (Fig. 8B,C). This is, on average, 20 ms later than the estimated onset of a significant odor code in the PN ensemble at the MB input (supplemental Fig. 2, available at www.jneurosci.org as supplemental material) (Krofczik et al., 2008). In addition, ∼80 ms later (141 ms after odor onset), the CS+ representation significantly changed compared to all other stimuli, reaching 90% of its maximum at ∼177 ms. Thus, during the retention test the CS+ representation in the EN ensemble activity is pushed apart from the representations of the CS− and all control odors, an effect that rapidly becomes significant within <150 ms after the odor stimulus reached the antennae, allowing enough time for neural processing underlying behavioral decision making and response initiation as expressed in the extension of the proboscis (∼470 ms after odor onset) (cf. supplemental material, section 3, available at www.jneurosci.org).
Odor representation at the mushroom body output
The MB calyx of the honeybee is one of the convergence sites of second-order olfactory neurons and modulatory neurons like the octopamine immunoreactive neuron (VUMmx1) known to code the appetitive meaning of olfactory stimuli (Hammer, 1993). MB intrinsic neurons (KCs) change their responses to both the rewarded and the unrewarded odor (Szyszka et al., 2005), and the MB as a whole was found to be involved in the consolidation of long-lasting olfactory memory in both honeybees (Menzel et al., 1974; Menzel and Müller, 1996) and Drosophila (Heisenberg et al., 1985). The sparse coding in the temporal and spatial domain suggests that KCs require multiple coincident input to be activated, including inhibitory input (Szyszka et al., 2005), very much as was found by direct KC recordings in locust (Perez-Orive et al., 2002; Stopfer et al., 2003; Jortner et al., 2007). It was therefore concluded and proven for locusts that the response profiles of KCs are narrowly tuned to combined sensory input (Perez-Orive et al., 2002).
Questions arising from this view of neural processing in the MB relate to both the combinatorial sensory code across KCs and their associative plasticity. We address these questions in the honeybee by recording with extracellular electrodes from MB ENs in the ventral aspect of the α lobe. These ENs receive input from a large number of KCs and convey information to various parts of the brain, including the premotor descending neuropil of the lateral horn (Rybak and Menzel, 1993; cf. Material and Methods). It is thus less likely that they are selectively tuned to particular stimulus combinations; rather, they may reflect the history of experience with the stimuli. Evidence for this view comes from recordings from an identified EN in this region, the PE1 neuron, which was found to respond to a large range of stimuli (olfactory, mechanosensory, visual), and changes its response profile as a consequence of olfactory learning (Mauelshagen, 1993; Menzel and Manz, 2005; Okada et al., 2007). Here we focused on other ENs. We found that in unconditioned bees about three-quarters of the single units that we extracted from our EN recordings responded to at least one of the tested odors. Most of these units displayed a broad response profile (Fig. 2). The lifetime sparseness of switched (SL ≈ 0.2) and modulated ENs (SL ≈ 0.1) at the MB output in untrained bees is very low. PNs at the input of the MB, for example in Drosophila, show much higher values (SL ≈ 0.2–0.7, Wilson et al., 2004), and third-order olfactory neurons, the MB Kenyon cells, are coding odors in a sparse manner in locust (Perez-Orive et al., 2002), associated with high values of SL. Our results demonstrate that ENs receiving direct input from KCs show a broad tuning profile to different odors and appear not to represent single features of an odor stimulus.
After associating one odor with reward, nearly half of the units changed their odor response spectra; i.e., they dropped responses and/or they developed new responses to one or several odors. All switches observed with respect to the CS+ odor were recruitments (Fig. 4). We interpret this as a recruitment of neuronal resources to specifically encode the rewarded stimulus. The other half of the units did not change their qualitative odor profile but showed quantitative changes regarding their response strength. These “modulated” units increased and/or decreased their response rates to different odors. Thereby rewarded odors evoked increasing response strengths, whereas CS− and control odors evoked decreased or increased response strengths in approximately equal proportions (Fig. 5). The dichotomy of “switched” and “modulated” units could result from morphological distant EN types, since we rarely observed switched and modulated units in the same recording (only in 2 out of 20 bees). Thus, we found little odor identity coding but strong odor–reward association encoding after consolidation at the MB output.
Mechanisms of associative plasticity
KCs are assumed to have narrow and specifically tuned response profiles to combinations of sensory inputs across modalities [Drosophila: Wang et al. (2004) and Turner et al. (2008); locust: Perez-Orive et al. (2002), Stopfer et al. (2003), and Jortner et al. (2007); honeybee: Szyszka et al. (2005)]. Differential conditioning after pre-exposure evoked a recovery from the repetition-induced decrease in KC responses to a rewarded odor (CS+), while the responses to a nonrewarded odor (CS−) decreased further and some of the KC initially activated by the CS− odor even lost their responses (Szyszka et al., 2008). This associative effect may result from combined excitatory input [e.g., from the VUMmx1 neuron (Hammer, 1993)] and inhibitory input [e.g., from GABA-immunoreactive (ir) inhibitory feedback neurons of the protocerebro-calycal-tract (PCTs) (Bicker et al., 1985; Grünewald, 1999a)]; both types of neurons undergo learning-induced plasticity [VUMmx1: Hammer (1993); PCTs: Grünewald (1999b)]. It is conceivable that associative changes of the balance between excitation and inhibition favor the focus of excitation in small subpopulations of KCs and thus may provide more specific output to ENs. Thus, plasticity at the level of the MB input may well be responsible for our observed changes in the MB output neurons.
In addition, intrinsic properties of ENs may be responsible for their learning-related plasticity. Long-term potentiation (LTP) was found after electric stimulation of KCs and coincident depolarization of the PE1 neuron (Menzel and Manz, 2005) in honeybees, and spike time-dependent plasticity (STDP) was demonstrated for the synapses between KCs and β-lobe neurons in the locust (Cassenaer and Laurent, 2007). Both mechanisms imply immediate changes following odor–reward pairing. However, during the 10 trials of the acquisition phase, we did not observe associative effects. Neither the switched nor the modulated units increased their response strength to the CS+ presentation immediately during the acquisition (Fig. 6), although both unit types responded to the US, indicating an internal representation of the reward stimulus (Fig. 6B). Thus, we conclude that fast mechanisms of LTP such as STDP cannot explain our results. This is supported by experiments done in moths. There, Ito et al. (2008) could show that the most reliable and robust learning occurred if CS and US were presented without overlap. We tested the bees 3 h after acquisition. The very same time window allowed the formation of a translation-dependent, delayed memory trace in KC axons of Drosophila (Yu et al., 2006). In honeybees, a prolonged activation of cAMP-dependent protein kinase during conditioning induced long-term memory (Müller, 2000). Thus, we think that the associative effect reflected in recruitment of switched and increasing response strength of modulated EN units at the MB output after reward conditioning is a delayed phenomenon that involves mechanisms of memory consolidation. Furthermore, the established increased response strength after consolidation maintained on an increased level during all 10 retention tests (Fig. 6C). Neuronal extinction, which by itself would be a short-term learning effect, was not observed.
Rapid reward prediction at the MB output
A rapid representation of the behaviorally relevant (rewarded) stimulus may be a prerequisite for behavioral decision making based on the long-term associative memory. Figure 8 shows the dynamic of this process in an ensemble consisting of “switching” and “modulated” ENs. Odor identity representation in the EN population becomes significant within ∼60 ms after stimulus onset. Starting ∼140 ms after stimulus onset, the CS+ representation in the EN ensemble separates from those of the control odors and the CS− (Fig. 8C), indicating an encoding of the expected reward that persists throughout stimulation at the level of the MB output (Fig. 8B). The maximum is reached within ≤200 ms. At the MB input, odor identity coding by uniglomerular projection neurons reaches significance within ∼40–50 ms, and its maximum is reached after ∼150 ms (Krofczik et al., 2008) (cf. supplemental Fig. 2, available at www.jneurosci.org as supplemental material). Thus, odor representation is established at the AL output and the MB output with a short delay of ∼10–20 ms.
Less than 300 ms later, the motor activity in the M17 increased with an average response latency of 470 ms (cf. supplemental Fig. 2, available at www.jneurosci.org as supplemental material). This value is close to the response intervals as previously reported by Wright et al. (2009). Thus, in honeybees processing from odor stimulation through perception, stimulus evaluation based on the long-term associative memory, behavioral decision making, and motor execution (PER) occurs within 400–500 ms.
Together, our findings corroborate the view that the MB is involved in computing the reward-associated stimulus. At the MB output, we found mainly two types of neuronal behaviors switched and modulated units. Both unit types showed an internal representation of the US and the CS, indicating a convergence of the two pathways. The associative effect (recruitment in switched units, increased activity in modulated units) to the CS+ presentation was not observable during the acquisition, but only during the retention tests 3 h after conditioning. This might indicate the involvement of consolidation mechanisms in the odor representation changes for the associated stimulus at the MB output.
This work was funded by the Bundesministerium für Bildung und Forschung through Grant 01GQ0413 to the Bernstein Center for Computational Neuroscience Berlin and Grant 01GQ0941 to the Bernstein Focus Learning and Memory in Decision Making. We are grateful to Ryuichi Okada for his advice with respect to the recording technique. We thank Jürgen Rybak for discussions on the methods used to determine the recording position, Evren Pamir for his contribution to the initial steps of our data analysis, and Bernhard Komischke and Michael Schmuker for helpful comments on an early version of this manuscript. Finally, we thank Brian Smith for his comments.
- Correspondence should be addressed to Martin Fritz Strube-Bloss at his present address: School of Life Sciences, Arizona State University, P.O. Box 874501, Tempe, AZ 85287-4501. or .