Abstract
Sensory cortices, even of primary regions, are not purely unisensory. Rather, cortical neurons in sensory cortex show various forms of multisensory interactions. While some multisensory interactions naturally co-occur, the combination of others will co-occur through experience. In real life, learning and experience will result in conjunction with seemingly disparate sensory information that ultimately becomes behaviorally relevant, impacting perception, cognition, and action. Here we describe a novel auditory discrimination task in mice, designed to manipulate the expectation of upcoming trials using olfactory cues. We show that, after learning, female mice display a transient period of several days during which they exploit odor-mediated expectations for making correct decisions. Using two-photon calcium imaging of single neurons in auditory cortex (ACx) during behavior, we found that the behavioral effects of odor-mediated expectations are accompanied by an odor-induced modulation of neuronal activity. Further, we find that these effects are manifested differentially, based on the response preference of individual cells. A significant portion of effects, but not all, are consistent with a predictive coding framework. Our data show that learning novel odor–sound associations evoke changes in ACx. We suggest that behaviorally relevant multisensory environments mediate contextual effects as early as ACx.
SIGNIFICANCE STATEMENT Natural environments are composed of multisensory objects. It remains unclear whether and how animals learn the regularities of congruent multisensory associations and how these may impact behavior and neural activity. We tested how learned odor–sound associations affected single-neuron responses in auditory cortex. We introduce a novel auditory discrimination task for mice in which odors set different contexts of expectation to upcoming trials. We show that, although the task can be solved purely by sounds, odor-mediated expectation impacts performance. We further show that odors cause a modulation of neuronal activity in auditory cortex, which is correlated with behavior. These results suggest that learning prompts an interaction of odor and sound information as early as sensory cortex.
Introduction
Natural environments are inherently multisensory. By continuous exposure to natural scenes, our senses are constantly bombarded by stimuli from several modalities. By repeated exposure or learning, multisensory information becomes behaviorally meaningful, whereas one stimulus provides contextual information to another. It is, therefore, not surprising that nervous systems of creatures as simple as insects and as complex as humans are endowed with mechanisms that integrate information from several senses, and are well adapted to perceive multisensory scenes (Stein and Meredith, 1993; Leonard and Masek, 2014; Currier and Nagel, 2020). How neural circuits compute and integrate information from the various senses in support of behavior remains a fundamental, yet underexplored, topic in neuroscience.
Multisensory (or cross-modal) interactions are evident in our everyday perception. Taste, for example, emerges from the synthesis of gustatory, olfactory, tactile, and visual cues (Spence, 2013), and speech perception is highly influenced by concomitantly observing lip movements (McGurk and MacDonald, 1976). Moreover, the benefits of cross-modal interactions extend beyond proper perception of multisensory objects. In some cases, associative information from one modality might generate context and, thereby, modulate perception by the way an animal attends, expects, and interprets information from other modalities. Odors, for example, are a prime case for such modulation because olfactory signals can provide strong contextual cues of the current state of the environment. For example, scent markings are used as social communication signals to mark a territory or provide a signature of social dominance (Arakawa et al., 2008). An animal passing by an environment sprayed with the smell of a predator will become differentially attentive to environmental sounds to avoid predation. Similarly, conspecific urine smell conveys information on reproductive state and has been shown to impact reproductive success (Thonhauser et al., 2013; Coombes et al., 2018). Odors are potent contextual cues likely because they remain in the environment for relatively long periods of time (Doty, 1986), and since the activation of olfactory circuits is directly linked to the limbic system (Sokolowski and Corbin, 2012). In humans, odors have also been shown to induce context-dependent effects on long-term memory (Willander and Larsson, 2007).
Odors have been shown to have contextual effects that impact the auditory modality. Parental behavior toward newborns is one such example. In rodents, pup odors play a role in a multisensory behavior called “pup retrieval.” Pups emit ultrasonic vocalizations (USVs) when they are isolated from the nest. As a result, the mother retrieves the pups back to the nest (Ehret, 2005; Elyada and Mizrahi, 2015). This behavior is multisensory in nature. When pup vocalizations are synthetically played from a speaker, mothers approach the speaker primarily when it is in the presence of pup odor, suggesting that both odors and sounds are important for this behavior (Okabe et al., 2013). Moreover, responses of single neurons to USVs and other sounds in the mouse auditory cortex (ACx) have been shown to be modulated by the presence of pup odors (Cohen et al., 2011). Predator odors, too, have been shown to modulate neuronal responses to auditory stimuli in mice (Halene et al., 2009).
The above-mentioned examples of odors acting as context for other senses still remain underexplored. Moreover, they are often restricted to behaviors and/or odors with innate valence or that naturally cooccur with other sensory stimuli as multisensory objects. It remains unclear whether and how animals learn the statistical dependencies of odors and sounds in the environment and how such associations may impact behavior. Even less is known about how odors change the underlying neural activity of the relevant circuits. To that end, we designed a sound discrimination task using odors as contextual expectation cues for trial identity. We hypothesized that different behavioral contexts associated with odors will result in the modulation of behavior and neuronal activity in the first cortical station of sound processing, the ACx. Our results show that indeed learned odor–sound associations modulate behavioral choices, which are correlated with changes in single-neuron responses in ACx. The nature of modulation varied among neurons and was partially consistent with the theory of predictive coding.
Materials and Methods
Surgical procedures and imaging.
All experimental procedures were conducted in accordance with the Hebrew University Animal Care and Use Committee. We used a virus expressing a calcium indicator in the cytoplasm and a red fluorescent protein in the nucleus (AAV9-hsyn-GCaMP6s-P2A-nls-dTomato) that was produced at The Edmond and Lily Safra Center for Brain Sciences virus core facility (https://elsc.huji.ac.il/research-and-facilities/expertise-centers/elsc-vector-core-facility/) and was assessed at a titer of 1012. The virus (200 nl) was injected using NanoJect 2 to the ACx of the left hemisphere of female juvenile BALB/C mice [age range, postanatal day 21 (P21) to P24]. The injection site was sealed with bonewax. During the same procedure, a head bar was fixed to the top of the skull using dental cement. A 3 mm chronic glass window was implanted above the injection site according to published protocols (Goldey et al., 2014) at 21–28 d after virus injection. Both procedures were performed under 2% isoflurane anesthesia. The hair was initially removed from the surgery area using a commercial hair removal cream and rinsed with rubbing alcohol. Lidocaine was injected under the skin as local analgesic. Mice were injected subcutaneously with carprofen (4 mg/kg) after each procedure.
Sound presentation.
Pure-tone sound stimuli were 100 ms in duration and presented through a free field speaker (model ES1, TDT) positioned 5 cm from the right ear of the animals. The speaker was driven at a 500 kHz sampling rate via a driver (model ED1, TDT). Sound intensity was calibrated to 75 ± 2 dB SPL for all presented sound frequencies.
Odor presentation.
We used the odors ethyl-butyrate, α-pinine, and isoamyl-acetate (Sigma-Aldrich) and diluted them with mineral oil to an equal vapor pressure of 10 ppm. These odors have been shown to have neutral valence (Root et al., 2014). Odors were delivered to the snout of the mouse through a custom-built olfactometer, at a flow rate of 0.1 L/m using a mass flow controller (Vinograd et al., 2017). Odors were continuously removed around the headspace by air suction. Odor delivery was calibrated with a photoionization detector (miniPID, Aurora Scientific) to verify that no odor trace remained in subsequent trials. Odors were still present and overlapped completely during the presentation of sounds.
Intrinsic signal imaging.
To identify the primary auditory cortex, we imaged the brain at low resolution using a PhotonFocus CMOS camera, while directly illuminating its surface with an LED light (wavelength, 617 nm). We played 10 repetitions of tone clouds with 2 s duration and a center frequency of 4, 7, 13, or 24 kHz, consisting of 30 tones with 50 ms duration logarithmically spaced between ±10% of the center frequency.
Two-photon calcium imaging.
We imaged GCaMP6s-labeled neurons in layer 2/3 for 28 training sessions in four mice (of the five mice trained behaviorally) using a custom-built (Flickinger et al., 2010) galvo-mirror scanning two-photon microscope with a frame rate of 7.2 Hz. Two-photon excitation (950 nm) was delivered through a DeepSee femtosecond laser (Mai Tai, SpectraPhysics). Imaging was performed through a water-immersion objective (0.8 numerical aperture; model CF175, Nikon) and detected through GaAsP Photomultiplier Tubes (Hamamatsu). The imaging field size was set to 260 × 260 μm over a 512 × 210 pixel window. We used Scanimage (Pologruto et al., 2003) software for acquisition and online drift correction (using the red channel).
Sound selection.
Imaging fields were first selected based on the optical quality of imaging. Once chosen, for every mouse, we played a series of pure tones of frequencies ranging from 4 to 24 kHz and recorded neuronal responses from that field. Then, we chose two frequencies 0.5–1 octave apart such that a majority of neurons in the field were responsive to one or both frequencies.
Behavioral timeline.
Mice were initially trained on a Go-NoGo task without olfactory cues. Hit trials were rewarded with 5–10 μl of sweetened water (5% sucrose). Reward was delayed for at least 1 s after stimulus onset to allow for sufficient neuronal recording time not driven by possible responses to the reward. False alarm (FA) trials were punished by 2 s of white noise, without delay. After mice reached high performance in the task (d′ > 2), either the Go or NoGo frequencies were shifted closer to the other frequency to increase task difficulty. We shifted the frequencies daily while continuing to probe the mouse performance. When performance stabilized at a moderate level (1 < d′ < 2; see Behavioral analysis), we defined that we reached the level of task difficulty that may benefit from adding additional information to the task. Only after reaching this stage, odors were added. Table 1 shows the final frequencies used as Go and NoGo for each mouse.
The final frequencies used as Go and NoGo for each mouse
Mice initially tended to perform worse after the addition of odors and gradually improved. The first session when their performance exceeded a d′ value of 1 was considered the pre-odor bias session. From the next session onward, odor–sound probabilities were changed, as indicated in Table 2.
The odor–sound probabilities used in the experiments
We used a mild coupling between odor and sound to prevent mice from ignoring sounds and using odors instead. The identities of the three odors were different for each mouse such that the odor for which the bias was highest on the pre-odor bias session was chosen as odor 3 (predicting NoGo with higher probability). Each mouse went through six odor bias behavioral sessions, not necessarily on consecutive calendar days.
Behavioral analysis.
We excluded trials in which mice began licking during odor presentation, before sound onset. The criterion to stop a session was a drop in licking to <10% of trials in which the mouse licked in the last 50 trials.
For each behavioral session, we calculated a hit rate (the response probability for Go trials) and a FA rate (the response probability for NoGo trials). To compensate for individual biases, we quantified accuracy levels using a measure of discriminability from signal detection theory, d′ (Nevin, 1969). d′ is defined as the difference between the normal inverse cumulative distribution of the hit and FA rates,
Using odors for manipulation of expectation in an auditory discrimination task. a, The full experimental timeline started at P21–P24 and lasted for 10–15 weeks. b, d′ values of all five mice before learning when training on the set, after learning the easy task and after we increased task difficulty to maximum, just before the pre-odor bias (see Materials and Methods). Each shape is data from a different mouse. The red cross is the mean. c, Behavioral setup. d, Trial structure and outcomes for the Go-NoGo task with odors as preceding cues. Odor duration, 500 ms; sound duration, 100 ms; response window, 2 s. e, Odor–sound probabilities for all of the six combinations. The pre-odor bias and odor bias stages are shown on the top and bottom, respectively. Odors 1, 2, and 3 are labeled blue, gray, and red, respectively. Expectation condition for each odor–sound pair is labeled as expected (E; turquoise, p(sound/odor) = 2/3), neutral (N; gray, p(sound/odor) = 1/2), or unexpected (UE; purple, p(sound/odor) = 1/3). f, A representative example of 30 consecutive trials from one mouse engaged in an odor bias session. Black dots are licks, green dots are rewards, and red dots are punishments. Trial outcome and the expected probability of the stimulus per trial are indicated on the right. g, d′ values (mean ± SEM) for all behavioral sessions across all mice. The data are sorted based on sounds alone (solid line; Go, lick; NoGo, no lick) or based on odors alone (odor 1, lick; odor 3, no lick). d′ for sound alone was significantly higher than for odor alone across all sessions (one-tailed signed-rank test: *p < 0.05, df = 4, es = 6.55, 3.56, 1.4, 1.26, 2.06, 5.08, 3.88).
Odor cues affect sound discrimination performance. a, Hit rates for the different odors of one example mouse for the pre-odor bias session (left) and one of the odor bias session (session 3, right). b, Difference in hit rate between odors 1 and 2 (blue), or odors 3 and 2 (red), for all mice across all sessions (mean ± SEM; one-tailed signed-rank test: n.s., no significant difference; df = 4; es = 0.01, −0.75, 0.48, 0.29, 0.53, −1.09, −0.44). c, FA rates for the same mouse and sessions as in a. d, Same as b but for the difference of FA rates. Starting from odor bias session 2, the FA rate was significantly higher for odor 1 as compared to odor 3 (one-tailed signed-rank test: *p < 0.05; df = 4; es = 1.08, 0.19, 1.91, 1.88, 3.06, 1.23, 2.5). e, Lick bias for the same mouse and sessions as in a. f, Same as b but for lick bias. On Odor bias sessions 2–4, lick bias was significantly higher for odor 1 as compared to odor 3 (one-tailed signed-rank test: *p < 0.05; df = 4; es = 0.72, 0.8, 1.55, 2.15, 3.9, 0.04, 0.98). Shaded gray area marks “effective biasing sessions.” g, d′ for the same mouse and sessions as in a. In the pre-odor bias stage, all odors were neutral, though color codes match their future expectation condition. At the odor bias stage odor 1 → Go and odor 3 → NoGo became expected (E; turquoise), odor2 → NoGo and odor2 → Go remained Neutral (N; gray), and odor1 → NoGo and odor3 → Go became unexpected (UE; magenta). h, Difference in d′ between either the expected or the unexpected condition and the neutral condition for all mice across all sessions (mean ± SEM). On odor bias sessions 2–4, d′ was significantly higher for the expected condition (one-tailed signed-rank test: *p < 0.05; df = 4; es = 0.72, 0.8, 1.55, 2.15, 3.9, 0.04, 0.98). i, All “effective biasing sessions” and mice for (from left to right) hit rate, FA rate, bias, and d′ for the different expectation conditions (mean ± SEM; one-tailed paired t test with Bonferroni’s correction: *p < 0.05, **p < 0.005, ***p < 0.0005; df = 14; es(Hit rate 1,2) = 0.42, es(Hit rate 2,3) = 0.23, es(Hit rate 1,3) = 0.65, es(FA 1,2) = 1.13, es(FA 2,3) = 1.16, es(FA 1,3) = 2.28, es(bias 1,2) = 1.58, es(bias 2,3) = 0.71, es(bias 1,3) = 2.27, es(d′ 1,2) = 1.58, es(d′ 2,3) = 0.71, es(d′ 1,3) = 2.27).
Two-photon calcium imaging of the ACx of mice engaged in the task. a, Experimental setup. ISI, Intrinsic signal imaging; 2P; two-photon calcium imaging. b, Top, Image of a blood vessel map from the chronic window. Bottom, Mean intrinsic signal response to a 4 kHz tone cloud. The dotted line marks the rough boundary of ACx in this mouse. c, Example 2P micrograph of a representative neuronal field of view. d, Mean calcium response traces (shaded area is the SEM) of 30 example neurons (rows) from the neuronal field shown in b to Go (left column) and NoGo (right column) following odor 1 (blue), odor 2 (gray), and odor 3 (red). Calibration: 0.1 ΔF/F; 1 s. Gray vertical lines indicate the time of sound presentation. The table on the top shows the odor–sound combination of each column. e, Mean calcium response traces (shaded area, SEM) of an example neuron with a stable response profile to Go (left column) and NoGo (right column) following odor 1 (blue), odor 2 (gray), and odor 3 (red) throughout all behavioral sessions. Y-scale, 0.1 ΔF/F. Green rectangle indicates odor presentation (duration, 500 ms). Black/white rectangle indicates Go/NoGo presentation respectively (duration, 100 ms). Asterisks indicate a statistically significant response. f, same as e for a neuron with an unstable response profile.
We used an additional measure from signal detection theory which is the lick bias (Fig. 2e). Bias is the general tendency for a response, independent of d′, and is defined as
Calcium imaging analysis.
Image stacks were corrected online for motion using the red channel as a reference. Regions of interest (ROIs) were selected manually for each cell in each session. Raw fluorescence time series F(t) were obtained for each cell by averaging across pixels within each ROI. Baseline fluorescence F0 was computed by taking the mean F(t) before each stimulus. The change in fluorescence relative to baseline, ΔF/F0, was computed by taking the difference between F and F0 and dividing it by F0. A neuronal response to a single trial was calculated as the average of the ΔF/F0 of that neuron, 1 s following sound onset. To avoid analyzing responses that follow a potential change in fluorescence resulting from premotor activity or any other non-task-related input, trials in which the relative SD of F0 was >10% were excluded. Neuron–session pairs that responded to five trials or less of some odor–sound pair were excluded from the analysis. A minority (22% of responsive neuron–session pairs) of neuron–session pairs responded with a decrease in fluorescence to some odor–sound pairs. These responses were excluded from the analysis of this work.
Responsive neurons were classified as neurons with responses significantly >0 (signed-rank test, p < 0.05) to at least one odor–sound pair. Analysis was restricted to responsive neuron-session pairs and correct trials (Hit, CR) since Miss trials were scarce and neuronal responses to FA might be contaminated by the response to the noise punishment. Neuron–session pairs were classified as Go-preferring neurons if their strongest response was to an odor–Go pair, and as NoGo-preferring neurons if their strongest response was to an odor–NoGo pair. Of note, single-neuron responses to sounds were variable throughout sessions, such that neurons could even be classified as responsive in one session and as unresponsive in another (Fig. 3e, for examples). Therefore, we avoided a between-session comparison of the same neurons and treated different sessions separately. In the analysis shown in Figures 4d and 5, in which we pooled all effective bias sessions together, the responses of neurons that were responsive on more than one session were taken only from the first session in which they responded in each population. This was done to avoid multiple sampling of the same neuron.
Discrimination indices of ACx neurons correlate with behavioral performance. a, Mean calcium response traces (shaded area, SEM) of five example neurons to Go (left column) and NoGo (right column) following odor 1 (blue) and odor 3 (red). Y-scale, 0.1 ΔF/F. Green rectangle indicates odor presentation (500 ms). Black/white rectangle indicates Go/NoGo presentation, respectively (100 ms). DI for expected and unexpected trials is indicated in turquoise and purple text, respectively, for each neuron. b, Top, Mean (±SEM) expected (turquoise) and unexpected (purple) DI of all responsive neurons throughout all behavioral sessions (signed-rank test: **p < 0.005, ***p < 0.0005; df = 218, 233, 209, 234, 175, 180, 127; es = 0.06, 0.1, 0.31, 0.26, 0.23, 0.02, 0.04). Bottom, Mean difference between expected and unexpected DIs (ΔDI) from the top. c, Mean ΔDI over all responsive neurons within a neuronal field of view per session as a function of d′ CI of that session, for all behavioral sessions (Pearson correlation: R = −0.43, p < 0.05). Each dot is a mouse–session pair; red line, linear fit. d, Left, DI of Go-preferring neurons for the three expectation conditions (n = 213) for all “effective odor bias sessions” together (shaded area in ‘b’). Right, Same for NoGo-preferring neurons (n = 153). For each box plot, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the “+” symbol (two-tailed signed-rank test with Bonferroni’s correction: ***p < 0.0005; df(Go) = 212; es(Go E,N) = 0.1, es(Go N,UE) = 0.25, es(Go E, UE) = 0.34; df(NoGo) = 212; es(NoGo E,N) = 0.1, es(NoGo N,UE) = 0.05, es(NoGo E, UE) = 0.14).
Cortical responses are modulated by odor–cue expectations. a, Top, Mean calcium response traces (shaded area, SEM) of Go-preferring neurons (n = 213) to the 6 odor–sound stimuli on Odor bias sessions 2–4. Y-scale, 0.05 ΔF/F. Green rectangle indicates odor presentation (500 ms). Black\white rectangle indicates Go/NoGo presentation, respectively (100 ms). Bottom, Individual mean responses of all neurons. Black line indicates the mean (two-tailed t test with Bonferroni’s correction: *p < 0.05, **p < 0.005, ***p < 0.0005; df = 212; es(Go 1,2) = 0.02, es(Go 2,3) = 0.14, es(Go E, 1,3) = 0.13, es(NoGo 1,2) = 0.11 es(NoGo 2,3) = 0.18, es(NoGo 1,3) = 0.27). b, Same as a for NoGo-preferring neurons (n = 153; df = 152; es(Go 1,2) = 0.06, es(Go 2,3) = 0.13, es(Go E, 1,3) = 0.22, es(NoGo 1,2) = 0.23 es(NoGo 2,3) = 0.01, es(NoGo 1,3) = 0.23). c, Same as a for the pre-odor bias session (n = 167; df = 166; es(Go 1,2) = 0.23, es(Go 2,3) = 0.18, es(Go E, 1,3) = 0.07, es(NoGo 1,2) = 0.05 es(NoGo 2,3) = 0.13, es(NoGo 1,3) = 0.09). d, Same as b for the pre-odor bias session (n = 52; df = 51; es(Go 1,2) = 0.02, es(Go 2,3) = 0.22, es(Go E, 1,3) = 0.2, es(NoGo 1,2) = 0.02 es(NoGo 2,3) = 0.03, es(NoGo 1,3) = 0.092).
To measure how well neuronal populations could discriminate between Go and NoGo, we calculated a receiver operating characteristic curve between the distributions of the responses of each neuron to these sounds and calculated its area under the curve (AUC). AUC values close to 0.5 indicate low discrimination, whereas values away from 0.5 indicate high discrimination. To compensate for neuronal preferences of either sound, we calculated the discrimination index (DI) as follows:
Discrimination indices were calculated separately for Go–NoGo pairs in the expected, neutral, and unexpected conditions.
Statistical analysis.
All analysis was performed in custom-written MATLAB code. We used either the statistical signed-rank test or Student’s t test as indicated in the text. When the comparison was hypothesis driven, we used a one-tailed test. Otherwise, we used a two-tailed test. A multiple-comparison Bonferroni correction was applied when necessary. Since all tests are paired, degrees of freedom were always calculated as n – 1. Effect size (es) was calculated as Cohen’s d [
Data availability.
All the data and codes used in this study are available from the corresponding author on request or can be downloaded from https://github.com/MizrahiTeam.
Results
A behavioral paradigm to induce odor-mediated expectation
To find out whether learned odor-mediated expectation can modulate sound perception, we started by designing an auditory discrimination task with preceding odors as cues. We trained head-fixed mice on an auditory Go–NoGo discrimination task with one of three possible odors preceding the sounds. Mice were rewarded for licking the spout in response to one sound frequency (Go) by a sweet drop of water and punished by a white noise sound for licking in response to a second sound frequency (NoGo). No-lick trials were neither punished nor rewarded [Fig. 1c,d; see also Materials and Methods (also see Fig. 1a,b, for the full training protocol)]. To attain mice that are engaged in a difficult task, we used sounds (pure tones) separated by a frequency difference just above the discrimination threshold of the mouse (Maor et al., 2020; see also Materials and Methods). As odors, we used monomolecular odorants with neutral valence (Root et al., 2014). Sound stimuli, whether Go or NoGo, were initially preceded by one of the odors with equal probabilities (Fig. 1e; p(sound|odor) = 1/2 for all six possible combinations). At this stage, which we called “pre-odor bias”, all odors were merely a cue that signals the initiation of a new trial. Thus, at this stage, odors did not convey any information about the identity of the upcoming trial in the task (Fig. 1e, pre-odor bias). In the pre-odor bias stage mice performed the sound discrimination based strictly on the sounds themselves, not the odors (Fig. 1g, pre-odor bias; d′sound_alone = 1.5 ± 0.1; d′odor_alone = 0 ± 0.1; mean ± SEM; n = 5 mice).
To manipulate expectations of trial types, we rearranged the probabilities of specific odor → sound pairs, a stage we called “odor bias”. Odor → sound pairs were changed as follows: (1) odor 1 (blue in all figures) was more likely to be followed by a Go sound (p(Go|Odor1) = 2/3, p(NoGo|Odor1) = 1/3); (2) odor 2 (gray in all figures) was followed by either Go or NoGo sounds with equal probabilities (p(Go|Odor2) = p(NoGo|Odor2) =1/2); and (3) odor 3 (red in all figures) was more likely to be followed by a NoGo sound (p(Go|Odor3) = 1/3, p(NoGo|Odor3) = 2/3; Fig. 1e, odor bias). We measured mouse performance for sound discrimination along six consecutive sessions when odors were now potentially informative about the identity of an upcoming trial. In the odor bias stage, two of the odor → sound pairs were expected, two pairs were unexpected, and two remained neutral (Fig. 1e). On average, mice performed 404 ± 18 trials per session (mean ± SEM), allowing us to measure behavioral responses to all six odor → sound combinations, which were presented in random order (Fig. 1f, a representative snapshot of 30 consecutive trials). Despite odors now being informative, mice still used the sound information significantly more than the odors for solving the task (Fig. 1g, odor bias sessions 1–6).
We next asked to what extent the manipulated probabilities of odor → sound pairs changed the perceived expectation of trial type? To answer this question, we analyzed expectation by assessing mouse performance in the different stages (i.e., pre-odor bias vs odor bias) and by comparing behavioral responses in expected versus unexpected trials. Notably, by subtracting responses of neutral odors, we balanced off any nonspecific behavioral effects.
First, we analyzed lick rates for Go and NoGo sounds (i.e., Hit rate and FA rate, respectively). Expectation did not affect Hit rates, which remained consistently high both in the pre-odor bias stage and throughout the odor bias sessions (Fig. 2a,b). In contrast, we found clear differences in FA rates contingent on the expectation of a Go trial. Specifically, FA rates increased for odor 1, while they concomitantly decreased for odor 3 (Fig. 2c,d). This change in FA rates was not evident in the first session after switching the odor → sound probabilities, but only after the second session and onward (Fig. 2d). This finding suggests that mice became sensitive to the change in odor → sound probabilities only after session 2 during the odor bias stage.
Second, we measured lick bias and sound discriminability (d′). Lick bias is a general measurement of the tendency to lick regardless of the sound (Nevin, 1969). Lick bias was higher for odor 1 and lower for odor 3 but only during odor bias sessions 2–4 (Fig. 2e,f). To test the role of expectation on behavioral performance, we compared d′ for expected, neutral, and unexpected Go–NoGo pairs (see Fig. 1e for precise probabilities). d′ increased when sounds were expected (Fig. 2g, green bars) and decreased when sounds were unexpected (Fig. 2g, purple bars). Here, too, the significant differences were evident during odor bias sessions 2–4 (Fig. 2h). These results show that our manipulation had a strong yet transient effect on behavior, appearing at session 2, maintained for at least three sessions (sessions 2–4), and waning down at sessions 5–6.
In summary, by manipulating the statistics of odor cues that precede specific sounds, we developed a behavioral paradigm that allows us to test the effects of learned olfactory cues in different contexts during an auditory discrimination task. The behavioral effects were clear, yet transient. The late decline of the effect may be because of the learning (by session 5) that odors are in fact unnecessary for solving this task. Since we cannot rule out that mice learned to ignore the odor probabilities by session 5, we focused our analysis only on sessions 2–4 [Fig. 2f,h, shaded gray (collectively referred to herein as “effective biasing sessions”)]. During the effective biasing sessions, the behavioral effects are monotonic with odor → sound probabilities for FA rate, lick bias, and d′ (Fig. 2i).
Cortical neurons show increased discrimination when sounds are unexpected
We used two-photon calcium imaging in the ACx of mice engaged in the above-mentioned task to measure how the regularities of odor → sound probabilities modulate neuronal activity in ACx. We injected mice, unilaterally, with AAV9-hSyn-GCaMP6s-P2A-nls-dTomato into the left ACx and prepared them for imaging (see Materials and Methods). We located the rough borders of the ACx using intrinsic signal imaging of the cortical sheet (Fig. 3a,b; see Materials and Methods) and then zoomed-in to image single-neuron responses (Fig. 3c). We imaged ACx at depths corresponding to L2/3 (range, 150–350 μm from the pial surface), and successfully imaged four of the five mice that were described above. We imaged calcium responses of single neurons during behavior both in the pre-odor bias stage as well as throughout all sessions of the odor bias stage (Movie 1, representative example of raw data in a behaving mouse).
Two-photon calcium imaging during behavior. Left, Raw imaging data from the mouse shown on the right. Red, dTomato; green, GCaMP6s. Right, Movie of a head-fixed mouse engaged in the behavioral task and imaged; top right, trial type; bottom left, information about odors, sounds rewards, and punishments.
Each behavioral session lasted 45–75 min, during which we imaged neuronal responses to all six combinations of odor → sound stimuli. As expected from ACx (Rothschild et al., 2010; Feigin et al., 2021), neuronal responses to sounds were highly heterogeneous (Figs. 3d–f, 4a). Some neurons responded only to the Go sound (Fig. 4a, neuron 4), NoGo sound (Fig. 4a, neuron 3), or both sounds to different degrees (Fig. 4a, neurons 1, 2, and 5). Moreover, while some neurons responded stably to the same stimulus in every session, others were unstable and responsive to each stimulus only during some sessions. Notably, heterogenous response patterns across days are consistent with what had been referred to as representational drift (Rule et al., 2019) and has also been shown to be evident in ACx (Aschauer et al., 2022; Suri and Rothschild, 2022). We therefore analyzed only responsive neurons and only in those sessions they were responsive (i.e. neuron-session pairs). In total, we analyzed a dataset composed of 301 neurons in 1383 neuron–session pairs. Since miss trials were rare, and FA responses could have included a response to the white noise punishment, our primary analysis was focused on neuronal responses only between similar correct trials (i.e., Hit vs Hit and CR vs CR). This ensured that we analyzed how odor-mediated changes affect cortical responses when actions are similar. Central to our hypothesis about the role of odors as contextual cues, we found neurons that responded to a given sound distinctly based on the preceding odor stimulus (Figs. 3d–f, 4a). This shows that learned odor → sound associations modulated the representation of the same sound when motor responses and behavioral outcomes were identical.
To evaluate how the different odor → sound probabilities affected the way neurons discriminated between the sounds during different choices, we calculated a DI between Go and NoGo, in expected/unexpected conditions (see Materials and Methods). Some neurons were more discriminative in expected trials (Fig. 4a: odor 1 → Go and odor 3 → NoGo, in neurons 4 and 5, respectively), while others in unexpected trials (Fig. 4a: odor 3 → Go and odor 1 → NoGo, neurons 1 and 3, respectively). The mean DI of all single neurons was significantly higher in the unexpected condition, only during the effective biasing sessions. Strikingly, the sessions in which odor-mediated expectations had pronounced physiological effects correspond exactly to the sessions of the behavioral effects (Fig. 4b, shaded area). Notably, the changes in neuronal DI during the effective odor bias sessions were opposite in direction to the behavioral ones (compare Figs. 4b, 2h), which is partially explained by the fact that only correct trials were considered in the physiological analysis (see more on this issue in the Discussion). To test for a correlation between the behavioral effects and the neuronal effects, in each mouse we plotted the mean DI difference (ΔDI) per session versus the difference in d′ (d′ CI; see Materials and Methods) between expected and unexpected trials. Plotting this relationship across all behavioral sessions individually revealed a negative correlation (Fig. 4c). This result suggests that ACx is involved in discriminating sounds during behavior, and particularly so when they are unexpected.
We then tested whether the difference in DI is a feature of neurons with a specific response profile. We thus analyzed DI in single neurons based on their sound–frequency preference—either as Go-preferring or NoGo-preferring neurons. The mean DI of all neurons in the effective biasing sessions (such that in each population, each neuron was sampled only once; see Materials and Methods) revealed a statistically significant effect in DI as a feature of Go-preferring neurons, but not NoGo-preferring neurons (Fig. 4d). Furthermore, by adding the neutral expectation condition to the analysis, we found that only the unexpected condition in the Go-preferring neurons contributes to the effect (Fig. 4d, left graph). Note that in the behavior we found a difference across all conditions (Fig. 2i), which argues that the neuronal responses we measured from ACx cannot explain the full behavioral manifestation induced by the odor → sound associations.
Since neuronal responses were affected by the preceding odors in various ways (Figs. 3d–f, 4a), we next sought to account for the differences in DI during the effective biasing sessions, between the expected and unexpected conditions. We analyzed responses of Go-preferring and NoGo-preferring neurons separately. Interestingly, the average population response amplitudes differed following the different odors, differentially between neuronal populations. Specifically, Go-preferring neurons responded weaker to NoGo when it was unexpected [Fig. 5a (also see Fig. 4a, neuron 2)]. This result readily explains the increased DI in Go neurons. NoGo-preferring neurons, however, responded stronger to both stimuli when they were unexpected (Fig. 5b, bottom right). No such changes were evident during the pre-odor bias session (Fig. 5c,d). These results suggest that odor 1, which is associated with a Go trial (either sound and/or its reward), has, on average, increased the selectivity of both neuronal populations to their preferred stimulus.
Discussion
It is well established that the so called “unisensory” cortices are strongly modulated by stimuli from other sensory modalities (Schroeder and Foxe, 2005; Ghazanfar and Schroeder, 2006; Stein and Stanford, 2008). This is true for all sensory cortices and includes their primary subregions (Morgan et al., 2008; Maier et al., 2015; Murray et al., 2016; Clemens et al., 2018). In ACx, several physiological studies have revealed that neurons integrate auditory–visual or auditory–somatosensory cues, which are conveyed through direct anatomic and functional connections among the regions (Murray et al., 2005; Bizley et al., 2007, 2016; Kayser et al., 2007, 2009; Lakatos et al., 2007). Those forms of multisensory integration have been suggested to complement auditory processing and modulate the way an animal perceives its natural acoustic environment (Ghazanfar and Schroeder, 2006; Stein and Stanford, 2008). For example, in humans, who are highly visually guided, audiovisual integration has been linked to improved speech perception, localization accuracy, and reaction times to auditory cues (Schröger and Widmann, 1998; Sekiyama et al., 2003; Besle et al., 2008; Schroeder et al., 2008). ACx in humans and other animals has also been shown to be modulated by other senses like touch (Kayser et al., 2005; Schürmann et al., 2006). In mouse ACx, multisensory interactions between sounds and touch have also been suggested to play a role in biologically meaningful ways, like during social interactions (Rao et al., 2014). Here, we asked to what extent is ACx affected by contextual cues from other senses that seem intuitively distant, like olfaction?
Olfactory cues form cross-modal associations with all other senses (Deroy et al., 2013). However, most examples of such associations are related to natural contingencies, like between odors and flavors. Multisensory interactions between odors and flavors are intuitively explained by the mere statistical regularities of the environment (e.g., caramel odor is always congruent with its sweet taste). But some cross modal associations with odors are surprising, like those between odors and touch or odors and colors in humans (Demattè et al., 2006a, b). Such surprising congruencies come from human psychology and are often explained as anecdotal associations arising from metaphorical/synesthetic transfers among the senses. Yet, real life experience and learning offer rich substrates for forming associations between any stimuli that co-occur in the environment in a meaningful manner.
One example of a natural form of odor–sound contingency is one that develops during parenthood. By measuring single-neuron responses to sounds in ACx of mouse dams, we previously showed that exposure to the body odor of pups has a modulatory effect on sound-evoked responses (Cohen et al., 2011; Cohen and Mizrahi, 2015). Although the neural circuit underlying those effects remains unknown, they likely involve innate circuits (e.g., the medial preoptic area and amygdala) that receive strong inputs from pup odors and directly shape maternal behavior (Dulac et al., 2014). Notably, maternal plasticity can be argued as a different case than the one we studied here, as it involves innate circuits and behaviors, while our task is based purely on learned association. By artificially creating a learned association between specific odors and specific sounds with no innate preference, we ensured that contextual odor information is first learned during the task (Fig. 1). Given this experimental design and choice of odors and sounds, we hypothesized that any contingencies formed will likely arise from cortical association areas and engage in cognitive processes like attention and expectation (Talsma et al., 2010; Rohe and Noppeney, 2016).
Our behavioral design required fine tuning such that an increased task difficulty will be easier to bias by the predictive cues. Since mice rely heavily on olfaction (Howard et al., 1968; Rokni et al., 2014), we trained mice to their perceptual limit on a strictly auditory task and only then started introducing odors (Fig. 1a). In addition, we made sure that odors will be sufficiently less informative than sounds (i.e., while the Go sound predicts a reward with 100% certainty, odor 1 predicts it with 66% certainty). Fine-tuning these measures ensured that mice relied more on sounds compared to odors (Fig. 1g). The fact that odors were not necessary for solving the task could explain, at least in part, the transient nature of the behavioral effect that we observed (Fig. 2f,h). Nonetheless, odor expectation cues had a clear behavioral effect in a time window of several days, which was also correlated with neuronal changes (Fig. 4b,c). We interpret this correlation as evidence that odor–cue contingency changed the learned auditory behavior and that neural changes in ACx might be informative for the task. But how?
A somewhat counterintuitive finding is that neurons showed increased discrimination during unexpected trials, but mice showed decreased performance in those trials (Figs. 2i, 4). One possibility is that the choice of the mouse is affected by the activity of other brain regions in addition to ACx that weigh the odor cues more heavily and that choices were incorrect in those trials despite ACx being more discriminative. However, since we analyzed only correct trials, another possible explanation is that information in ACx is used for responding correctly, although the trial was unexpected. Specifically, increased discrimination was observed for Go-preferring neurons, which decreased their response to the NoGo sound when it was unexpected. Thus, this attenuated neural response on these (correct) trials, might have contributed to the accurate behavioral choice.
One of our main physiological findings is an increased response of NoGo-preferring neurons to unexpected trials (Fig. 5b). This type of a response is reminiscent of the response profile of ACx neurons to the oddball paradigm (Ulanovsky et al., 2003). In the classical auditory oddball paradigm, two sounds are repeated in sequence such that one of them (the standard) appears with higher probability than the other (the rare). ACx neurons tend to respond more strongly to the same sound when it is rare (and therefore less expected) than when it is standard. This is similar to NoGo neurons in this work responding more strongly to the same sound when it is unexpected. Notably, however, we think that these two phenomena likely do not have a common mechanism, since the characteristic response of ACx neurons to the oddball paradigm is thought to be a result of a local feedforward computation (Mill et al., 2011, 2012; Taaseh et al., 2011). The responses described here, however, require that information about odor-mediated expectations will arise from regions outside the ACx.
Interestingly, heightened responses to rare sounds in the oddball paradigm are termed “prediction error signals” (Rubin et al., 2016)—a term commonly associated with the theory of predictive processing (Rao and Ballard, 1999; Friston, 2005; Keller and Mrsic-Flogel, 2018). Indeed, according to this theory, predictions of upcoming stimuli arrive at sensory cortices from high-order (top-down) cortical regions and compared with bottom-up input. When these do not match, neurons respond with a prediction error signal. Our finding of NoGo-preferring neurons responding more strongly to unexpected stimuli fits well with the theory of predictive processing, but the fact that the Go-preferring neurons did not show such a response pattern argues that computations in ACx are more diverse than simply prediction errors.
We speculate that the information of odor–cue expectation arises from brain regions that integrate information from both the auditory and olfactory modalities. One such candidate is the orbitofrontal cortex, which has been shown to respond to odors and sounds, as well as to innervate ACx (Rolls, 2004; Winkowski et al., 2018). In addition, the orbitofrontal cortex is generally thought to be involved in the assignment of values to sensory stimuli during associative learning (Padoa-Schioppa and Assad, 2006; Schoenbaum et al., 2009). Whether orbitofrontal cortex, or any brain region, is indeed involved warrants future investigation.
We cannot rule out that the effects we measured here are not purely sensory. Indeed, odor-induced expectations are not necessarily limited to the Go/NoGo sounds but likely contain information more generally—pertinent to the meaning of Go/NoGo as a whole. First, odor–cue expectations might carry information about the reward component as well (Schultz et al., 1998). Second, it is well established that cortical activity in sensory systems, including primary sensory cortices, includes motor components (Musall et al., 2019; Steinmetz et al., 2019). However, since odors modulated the responses to the NoGo sound, for which the mouse did not proactively respond by a motor action, the effect on sound processing remains probable, though not exclusive. Teasing out the individual components of the effects warrants additional experiments that will better isolate each component individually.
Footnotes
This work was supported by ERC (European Research Council) Consolidator Grant 616063 (to A.M.), Israeli Science Foundation Grant 2453/18 (to A.M.), and the Gatsby Charitable Foundation. Some elements in Figures 1 and 3 were created with graphical features from BioRender.com. This work is dedicated to the memory of Mrs. Lily Safra, a great supporter of brain research. We thank Eran Lottem, Leon Deouell, Ido Maor, and members of the Mizrahi laboratory for comments on the manuscript. We also thank Yishai Elyada for technical help in setting up a first version of the microscope and for providing other technical help. In addition, we thank Maya Sherman and Yishai Elyada for virus preparation and calibration.
The authors declare no competing financial interests.
- Correspondence should be addressed to Adi Mizrahi at Mizrahi.adi{at}mail.huji.ac.il
