Abstract
The posterior parietal cortex (PPC) plays a key role in integrating sensory inputs from different modalities to support adaptive behavior. Neuronal activity in PPC reflects perceptual decision-making across behavioral tasks, but the mechanistic involvement of PPC is unclear. In an audiovisual change detection task, we tested the hypothesis that PPC is required to arbitrate between the noisy inputs from the two different modalities and help decide in which modality a sensory change occurred. In trained male mice, we found extensive single-neuron and population-level encoding of task-relevant visual and auditory stimuli, trial history, as well as upcoming behavioral responses. However, despite these rich neural correlates, which would theoretically be sufficient to solve the task, optogenetic inactivation of PPC did not affect visual or auditory performance. Thus, despite neural correlates faithfully tracking sensory variables and predicting behavioral responses, PPC was not relevant for audiovisual change detection. This functional dissociation questions the role of sensory- and task-related activity in parietal associative circuits during audiovisual change detection. Furthermore, our results highlight the necessity to dissociate functional correlates from mechanistic involvement when exploring the neural basis of perception and behavior.
SIGNIFICANCE STATEMENT The posterior parietal cortex (PPC) is active during many daily tasks, but capturing its function has remained challenging. Specifically, it is proposed to function as an integration hub for multisensory inputs. Here, we tested the hypothesis that, rather than classical cue integration, mouse PPC is involved in the segregation and discrimination of sensory modalities. Surprisingly, although neural activity tracked current and past sensory stimuli and reflected the ongoing decision-making process, optogenetic inactivation did not affect task performance. Thus, we show an apparent redundancy of sensory and task-related activity in mouse PPC. These results narrow down the function of parietal circuits, as well as direct the search for those neural dynamics that causally drive perceptual decision-making.
Introduction
The construction of context-dependent representations of sensory inputs is required to inform adaptive decision-making. This process involves multiple functions, such as processing of stimuli coming from different sensory modalities, evidence accumulation and integration with past information, and finally transformation of sensory information into an appropriate motor plan. The parietal cortex has been identified as a key hub for these functions through classical lesion studies in humans (Holmes, 1918; Bender and Teuber, 1947; Denny-Brown et al., 1952), including seminal studies on hemineglect (Vallar, 1998; Kerkhoff, 2001), and through extensive work in primates (Robinson and Goldberg, 1978; Platt and Glimcher, 1999; Freedman and Assad, 2006; Cui and Andersen, 2007; Andersen and Cui, 2009; Bisley and Goldberg, 2010). In rodents, a key animal model to study the circuit-level mechanisms of cognitive processes, the posterior parietal cortex (PPC) is located at the nexus of visual, auditory, and somatosensory cortices (Wilber et al., 2015; Hovde et al., 2019) and is bidirectionally connected to sensory and motor areas, as well as to other associative cortical areas (Oh et al., 2014; Shadi et al., 2020), homologous to primate PPC (Whitlock et al., 2008; Wilber et al., 2015). Neurons in rodent PPC respond to visual, auditory, and somatosensory inputs (Wallace et al., 2004; Olcese et al., 2013; Raposo et al., 2014; Mohan et al., 2018; Nikbakht et al., 2018). Sensory-evoked PPC responses have been described in naive animals (Olcese et al., 2013), but PPC is mostly recruited during task engagement (Pho et al., 2018). Neuronal activity in PPC has been shown to reflect aspects of perceptual decision-making, such as an animal's decision (Funamizu et al., 2016; Goard et al., 2016; Driscoll et al., 2017; Runyan et al., 2017; Song et al., 2017; Krumin et al., 2018; Pho et al., 2018), accumulated evidence (Hanks et al., 2015), trial history (Hwang et al., 2017; Akrami et al., 2018), and working memory (Harvey et al., 2012). This combination of sensory-, task-, and choice-related activity suggests an important role in perceptual decision-making, but under which conditions PPC is causally involved is less clear. Several studies find PPC necessary for making decisions based on visual stimuli (Harvey et al., 2012; Goard et al., 2016; Driscoll et al., 2017; Licata et al., 2017), but not for tactile (Guo et al., 2014) or auditory information (Erlich et al., 2015; Licata et al., 2017). Other studies have suggested that PPC is causally involved in both visual and auditory tasks, but only when additional cognitive processes beyond simple sensorimotor associations are required (Harvey et al., 2012; Funamizu et al., 2016; Hwang et al., 2017; Licata et al., 2017; Song et al., 2017; Akrami et al., 2018; Zhong et al., 2019).
As PPC receives converging inputs from auditory and visual cortices and is strongly connected to (pre)motor areas, we tested the hypothesis that PPC is required to solve tasks in which signals from two different modalities have to be compared to reach a behavioral decision. We designed an audiovisual change detection task for which PPC is conjectured to arbitrate between noisy inputs from different sensory channels and help decide in which modality a sensory change occurred. In line with our hypothesis, we found that PPC displayed rich visual, auditory, and decision-related activity that reflected sensory processing and predicted task performance. However, optogenetic inactivation of PPC did not affect either visual or auditory change detection. These results fundamentally question which function sensory- and task-related activity in PPC may fulfill, and to what extent the presence of rich sensory and task correlates reflects the role in perception and behavior of a cortical area.
Materials and Methods
Data and code availability
The data and code that support the findings of this study are available from authors M.N.O.L., C.M.A.P. and U.O., on reasonable request.
Animals
All animal experiments were approved by the Dutch Commission for Animal Experiments and by the Animal Welfare Body of the University of Amsterdam. A total of 24 male mice was used from two transgenic mouse lines: PVcre (JAX 008069) and F1 offspring of Pvcre (JAX 008069) and Ai9-TdTomato cre-reporter mice (JAX 007909). Mice were at least 8 weeks of age at the start of experiments and group-housed under a reversed day-night schedule (lights were switched off at 8:00 and back on at 20:00). All experimental procedures were performed during the dark period.
Head bar implantation
Before the start of any experiment, mice were implanted with a custom-made titanium head bar to allow head fixation. Mice were anesthetized with isoflurane and fixed in a stereotaxic apparatus. A circular head bar was positioned to include V1 and PPC bilaterally and glued and cemented to the exposed skull. Areas of interest were located based on stereotaxic coordinates (V1 relative to lambda: AP 0.0, ML ± 3.0, PPC relative to bregma: AP 1.9, ML ± 1.6) (Goard et al., 2016; Song et al., 2017; Le Merre et al., 2018). Mice were allowed to recover for 2-7 d after implantation and were then habituated to handling and head-fixation before the start of the training procedure.
Audiovisual change detection task
Throughout experiments, mice were water-deprived and earned their daily ration of liquid by performing the behavioral task. Mice were head-fixed, and two lick spouts were positioned symmetrically on the left and right side within reach of their tongue. Licks were detected by capacitance-based (training setups) or piezo-electric-based detectors (recording setup). Upon correct licking, 5-8 µl of liquid reward (Infant formula, Nutrilon) was delivered through the lick spout using gravitational force and solenoid pinch valves (Biochem Fluidics).
Stimuli were continuously presented throughout behavioral sessions. Visual stimuli were drifting square-wave gratings with a temporal frequency of 1.5 Hz and spatial frequency of 0.08 cpd at 70% contrast. Stimuli were presented with a 60 Hz refresh rate on an 18.5 inch monitor positioned at a straight angle with the body axis from the mouse at 21 cm from the eyes. In trials with a visual change, the orientation of the drifting grating was instantaneously changed (e.g., from 150° to 180°) while preserving the phase. The auditory stimulus was a stationary Shepard tone (Shepard, 1964) composed of a center tone and multiple harmonics (2 lower and 2 higher harmonics). The center tones ranged a full octave spanning from 213 Hz (8372 Hz) to 214 Hz (16,744 Hz). For each given Shepard tone in this stimulus set, the weight of the center and harmonic tones are taken from a fixed Gaussian weight distribution over all center and harmonic tones, in this case centered at 213.5 (11,585 Hz). Stimuli were presented with a sampling rate of 192 kHz. Stimuli were high-pass filtered (Beyma F100, Crossover Frequency 5-7 kHz) and delivered through two bullet tweeters (300 W) directly below the screen. Sound pressure level was calibrated at the position of the mouse, and volume was adjusted per mouse to the minimum volume that maximized performance (average ±70 dB). In trials with an auditory change, the stimulus was modified instantaneously from one Shepard tone to another with a different center frequency and associated harmonics, while preserving the phase across all compound tones. For example, an auditory change of ¼ octave would jump from 213 to 213.25. The amount of change determined stimulus difficulty (see also below).
Animals were trained to respond in a lateralized manner to sensory changes in each modality: lick to one side to report visual changes, to the other side for auditory changes (modality-side pairing was counterbalanced across mice). In other words, mice were required to simultaneously monitor both the auditory and visual modality and identify the sensory modality in which a change occurred.
Trials were separated by a random intertrial interval (taken from an exponential distribution with a mean of 6 s, minimum 3 s, and maximum 20 s). Trial types were pseudorandomly ordered by block-shuffling: every block of 10 trials contained a mixture of trial types in a fixed proportion but random order (8% catch trials = no change, 46% visual trials, 46% auditory trials). For instance, each block contained on average 46% of visual trials randomly interspersed among the other trial types. Directly after stimulus change, a response window of 1500 ms followed in which mice could obtain a reward by licking the correct side (no reward available in catch trials). The first lick on the correct side during the response window was immediately rewarded (median reaction time was 324 ms across all auditory hits and 407 ms across all visual hits).
It is not likely that mice solved the task using short-term memory or prolonged evidence accumulation, as mice could respond immediately and the observed reaction times are shorter than the time windows usually considered in behavioral tasks explicitly including evidence accumulation or working memory (∼1000 ms) (Brunton et al., 2013; Akrami et al., 2018; Odoemene et al., 2018). However, it can still be argued that this task could be solved, for instance, by keeping the previous orientation in memory and comparing it with the previous one.
For each trained animal, we presented five levels of auditory and visual amount of change that spanned the perceptual range. We fitted this psychophysical data (see below) to establish the perceptual threshold for the visual and auditory domains for each animal. To sample sufficient trials per condition in recording sessions, we used two levels of change (threshold and maximal). For noncontingently exposed (NE) mice, we used threshold values that matched those from trained animals.
As visual and auditory feature changes were associated with different motor actions (in trained but not in NE animals), a simultaneous auditory and visual change would present the animal with conflicting signals (the visual change predicts reward for licking left, auditory change predicts reward for licking right). In a subset of sessions, we introduced these conflict trials (25% of the trials, replacing unimodal trials) and registered the choice (side of the first lick). Both sides were rewarded in conflicting trials.
We compared multisensory trained animals (MST, n = 17) with NE animals (n = 7). For NE animals, the sensory environment was identical; both the auditory and visual stimuli were continuously presented with the same distribution of trial types and temporal statistics as for trained animals. However, these animals received rewards if they licked during a hidden response window that was temporally decorrelated from the stimuli. Spontaneous licks were therefore occasionally rewarded, and this allowed us to compare intermittent licks, rewards, and stimuli between trained and NE mice. Each session was terminated after 20 trials of unresponsiveness, and these last 20 trials were always discarded from all analyses.
We further excluded a subset of sessions from trained animals with poor behavioral performance because of (1) a high false alarm rate to auditory or visual spout (>50%, one session excluded), (2) a high lapse rate on easiest auditory or visual trials (a threshold of >30% hit rate on both visual and audio trials with a maximal amount of change had to be met, two sessions excluded for low auditory performance, nine sessions for low visual performance). In this two-alternative unforced choice task, performance at chance level is not 50% with three response options (lick to visual spout, auditory spout, no lick) and depends on spontaneous lick rates and trial type distribution.
Viral injection
Once animals were trained to asymptotic performance, we aimed at optogenetically inactivating PPC by locally expressing Channelrhodopsin-2 in a cre-dependent manner in PVcre mice, therefore driving inhibitory interneurons. Mice were subcutaneously injected with the analgesic buprenorphine (0.025 mg/kg) and maintained under isoflurane anesthesia (induction at 3%, maintenance at 1.5%-2%) throughout the surgery. Small craniotomies (∼100 µm) were made over the area of interest (V1 or PPC) using a dental drill. A glass micropipette backfilled with AAV2.1-EF1a-double floxed-hChR2(H134R)-EYFP-WPRE-HGHpA (titer: 7 × 1012 vg/ml, 20 298-AAV1 Addgene) was slowly lowered in the cortex, and 25 nl was injected at 700 µm and 400 µm below the dura each using a Nanoject pressure injection system (Drummond Scientific). Additional experiments continued after 3 weeks to allow for viral expression. A potential concern is that expression of ChR2 in deep parvalbumin-expressing (PV) pyramidal neurons (Tanahira et al., 2009) led to an increase rather than a decrease of activity in deep layers. We observed no such expression in deep pyramidal neurons of PPC (see Fig. 6B).
Neuronal recordings in PPC
We performed craniotomies on the day before starting extracellular recording sessions. Mice were anesthetized with isoflurane and small (∼300-500 µm) craniotomies over the areas of interest were made using a dental drill leaving the dura intact. Craniotomies were made in the left hemisphere over V1, PPC, as well as primary auditory cortex, and medial PFC. Only data from V1 and PPC were analyzed for this study. The data presented from these areas were partly recorded in the same animals. Data regarding optogenetic manipulations of V1 and PPC activity were gathered in different animals.
Extracellular recordings were performed on consecutive days with a maximum of 4 d to minimize damage to the cortical tissue. Microelectrode arrays of 32 or 64 channels (NeuroNexus, A1x32-Poly2-10 mm-50s-177, A1x64-Poly2-6 mm-23s-160) were slowly inserted perpendicularly to the cortical surface until all recording sites were in contact with the tissue. To allow for tissue stabilization, the start of the behavioral task commenced at least 15 min after array insertion. For the recording session on the final day before perfusion, the array was covered in DiI (Thermo Fisher Scientific) to facilitate post hoc visualization of the electrode tract. Neurophysiological signals were pre-amplified, bandpass filtered (0.1 Hz to 9 kHz), and acquired continuously at 32 kHz with a Digital Lynx 128 channel system (Neuralynx).
Optogenetics
In sessions with optogenetic interventions, a random subset of trials (50% of trials) was associated with photostimulation. Photostimulation started at stimulus onset and continued until the animal made a choice. Two fiber-optic cannulas (ID 200 µm, NA 0.48, Doric lenses) were connected to a 473 nm laser (Eksma Optics, DPSS 473 nm H300) and positioned within 1 mm directly over the thinned skull at the area of interest. We performed extracellular recordings simultaneous with photostimulation in all mice to verify the effectiveness of inactivation, and we adjusted the laser power for each animal to the minimum power that maximally inhibited neural activity. The horizontal offset between the fiber tip and insertion site of the microelectrode array was minimized within the limited space constraints and measured ∼200-400 µm (for comparison: PPC measures ∼1.0-1.5 by 1.0-1.5 mm, depending on delineation) (Lyamzin and Benucci, 2019). The range of laser powers used was the same for PPC animals compared with V1 animals (2-15 mW at the tip of each fiber-optic cannula, corresponding to 15.9-119.3 mW mm−2). This laser power in combination with our optogenetic approach was previously shown to lead to effective spatial inhibition across our infected target area (Li et al., 2019). To allow rapid control over light delivery, laser beam continuity was controlled by a shutter (Vincent Associates LS6 Uniblitz). We stimulated with 10 ms pulses at 20 Hz (40 ms off, 20% duty cycle). For the photoinactivation of V1, a stimulation scheme was used in which the pulse and interpulse duration was variable with an average of 20 Hz and 75% duty cycle. The higher duty cycle of V1 inhibition versus PPC inhibition (75% vs 20%) is unlikely to explain the difference in effect on task performance for two reasons. First, when we investigated spiking activity relative to single laser pulses, we observed no rebound activity during the interpulse interval (see Fig. 6D). Second, the same stimulation protocol has been used in the same laboratory to effectively silence higher visual areas to study the impact on V1 (Oude Lohuis et al., 2021). To prevent light from reaching the eye of the mouse, the fiber-optic cannulas were sealed with black tape, leaving only the tip exposed. Furthermore, animals performed the task in an environment with ambient blue pulsating light.
Pupil monitoring
The left eye was illuminated with an off-axis infrared light source (IR-LEDs, 850 nm) positioned to yield high contrast illumination of both the eye and whisker pad. A near-infrared monochrome camera (CV-A50 IR, JAI) coupled with a zoom lens (Navitar 50 mm F/2.8 2/3 inch 10MP) was positioned at ∼30 cm from the mouse to capture a view of the lick spouts and face of the mouse. A frame-grabber acquired images of 752 × 582 pixels at 25 frames per second. The pupil size and position were extracted from the obtained videos by labeling the pupil center and 6 radially symmetric points on the edge of the pupil using DeepLabCut (Mathis et al., 2018), and pupil size was quantified as the surface area of an ellipse fit to these points.
Histology
At the end of the experiment, mice were overdosed with pentobarbital and perfused (4% PFA in PBS). The brains were recovered for histology to verify viral expression and recording sites. We cut coronal and flattened cortical sections as described previously (Lauer et al., 2018). For coronal sections, area borders were drawn by aligning and overlaying the reference section from the atlas (Paxinos and Franklin, 2004). For flattened cortical sections, areas were identified based on cell densities aligned to reference maps (Gămănuţ et al., 2018).
Data analysis
Unless otherwise stated, all data were analyzed using custom-made software written in MATLAB (The MathWorks).
Behavior: analysis of performance
Behavioral hit rates were fit with a multialternative signal detection model (Sridharan et al., 2014). This model extends signal detection theory (Green and Swets, 1966) for multiple signals and has been designed to accurately and parsimoniously account for observer behavior in a detection task with multiple signals. In our behavioral task, these are the auditory and visual signals to be reported at different lick spouts. In this model, the decision is based on a bivariate decision variable whose components encode sensory evidence in each modality, and decision space is partitioned in three regions (miss: neither evidence is strong enough, auditory response, and visual response). In a given trial, the observer either chooses to report nothing (no licking) or report visual or auditory stimuli (by licking left or right) if the decision variable exceeds a particular cutoff value, the “criterion” for each signal (the animal's internal signal threshold for responding, in terms of signal detection framework). We fit two versions of this model. In sessions with two levels of change per modality (threshold and maximum), we fit the d′ and criterion (c) to the behavioral response rates for each stimulus difficulty separately. This consists of fitting four free parameters: the d′ parameters (d′vis, d′aud) and criterion parameters (cvis, caud). In sessions with four levels of change per modality, we fit the behavioral response rates by fitting a fixed criterion and a d′ at each stimulus difficulty that was described by psychophysical function (three-parameter hyperbolic function). The d′ at each stimulus difficulty follows from:
For a detailed description of how the d′ and criterion subsequently relate to response rates, we refer the reader to Sridharan et al. (2014). This analysis was used before the start of experimental sessions to establish for each animal the perceptual threshold and therefore auditory and visual stimulus difficulty to use, as well as quantify behavioral performance on control and optogenetic trials.
Behavior: regression model of choice
To test whether PPC inactivation affected other factors beyond performance averaged over trial types, we constructed a regression model of behavioral choice (Busse et al., 2011; Hwang et al., 2017; Akrami et al., 2018). We selected a multinomial logistic regression model, as in our behavioral task the animal was presented with three discrete choice options (lick to visual spout, lick to auditory spout, or no-go). In this model, each regressor has two weights, where positive weights increase the probability of responding relative to a third reference option. We chose no-go (not licking) as the reference option, such that the linear sum of weights across regressors determine the probabilities of the choice on a given trial through the following:
Where b is a bias parameter, β are sets of regression coefficients (or weights), and X is the matrix with the values of regressor variables for every trial. As regressors, we used information from the current and the previous trial. Only information from up to one trial in the past was used, as this captured the strongest effects on behavioral choice and allowed us to test the effects of PPC inactivation. First, to construct a null model, we included a random predictor with random values between 0 and 1 (abbreviated to N in Fig. 9). This null model can already predict behavioral choice above chance by matching choice fractions. Second, we included the within-session trial number to account for nonstationarity in behavioral choice because of satiation (T). Third, for sensory information, we used the amount of visual and auditory change (degree of grating orientation and pitch change, respectively) as a scalar variable (both log-transformed to account for logarithmic sensitivity in sensory systems) (Sv, SA), and sensory stimuli on the previous trial (SV-1, SA-1). Fourth, reward on the previous trial was captured in two binary regressors per modality (RV-1, RA-1; 0 = not-rewarded, 1 = rewarded). Fourth, we included choice history (C-1). Because this was a detection task with sensory stimuli at or near the perceptual threshold, licks outside “trials” to the auditory or visual lick spout could be interpreted as reports of perception. Therefore, we included for choice history a binary predictor that reflected the side of the last lick (0 = last auditory lick; 1 = last visual lick). Last, to capture the effect of PPC photoinactivation on choice, a binary predictor of photostimulation was included (O).
All sessions of single animals were concatenated as if it were one session (N = 7.2 sessions on average). The model was fit on this concatenated trial data per animal (N = 17 animals, 58.652 trials in total) using the glmnet package in MATLAB (Friedman et al., 2010) with elastic-net regularization (α = 0.95) and threefold cross-validation. Regularization parameter λ was maximized while not decreasing cross-validated performance. Model fit quality was assessed as the fraction of held-out test trials in which the estimated choice (choice with the highest probability) matched the actual choice. We term this cross-validated model performance, and it is shown in the figure in partial models. We used a subset of variables and performed the same regression procedure: for instance, a model with and without including photostimulation as a predictor to assess the effect of PPC inactivation on model performance. The analysis of the effects of photoinactivation was done on the 5 animals with PPC inactivation.
Neural data processing
Before spike sorting, the median of the raw trace of nearby channels (within 400 µm) was subtracted to remove common noise artifacts. Automated spike sorting and manual curation were done using Klusta and the Phy GUI, respectively (Rossant et al., 2016). During manual curation, each putative single unit was inspected based on its waveform, autocorrelation function, and its firing pattern across channels and time. High-quality single units were included as having (1) an isolation distance >10 (Schmitzer-Torbert et al., 2005); (2) <0.1% of their spikes within the refractory period of 1.5 ms (Vinck et al., 2016; Bos et al., 2017); and (3) stable presence throughout the session. This latter was quantified by binning the firing across the entire session (∼50 min) in 100 time bins and only including neurons that spiked in >90 time bins. We recorded a total of 671 neurons from 14 animals over 32 sessions that met our criteria.
For Figure 1D, F, spike times were binned in 1 ms bins, convolved with a Gaussian window with 50 ms SD, and z-scored by subtracting the mean baseline activity and dividing by the SD of all baseline periods (−1 to −0.2 s before stimulus). For the single neuron encoding model, spikes were binned in 25 ms bins and convolved with a Gaussian window with a 50 ms SD.
Single-neuron encoding model
We constructed an encoding model that allowed us to model, for single neurons, the time-dependent effects of all measured variables related to the task and the animal's behavior simultaneously on single-trial neuronal activity. This approach is particularly useful to disentangle the different events that contribute to heterogeneous responses in associative regions, such as parietal cortex (Park et al., 2014).
We included six categories of predictors: visual stimuli, auditory stimuli, reward, licking movement, pupil size, and trial history. Binary variables (stimulus present or not, last trial rewarded or not; all variables except for pupil size were binary) were modeled with a series of temporal basis functions (raised cosines) that spanned the relevant epoch of influence to fit time-dependent modulation of neuronal responses by these predictors.
For the auditory and visual predictors, we used two kernels with 100 ms SD that spanned the first 200 ms after stimulus to capture the early spiking activity and 10 kernels with 200 ms SD that spanned from 0 to 2000 ms after stimulus to capture the late, sustained response. A separate predictor set was used per combination of orientation/frequency × amount of change. For reward variables, we used 10 kernels with 200 ms SD that spanned from 0 to 2000 ms relative to stimulus change in hit trials (visual hit, audio hit) and 10 predictors that spanned −500 ms to 1500 ms relative to reward. We used reward as a single term to refer to hit-trial specific activity (rewarded visual and auditory hits). This definition was used also in tensor component analysis (TCA) and population decoding (see below). This definition captures activity related to correct detection and report as well as reward-related activity, and this encoding model dissociated this from other confounds (e.g., licking and arousal) by exploiting the trial-by-trial variability in timing. For licking movement, we used three kernels that spanned −200 to 400 ms relative to each lick, split by lick side. To capture arousal effects, the z-scored pupil area was included in the predictor set: with original timing and two temporal offsets (−800 and −400 ms). We included three binary history-dependent predictors, capturing the modality of the previous trial (visual or auditory), reward (hits or not), and choice (lick to visual or auditory spout). Last, the trial number was included to account for nonstationarity in firing rate across the entire session because of, for example, motivational signals and electrode drift, but was not reported in figures.
The number, width, and spacing of temporal basis functions were selected by optimizing the variance explained on a diverse subset of representative neurons (Runyan et al., 2017). For example, the use of trial-spanning history predictors was based on the fact that history had an offset effect throughout the trial duration in example neurons (Fig. 3). Explained variance (EV) both on trial averages and single trials was comparable to previously reported studies (Runyan et al., 2017; Steinmetz et al., 2019).
This resulted in a predictor matrix of size P × T for each neuron, where P is the number of predictors and T is the number of total time bins. The encoding model was fit on concatenated single trials, and T is therefore the number of trials (typically 200-500 trials) multiplied by the number of time bins per trial (100 time bins; −0.5 to 2 s relative to stimulus change, 25 ms time bins). The encoding model (a GLM) was fit with a Poisson link function to single neuron activity with the glmnet package in MATLAB (Friedman et al., 2010). We used elastic-net regularization (α = 0.95) and fivefold cross-validation. To maximally punish weights without losing model fit quality, λ was maximized while keeping the cross-validated error within 1 SE of the minimum. We quantified model performance by assessing the fivefold cross-validated EV in two ways. First, we computed EV over all concatenated firing rate bins (over all single trials). Second, we computed EV on the concatenated firing rate bins of the average firing rate for four main trial type conditions with most trial counts (visual and auditory hits and misses) (Runyan et al., 2017; Musall et al., 2019).
To quantify the contribution of subsets of predictors, we calculated the single-trial EV for a firing rate prediction based on only those predictors. In other words, we computed how much of the firing rate variance is explained by only considering the weights from, for example, visual variables. This value was compared with a shuffled distribution where the EV was computed using the predicted firing rate and the actual firing rate from shuffled trials (N = 1000 shuffles). A neuron was deemed to significantly encode this variable if it exceeded the 99% percentile of this shuffled distribution. This permutation test approximately labeled variable encoding in single neurons if at least 1% of the single-trial variance was explained, as we found very similar results when we simply thresholded on 1% EV.
To quantify whether the joint encoding of variables was significantly different from a random distribution across neurons, we shuffled (N = 1000) the vectors of neuron indices that significantly encoded visual, auditory, and reward variables relative to each other and recomputed joint or unique encoding (i.e., recomputing the Venn diagram). For instance, with 50% of neurons encoding visual and 50% encoding auditory variables, this shuffling procedure would generate percentages of joint audiovisual encoding neurons at ∼25%, against which the actual percentage was tested (exceeding the 2.5% or 97.5% percentile, corresponding to a two-sided test with p < 0.05).
Population decoding analysis
We tested whether PPC ensembles were responsive to audiovisual stimuli and reward by training decoders to classify (1) audio versus catch trials, (2) visual versus catch trials, (3) visual versus audio trials, and (4) rewarded versus nonrewarded trials. In 1, 2, and 3, we included only trials with large stimulus changes. Decoding was performed on recordings that contained at least 15 neurons and 15 trials per class. Spikes were binned using a sliding window of 100 ms with 50 ms increments, whereas for the insets, showing broader temporal dynamics, we used a window of 500 ms moved with 250 ms increments. Trials were aligned to the moment of stimulus change (Fig. 4A–H) or the timing of the first response lick (i.e., the first lick occurring at least 100 ms after stimulus change, Fig. 4I,J). When trials were aligned to stimulus change, temporal bins containing data before and after stimulus change (t = 0) were excluded from the analysis. Decoding was performed using a random forest classifier with 200 trees, as implemented in Scikit-learn (Pedregosa et al., 2011), and we used a 5 × 5 cross-validation routine with stratified folds (which preserves the proportions of samples of the two classes in each fold). The average accuracy obtained in the cross-validation routine was corrected by subtracting the average accuracy on 300 surrogate datasets in which the trial labels were randomly permuted to obtain the improvement in decoding accuracy beyond chance level. For every time point, we tested whether the corrected accuracy was significantly different from 0 using Wilcoxon signed-rank test; p values were corrected for multiple comparisons using the Benjamini–Yekutieli method, with a significance level of 0.05. We additionally trained decoders to classify the orientation of the drifting gratings and the frequency of the audio tone (Fig. 4G,H). Before decoding visual and auditory stimulus identity, we grouped the two pairs of orientations/frequencies, to obtain a two-class classification problem, and included only trials in which the stimulus changed from one pair of stimuli to the other (max changes).
Absolute and shuffled population decoding accuracya
Tensor component analysis (TCA)
We applied TCA (Williams et al., 2018) to recordings that contained at least 15 neurons. As the objective and similarity plot did not demarcate a fixed number of components to decompose our data, we chose a rank of 8 and tested the robustness of our analysis to different choices of rank (5 and 10 components, data not shown). As a way of identifying components representing noise and drift in the recordings, we filtered out components that were expressed in <20% of units or <20% of trials. For each component, we measured with a ROC-AUC score how well the trial factors discriminated (1) audio trials versus catch trials, (2) visual trials versus catch trials, and (3) correct trials versus incorrect trials. For each AUC score, we computed an associated p value by generating a null distribution of AUC values calculated on randomly permuted trial labels and computing the fraction of values in the null distribution which were equal or larger than the observed AUC score. AUC scores were considered significant with p < 0.05. For Figure 5B, C, components were labeled based on the contrast associated with the highest AUC score and included if the highest AUC score had an associated p < 0.05.
Statistics
We used Bayesian statistics throughout the manuscript (Jeffreys, 1939; Rouder et al., 2009) to facilitate intuitive interpretation of the strength of evidence as well as establish evidence of the absence of effects (Keysers et al., 2020). Bayesian statistics assess the likelihood of the data under both the null and the alternative hypotheses (H0 and H1, respectively). In most cases, we report the Bayes factor that corresponds to the ratio of likelihoods p(data|H1)/p(data|H0), abbreviated to BF. For instance, BF = 10 would mean that the data are 10 times more likely under H1 than H0 providing very strong support for H1, while BF = 0.1 would mean that the data are 10 times more likely under H0 than H1 providing very strong support for H0. Generally, a BF between 1/3 and 3 indicates that the data are similarly likely under H1 and H0 and that the data thus does not adjudicate which is more likely. A BF below 1/3 or >3 is interpreted as supporting H0 or H1, respectively, corresponding roughly to p < 0.05 for moderate sample sizes (Jeffreys, 1939). Evidence of absence of an effect, where the null hypothesis is more likely given the data, is denoted by the hashtag symbol (#) in figures, next to the standard asterisk symbol (*) for evidence for the alternative hypothesis.
We also performed classical frequentist statistics (i.e., calculating the probability of observing the data given a hypothesis) for each test and found nearly identical results, except for one statistical test in Figure 8C, where the auditory threshold was slightly affected by PPC inactivation.
We made four exceptions and used the classical frequentist test when we required a significance threshold without an available Bayesian alternative: (1) χ2 test for fractions for significant increased and decreased fractions of PPC neurons (Fig. 1D) and (2) AUC values per TCA component versus a shuffled distribution (Fig. 5), (3) significant time points of decoding performance (Fig. 4), and (4) significant (joint) encoding of stimulus and behavioral variables versus shuffled distributions (Fig. 3).
Results
We trained mice to continuously and simultaneously monitor audiovisual stimuli and respond to one lick spout for changes in auditory frequency and the other spout for changes in the orientation of a drifting grating (Fig. 1A). In other words, mice were required to identify the sensory modality in which a change occurred and could respond as soon as a change was detected. Behavioral performance was analyzed using a multi-alternative signal detection model (Sridharan et al., 2014). This model extends classical signal detection theory to distinguish perceptual sensitivity from choice bias in detection tasks with multiple signals (in this case, visual and auditory changes; see Materials and Methods). Trained animals selectively reported both orientation and frequency changes and performance increased as a function of the amount of change (Fig. 1A). Animals were mostly tested in these unisensory trials, in which either a visual or an auditory change were presented, but never simultaneously. We also presented some truly multisensory trials, which will be discussed later. Mice lateralized their licks well, with occasional licks to the incorrect spout (10.5% errors at the visual lick spout, 14.9% errors at the auditory lick spout).
PPC is recruited during audiovisual change detection. A, An auditory stimulus (Shepard tone) and a visual stimulus (drifting grating) were continuously presented. Head-fixed mice had to report frequency changes (e.g., lick left on auditory trials) and orientation changes (lick right on visual trials). Gray lines indicate the psychometric fit of behavioral hit rates for individual sessions for an example mouse. Colored lines indicate the average across sessions (n = 7 sessions). Dotted lines indicate licks to the incorrect spout. Licks at zero orientation and frequency change are false alarms (middle of panels). The fraction of trials without a licking response is not shown but can be inferred from the figure as they sum to 100% together with shown response rates. B, We recorded single-unit activity in PPC. The image represents histologic verification with DAPI staining in blue and electrode tract stained with DiI (in red) overlaid with reference section from Paxinos and Franklin (2004). LTPa, Lateral parietal association area; V2L, lateral secondary visual cortex; RSA/G, retrosplenial cortex (a)granular part. C, Flattened cortical section in a PvCre-tdTomato mouse providing landmarks through cell densities in red (Gămănuţ et al., 2018), with additional DiI-stained electrode tracts in V1 and PPC (also red): V1, Primary visual cortex; LM, lateromedial; PM, posteromedial; A, anterior; AM, anteromedial; RSP, retrosplenial; S1bf, barrel field of primary somatosensory cortex. D, Heatmap of trial-averaged z-scored activity over all recorded neurons aligned to stimulus change. Each row corresponds to one PPC neuron recorded in NE animals (left panels; n = 218 NE neurons) and MST animals (right panels; n = 453 MST neurons), sorted by poststimulus activity. E, Fraction of neurons with significantly increased or decreased firing rate (Bayesian paired one-sided t test BF > 3, prestimulus −500 to 0 ms, poststimulus 0-500 ms; BF corresponds to the ratio of likelihoods of the alternative and null hypothesis) is increased in MST versus NE animals (χ2 two-sample tests: visual increase, p < 0.001; visual decrease, p = 0.003; auditory increase, p = 0.031; but not for auditory decrease, p = 0.998). Top left, Inset, Closeup of the visually responding fractions in NE mice. F, z-scored firing rate for neurons with increased and decreased rate for visual and auditory trials separately shows increased amplitude of modulation for trained versus naive mice. Black tick mark represents median reaction time. ***p<0.001. n.s. not significant.
After establishing psychophysical performance, we only used two levels of change for each modality for subsequent sessions: threshold (individually titrated per animal) and maximal. We first wondered what types of task-relevant activity the PPC displays. We compared this cohort of MST mice to another cohort of animals that was NE to the stimuli. NE mice were not trained to detect changes but were pseudorandomly rewarded for spontaneous licks. We recorded single-unit activity in mouse PPC (Fig. 1B,C), which corresponds to cortical areas also referred to as mouse lateral parietal association cortex (Paxinos and Franklin, 2004) or anteromedial visual cortex (Driscoll et al., 2017; Wang and Burkhalter, 2007). We found that the fraction of neurons responding to visual stimuli with changes in firing rate significantly increased from 5% in naive animals to 29.3% in trained animals, and from 31.6% to 39.3% for auditory stimuli (Fig. 1D,E). In contrast to what is commonly reported in primary sensory areas, sensory stimuli both increased and decreased firing activity of PPC neurons (Fig. 1F). The coupling between modality and rewarded lick spout (left or right) was counterbalanced across mice. We found, however, no systematic relationship for both modalities between the associated lick spout and evoked responses in contralateral PPC (Fig. 2). We therefore show that, in addition to visual stimuli (Pho et al., 2018), also auditory stimuli evoke activity increments and decrements in PPC during task performance compared with passive stimulation.
Increased auditory and visual responsiveness regardless of modality-side pairing. The modality-reward side pairing (which lick spout was associated with reporting changes in which modality) was counterbalanced across MST mice, while PPC was always recorded in the left hemisphere. To investigate whether the modality-reward side pairing was associated with different dynamics (e.g., increased auditory responsiveness in PPC if the auditory lick spout was contralateral), we repeated Figure 1E, F for neurons based on the pairing of the animals from which they were recorded. Data from NE mice were the same, as modalities were not associated with specific lick spouts. Results were qualitatively similar for both sets of neurons. A, Same as in Figure 1F, but for PPC neurons contralateral to auditory lick spout. B, Same as in Figure 1F, but for PPC neurons contralateral to visual lick spout. C, Same as in Figure 1E, but for PPC neurons contralateral to auditory lick spout. χ2 two-sample tests: visual increase, p < 0.001; visual decrease, p = 0.0149; auditory increase, p = 0.018; but not for auditory decrease, p = 0.424. D, Same as in Figure 1E, but for PPC neurons contralateral to visual lick spout (χ2 two-sample tests: visual increase, p < 0.001; visual decrease, p < 0.001; auditory increase, p < 0.001; but not for auditory decrease, p = 0.143). E, We tested directly whether more neurons were modulated during trials associated with a contralateral response (left PPC neurons during trials rewarded at the right lick spout) versus ipsilateral trials. The rise in the fraction of significantly responsive neurons was not significantly different based on laterality (n = 8 fractions, BF = 0.35). *p<0.05. ***p<0.001. n.s. not significant.
Sensory and behavioral correlates are heterogeneously distributed across single parietal neurons
We next focused on characterizing single-neuron responses in relation to stimuli and behavioral decisions. In trained animals, these were highly heterogeneous (Fig. 3A–C). We found neurons that responded specifically to visual (Example 1), or auditory stimuli (Example 2), or were bimodally responsive (Examples 3, 4). Additionally, the firing rate of many neurons was modulated during rewarded, but not unrewarded trials (Examples 5, 6) or showed a mix of stimulus and reward-related activity (Example 7; “reward” refers specifically to hit trials and thus captures activity related to stimulus detection, report, and reward; see Materials and Methods). Firing rates were further modulated by spontaneous licks outside trials (Example 8), as well as by the previous trial's outcome (reward or not) and choice (left or right lick), indicating a history effect (Examples 9, 10). To capture and quantify the encoding of such heterogeneous sensory and task-related variables in the firing rate of single PPC neurons, we implemented a kernel-based regression model (Park et al., 2014; Runyan et al., 2017). This model accurately predicted single neuron firing rates (Fig. 3A, dashed lines), explaining on average 46.1% of trial-averaged and 11.6% of trial-to-trial firing rate variance during 0-500 ms after stimulus. This trial-to-trial variance was jointly explained by visual (12.1%) and auditory (23.4%) stimulus features, as well as reward (37.6%), licking movement (7.8%), pupil size (16.3%), and trial history (stimulus, choice, and reward on the previous trial; 2.9%) (Fig. 3D). We investigated whether these variables were encoded by distinct or the same neurons and focused on the variables most relevant to trial-by-trial performance (vision, audition, and reward). We found a mixed profile, with some neurons coding a single variable, while other neurons encoded two or all three of these variables simultaneously (Fig. 3E), in line with the example neurons shown in Figure 3A. Visual, auditory, and reward coding were smoothly distributed across the neuronal population, without any clear clustering or organization (Fig. 3F,G). Moreover, the number of neurons encoding all three variables was higher than expected based on chance (Fig. 3H). Thus, single-neuron correlates of audiovisual change detection were heterogeneously distributed, and not segregated, across the neuronal population.
Heterogeneous task-related responses in single parietal neurons. A, Peristimulus time histograms (PSTH) for 7 example neurons for visual and auditory trials split by decision (hit or miss; licks to incorrect spout omitted). Dashed line indicates the predicted firing rate based on a kernel-based regression model. Neurons responded to visual (1) and auditory changes (2), or changes in both modalities (3, 4) and were modulated by decision (1, 5, 6, 7). Black tick marks represent mean reaction time in that session. B, PSTHs for example neuron (8) aligned to spontaneous licks during intertrial intervals (or random nonlicking time points). C, PSTHs for two other example neurons conditioned on the previous trial: reward (Example 9) and choice (Example 10), indicating history effects. We found no clear examples of neurons encoding stimulus history. D, EV of the trial-to-trial firing rate across 0-500 ms after stimulus (this window includes lick response and reward delivery) for each category of predictors (Bayesian ANOVA, main effect of predictor group, BF = 7.9 × 1013). Data are mean ± SEM. E, Venn diagram showing the percentage of all PPC neurons that significantly encoded visual, auditory, or reward, but also all combinations thereof (tested against random trial permutations, p < 0.01, corresponding roughly to >1% EV). Gray numbers indicate where example neurons in A–C are located. F, Scatter plot of EV by vision, audition, or reward. Each dot represents a single neuron, with the RGB color-scaled by the amount of EV for each variable (light represents high EV; dark represents low EV). Logarithmic axes were used to capture the spread in EV. The absence of clear clustering indicates heterogeneous mixed selectivity across the PPC population. G, Average variance explained by each predictor for neurons based on cortical depth, showing that EV was not because of a subset of neurons localized to specific layers. Lines indicate mean ± SEM. N = 483 neurons. H, We tested whether the distribution of visual, auditory, and reward encoding across neurons was different from a random distribution based on chance (given the number of neurons significantly encoding each of these variables). Each inset in the Venn diagram shows the shuffled distribution and the actual percentage (in red if exceeding the 1% or 99% percentile of the shuffle distribution). The fractions of neurons exclusively encoding auditory stimulus features (14%) or reward (11%) were significantly lower than shuffled, while the percentage of neurons encoding all three variables (11%) was higher than that expected based on a random distribution of encoding across PPC neurons. So, joint encoding of auditory stimuli, visual stimuli, and reward was more common than exclusive coding of auditory stimulus and reward, and also more common than what would have been expected from a random distribution of encoding variables across neurons.
Structure and content of parietal ensemble activity
To further investigate how information is globally encoded in PPC, we focused on how parietal population activity coded task-relevant variables relative to stimulus onset and decision. We used population decoding on ensemble activity recorded in individual experimental sessions and found that PPC activity sufficed to decode the presence of a sensory change (Fig. 4A–C; significant decoding in 12 of 14 sessions for audition, 8 of 14 for vision), as well as the sensory modality of this change (Fig. 4D–F; significant in 13 of 14 sessions for sensory modality). This suggests that parietal activity does not indiscriminately respond to any sensory change, but that different subspaces encode visual versus auditory information (Raposo et al., 2014). It was even possible to decode the sensory feature (specific orientation or frequency) in a subset of sessions (Fig. 4G,H; significant in 5 of 13 sessions orientation, and 5 of 11 for tone frequency). Furthermore, population activity sufficed to decode the animal's decision (Fig. 4F,I,J, significant in 14 of 14 sessions for reward), already ∼250 ms before reaction time (Fig. 4I, dotted line).
Parietal ensemble activity suffices to decode stimulus and decision. A, Accuracy of population decoding relative to chance plotted over time for random forest classifiers trained to discriminate visual versus catch trials (blue) and auditory versus catch trials (red). Trials were aligned to the time of stimulus change. Accuracy above chance was obtained by subtracting from the decoding accuracy the average decoding accuracy obtained after repeatedly shuffling the trial labels (for absolute decoding accuracies, see Table 1). Diamonds represent time points at which the distribution of decoding accuracy across recordings was significantly different from chance (Wilcoxon signed-rank test, p < 0.01). Shaded bands represent 95% CIs. Vertical gray dotted line and shaded band represent the median and interquartile range of lick times, respectively. Inset, Same decoding over a longer time window with larger temporal bins (similar in D and I). B, Decoding of auditory versus catch and visual versus catch at t = 0.25 s (time bin extending from 0 to 0.5 s). Each dot is a session, where the p value for each session was computed as the fraction of decoding accuracies obtained on shuffled data, which were larger than the observed accuracy (similar in E and H). C, Same as in A, but for individual recording sessions and longer time windows. Average decoding performance was not driven by a subset of sessions but was possible from nearly all recorded PPC populations. D, Same as in A, but for sensory modality and reward, that is, discriminating visual versus auditory trials (purple), and rewarded versus unrewarded trials (green). E, Same as in B, but for sensory modality (visual vs auditory trials) and reward (rewarded vs unrewarded trials). Decoding performance was significantly above chance (p < 0.05) in 13 of 14 sessions for sensory modality, and 14 of 14 for reward. F, Same as in D, but for individual recording sessions and longer time windows. G, Population decoding of drifting grating orientation (blue) and Shepard tone frequency (red) over time (same as in A, but with large temporal bins). Decoding aggregated across sessions was not significant overall (Wilcoxon signed-rank test, p > 0.05 for virtually all time points). H, Same as in B, but for decoding of drifting grating orientation (blue) and Shepard tone frequency (red). Decoding performance was significantly above chance (p < 0.05) in 5 of 13 sessions for visual orientation frequency, and 5 of 11 for tone frequency. This shows that, in a subset of recordings, not only the presence of an auditory of visual change was encoded, but also the identity of the postchange stimulus. I, Same as in A, but for decoding rewarded versus unrewarded trials when aligned to first lick (licking activity/reward expectancy allows above-chance decoding before reaction time). Vertical dashed green line indicates the time point at which decoding accuracy reached half of the maximum accuracy (t = −0.25 s). J, Same as in I, but for individual recording sessions and longer time windows.
In addition to supervised decoding, we also used an unsupervised dimensionality reduction method (Williams et al., 2018) to test whether the dominant low-dimensional neural dynamics of PPC activity were task-relevant. This analysis showed that the distributed patterns of task-related neuronal activity were well described by latent components that closely corresponded to auditory and visual responses and decision-related behavior (Fig. 5). Thus, well before reaction time, parietal population activity shows rich visual, auditory, and decision representations, including task-relevant information about which sensory modality shows a change, which are, theoretically, sufficient for downstream areas to execute the motor decision.
Latent dimensions of parietal population activity correspond to task-relevant components. To capture, visualize, and quantify the dominant low-dimensional neural dynamics of population activity in PPC, we used an unsupervised dimensionality reduction technique, TCA (Williams et al., 2018). TCA decomposes population activity into a limited set of components, each of which corresponds to an assembly of cells with rapid, common within-trial dynamics expressed across a variable set of trials. A, Three selected components from a TCA analysis of two example sessions (8-component model; see Materials and Methods). Each row corresponds to a component (#1-3) that captures a subpopulation of neurons (unit factors, left column), that share a temporal response profile within the trial (temporal factors, central column), which is in turn expressed differently across trials (trial factors, right column). All factors are unitless. Neurons are ranked based on their contribution to the first plotted component, then the second, etc. Dotted line and the shaded gray (middle) panel indicate the median and interquartile range of reaction times, respectively. The TCA decomposition is performed with no information about trial types, which are colored a posteriori. Nonetheless, the identified latent components show selective expression in auditory (#1), visual (#2), and hit trials (#3) in both example sessions, showing that the main dimensions of population activity are task-related. B, AUC scores measure how individual components are selectively expressed in auditory versus catch trials (red), visual versus catch trials (blue), and rewarded versus not rewarded trials (green). Darker dots represent components whose AUC scores were significantly larger than chance (tested against random trial permutations, p < 0.05). Boxes represent the quartile and 1.5 times the interquartile range of the significant AUC scores. White-filled dots locate the AUC scores of the three components shown in the top TCA decomposition of A. The donut plot represents the fraction of components that have a significant AUC score for each contrast, across all recordings. #BF < 1/3. C, Averaging the time factors of all selective components from different sessions (i.e., those with a significant AUC score, corresponding to the components marked in darker dots in B) showed a distinct temporal profile with a progression of first auditory, then visual, and last, decision components. Shaded bands represent 95% CIs. Vertical dashed lines indicate the peak time of the averaged temporal factors.
Sensory and task-related representations in PPC are not required for task performance
Together, when animals are trained to detect auditory and visual changes, PPC multiplexes sensory and task-related variables, thus suggesting a potential role in linking task-relevant auditory and visual inputs to adaptive decisions. To investigate the causal nature of this link, we optogenetically inactivated PPC bilaterally (by locally enhancing inhibition by PV interneurons). We verified that viral expression spanned PPC across the anteroposterior and mediolateral axes and across cortical layers (Fig. 6A–C) and that photostimulation excited a fraction of PPC neurons (putative ChR2-expressing PV cells) while effectively suppressing ongoing activity (Fig. 6D–F). Optogenetic inactivation was less powerful in the deepest layers compared with more superficial ones (Fig. 6F), also compared with inactivation of V1 achieved using the same methodological approach (Fig. 6G). As opposed to the major silencing effect on PPC itself, optogenetic inhibition of PPC had a slight excitatory effect on activity in V1 (Fig. 6H).
Optogenetic inactivation of PPC. A, Coronal section showing viral expression in bilateral PPC and the electrode tract stained with DiI for simultaneous physiological validation. B, Viral infection was present throughout layers of PPC. No evidence was obtained for infection of deep pyramidal neurons with large somata, as reported for S1 (Tanahira et al., 2009). C, Series of coronal sections from one mouse showing the extent of viral infection of PPC along the anteroposterior and mediolateral axis (compare Paxinos and Franklin, 2004). Numbers indicate approximate anteroposterior offset from bregma. When aligning to the Allen Mouse Brain Atlas, expression starts in the anteromedial visual cortex (AM) and ends in the posterior parietal association area (PTLp). D, We inactivated PPC by local activation of PV interneurons and photostimulation with an optic fiber (2-10 mW, 10 ms pulses at 25 Hz) directly over the thinned skull. Top, High-pass filtered voltage (>500 Hz) at an example electrode site (recorded at ∼450 μm below dura) with effective inhibition of spiking activity (i.e., multiunit activity) on photostimulation. Bottom, Closeup of individual pulses during 1 s photostimulation. E, The average waveform was narrower for PPC neurons that showed increased (red) versus decreased (blue) firing rate on photoinactivation of PPC. This is in line with putative narrow-spiking PV cells being recruited by photoinhibition to suppress broad-spiking pyramidal cells. Inset, Significant difference in the peak-to-trough delay (0.81 vs 0.70 ms, 90 vs 38 neurons, BF = 4.47). *BF > 3. F, Effectiveness of optical manipulation across cortical depth is shown as the percentage of the baseline firing rate. Neurons were separated into inhibited (n = 95 neurons; blue) and excited (n = 39 neurons; red) populations. We clipped the range of modulation ratios at 200% (some neurons were strongly recruited by photostimulation). Lines and shading represent mean ± SEM. G, Same as in F, but for V1 inhibition (n = 150 inhibited neurons in blue, 68 excited neurons in red). Compared with PPC, optogenetic inhibition in V1 was more effective in the deeper layers. H, To investigate the spatial extent of PPC inactivation, we analyzed V1 activity during PPC photostimulation. Optogenetic inhibition of PPC had a slight excitatory effect on activity in V1 (Bayesian t test photostimulation vs baseline, n = 213 V1 neurons, Cohen's d = 0.41, BF = 4.08), as opposed to the major silencing effect on PPC itself. At the minimum, this suggests that PPC inactivation does not affect V1 in a major way, in line with the extent of viral expression seen in C. *BF > 3.
Surprisingly, optogenetic silencing of PPC had significant effects on neither visual nor auditory change detection, with evidence for the absence of an effect (Keysers et al., 2020) in both visual and auditory conditions (Fig. 7A–C). We additionally inactivated V1 in a separate cohort of animals. Bilateral inactivation of V1 strongly reduced detection of visual orientation changes, but not auditory frequency changes, consistent with the primary role of V1 in visual feature processing (Glickfeld et al., 2013; Resulaj et al., 2018; Zatka-Haas et al., 2021).
Optogenetic inactivation of PPC does not affect visual or auditory change detection. A, Bilateral V1 inhibition reduced visual detection performance as quantified by d′ with a two-alternative signal detection model (Bayesian paired t test, 19 sessions; threshold visual changes: Cohen's d = 1.162, BF = 135.55; maximal visual changes: Cohen's d = 1.251, BF = 593.51), with no effect on auditory performance (threshold auditory changes: Cohen's d = 0.231, BF = 0.49; maximal auditory changes: Cohen's d = 0.132, BF = 0.30). Bilateral PPC inhibition had no effect on either visual or auditory performance (Bayesian paired t test, 15 sessions; threshold visual changes: Cohen's d = −0.197, BF = 0.34; maximal visual changes: Cohen's d = −0.110, BF = 0.30; threshold auditory changes; Cohen's d = −0.178, BF = 0.27; maximal auditory changes: Cohen's d = 0.417, BF = 0.32). B, Behavioral response rates averaged over V1 inactivation sessions for control and photostimulation trials. Error bars indicate SEM across sessions. C, Same as in B, but for PPC silencing, where behavioral hit rates are unaffected. D, PPC inactivation did not affect reaction times. Boxplot represents median and interquartile ranges. Auditory hits: 257 ms (control, median reaction time), 238 ms (opto), 1454 versus 915 trials, Cohen's d = −0.036, BF = 0.033; visual hits: 391 ms (control), 392 ms (opto), 1115 versus 735 trials, Cohen's d = −0.001, BF = 0.016. Outliers were omitted from the plot for visual clarity. E, Average d′ with and without PPC inactivation for each animal. Same data as in A, right. F, The change in d′ by PPC inactivation was not related to the threshold for visual or auditory report, that is, the criterion parameter in signal detection theory (Bayesian correlation; visual: r = −0.06, BF = 0.18; auditory: r = 0.01, BF = 0.17). Each dot represents a session. *BF > 3. **BF > 10. ***BF > 30. ****BF > 100. #BF < 1/3 (evidence for the absence of an effect).
PPC inactivation could affect other aspects of behavioral performance, despite overall preserved visual and auditory hit rates. However, we found that all tested aspects of behavioral performance persisted with PPC inactivation. First, we found no effect on reaction times (Fig. 7D). Second, we tested the hypothesis that PPC inactivation could be restricted to a subset of animals showing a particular behavioral strategy. However, we found both small increases and decreases in task performance in individual mice and no relationship between changes in task performance and the bias to report visual or auditory stimuli (Fig. 7E,F). Third, although detection of stimuli at perceptual threshold (those most sensitive to perturbation) was preserved (Fig. 7A–C), psychometric performance could potentially be affected in other subtle ways. However, when we inactivated PPC at various levels of auditory and visual stimulus difficulty levels, hit rates were unaffected (Fig. 8A). We fit each session with a psychometric version of our signal detection model and found no evidence for an effect, but also no evidence for the absence of an effect, on sensitivity, threshold, and bias (Fig. 8B,C).
Inactivating PPC does not affect psychometric parameters in the audiovisual detection task. A, Same as in Figure 7C, but now for PPC inactivation during sessions with four levels of auditory and visual change (n = 9 sessions). #BF<1/3 (evidence for the absence of an effect). B, We fitted a psychometric two-alternative signal detection model on control and photostimulation trials of individual sessions separately. Thin lines indicate single session fits. Thick lines indicate the median fit. Based on the same data as in A. C, For each of the eight model parameters, we found no evidence for an effect on fitted parameter value comparing control and photostimulation trials with a Bayesian paired t test (n = 9 sessions): auditory parameters: d′, Cohen's d = −0.196, BF = 0.363; sensitivity: Cohen's d = 0.668, BF = 1.469; threshold: Cohen's d = −0.534, BF = 0.898; criterion: Cohen's d = 0.761, BF = 2.108; visual parameters: d′: Cohen's d = −0.160, BF = 0.344; sensitivity: Cohen's d = 0.134, BF = 0.333; threshold: Cohen's d = −0.419, BF = 0.617; criterion: Cohen's d = −0.154, BF = 0.342. For four of these parameters, BF was close to (but not below) 1/3, suggesting evidence of absence of an effect.
Behavioral choice is history-dependent, but this effect is not dependent on PPC
Previously, PPC was shown to mediate history-dependent effects on behavioral choice (Hwang et al., 2017; Akrami et al., 2018). To investigate this, we constructed a multinomial logistic regression model that uses information about the current and previous trial to predict the animal's choice (three response options: lick to visual spout, lick to auditory spout, or no lick). To test the contribution of individual sensory and behavioral features, we constructed 10 models with different regressor combinations (Fig. 9A). Data were fit on concatenated sessions of individual animals, and model predictions were evaluated on held-out test data (threefold cross-validation). Relative to a null model, including sensory history improved model performance slightly, while including reward history and choice history had larger effects (Fig. 9B). Including information about the sensory stimulus of the current trial (amount of visual and auditory change) boosted performance significantly, confirming animals mostly base their choice on the current stimulus.
Behavioral choice is history-dependent, but this effect is not dependent on PPC. A, Overview of 10 models with different combinations of regressors. A null model served as a baseline comparison, and Models 2-6 included different single predictors, while Models 7-10 contained multiple predictors. B, Cross-validated performance (% correctly predicted choices on held-out test data) for the 8 model variants (without photoinactivation of PPC). Each dot represents the average performance over 3 folds for one animal. Models differed in their predictive performance (Bayesian ANOVA, BF relative to intercept only = 5.6 × 1032). Models 2-6 all performed better than the null model (trial number: Cohen's d = 0.67, BF = 41.92; sensory history: Cohen's d = 1.34, BF = 4.10; reward history: Cohen's d = 1.27, BF = 577.24; choice history: Cohen's d = 3.51, BF = 350.15; sensory evidence: Cohen's d = 4.25, BF = 6.6 × 107). C, Weights of the different predictors on probability ratios of licking to auditory and visual spouts relative to no-lick in the full model (Model 8). Error bars indicate mean ± SEM. *BF > 3. **BF > 10. ***BF > 30. ****BF > 100. #BF < 1/3. D, The effect of PPC inactivation on choice behavior was tested in four different ways. First, we compared model performance with and without including photostimulation as a regressor in the sensory and motor history model (Model 7 vs Model 9, Bayesian t test; Cohen's d = −0.519, BF = 0.643). Each dot represents one animal. E, Same as in D, but for the full model with and without photostimulation (Model 8 vs Model 10, Bayesian t test; Cohen's d = −0.006, BF = 0.397). F, Second, as in Hwang et al. (2017), we compared model performance when training on control trials and testing on control or photostimulation trials (Bayesian t test; cross-validated correct trials on control vs opto: Cohen's d = 0.217, BF = 0.437). G, Third, as in Akrami et al. (2018), we tested model performance (Model 8) when training and testing separately on control or photostimulation trials only (Bayesian t test; cross-validated correct trials, control vs opto: Cohen's d = 0.454, BF = 0.582). H, Finally, as in Akrami et al. (2018), we compared the sum of motor and reward weights when training and testing separately on control or photostimulation trials only (Bayesian t test; sum of history weights control vs opto: Cohen's d = −0.214, BF = 0.436). n.s. not significant.
We investigated the regressor weights in the full model, with positive auditory or visual weights indicating a higher probability of licking the auditory or visual lick spout, respectively, relative to not licking (Fig. 9C). Trial number had a small negative weight on both lick probabilities, in line with growing satiety. The other weights were modality-specific. Auditory and visual stimuli on the current trial had strong positive weights, sensory history a slight negative weight, and previous reward a positive weight. The previous choice had opposite weights suggesting animals repeated their choice.
We next tested the effect of PPC inactivation on history-dependent behavioral choice in four different ways but found no evidence for effects of PPC inactivation on how sensory information of the current trial is used, nor how trial history affects behavior (Fig. 9D–H) (Hwang et al., 2017; Akrami et al., 2018).
In sum, although we found that behavioral choice was influenced by reward and choice history (but not sensory history), these history effects were unchanged by PPC inactivation.
Last, we presented a subset of multimodal trials in which both modalities changed simultaneously, which implies a conflicting situation (i.e., mice could lick either left or right, based on which sensory modality prevails). In contrast to Song et al. (2017), who found that PPC inactivation shifted behavioral choice from auditory to visual report, we observed no such effect (Fig. 10). Therefore, despite robust encoding of sensory- and task-related variables well before response onset, we found no causal relevance of PPC in audiovisual change detection.
Behavioral choice during audiovisual trials is preserved during PPC inactivation. A, As visual and auditory feature changes were associated with rewards at different lick spouts and thus different motor actions, the modalities acted as competing inputs. A simultaneous change presents the animal with a conflicting situation (i.e., mice could lick either left or right, based on which sensory modality prevailed). A reward was given for licking either spout. B, We tested whether PPC inactivation specifically affected the auditory or visual choice fraction in conflict trials; this was not the case (Bayesian t test, N = 15 sessions; auditory choice: Cohen's d = 0.21, BF = 0.289; visual choice: Cohen's d = −0.404, BF = 0.30). #BF < 1/3 and denotes evidence for the absence of an effect. C, We found no effect of PPC inactivation on reaction times only during conflict trials. Auditory choice trials: 253 ms (control), 255 ms (opto), 507 versus 290 trials, Cohen's d = −0.027, BF = 0.018. Visual choice trials: 368 ms (control), 390 ms (opto), 284 versus 211 trials, Cohen's d = −0.031, BF = 0.018. Boxplots represent median and interquartile ranges.
Discussion
We tested the hypothesis that PPC is required to arbitrate between sensory modalities in a change detection task that required mice to segregate and identify sensory modalities. Training mice on this task led to qualitatively rich and heterogeneous auditory, visual, and decision-related activity changes in PPC (Figs. 1–5), which was however not causal to performing the task (Figs. 7–10).
Previous studies reported a similar absence of behavioral consequences despite strong decision-related activity in PPC (Raposo et al., 2014; Erlich et al., 2015; Hanks et al., 2015; Katz et al., 2016; Licata et al., 2017). Our results further reveal a surprising dissociation between rich representations in PPC (preceding reaction time) and causal involvement in task performance. This is especially surprising given the dense interconnection of PPC with both primary sensory cortices (V1 and A1) and nearby associative areas, such as anterolateral visual cortex (Meijer et al., 2020), and premotor areas, such as supplementary motor cortex, which show multisensory task correlates and are causally involved in task performance (Erlich et al., 2015; Barthas and Kwan, 2017; Allen et al., 2017; Coen et al., 2021). Therefore, despite its uniquely suited anatomic and functional properties, at the interface between visual and auditory processing and premotor cortices, PPC is required for neither auditory nor visual change detection. A different result might have been obtained if visual and auditory had to be integrated. Another possible explanation, however, is that sensory and decision-related activity in PPC may reflect secondary processes not directly subserving behavioral decision-making in our task, or functional processing of an order of complexity that is not required to solve the task, as this may be solved by lower-order (e.g., subcortical) structures. Interestingly, the large increase in auditory responses that we reported in trained mice conforms with similar reports in rodent association cortices (Raposo et al., 2014; Zhong et al., 2019; Meijer et al., 2020) but is different from what has been reported in primates (Grunewald et al., 1999; Cohen, 2009). In particular, limited auditory responses have been reported in primate lateral intraparietal area on engagement in an auditory task (Grunewald et al., 1999). This may indicate different circuitry for auditory processing in rodent versus primate PPC, but also the fact that primate experiments were performed in an area (lateral intraparietal area) that bears no homolog in rodents. The increase in visual and auditory responses that we observed could result both from extensive behavioral training as well as active task engagement, as both were confounded in our experimental comparison between trained and naive mice.
Considering the broader literature on rodent PPC, under which task conditions would parietal circuitry be causally involved? In recent studies, stimuli and context were fixed while task demands were varied, and PPC, among other regions, became necessary only when evidence accumulation, working memory, or in any case an extension in the time scale of the task were required (Harvey et al., 2012; Pinto et al., 2019; Arlt et al., 2021). Our results fit with this view and suggest that PPC is dispensable for tasks that have fixed one-to-one sensorimotor mappings (but see Arlt et al., 2021). Along these lines, PPC is thought to participate in a larger cortical network constructing contextual representations of multimodal inputs. These representations likely serve to maintain information across a delay (working memory) or across trials (history effects) or integrate information with previous experience (learning) and are not crucial for the detection of sensory stimuli. An interesting parallel interpretation of the richness of neural PPC correlates in the absence of a causal role in audiovisual change detection relates to the involvement of the human PPC in hemineglect (Vallar, 1998; Driver and Vuilleumier, 2001; Kerkhoff, 2001), pointing to a function of the PPC in generating the multisensory spatial survey that we associate with consciousness and spatial attention (Pennartz, 2015; Pennartz et al., 2019). Intriguingly, our results show that PPC displays classical neural correlates of consciousness (Koch et al., 2016), without being causally involved in the reportability of sensory stimuli. This indicates the need to critically evaluate the potential relevance of neuron-level correlates of sensory detection in the context of consciousness research, but see Vugt et al. (2018) and Nieder et al. (2020) for recent studies in which such correlates are interpreted as markers of consciousness. Overall, based on this functional angle, PPC activity may become behaviorally relevant when utilization of this contextual information is necessary, for example, in guiding decisions based on unfamiliar sensory stimuli or during learning (Wander et al., 2013; Wilber et al., 2017; Zhong et al., 2019), as also reported in primary somatosensory cortex (Hong et al., 2018) or in a more complex task requiring a multisensory spatial survey of the subject in its environment.
Another possible interpretation is that choice-related activity in PPC may reflect, at least to some extent, movement variables (Whitlock et al., 2012; Mimica et al., 2018), which are increasingly reported as affecting widespread cortical circuits in rodents (Musall et al., 2019; Stringer et al., 2019; Salkoff et al., 2020). Indeed, movement variables did explain a fraction of the variance across the PPC population activity (Fig. 3D). Alternatively, the modality identification task tested here may be solved by a distributed network of nodes in which the PPC participates in a causally redundant manner or that PPC's role is redistributed through fast-acting compensatory mechanisms (Sigler et al., 2009; Mejias and Wang, 2019). The inconsistency between our reported absence of an effect of PPC inactivation on auditory dominance with Song et al. (2017) may lie in the fact that we used instantaneous sensory changes, which do not result in an extended temporal window of conflicting signals. Finally, a potential concern is that our protocol for optogenetic inactivation spared a limited functionality of the PPC in its deep layers sufficient for full task performance (Fig. 6F). Although viral infection expression encompassed PPC across the full mediolateral axis of PPC and reached the deepest layers (Fig. 6A–C), and we effectively suppressed visual perception using the same experimental approach in V1, we cannot exclude that a portion of PPC might have been spared and might have been sufficient to prevent behavioral impairment.
In conclusion, we showed that neuronal responses in mouse PPC during audiovisual change detection, despite being multisensory and anticipating task-related responses, are not causally related to task performance. This sheds light on the function and architecture of parietal associative circuits and emphasizes the importance of cautiously interpreting the causal relevance of neural activity for task performance.
Footnotes
This work was supported by the European Union's Horizon 2020 Framework Program for Research and Innovation under the Specific Grant Agreement 720270 (Human Brain Project SGA1) to C.M.A.P.; Grant Agreement 785907 (Human Brain Project SGA2) and 945539 (Human Brain Project SGA3) to C.M.A.P. and U.O.; FLAG-ERA JTC 2015 project CANON (cofinanced by the Netherlands Organization for Scientific Research) to U.O.; and FLAG-ERA JTC 2019 project DOMINO (cofinanced by the Netherlands Organization for Scientific Research) to U.O. We thank D. Sridharan for providing code for the multialternative detection model; A. Williams for code for tensor component analysis; C. Rossant, collaborators, and members of the Cortex Lab (UCL) for Klusta and Phy spike sorting software; and A. Mathis and collaborators for DeepLabCut software.
The authors declare no competing financial interests.
- Correspondence should be addressed to Matthijs N. Oude Lohuis at matthijsoudelohuis{at}gmail.com or Cyriel M.A. Pennartz at c.m.a.pennartz{at}uva.nl or Umberto Olcese at u.olcese{at}uva.nl