Maintaining Auditory Working Memory Representations beyond Sensory Cortices Makes Working Memory Work

Since the concept of working memory was proposed, where the memorized information is temporarily maintained in the brain has been a central issue in the field. Early neurophysiological in primates and human neuroimaging work suggested that the prefrontal and parietal lobes are the potential neural substrate for working memory based on the observations that cells or neural circuits in these regions showed persistent activity (and in a load-dependent manner) during the delay period of working memory tasks. However, evidence showing associations between frontoparietal and sensory regions in working memory tasks (e.g., coactivation) led researchers to put forward a sensory recruitment account. This model assumes that the sensory cortices that encode specific features for to-be-remembered information are also involved in maintenance of those features in working memory tasks, with prefrontal and parietal regions proposed to have roles not in storage but in executive functions (e.g., controlling attention to prioritize maintenance of task-relevant information or resolve interference from task-irrelevant information). More recently, noninvasive human neuroimaging studies with multivariate decoding techniques (e.g., multivoxel pattern analysis) provided intriguing evidence on this issue. Multivariate decoding methods are thought to be more powerful in addressing questions regarding the contents of memory because they relate distinguishable activation patterns (indexed by an above-chance classification accuracy) to different feature/content conditions; thus, they can decode the contents of working memory from activity patterns in a given brain region. Using this approach, researchers found that, during the delay period in visual working memory tasks, visual information could be read out from local activity patterns in visual sensory areas (e.g., Harrison and Tong, 2009). This evidence provides support for the sensory recruitment account of working memory.

However, evidence challenging this view has been reported. Researchers have argued that an essential feature of working memory is the storage of task-relevant information in the face of task-irrelevant distractors. Thus, testing resistance to interference from distractors provides a way to examine which regions play central roles in working memory storage. For example, Bettencourt and Xu (2016) showed that the contents of visual working memory could not be decoded from activity in the sensory cortex if distractors were present during the retention period, but decoding from activity from downstream/hierarchical regions in frontoparietal (e.g., intraparietal sulcus) was successful even when distractors were presented. Other researchers reported the coexistence of working memory decoding in visual sensory and frontoparietal regions when distractors were present during the working memory delay (e.g., Rademaker et al., 2019), clouding the issue regarding which region is essential for working memory storage in the face of distraction. While the debate continues, there are limitations to the research. Most studies have been done in the visual domain; moreover, these studies have rarely examined the distractor effect on working memory decoding when participants have to process the distractors.

To address some of the limitations of previous studies, Deutsch et al. (2023) tested whether representations of auditory stimuli were maintained in auditory sensory cortex (e.g., Heschl’s gyrus) over the delay period of an auditory working memory task. Specifically, they asked whether a simple feature of auditory stimuli (i.e., high vs low pitch of complex harmonic sounds an octave apart) could be decoded using multivoxel pattern analysis from the sensory cortices during the delay period under different distraction conditions. In the working memory task, a target sound was presented to participants who had to remember the pitch of the sound over a 12.8 s delay period, and then judge whether the pitch of a probe sound was higher or lower than that of the remembered sound. On some trials, distractors were present during the delay period, and participants were instructed to either ignore or actively process the distractors (i.e., detect whether there were any volume reductions with distractors) on different trials. The authors found that, in the no-distractor condition, a “perception classifier” that was trained on the same auditory stimuli with no working memory demand successfully decoded pitch from the auditory sensory area during the delay period of the working memory task. Successful decoding was also obtained from a “working memory classifier” trained on the working memory delay-period activity. However, when the delay period was filled with similar auditory distractors, regardless of whether participants had to ignore or actively attend to those distractors, decoding in the auditory area dropped to a chance level, and participants’ behavioral performance on the target memory sound decreased.

Participants’ above-chance behavioral performance in both passive and active distraction conditions, despite the failure of decoding from sensory cortical activity in these conditions, suggests that the target sound was maintained in another brain area when distractors were present. To explore whether any neural substrate beyond the auditory area maintained working memory contents, Deutsch et al. (2023) conducted multivoxel pattern analysis in some regions previously implicated in working memory storage. In the active distraction condition, pitch information could be decoded from the inferior frontal and the superior parietal areas by the perception classifier, and decoding with the working memory classifier was also successful for the inferior frontal region. This suggests that these regions maintain auditory working memory content in the face of distraction.

The findings of Deutsch et al. (2023) showed that the decoding of auditory working memory contents in the sensory areas is not distractor-resistant. One explanation the authors provide is that, unlike the visual sensory area, auditory cortex could not maintain memory and distractor representations concurrently, or the memory representation was suppressed by the presence of distractors because attention was drawn away from the memory item. However, this latter possibility does not seem to explain the absence of decoding in the passive distraction condition in which participants were instructed to ignore the distractors. Considering the successful decoding in the inferior frontal region in the active distraction condition, one may expect that the inferior frontal region could also maintain pitch information in the passive distraction condition. However, as the authors explained that the presence of decoding in the inferior frontal region reflects a shift in abstraction of working memory representation or task-oriented codes in the high-level regions (Christophel et al., 2017), it is possible that, under the high demanding condition, participants maintained pitch information via some task strategies (e.g., humming high vs low pitch) which gave rise to successful decoding in the inferior frontal lobe. Then, in the passive distraction condition, participants did not have to use such a strategy, and decoding was absent in this frontal region. Thus, the question of where the auditory working memory representation was maintained in the passive distraction condition remains open.

Contrary to the sensory recruitment account, researchers have provided another explanation for the absence of decoding evidence in the sensory cortex in the face of distractors (e.g., Xu, 2017). Based on work in visual working memory, Xu (2017) claims that the sensory visual cortex does not play an essential role in working memory storage. The distractor effect on decoding evidence (i.e., a decrement of decoding to chance level) in the sensory cortex, as well as above-chance decoding in a downstream area (e.g., intraparietal sulcus), suggests that, in the face of distractors, the target memory item was transferred and maintained in the downstream area. Maintaining working memory representations in a nonsensory region has an ecological advantage in that it allows working memory maintenance while freeing up the sensory system for processing upcoming stimuli. Interestingly, recent studies on auditory working memory with multivariate decoding techniques all found that auditory working memory content (e.g., frequencies of tones) could be decoded from frontoparietal regions beyond the auditory sensory area (Kumar et al., 2016; Uluç et al., 2018; Czoschke et al., 2021). Similarly, using auditory verbal stimuli with another multivariate decoding approach (i.e., representational similarity analysis), Yue and Martin (2021) found that phonological features could be decoded from the left supramarginal gyrus, but not the phonological “sensory” area (i.e., the left superior temporal lobe that encodes phonological features) during the delay period of a phonological working memory task. More recently, researchers found that the auditory working memory content could be decoded from functional connectivity patterns between auditory sensory and hierarchical areas (e.g., left supramarginal gyrus) (Ahveninen et al., 2023). All these results offer a possibility of a neural basis for a transfer of auditory working memory representations from sensory areas to a downstream area in the face of distractors. Future studies need to test the distractor interference effect on auditory working memory in those association areas. Although Deutsch et al. (2023) conducted a whole-brain searchlight analysis to identify additional regions showing distractor-resistant working memory decoding evidence and did not find any, they explained that this was probably because of the limited sensitivity of decoding in the searchlight procedure.

Deutsch et al. (2023) also suggest that, under distraction conditions, the memory information in the sensory area may be stored in a silent- or hidden-state that could not be decoded from fMRI data (Stokes, 2015). Evidence supporting this notion mainly comes from studies in the visual domain (e.g., Rose et al., 2016; Wolff et al., 2017). Although this view is appealing, evidence showing that this principle applies to auditory working memory in auditory areas is still limited (but see Mamashli et al., 2021). From a theoretical perspective, the model adopting this hypothesis (e.g., the sensory recruitment hypothesis) has to explain how neural mechanisms in the sensory cortex could both maintain the memory representation and process distractors, as in the active distraction condition in the Deutsch et al. (2023) study, particularly when the target memory item and distractors share some common features (e.g., pitch).

In conclusion, the study by Deutsch et al. (2023) sheds light on unique characteristics of auditory working memory by showing that activity in auditory sensory areas may be insufficient to maintain working memory representations in the face of distractors. To make auditory working memory resistant to distractors, regions beyond the auditory sensory cortex may be required. These areas should be explored in further research.