Using fMRI representations of single objects to predict multiple objects in working memory in human occipitotemporal and posterior parietal cortices

Abstract

Research in visual perception has shown that in sensory areas, neural responses to a pair of objects presented together can be approximated by the linear average of the responses of each object shown alone. In this study, we ask if such an averaging relationship is unique to perceptual representations or if it also applies to representations maintained in visual working memory (VWM). By examining fMRI response pattern averaging across two experiments in both male and female human participants, we found that after properly accounting for task factors such as load, an averaging relationship also applies to representations formed in VWM. Specifically, VWM representations for two items can be approximated by the linear average of the VWM representations of each component item in both human occipitotemporal cortex (including early visual areas) and posterior parietal cortex. Although response averaging was originally proposed as a mechanism to combat distortion in representation due to neuronal response saturation in perception, the present study shows that even when response amplitudes were much lower in VWM compared to those in visual perception, an averaging relationship is still present for neural representations formed in VWM. This likely stems from the need to reduce interference among the concurrently stored items in VWM to maintain their representational independence. As an experimental method, response averaging may constitute an efficient yet simple tool to probe response independence in the human brain beyond perception and VWM.

Significance Statement In visual perception, neural responses of two simultaneously presented objects can be approximated by the linear average of the responses of each object shown in isolation. Here, we ask if such an averaging relationship is unique to perceptual representations or if it also applies to representations formed in visual working memory (VWM). Across two fMRI experiments, we found that when the storage load was properly equated, VWM representations for a pair of objects can be approximated by those of its component objects in both human occipitotemporal cortex (including early visual areas) and posterior parietal cortex. We additionally argue that response averaging may be used as an effective tool to probe response independence in the human brain.