The Human Brain Encodes a Chronicle of Visual Events at each Instant of Time thanks to the Multiplexing of Traveling Waves

Abstract

The human brain continuously processes streams of visual input. Yet, a single image typically triggers neural responses that extend beyond one second. To understand how the brain encodes and maintains successive images, we analyzed with electro-encephalography the brain activity of human subjects of either sex, while they watched ∼5,000 visual stimuli presented within fast sequences. First, we confirm that each stimulus can be decoded from brain activity for ∼1 sec, and demonstrate that the brain simultaneously represents multiple images at each time instant. Second, we source-localize the corresponding brain responses in the expected visual hierarchy, and show that distinct brain regions represent different snapshots of past stimulations. Third, we propose a simple framework to further characterize the dynamical system of these traveling waves. Our results show that a chain of neural circuits, which consist of (i) a hidden maintenance mechanism, and (ii) an observable update mechanism, accounts for the dynamics of macroscopic brain representations elicited by successive visual stimuli. Together, these results detail a simple architecture explaining how successive visual events and their respective timings can be simultaneously represented in brain activity.

SIGNIFICANCE STATEMENT

Our retinas are continuously bombarded with a rich flux of visual input. Yet, how our brain continuously processes such visual streams is a major challenge to neuroscience. Here, we developed techniques to decode and track, from human brain activity, multiple images flashed in rapid succession. Our results show that the brain simultaneously represents multiple successive images at each time instant by multiplexing them along a neural cascade. Dynamical modeling shows these results can be explained by a hierarchy of neural assemblies which continuously propagates multiple visual contents. Overall, this study sheds new light on the biological basis of our visual experience.