Our visual system can operate at fascinating speeds. Psychophysical experiments teach us that the processing of complex natural images and visual object recognition require a mere split second. Even in everyday life, our gaze seldom rests for long on any particular spot of the visual scene before a sudden movement of the eyes or the head shifts it to a new location. These observations challenge our understanding of how neurons in the visual system of the brain represent, process, and transmit the relevant visual information quickly enough. This article argues that the speed of visual processing provides an adjuvant framework for studying the neural code in the visual system. In the retina, which constitutes the first stage of visual processing, recent experiments have highlighted response features that allow for particularly rapid information transmission. This sets the stage for discussing some of the fundamental questions in the research of neural coding. How do downstream brain regions read out signals from the retina and combine them with intrinsic signals that accompany eye movements? And, how do the neural response features ultimately affect perception and behavior?