When Does the Visual System Need to Look Back?

Visual perception, and in particular object recognition, has traditionally been conceived of as a hierarchical process with a dichotomy between feedforward and feedback. Going forward in the ventral stream (V1/V2→V4→PIT→AIT), cell responses gradually become size and position tolerant, as well as selective to increasingly complex features. In this framework, the role of the early retinotopic visual cortex (i.e., V1/V2) is limited to basic computations of the visual input (e.g., edge detection) to feed higher areas for more complex processing. A single forward sweep through this hierarchy may be sufficient to perform object categorization (Serre et al., 2007). Indeed, rapid categorization experiments showed that humans and monkeys can produce extremely fast and reliable responses about the category of objects in natural scenes (Thorpe et al., 1996). It is important to keep in mind that this original hypothesis comes from a very specific observation: the first reliable behavioral responses start as early as 250 ms after image display. With roughly 10 synapses from the retina to high-level visual areas, and considering that transfer of information through each synapse takes 10–20 ms, behavioral responses occurring only 250 ms after image presentation are produced too early for recurrent processing to have a role. Nevertheless, a crucial point to note is that there is no reason to argue that later behavioral responses need to rely only on the first feedforward sweep. Late responses may take advantage of recurrent processing, for instance where the object is embedded in a cluttered background or is partially occluded, or simply when subjects are not respecting the instruction to respond as fast as possible. Indeed, a large number of feedback connections transmitting information from higher to lower levels of the hierarchy have been found anatomically in the visual system. Their functional role, so-called recurrent processing, is unaccounted for in this framework.

A recent study by Koivisto et al. (2011), published in The Journal of Neuroscience, set out to directly test whether recurrent processing was needed for natural scene categorization. The authors used transcranial magnetic stimulation (TMS) in combination with a rapid animal versus non-animal categorization task, a paradigm repeatedly used by Thorpe and colleagues as evidence for the role of feedforward processing for scene perception (e.g., Thorpe et al., 1996). Koivisto et al. demonstrated that V1/V2 activity continues to have an impact on the speed of categorization after the first feedforward sweep is over, which suggests that recurrent processing and/or sustained activity involving these early areas may be involved in the process of natural scene categorization. More specifically, a TMS pulse on V1/V2 with a 90 ms stimulus onset asynchrony (SOA = delay between image presentation and TMS pulse) significantly delayed the reaction time (RT), whereas a TMS pulse applied on the lateral occipital area (LO) did not affect RT for SOAs <150 ms; this is consistent with a sequential transfer of information from lower levels (V1/V2) to higher levels (LO) of the hierarchy. Somewhat less expected, pulses applied on V1/V2 at SOAs >90 ms also induced a delay in RT, as though activity in V1/V2 remained functionally important for categorization once the first round of computations was over.

Incidentally, in the Koivisto et al. study, median reaction times were far longer (between 550 and 600 ms) than those generally observed in rapid natural scene categorization tasks (between 350 and 450 ms). Thus, the responses that they recorded and analyzed were late responses, where an influence of recurrent processing might be expected, and was found (effect of a late TMS pulse on V1/V2). In any case, even if an effect was found on reaction times, the absence of any effect on accuracy suggests only a marginal role for recurrent processing in the categorization task itself.

An additional aspect of the Koivisto et al. study is that participants had to make a confidence rating for their responses (from 0 for a guess to 2 for a confident decision). These ratings can be seen as a quantitative measure of the subjective visibility. As expected, TMS made subjects less confident in their responses, possibly inducing a form of backward masking reducing subjective visibility, but not strong enough to significantly disturb the accuracy of the response. TMS of V1/V2 affected confidence at shorter SOAs than TMS of LO, which is compatible with a disruption of the first feedforward sweep. More interestingly, V1/V2 stimulation resulted in a larger disturbance of subjective visibility than LO stimulation, and the effect of TMS on V1/V2 was still seen at SOAs longer than for LO. While the effect size is rather small (maximum drop of ∼7% in the rating value at 120 ms), it is significant and there is a correlation between the level of suppression and the intensity of the pulse (illustrated in the paper at a SOA of 120 ms). All in all, this suggests that the activity in the primary visual cortex plays a functional role in subjective visibility, even after activity in LO has ceased being important. This activity could be caused by feedback from higher areas as well as by sustained activity in the early visual cortex. The sustained activity explanation might be supported by the single long effect of TMS observed in this study, in comparison to the two discrete periods reported in Camprodon et al. (2010). As a conclusion, even though this study failed to provide convincing evidence for the involvement of recurrent processing in fast categorization, it experimentally establishes a crucial link between recurrent/sustained activity in primary visual cortex and visual awareness.

The Koivisto et al. study is one of the rare experimental demonstrations in support of the theoretical works claiming an active role for V1 in visual awareness. Mumford (1991) was the first to suggest that a low-level visual area (the thalamus in his case) could be the center for visual awareness, functioning as an active blackboard for cortical reentrant activity. During the first feedforward pass, the visual signal is routed to different cortical areas for specific analysis by “experts.” Then, the massive feedback projection allows these experts to “write back” the results of their analyses on the thalamic blackboard, whose role will be to bind them and generate visual awareness. The lack of direct proof of such a role for the thalamus, as well as converging evidence for a similar mechanism in primary visual cortex, has led several authors to transfer the blackboard location to V1 (Lamme and Roelfsema, 2000; Bullier, 2001). However, even if Lamme and Roelfsema (2000) have argued that feedback activity was sufficient to give rise to awareness, the mechanisms by which the reinjection from high- to low-level visual areas could give rise to awareness are unknown.

Moreover, until recently, convincing experimental evidence for a functional role of feedback to early visual areas came only from the dorsal stream, more specifically from V1/MT interactions. Using TMS, it has been shown that activity in V1 after a TMS pulse over MT was critical for conscious perception of a moving phosphene (Pascual-Leone and Walsh, 2001). Hupé et al. (1998) showed that reversible inactivation of MT in anesthetized monkeys rapidly modulates the activity of V1 cells depending on figure–ground information. The authors hypothesized that this fast feedback was made possible by the fast-conducting axons of the magnocellular pathway, which could get information to the top of the hierarchy and reinject it just in time for the arrival of the parvocellular information. Interestingly, the magnocellular path does not project only to the dorsal stream, so there is no a priori reason to think that rapid feedback as observed by Hupé et al. could not be found in the ventral stream. However, experimental evidence for the role of reentrant activity to V1 in supporting conscious object recognition has been lacking. The Koivisto et al. study, together with a paper by Camprodon et al. (2010), are the first experimental attempts to test the implication of recurrent processing in the ventral stream for conscious object recognition. Now that microstimulation in monkeys has encountered a large success to document top-down effects reminiscent of attentional modulation (Moore and Armstrong, 2003), we hope that these encouraging results using TMS will trigger more experimental investigations to elucidate the functional role of feedback.