Subjective Experience of Motion or Attentional Selection of a Categorical Decision

“As the bus appeared round the corner, the passengers waiting at the station hailed it to stop.” This simple scenario provides many of the ingredients needed to discuss the subtle difficulties of understanding the relationship between subjective awareness and neural brain activity. The brain activity specifically correlated with this event could be the neural correlate of (1) “the bus appearing” (e.g., stimulus-evoked visual processing), (2) “hailing the bus” (e.g., response selection, motor planning, and execution), (3) subjective awareness of either 1 or 2, or, indeed, (4) a nonexclusive combination of any of the three. Therefore, Frith and colleagues (1999) suggested that to study the processes involved specifically in subjective awareness of a visual stimulus, one should make sure that the neural activity is neither a correlate of the physical changes in the stimulus nor a component of the behavior by keeping sensory stimulation and required behavior constant.

Using bistable stimuli (Leopold and Logothetis, 1999) is particularly useful for studying subjective awareness. A bistable stimulus provides conflicting or ambiguous information, giving rise to two different perceptual states, e.g., the face–vase illusion (Rubin, 1915) (Fig. 1A). This permits investigation of the neural correlates of the shifts of subjective awareness between two competing states while keeping the neural correlates of the visual input constant. The use of a bistable stimulus, however, does not automatically guarantee control on neural correlates of behavior. A comparison of two studies on neural correlates of subjective awareness of ambiguous apparent motion, one in macaque brain (Williams et al., 2003) and the other in human brain (Serences and Boynton, 2007b), demonstrates this point.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

A, Edgar Rubin's face–vase illusion. B, Apparent motion stimuli used by Williams et al. (2003). Each frame consisted of a regular grid of dots. Motion was created by displacing the entire grid by a fixed distance from one frame (t1) to another (t2). Any displacements smaller than one-half the distance (D) between two adjacent dots resulted in unambiguous motion (left). When the displacement was exactly equal to D/2 (right), direction of motion becomes bistable. C, Random moving dots used by Serences and Boynton (2007b). In the unambiguous displays (left), all dots moved coherently in one direction. In the ambiguous displays (right), direction of motion of each dot was chosen randomly, giving rise to no particular physical direction of motion.

Williams et al. (2003) studied the responses of direction-selective neurons to unambiguous and ambiguous motion in three different regions of the parietal cortex in the macaque brain. In each trial, the monkey was presented with two stimuli in succession: an initial bistable (hence, ambiguous) one followed by an unambiguous one (Fig. 1B). The monkey's task was to indicate, at the end of the trial, whether the two stimuli had the same or different directions. Crucially, no behavior was required from the monkey while observing the initial ambiguous stimulus. Moreover, the experimental design ensured that the perception of motion direction during the ambiguous interval was not influenced by selective attention to one particular direction of motion. Neurons in middle temporal (MT) and medial superior temporal (MST) areas showed strong stimulus-specific responses to unambiguous motion but little (MST) or no (MT) percept-specific responses to bistable motion. On the other hand, neurons in lateral intraparietal area (LIP) showed a modest stimulus-specific but a much stronger percept-specific response: the activity level of the LIP neurons at the onset of the bistable stimulus reflected the monkey's subsequent choice. Williams et al. (2003) concluded that LIP activity reflected the monkeys' subjective perception of motion direction.

Addressing a similar question in the human brain, Serences and Boynton (2007b) used functional magnetic resonance imaging (fMRI) to measure brain activity of human observers while they reported the direction of ambiguous (i.e., incoherent) random dot stimuli (Fig. 1C). They used a recently developed method for multivariate pattern decoding (MVPD) of the blood oxygenation level-dependent (BOLD) signal (Haynes and Rees, 2005; Kamitani and Tong, 2005). MVPD differs from conventional fMRI analysis in that activity within a brain region is not quantified; instead, the focus is on the pattern of signal in a population of voxels and its ability to predict experimental conditions. If a combination of voxels has a similar pattern of activation at two discrete time points, these points are classified as belonging to the same condition. The ability to predict one versus another experimental condition significantly above chance performance implies that the underlying brain activation patterns are distinguishable and therefore feature (e.g., orientation, direction of motion) selective. MVPD has also been demonstrated to probe selective attention to features (Kamitani and Tong, 2006; Serences and Boynton, 2007a), which was formerly believed to be beyond the spatial resolution of fMRI.

Serences and Boynton (2007b) used MVPD to dissociate the stimulus-evoked and the subjective percept-specific neural correlates of motion perception by decoding the direction of motion of the unambiguous (fully coherent) and ambiguous (incoherent) random dot stimuli, respectively. In two-thirds of trials, dots moved coherently, and the direction of motion was unambiguous (Fig. 1C, left); in the remaining trials, the dots moved randomly, so the direction was ambiguous (Fig. 1C, right). Observers were instructed to monitor motion direction in all trials. The direction of unambiguous motion could be classified above chance level based on BOLD signal in early visual areas V1–V3, motion sensitive area hMT+, and intraparietal sulcus (IPS), but not frontal eye fields. These results replicate previous reports of direction-selective neural processes in these areas using the same technique (Kamitani and Tong, 2005). In addition, the results from IPS parallel previous reports of direction selectivity in the homologous regions of macaque brain (Shadlen and Newsome, 2001; Williams et al., 2003).

To look for percept-specific neural processes, Serences and Boynton (2007b) asked whether and in which areas MVPD could classify the attributed direction of ambiguous motion, i.e., for each trial, predict the label the observer assigned to the zero-coherence display. Their results showed that successful classification of ambiguous motion was possible only in area hMT+ but not in earlier retinotopic areas V1–V3 and, importantly, not in IPS. They concluded that “… it is ultimately the activation pattern in hMT+ that is read out to generate the perceptual experience of motion” (Serences and Boynton, 2007b).

Both studies summarized above (Williams et al., 2003; Serences and Boynton, 2007b) claim to have identified the neural correlates of subjective experience of motion in different species. The study by Williams et al. (2003) points to areas LIP and MST in the macaque brain, whereas the Serences and Boyton (2007b) study underlines area hMT+ in the human brain. Serences and Boynton (2007b) point out that it remains elusive why hMT+ but not IPS would encode conscious perceptual choice for motion direction in the human brain. Given the assumption that area hMT+ in the human brain comprises the homologues of both MT and MST of the macaque brain and considering the possibility that human IPS [as defined by Serences and Boynton (2007b)] may not be the exact homolog of macaque LIP, one might argue that the two works show enough similarities to overlook the different results.

Applying the framework suggested by Frith et al. (1999) as described above, however, shows an interesting difference: if one compares the criteria for identifying neural correlates of motion direction perception in these studies, one realizes that both controlled for stimulus-evoked processes, but Serences and Boynton (2007b) did not control for behavior. The monkeys were not required to respond to the ambiguous motion but only to decide whether a subsequent unambiguous motion matched it or not. Thus, the monkeys' behavioral state at the end of the ambiguous motion interval was invariant in relation to the response categories (match vs mismatch) and, importantly, was orthogonal to the perceived direction of the bistable stimulus (Williams et al., 2003). The human observers had to report the motion direction whenever there was a slight change in the speed of motion, which happened three to four times during each trial. They were instructed that there was some coherent motion in the ambiguous displays too, similar to unambiguous ones. Therefore, one may argue that in the ambiguous condition, subjects chose a direction (i.e., one of the two available decision categories) and actively attended to that direction while awaiting the moment of speed change. Thus, a strong attentional bias for the chosen direction of motion would be expected in the ambiguous trials. Attention provides a link between the stimulus features and the behaviorally relevant decision categories in extrastriate cortex (Mirabella et al., 2007). As such, unlike the study by Williams et al. (2003), the design used by Serences and Boynton (2007b) does not distinguish between the attended and perceived direction of motion, because it does not control for neural correlates of behavior/decision.

The MVPD method can accurately decode the attended direction of motion in area hMT+ (Kamitani and Tong, 2006), even in the complete absence of stimulation (Serences and Boynton, 2007a). Hence, it is conceivable that MVPD applied to BOLD activation induced by the zero-coherence moving dot display by Serences and Boynton (2007b) might have decoded the attended (rather than the perceived) direction of motion, replicating the earlier findings rather than indicating the neural correlates of subjective perception. Future research on this topic is necessary to clarify this question.