Visual temporal window of integration as revealed by the visual mismatch negativity event-related potential to stimulus omissions

Abstract

We studied whether, similarly to the auditory modality, short-period temporal integration processes occur in vision. Event-related potentials (ERP) were recorded for occasional stimulus omissions from sequences of patterned visual stimuli. A posterior negative component emerged only when the constant stimulus onset asynchrony (SOA) was shorter than 150 ms. This upper limit is comparable with the duration of the temporal window of integration observed in the auditory modality (including experiments studying the effects of stimulus omissions). Parameters of the posterior negativity were highly similar irrespective of whether the stimuli were task-relevant or not (Experiment 1). Thus, we identified this potential as the visual mismatch negativity (vMMN) component, which reflects task-independent detection of violating regularities of the stimulation. vMMN was followed by an anterior positivity (the P3a), indicating attentional shifts induced by the stimulus omissions. In Experiment 2, a posterior negativity similar to that observed in Experiment 1 emerged after the termination of short trains of stimuli, again only when the SOA was shorter than 150 ms. These results support the notion of a temporal integration window in the visual modality, the duration of which is between 150 and 180 ms.

Introduction

Although stimuli arrive continuously to our senses, extracting the features of external objects requires samples of some definite duration to ensure perceptual stability (temporal quantization). In audition, various analysis periods have been observed (Nelken et al., 2003, Poeppel, 2003), the most notable of which is the temporal window of integration (TWI) lasting for ca. 200 ms from the onset of isolated sounds. Basic sound analysis processes, such as establishing loudness or the temporal order between short sounds, appear to be bound to the TWI period, which is also regarded as the first form of auditory memory storage in the brain (for a review, see Cowan, 1984). Contrary to the auditory modality, some researchers doubt the ecological validity of short-term stores in vision (Haber, 1983; see, however, Coltheart, 1983). Using a visual analogue of a well-established auditory event-related potential (ERP) phenomenon, the omission response (Yabe et al., 1997, Yabe et al., 1998, Yabe et al., 2001), the current study tested whether short-period temporal integration processes can be observed in the visual modality.

The question was studied using the mismatch negativity (MMN) component of ERPs, which is elicited when a sound violates some acoustic regularity of the preceding sound sequence. MMN is elicited by simple acoustic deviations in a repetitive sequence of sounds (e.g., by infrequently delivering a sound whose pitch differs from that of the repeating one; the auditory oddball paradigm; e.g., Schröger and Winkler, 1995) as well as by violations of more complex and even abstract regularities (e.g., occasional stimulus repetitions within a sequence of two alternating sounds; for reviews, see Näätänen and Winkler, 1999, Picton et al., 2000). However, MMN is not elicited by any acoustic change (e.g., by a sound following a long silent period, Cowan et al., 1993) but only when some regularity has been previously extracted from the sound sequence (Polich, 2003). On the other hand, MMN is elicited whether a deviant event is task-related or -irrelevant and it is largely insensitive to attentional manipulations (Näätänen, 1990, Sussman et al., 2003).

In the auditory modality, the omission response has been linked with the TWI because omissions elicit the mismatch negativity (MMN) component of the event-related brain potentials (ERP) when they are embedded in a sound sequence presented with a less than 200-ms stimulus onset asynchrony (SOA; onset-to-onset interval), but not when the SOA exceeds the duration of the TWI (Näätänen, 1990, Yabe et al., 1997, Yabe et al., 1998). Comparable results were obtained using paired presentation of tones (Tervaniemi et al., 1994). Two explanations, both based on temporal integration phenomena, have been offered to explain the omission MMN. Yabe and his colleagues suggest that sounds arriving within the TWI period are treated as a single chunk, which, however, retains the temporal structure of the acoustic events (Yabe et al., 1997, Yabe et al., 1998). If the SOA of a sequence is shorter than the duration of TWI, two sounds appear regularly within the TWI. Omissions result in the formation of a “deviant temporal window”, which contains only one sound. This deviation elicits the MMN. With longer SOAs, all integration windows contain only a single sound and, therefore, omissions do not cause deviation. It should be noted that the representations of the sounds encountered within a single TWI period can be decomposed at higher levels of perceptual processing (Czigler et al., 2003, Sussman et al., 2002). According to the alternative interpretation of omission MMN (Oceák et al., 2006), a sound following another sound within the TWI terminates the integration of the aftereffect of the first sound. As a consequence, the perceived loudness of sounds followed by another sound within the TWI is lower than that of the sounds preceding an omission (Cowan, 1987). Thus, infrequent omissions in a sound sequence presented with SOAs that are shorter than the TWI elicit a loudness-deviation MMN. Since loudness summation is limited to the TWI, no such effect occurs when the SOA exceeds the duration of the TWI.

In recent years, a visual analog of the MMN has been described. Visual mismatch negativities (vMMN) were elicited by deviations in spatial frequency (Heslenfeld, 2003), color (Czigler et al., 2002, Czigler et al., 2004, Horimoto et al., 2002, Kimura et al., 2006), motion direction (Kremláèek et al., 2001, Kremláèek et al., 2006, Pazo-Alvarez et al., 2004a, Pazo-Alvarez et al., 2004b), shape (Maekawa et al., 2005, Tales et al., 1999, Tales et al., 2002), line orientation (Astikainen et al., 2004, Fu et al., 2003), location (Berti and Schröger, 2004) and conjunction of color and spatial direction (Winkler et al., 2005) (for a review, see Pazo-Alvarez et al., 2003). The existence of vMMN provides the possibility to investigate whether omitting a visual stimulus from a sequence elicits event-related brain responses analogous to the auditory-omission-related response. The emergence of vMMN with short but not long SOAs would be taken to suggest that the implicit memory system underlying the detection of regularity violations in sequences of visual stimuli may show a critical period for unit formation, similarly to the assumed operation of TWI in audition.

In Experiment 1, we adapted the method developed by Yabe and his colleagues. Sequences of non-attended visual stimuli were presented with a constant SOA, and the event-related responses elicited by infrequent omissions were analyzed. By testing several SOAs, we aimed at assessing the critical duration of the assumed temporal integration period. In the Detection task, the participants performed a task unrelated to the sequence of non-attended stimuli. The task was the detection (in an RT situation) of the infrequent thickness change of a centrally presented cross. In order to assess the possibility of attentional effects, in the Divided-attention task, the stimulus omissions became task-relevant, that is, participants silently counted the number of stimulus omissions. In Experiment 2, we investigated whether the termination of a train of visual stimuli elicits similar brain responses.