Abstract
Normal visual perception readily overcomes suboptimal or degraded viewing conditions through perceptual filling-in processes, enhancing object recognition and discrimination abilities. This study used visual evoked potential (VEP) recordings in conjunction with electrical neuroimaging analyses to determine the spatiotemporal brain dynamics of boundary completion and shape discrimination processes in healthy humans performing the so-called “thin/fat” discrimination task (Ringach and Shapley, 1996) with stimuli producing illusory contours. First, results suggest that boundary completion processes occur independent of subjects' accuracy on the discrimination task. Modulation of the VEP to the presence versus absence of illusory contours [the IC effect (Murray et al., 2002)] was indistinguishable in terms of response magnitude and scalp topography over the 124–186 ms poststimulus period, regardless of whether task performance was correct. This suggests that failure on this discrimination task is not primarily a consequence of failed boundary completion. Second, the electrophysiological correlates of thin/fat shape discrimination processes are temporally dissociable from those of boundary completion, occurring during a substantially later phase of processing (∼330–406 ms). The earlier IC effect was unaffected by whether the perceived contour produced a thin or fat shape. In contrast, later time periods of the VEP modulated according to perceived shape only in the case of stimuli producing illusory contours, but not for control stimuli for which performance was at near-chance levels. Collectively, these data provide further support for a multistage model of object processing under degraded viewing conditions.
- illusory contour
- visual evoked potential
- VEP
- event-related potential
- ERP
- shape discrimination
- object recognition
- binding
Introduction
Object perception is a robust phenomenon that withstands degraded visual conditions (e.g., poor illumination, occlusion, and scotoma). Because of the two-dimensional nature of the retinal projection, such impediments produce object images with discontinuous boundaries or boundaries where no luminance difference is physically present. In experimental settings, these conditions have been reproduced with illusory contour (IC) stimuli, which give the perception of borders or surfaces that span inducing elements across regions of homogenous luminance (see Fig. 1).
Stimuli and behavioral results. Examples of IC, NC, and RC stimuli leading to thin and fat classifications are shown (see Materials and Methods for additional details). The bar graph indicates the mean performance accuracy (SEM indicated) on the thin/fat task at inducer rotations of 6 and 1° (solid and stippled bars, respectively) and for each contour type collapsed across thin and fat configurations (see Results for details).
Several experimental paradigms demonstrate how perceptually “filling in” incomplete visual information enhances shape discrimination. One had subjects indicate whether line segments had larger displacement along the horizontal or vertical axis relative to central fixation while varying whether boundary completion of the segments could occur (Murray et al., 2001). Another, the so-called “thin/fat” task, introduced by Ringach and Shapley (1996) and the focus of the present study, had subjects classify configurations of arrays of “pacman” inducer stimuli that either would (IC) or would not [no-contour (NC)] produce an illusory contour (see Fig. 1). During this task, boundary completion results in the perception of either elongated (thin) or extended (fat) IC squares. On both paradigms, performance significantly improved when boundary completion could occur, despite neither task explicitly necessitating selective processing of these IC shapes (Gold et al., 2000). Neuropsychological evidence from patients with visual neglect (Vuilleumier and Landis, 1998; Vuilleumier et al., 2001) or extinction syndromes (Mattingley et al., 1997) also suggests that filling-in processes occur preattentively, with patients benefiting from IC presence on visuospatial tasks despite their unawareness of IC presence. It is clear that filling-in and boundary completion processes pervade normal viewing and can assist organisms in (among other things) more successfully navigating their environments and detecting predators, prey, and obstacles. Still, the neurophysiological relationship between these processes and those of shape discrimination remains unresolved.
The present study details the relative timing and neural mechanisms of filling-in and shape discrimination. First, we investigated whether perceptual filling-in processes are automatic. Second, we assessed whether boundary completion and shape discrimination processes are dissociable. Visual evoked potentials (VEPs) were recorded from subjects performing Ringach and Shapley's (1996) task at two levels of discrimination difficulty. Importantly, this discrimination need not itself be dependent on contour perception. Three types of stimuli were used: outwardly-turned inducers not forming illusory contour percepts (NC), aligned inducers forming illusory contour shapes (IC), and aligned inducers framing luminance-defined surfaces [real contour (RC)]. We reasoned that if boundary completion processes are indeed “automatic,” then a differential response to the presence versus absence of illusory contours (the IC effect) should be observed regardless of whether the stimulus is ultimately classified accurately. That is, the IC effect should be observed for both correct and incorrect trials. Second, if shape discrimination is indeed dissociable from boundary completion, then neurophysiologic effects of the thin/fat discrimination should follow the IC effect and/or engage distinct brain networks, the latter of which would manifest as a topographic change in the electric field at the scalp.
Materials and Methods
Subjects
Eleven (eight female) neurologically normal, paid volunteers, 19–41 years of age (mean ± SD, 23.6 ± 7.0) participated. All but one were right-handed (Oldfield, 1971). All had normal or corrected-to-normal vision and provided written consent to the procedures as approved by the Institutional Review Board of the Nathan S. Kline Institute.
Stimuli and procedure
For each trial, an array of four Kanizsa-type (Kanizsa, 1976) “pacman” inducers was presented on a computer monitor (Eizo Flexscan 930; 1280 × 1024 pixel resolution) located 75 cm away from the subject. These circular inducers had a radius of 1.8° visual angle, were separated by 13° (center-to-center distance), and appeared at 9.2° eccentricity from central fixation. Subjects were instructed to centrally fixate a dot during stimulus presentation and to perform a two-alternative forced-choice discrimination between two classes of stimuli (details of this classification are provided below). There were a total of 12 possible stimulus configurations, yielding a 3 (contour type) × 2 (classification type) × 2 (level of difficulty) within-subjects design. Contour type could be illusory, real, or no-contour.
The two classes of stimuli were operationally defined by the rotation of the pacmen inducers in one of two directions, with the pacmen diagonally across from each other being rotated in the same direction and those along the other diagonal always being rotated by the same amount in the opposite direction. Using the conventions introduced by Ringach and Shapley (1996), α refers to the angle of rotation of the top left pacman in the stimulus array. In what will be referred to as “thin” classifications (independent of the type of contour), α is positive, whereas for “fat” classifications, α is negative (see Fig. 1). The terms thin and fat were at no point supplied to subjects. Nor were subjects explicitly directed to note the rotation of the inducers. Rather, subjects were presented with a card bearing visual exemplars of stimulus configurations that fell into one group (meriting a left-button press) or the other group (right-button press). We use the terms thin and fat here across all types of contour, including NCs, in the interest of brevity. We used α values of ±1 and ±6°, thereby testing subjects at two levels of difficulty (with 1° being the more difficult to discriminate).
ICs followed from the approximate alignment of the “mouths” of the pacmen inducers, which in turn led to the perception of elongated or extended squares with slightly curved sides. Real shapes were identical to those produced by the IC stimuli, except that the shapes' surfaces were explicitly defined by a luminance decrement superimposed on the lighter-gray background. Inducers were present in the RC condition so that they appeared as white circles behind the dark gray RC surface. The no-contour condition was identical to the IC condition except that inducers were rotated 180° ±α, which resulted in the “mouths” of the pacmen facing outwards to prevent the induction of an illusory surface.
Mean background luminance was 70.5 cd/m2, whereas mean luminance of the inducers was 88.7 cd/m2. In the RC condition, the real surface had a mean luminance of 59.1 cd/m2. For the IC condition, the support ratio was fixed so that 27% of the illusory shape's perimeter was physically indicated by the presence of the inducers.
Each trial was preceded by the presentation of a central fixation point for 700 ms. One of the 12 possible stimuli described above then appeared for 168 ms (the central fixation point remained present). After an interstimulus interval of 700 ms, subjects were then presented with a cue, which prompted them to press either the left or right button of the response pad to indicate their classification of the inducer array. A paper card illustrating examples of IC, NC, and RC stimuli that would result in a left versus right button press was available to subjects for their reference throughout the experiment. No accuracy feedback was provided during the experimental trials. We instructed subjects not to respond until the prompt appeared, to minimize the contribution of motor responses to the sensory VEP. In this manner, subjects also controlled stimulus delivery and could thus concentrate on the accuracy of their discrimination. Consequently, only accuracy data and not reaction times were available for statistical analyses.
A practice block, with accuracy feedback, was completed before the experimental session to acclimate subjects to the task and stimuli. The experimental session consisted of at least 28 blocks of trials (mean ± SD, 32.9 ± 2.5), each block containing 96 stimuli with equal probability of each of the 12 possible stimuli within a block. Subjects were encouraged to take breaks between blocks to maintain high concentration and prevent fatigue.
EEG acquisition and analyses
Continuous 128-channel EEG was acquired through Neuroscan Synamps (Neuroscan, El Paso, TX), referenced to the nose (impedances <5 kΩ), bandpass filtered at 0.05–100 Hz, and digitized at 500 Hz. Analyses were conducted using Cartool software (http://brainmapping.unige.ch/cartool.php). Peristimulus epochs (−150 to 700 ms) were averaged from each subject separately for each condition to compute the VEP. Baseline was defined as the 150 ms prestimulus period. Trials with blinks or eye movements were rejected off-line, using horizontal and vertical electrooculogram. An artifact criterion of ± 60 μV (n = 9) or ± 80 μV (n = 2) was applied at all other electrodes. Data from artifact electrodes from each subject and condition were interpolated (Perrin et al., 1987). Each VEP was then down-sampled to a common 111-channel montage, 40 Hz low-pass filtered, and recalculated against the average reference before group averaging.
Two separate series of analyses were conducted to address each hypothesis detailed in the Introduction. To test the question of the automaticity of illusory contour processes, EEG epochs were sorted according to the accuracy of each subject's performance. This generated a 2 contour type (IC and NC) × 2 accuracy level (correct and incorrect) design. The average number of accepted EEG epochs per VEP for this analysis was 328 ± 105. To test the temporal independence of illusory contour and shape discrimination processes, EEG epochs in response to IC and NC contour types were sorted according to thin and fat configurations. This generated a 2 contour type (IC and NC) × 2 configuration (thin and fat) design. The average (±SD) number of accepted EEG epochs per VEP for this analysis was 353 ± 93. Analyses of the RC condition were included here to register the timing of thin/fat shape discrimination effects when boundary completion processes need be involved (i.e., for ICs) relative to when it is not required (i.e., for RCs). The average (±SD) number of accepted EEG epochs per condition for this analysis was 347 ± 96. The luminance difference between RC and IC/NC stimuli hampers our ability to interpret any VEP differences as following from functional differences rather than physical attributes. Therefore, the RC stimuli were analyzed separatelya. EEG data were pooled across both levels of difficulty (i.e., α ± 1° and α ± 6°), because preliminary analyses revealed difficulty-related differences for IC stimuli only after 500 ms poststimulus onset and failed to reveal such differences for NC stimuli.
For all series of analyses, the initial 500 ms poststimulus period of the VEPs for each condition was submitted to two independent analyses of the electrical field at the scalp. The methods applied here have been described in detail previously (Murray et al., 2004a,b; Foxe et al., 2005) and have several advantages over canonical waveform analyses (for review, see Michel et al., 2004a). For example, the reference-dependent nature of event-related potential waveform data means that statistical outcomes will change with the choice of the reference electrode (Lehmann, 1987; Dien and Santuzzi, 2005). Additionally, such analyses often require the preselection of time windows and scalp sites that will be submitted to statistical tests. Such a procedure unnecessarily introduces a degree of experimenter bias. Nonetheless, to minimize the possibility of missed effects (type II errors), we further analyzed waveform data from all electrodes as a function of time poststimulus onset in a series of pair-wise comparisons (t tests). Correction was made for temporal auto-correlation at individual electrodes (Guthrie and Buchwald, 1991) through the application of a 20 ms temporal criterion for the persistence of differential effects. This type of analysis has been reported previously in the study of illusory contour processes (Murray et al., 2002, 2004b) and in other studies (Molholm et al., 2002; Murray et al., 2004a, 2006; De Santis et al., 2006) and has been referred to as a statistical cluster plot (SCP). The x-, y-, and z-axes of these SCPs, respectively, represent time (poststimulus onset), electrode location, and the t test result (indicated by a color value) at each data point. We would emphasize that although these analyses give a visual impression of specific effects within the dataset, our conclusions are based solely on global measures of the electric field at the scalp and multifactorial analyses thereof. To provide the reader with a further sense of the correspondence between these two levels of data analysis (i.e., at individual electrodes and at global levels), we also analyzed area measures from VEP waveforms from individual scalp locations (i.e., the integral under the waveform as a function of time vs the 0 μv baseline).
The first analysis of the electric field at the scalp was a topographic pattern (i.e., map) analysis. Maps were compared over time within and between conditions, because topographic changes indicate differences in the active generators of the brain (Lehmann, 1987). This method is independent of the reference electrode and is insensitive to pure amplitude modulations across conditions (because topographies of normalized maps are compared). This method uses a k-means cluster analysis to determine, at the level of the group-average dataset, the predominant topographies as a function of time and experimental condition (Pascual-Marqui et al., 1995). The Appendix provides, in more general terms, a description of this analysis. A modified cross-validation criterion determined the optimal number of maps that explained the whole group-averaged dataset (Pascual-Marqui et al., 1995). This criterion is the ratio of the quantity of 1 minus the global explained variance divided by the degrees of freedom squared. In more intuitive terms, this criterion balances the additional variance explained by identifying more maps within a dataset against the consequent loss in degrees of freedom. The pattern of maps observed in the group-averaged data were statistically tested by comparing each of these maps with the moment-by-moment scalp topography of individual subjects' VEPs from each condition. Each time point of each VEP from each subject was labeled according to the map with which it best correlated spatially [spatial correlation has been defined by Brandeis et al. (1995)]. This method determines whether a given experimental condition is more often described by one map versus another and, therefore, whether different generator configurations better account for particular experimental conditions. In addition to testing for modulations in the electric field topography across conditions, this analysis also provides a more objective means of defining VEP components. That is, we here define a VEP component as a time period of stable scalp topography.
The second analysis used the instantaneous global field power (GFP) for each subject and stimulus condition to identify changes in electric field strength. GFP is equivalent to the spatial SD of the electric field at the scalp (Lehmann and Skrandies, 1980). The observation of a GFP modulation does not exclude the possibility of contemporaneous changes in the electric field topography or topographic modulations that nonetheless yield statistically indistinguishable GFP values. However, a GFP modulation without simultaneous topographic changes is most parsimoniously explained as amplitude modulation of statistically indistinguishable generators. GFP area measures were calculated (vs the 0 μV baseline) and submitted to a repeated measures ANOVA, using the within-subjects factors detailed above.
We estimated the sources in the brain underlying the VEPs from each condition, using the Local Auto-Regressive Averages (LAURA) distributed linear inverse solution (Grave de Peralta et al., 2001, 2004) (for a comparison of inverse solution methods, see Michel et al., 2004a). Like other minimum norm inverse solutions, LAURA is capable of dealing with multiple simultaneously active sources of a priori unknown location and makes no assumptions regarding the number or location of active sources. Like weighted minimum norm approaches, LAURA is a method for compensating for the tendency of minimum norm solutions to favor superficial over deep sources. In contrast with other approaches, LAURA incorporates a priori information that is based on biophysical constraints rather than purely mathematical operators (for full details and formulas, see Grave de Peralta et al., 2001, 2004). More specifically, LAURA applies the electromagnetic theory described in the Maxwell equations to describe the relationship between the brain activity at one point and its neighbors, because surface and intracranial EEG are generated by irrotational currents and are hence ohmic (Plonsey, 1982). This is expressed in terms of a local autoregressive estimator with coefficients depending on a power of the distance from a given point. In the present study, we used the cubic power of the inverse distance. In this way, LAURA selects the source configuration that best mimics the biophysical behavior of electric vector fields and produces a unique estimator of the current source density vector inside the brain. The solution space (i.e., the lead field matrix) was calculated on a realistic head model that included 4024 nodes, selected from a 6 × 6 × 6 mm grid of voxels equally distributed within the gray matter of the average brain of the Montreal Neurological Institute (MNI). In the present study, homogenous regression coefficients in all directions and within the whole solution space were used. The results of the above analyses defined time periods for which intracranial sources were estimated, providing visualizations of the likely underlying sources.
Results
Behavioral results
The percentage of correct classifications was recorded for each of the 12 stimulus types. These values were submitted to a 3 (contour type) × 2 (classification type; α > 0/α < 0) × 2 (difficulty level) repeated measures ANOVA. All reported statistics reflect Greenhouse–Geisser corrections where data did not satisfy criteria for sphericity.
There was a significant main effect of contour type (F(2,20) = 33.2; p < 0.0001), reflecting subjects' superior accuracy with RC (mean ± SEM, 84.9 ± 2.5%) versus IC (78.3 ± 2.9%) versus NC (56.2 ± 3.9%) stimuli. Follow-up, planned comparisons (paired t tests) confirmed this graded difference across stimulus types (RC vs IC, t(10) = 3.4, p < 0.005; RC vs NC, t(10) = 7.3, p < 0.0001; IC vs NC, t(10) = 4.8, p < 0.001). Consistent with previous research (Ringach and Shapley, 1996), we found that the presence of illusory contours and real contours is beneficial to task performanceb. The main effect of classification type was not significant (F(1,10) = 0.02; p = 0.90), indicating that subjects were equally accurate at discriminating thin and fat stimulus configurations, which were produced by inducer rotations of α > 0 and α < 0, respectively. Thus, electrophysiological differences between thin and fat classifications cannot be explained by differences in discrimination accuracy for the two classes of stimuli. The main effect of difficulty was also significant (F(1,10) = 95.8; p < 0.0001), indicating that subjects responded more accurately to stimuli with α = ±6° (81.5 ± 2.7%) than to α = ±1° (62.2 ± 2.5%). There was a significant interaction between contour type and difficulty level (F(2,20) = 6.6; p < 0.015). Follow-up comparisons revealed that difficulty significantly affected performance with ICs (88.8 ± 3.4% for α = ±6° vs 64.2 ± 2.9% for α = ±1°; t(10) = 9.6; p < 0.0001) and RCs (97.0 ± 1.2% for α = ±6° vs 70.3 ± 4.3% for α = ±1°; t(10) = 7.2; p < 0.0001) but had no effect on subjects' performance with NC stimuli (58.9 ± 6.5% for α = ±6° vs 52.1 ± 1.3% for α = ±1°; t(10) = 1.3; p > 0.20). No other interactions reached the 0.05 significance criterion. Figure 1 displays the overall percentage correct for each combination of stimulus type and level of difficulty, collapsed across thin and fat classification types.
Electrophysiological results
The automaticity of boundary completion: correct versus incorrect trials
The first level of analysis entailed a series of four pair-wise contrasts for correct and incorrect trials and also for IC and NC stimuli. The results of these contrasts are shown as SCPs (Fig. 2). Sensitivity to the presence versus absence of illusory contours (i.e., the IC effect) was observed over the 140–170 ms period when performance was ultimately correct and also when performance was incorrect (Fig. 2, top). In contrast, no reliable differences were observed for either IC or NC stimuli as a function of performance accuracy during the initial ∼300 ms, although effects on IC processing were observed at later time periods (Fig. 2, bottom). This pattern is also readily apparent after visual inspection of the group-averaged VEP and GFP waveforms (Fig. 3).
Statistical cluster plots. Color values indicate the result of point-wise t tests across poststimulus time (x-axis) and electrode (y-axis) positions. For clarity, only p values ≤0.01 are color encoded. Top, Results from IC versus NC contrasts for trials leading to correct and incorrect thin/fat discrimination (left and right panels, respectively). Bottom, Results from contrasts of correct versus incorrect trials for IC and NC stimuli (left and right panels, respectively).
Electrophysiological results illustrating the automaticity of illusory contour processes. a, Isocontour voltage maps depict stable electric field topographies obtained in the pattern analysis performed on the group-averaged data (period poststimulus indicated; left hemiscalp on left and nose upwards). b, Group-averaged VEP waveforms in response to IC and NC stimuli that lead to correct and incorrect behavior (color scheme indicated) from selected scalp sites. c, d, GFP waveforms and area measures over the 124–186 ms period from each stimulus condition (identical color scheme as in b). Arrows indicate sensitivity to illusory contours both when performance on the task was correct and incorrect. The asterisk indicates a significant main effect of contour type on GFP area (see Results for details). e, GFP difference waveforms show the IC effect for correct and incorrect trials (black and gray traces, respectively).
Two analyses of the global electric field at the scalp were then conducted to identify and compare the likely neurophysiological basis of illusory contour sensitivity both when performance was ultimately correct and also when it was incorrect. The first of these analyses, the spatiotemporal topographic pattern analysis, provided no indication of distinct maps for any the four stimulus conditions. Rather, VEPs from all conditions showed the same sequence of scalp topographies. Six different scalp topographies accounted for the collective 500 ms poststimulus periods across all conditions in this analysis. The global explained variance (GEV) of these maps was 97.26%. Different maps were observed over the 0–66, 68–122, 124–186, 188–250, 252–328, and 330–500 ms periods, and the same sequence was observed in each condition (Fig. 3a). These time periods and their topographies correspond well with previously described VEP components observed at the waveform level [e.g., P1, N1, P2 (Doniger et al., 2001; Murray et al., 2002, 2004b; Foxe et al., 2005)]. For example, the 68–122 ms period was characterized by a parieto-occipital positivity prototypical of the P1 component, and the 124–186 ms period was characterized by a parieto-occipital negativity prototypical of the N1 component (Martinez et al., 1999). These time periods of stable scalp topography were then used to define time windows for the analyses of GFP area measures (i.e., the integral under the GFP waveform as a function of time vs the 0 μv baseline), with the reasoning that periods of stable scalp topography are a more objective means for defining VEP components than the preselection of time windows (see Materials and Methods).
Over the 124–186 ms period, a 2 × 2 repeated measures ANOVA revealed a significant main effect of contour type (IC vs NC; F(1,10) = 18.1; p < 0.002), indicative of the larger GFP responses to IC than to NC stimuli; the IC effect). This IC effect is highly consistent with numerous previous studies demonstrating sensitivity to illusory contours during the period of the N1 component (Murray et al., 2002, 2004b; Halgren et al., 2003; Kruggel et al., 2001). Follow-up comparisons revealed that the IC effect was present both for trials leading to correct discriminations (t(10) = 3.26; p < 0.009) and also for those leading to incorrect discriminations (t(10) = 4.08; p < 0.002). Neither the main effect of accuracy nor the interaction between factors reached significance (in both cases, p > 0.45). In addition to GFP area, we likewise tested GFP peak latency over the 124–186 ms period. Neither main effect nor their interaction reached significance (in all cases, p > 0.25). Mean peak latency was 166.7.0 ± 4.9, 165.8 ± 5.3, 165.6 ± 4.7, and 173.8 ± 6.6 ms for the IC correct, IC incorrect, NC correct, and NC incorrect conditions, respectively. Over the ensuing 188–250 ms period, there were no significant main effects or interaction. Over the 252–328 ms period, there was again a main effect of contour type (F(1,10) = 4.73; p < 0.05), indicative of the larger GFP for IC than for NC stimuli. Over the 330–500 ms period, there was a main effect of accuracy (F(1,10) = 6.56; p < 0.03), with stronger GFP when performance was correct. No other main effects or interactions reached the 0.05 significance criterion.
To minimize the possibility of type II errors, we also analyzed area measures from a subset of 16 scalp locations (corresponding to P5, P6, P7, P8, PO5, PO6, PO7, PO8, F1, F2, F3, F4, AF1, AF2, AF3, and AF4) that were chosen based on the scalp topography over the 124–186 ms period (Fig. 3). Area measures from this time period were submitted to a 2 contour type (IC and NC) × 2 accuracy (correct and incorrect) × 2 scalp region (parieto-occipital vs frontal) × 2 hemiscalp × 4 electrode repeated measures ANOVA, and the results of multivariate tests are reported. There was a significant main effect of contour type (F(1,10) = 14.23; p < 0.004), indicative of the larger response to IC than to NC stimuli. Unsurprisingly, the main effect of scalp region was significant (F(1,10) = 46.00; p < 0.0001), indicative of the polarity inversion across the scalp. That is, both polarities of the electric field at the scalp are sampled by the electrodes included in this analysis, which is unsurprising, because the summed voltage across the scalp is zero at each instant in time (i.e., the definition of the average reference). There was also a main effect of hemiscalp (F(1,10) = 9.22; p < 0.013), indicative of the generally larger responses over the right versus left hemiscalp [for similar VEP findings, see Murray et al. (2002) and for comparable functional magnetic resonance imaging (fMRI) evidence, see Martuzzi et al. (2006)]. No other main effects reached our 0.05 significance criterion. Of the interactions, there was a significant interaction between contour type and scalp region (F(1,10) = 54.52; p < 0.0001), indicating that effects of contour type were larger over parieto-occipital versus frontal scalp sites, and between scalp region and hemiscalp F(1,10) = 6.05; p < 0.04), indicative of the bigger left-right difference at posterior versus frontal electrodes. Last, there was a significant interaction between contour type, accuracy, and hemiscalp (F(1,10) = 12.45; p < 0.005). Additional ANOVAs were therefore conducted as a function of hemiscalp to better isolate the basis for this interaction. For measurements from the left hemiscalp, the interaction between contour type and accuracy failed to meet our significance criterion; whereas for measurements from the right hemiscalp, there was a significant interaction between contour type and accuracy (F(1,10) = 12.73; p < 0.005). This followed from a larger IC effect when performance was incorrect rather than correct, although we would hasten to note that the IC effect was present both when performance was ultimately correct and incorrect (Fig. 2).
Source estimations from the 124–186 ms period were obtained in the following manner. For each subject and condition, VEP data were averaged across the 124–186 ms time period to generate a single time point of data. The LAURA inverse solution for these data (11 subjects × 4 conditions) was estimated for each of the 4024 nodes in the solution space (i.e., the gray matter of the MNI brain). These source estimations were then averaged across subjects for each of the four conditions (Fig. 4). Each condition yielded activity within bilateral lateral-occipital regions, consistent with previous studies of completion processes (Murray et al., 2002, 2004b; Pegna et al., 2002; Foxe et al., 2005). Stronger current densities were obtained for the IC versus NC stimuli both when performance was correct and incorrect. This can be seen in the mean difference of the source estimations.
Group-averaged LAURA distributed source estimations over the 124–186 ms period for both IC and NC stimuli as a function of performance accuracy on the thin/fat task as well as their difference (sagittal views of the left and right hemispheres are shown).
Temporal dissociation of filling-in and shape discrimination
As described in Materials and Methods, trials were also grouped according to thin versus fat inducer orientations (i.e., positive and negative α values) for both IC and NC stimuli. As above, two analyses of the global electric field at the scalp were conducted (given the high correspondence between results observed within statistical cluster plots and the results of multifactorial analyses of global measures, we display only the latter). The first, the spatiotemporal topographic pattern analysis, provided no indication of distinct maps for any stimulus condition. Rather, VEPs from all conditions showed the same sequence of scalp topographies. Seven different scalp topographies accounted for the collective 500 ms poststimulus periods of all conditions in this analysis (GEV, 97.54%). Different maps were observed over the 0–66, 68–122, 124–186, 188–254, 256–328, 330–406, and 408–500 ms periods in each condition (Fig. 5a). These time periods of stable scalp topography were then used to define time windows for the GFP analysis.
Electrophysiological results illustrating the temporal dissociation of illusory contour and shape discrimination processes. a, Isocontour voltage maps depict stable electric field topographies obtained in the pattern analysis performed on the group-averaged data (period poststimulus indicated; left hemiscalp on left and nose upwards). b, Group-averaged VEP waveforms in response to IC and NC stimuli that produce thin and fat shapes (color scheme indicated) from selected scalp sites. c, GFP waveforms (identical color scheme as in b). d, GFP difference waveforms illustrate the time course of differential responses to thin and fat classification types for IC and NC stimuli (black and gray traces, respectively). e, f, GFP area measures over the 124–186 and 330–406 ms periods from each stimulus condition (identical color scheme as in b). The arrows indicate that sensitivity to illusory contours precedes sensitivity to thin/fat shapes. The asterisks indicate significant effects (see Results for details).
A 2 × 2 repeated measures ANOVA using GFP area over the 124–186 ms period revealed a significant main effect of contour type (IC vs NC; F(1,10) = 15.56; p < 0.003), indicative of the larger GFP responses to IC than NC stimuli, the IC effect (Fig. 5). As would be expected, follow-up comparisons revealed that the IC effect was present both for trials producing thin configurations (t(10) = 3.126; p < 0.01) and for those producing fat configurations (t(10) = 3.55; p < 0.005). Neither the main effect of configuration nor the interaction between factors reached significance criterion (p > 0.15). This IC effect is identical to that described above when data were sorted according to performance accuracy. Over the ensuing 188–254, 256–328, and 408–500 ms periods, there were no significant main effects or interactions. Over the 330–406 ms period, there was a significant interaction between factors of stimulus type and classification type (F(1,10) = 11.85; p < 0.006). Follow-up comparisons revealed that there was a significant difference between classification types (i.e., thin and fat orientations of the inducers) for IC stimuli (t(10) = 3.90; p < 0.003) but not for NC stimuli (t(10) = −1.50; p > 0.15). This can also be observed at individual scalp sites (Fig. 5b). This pattern mirrors the behavioral findings, wherein subjects performed the thin/fat task at near-chance levels for NC stimuli (Fig. 1). To more directly relate GFP response modulations over this time period to task performance, we correlated the extent of performance improvement when IC stimuli were present versus absent (i.e., the difference in performance accuracy on IC vs NC trials collapsed across thin and fat classification types) with the corresponding GFP difference (i.e., the difference between IC and NC stimuli after collapsing across thin and fat classification types). In other words, we related the performance benefit obtained with IC stimuli to the equivalent GFP difference in the VEPs. There was a significant negative correlation between these variables (r(9) = −0.611; p = 0.046) (Fig. 6), indicating that larger performance improvements were paralleled by smaller GFP differences. This result thus provides an additional level of support to the notion that VEP modulations over this time period are related to shape discrimination performance with degraded visual stimuli, a point to which we return in the Discussion. Last, source estimations from the 330–406 ms period were obtained as above and are shown in Fig. 7. Each condition yielded activity within bilateral lateral-occipital regions that extended anteriorly into the lateral temporo-occipital junction and superior temporal sulcus, consistent with previous studies of shape discrimination processes (Foxe et al., 2005; Sehatpour et al., 2006).
Correlation between performance improvement on the thin/fat task with IC presence and the magnitude of modulations in GFP area over the 330–406 ms interval (see Results for details).
Group-averaged LAURA distributed source estimations over the 330–406 ms period for both IC and NC stimuli as a function of thin and fat classification types and the difference between classification types for IC and NC stimuli, separately (sagittal views of the left and right hemispheres are shown).
Analyses to this point suggest that boundary completion processes and thin/fat shape discrimination processes are temporally dissociable, with the former occurring over the 124–186 ms period and the latter over the 330–406 ms period. To situate the timing of thin/fat shape discrimination effects when filling-in processes are necessary relative to the timing when filling-in is not required, we also compared VEPs in response to RC stimuli forming thin and fat configurations. As above, the spatiotemporal topographic pattern analysis with RC stimuli provided no indication of distinct maps in response to either thin or fat configurations. Rather, VEPs from both configurations showed the same sequence of seven scalp topographies over the collective 500 ms poststimulus periods (GEV, 98.12). Different maps were observed over the 0–68, 70–124, 126–152, 154–192, 194–266, 268–328, and 330–500 ms periods (Fig. 8). These time periods of stable scalp topography were then used to define time windows for the GFP analysis. Only the GFP over 154–192 ms period significantly differed between thin and fat configurations (t(10) = 3.15; p = 0.01; p values over other time periods were all >0.10) (Fig. 8c). Thus, electrophysiological correlates of thin/fat shape discrimination are observed over the 154–192 ms period when real shapes are used and filling-in processes are unnecessary, in sharp contrast to the substantially delayed timing of similar processes when filling-in was necessary (330–406 ms).
Electrophysiological results illustrating the time course of thin/fat shape discrimination processes with RC stimuli. a, Isocontour voltage maps depict stable electric field topographies obtained in the pattern analysis performed on the group-averaged data (period poststimulus indicated; left hemiscalp on left and nose upwards). b, c, Group-averaged VEP and GFP waveforms in response to RC stimuli that produce thin and fat shapes (scalp site and color scheme are indicated; see Results for details).
Discussion
The present results support two hypotheses. First, boundary completion processes occur independent of subjects' accuracy on the thin/fat discrimination task. VEP modulations to the presence versus absence of illusory contours (the IC effect) occurred with indistinguishable response magnitude and intracranial generator configurations, regardless of whether performance was correct. Second, electrophysiological correlates of shape discrimination processes were temporally dissociable from those of boundary completion. The IC effect was unaffected by whether the perceived contour produced a thin or fat shape. In contrast, later time periods of the VEP (∼330–406 ms) modulated according to perceived shape only when stimuli produced illusory contours but not for control stimuli for which performance was at near-chance levels. Collectively, these data provide further support for a multistage model of object processing under degraded viewing conditions.
The first series of analyses indicated that indistinguishable IC effects, in terms of their magnitude and topography, were observed at ∼124–186 ms independent of subjects' ultimate accuracy on the discrimination task. We consider this as evidence that boundary completion proceeds automatically. Several lines of research indirectly support this conclusion (but see Maertens and Pollmann, 2005). In animals, illusory contour sensitivity has been observed electrophysiologically despite the use of anesthesia (von der Heydt et al., 1984; Grosof et al., 1993; Sheth et al., 1996; Lee and Nguyen, 2001). In humans, patients with visuospatial neglect and/or extinction showed improved performance on line bisection tasks when illusory contours were present, but not when they were absent, although these patients were unaware of the presence of the illusory contours (Mattingley et al., 1997; Vuilleumier and Landis, 1998; Vuilleumier et al., 2001). Explicit awareness of IC presence was unnecessary for improving perceptual judgments of other, attended stimuli. The present study extends these types of findings to show that healthy individuals process illusory contours independent of shape discrimination performance and via indistinguishable brain mechanisms. We consider these results as strong support for the automaticity of illusory contour processes.
Although we interpret this IC effect as an index of automatic boundary completion, the possibility that this effect instead reflects another type of processing cannot be unequivocally excluded. It has recently been proposed that the detection of salient regions may precede boundary completion processes, with the former taking place in lateral occipital (LOC) regions and the latter within V2/V1 (Stanley and Rubin, 2003; Yoshino et al., 2006). However, several lines of evidence would argue against this dissociation. A previous fMRI study observed differential responses within the LOC for illusory contours induced either by Kanizsa-type pacmen or by displaced gratings (Mendola et al., 1999), the latter of which lack salient regions. Similar findings were reported by von der Heydt and Peterhans (1989), wherein orientation-selective illusory contour responses were elicited by grating as well as notch stimuli. Our interpretation does not exclude the possibility that contours are “sharpened” in early visual areas through feedback mechanisms but does suggest that boundary completion processes themselves occur in the LOC. Future experiments involving intracranial recordings in humans or the combination of VEPs and transcranial magnetic stimulation (Brighina et al., 2003) will be required for more definitive conclusions, although preliminary evidence from our group (Shpaner et al., 2004) would argue that completion processes and detection of salient regions both occur nearly simultaneously in the LOC.
The second series of analyses indicated that the discrimination of thin versus fat illusory contour shapes was temporally distinct from neural sensitivity to the illusory contour itself. Although illusory contour sensitivity was observed over the ∼124–186 ms period, electrophysiological correlates of the thin/fat shape discrimination were observed at ∼330–406 ms, some 200 ms later. This pattern is highly consistent with a multistage model of object-related processing under degraded viewing conditions. The first stage, occurring over the ∼65–90 ms period, is sensitive to the spatial extent of inducers, independent of whether they form illusory contours (Murray et al., 2002). The second stage, occurring over the ∼90–190 ms period, is sensitive to boundary completion, independent of the spatial extent of the inducers (Murray et al., 2002, 2004b; Pegna et al., 2002; Foxe et al., 2005). Differential activity over this time period has also been observed when subjects successfully identified repetitions of highly fragmented line drawings of common objects (Doniger et al., 2001). Brain generators active over this period have been repeatedly localized to the LOC (Kruggel et al., 2001; Murray et al., 2002, 2004b; Halgren et al., 2003; Ritzl et al., 2003; Sehatpour et al., 2006). The third stage, occurring over the ∼240–400 ms period, is sensitive to perceptual closure processes supporting the recognition of fragmented line drawings of objects (Viggiano and Kutas, 1998; Doniger et al., 2000, 2001, 2002; Sehatpour et al., 2006) and to illusory contour processes when inducer stimuli are restricted to a single visual hemifield (Brandeis and Lehmann, 1989; Murray et al., 2002). In one paradigm, subjects were shown highly fragmented line drawings of common objects and were successively presented with more complete versions until object recognition and identification was achieved (Doniger et al., 2000, 2001). This progression was paralleled in the VEP by a negative potential (termed negativity for closure or Ncl) that peaked at ∼290 ms poststimulus onset and involved bilateral LOC generators (Sehatpour et al., 2006). More generally, we proposed that these latter two distinct processing periods (i.e., the timeframe of the N1 component and the timeframe of the Ncl) reflected distinct modes of object recognition within the LOC, with the former reflecting automatic “perceptual” level processes and the latter representing more effortful “conceptual” modes of object processing. The present results appear to provide additional support for this model through direct assessment within individual subjects of the relative independence of completion and discrimination processes (i.e., automatic and effortful modes of processing).
We advance this model only in terms of object processing under degraded viewing conditions. This model may not apply when the visual scene is intact. Indeed, numerous studies have now shown that the brain is capable of performing object discrimination and categorization within just 100 ms poststimulus onset (Thorpe et al., 1996; Mouchetant-Rostaing et al., 2000; Michel et al., 2004b; Murray et al., 2004a). Previous studies have likewise shown that the processing of RC stimuli is distinct from and sometimes faster (Tallon-Baudry et al., 1996; Pegna et al., 2002; Ritzl et al., 2003; Kaiser et al., 2004) than that of IC and NC stimuli. The present study provides further support for faster processing of RCs, in that electrophysiological correlates of thin/fat shape discrimination were observed at 154–192 ms for RC stimuli versus 330–406 ms for IC stimuli.
The present interpretation is also influenced by recent investigations of patients with schizophrenia, in whom there is a clear dissociation between electrophysiological signatures of illusory contour and perceptual closure processes. Although illusory contour processes during the N1 processing period were entirely intact, perceptual closure processes during the Ncl period were severely impaired, if not altogether absent (Foxe et al., 2001, 2005; Doniger et al., 2002). This dissociation was observed despite the fact that patients in both cases exhibited substantially impaired early sensory processing (55–90 ms), which manifested as attenuated activity within dorsal stream structures. One implication is that dorsal stream inputs may not be critical during the more automatic phase when illusory contour processing occurs but are critically important for later, more conceptual phases. These findings are also in agreement with the aforementioned behavioral results in patients with neglect, whose lesions involved regions of the parietal lobe. Thus, certain types or modes of object recognition processes may not critically rely on intact inputs from the dorsal visual processing stream. Determining whether this hypothesis extends to the present paradigm will require additional experimentation. Nonetheless, the present results do suggest that one stage of object-related processing is unrelated to behavioral outcome, whereas another is critically related to both boundary completion and subjects' accuracy. That is, a thin/fat VEP difference was only observed for IC and not NC stimulus configurations, with only the former yielding accuracy above chance levels.
Our results impact the interpretation of psychophysical studies that used backward masking paradigms in conjunction with the thin/fat paradigm. In their study, introducing this task, Ringach and Shapley (1996) varied stimulus presentation duration in combination with local and global masks to interfere with task performance. From their results, they proposed two phases that are involved in modal boundary completion. The first, lasting ∼117 ms, reflects the detection of local stimulus features (e.g., corners, endpoints, and boundary segments). The later stage, lasting an additional ∼140–200 ms, reflects integration processes that generate global percepts. More recently, Imber et al. (2005) revisited this masking paradigm and the validity of Ringach and Shapley's (1996) model. They found that illusory shapes could be successfully masked by the presentation of other illusory contour stimuli of varying sizes, despite the nonoverlapping positions of the mask and the target inducers. Imber et al. (2005) therefore proposed that the later phase identified by Ringach and Shapley (1996) may reflect processing stages involved in shape discrimination, rather than boundary completion. Our results provide clear electrophysiological support for this proposal.
Appendix
The k-means cluster analysis as applied to event-related potentials
An important issue in the analysis of event-related potentials is how to define the temporal intervals of a component and/or the temporal intervals for statistical analyses. This becomes increasingly challenging when high-density electrode montages are used. Also, the approach of averaging the measured potentials over a fixed and/or experimenter-defined time interval assumes that the electric field configuration is stable. Our approach derives from the principle of functional microstates, which was introduced by Dietrich Lehmann (Lehmann, 1987) (for review, see Michel et al., 2004a), which treats electroencephalographic data as a description of the electric field at the scalp. These electric fields are in turn characterized by their topography, and the objective of the k-means cluster analysis is to identify the sequence of topographies as a function of time and also as a function of experimental variables (e.g., conditions or subject groups, etc.). This Appendix is supplied as a description, in layman terms, of each step of the k-means cluster analysis. Additional material, including a tutorial film, can be viewed and/or downloaded on-line (http://brainmapping.unige.ch/docs/Murray-Supplementary.pps). Additional mathematic details can be found in the study by Pascual-Marqui et al. (1995) or in the description within the freely available software Cartool (http://brainmapping.unige.ch/cartool.php).
Step 1
First, a concatenated dataset is defined using the group-averaged VEPs across all conditions of the experiment. For example, supposing that an experiment contains two conditions and that the VEP to each condition consists of 100 data points, then the concatenated dataset would be the collective 200 data points, because the group-averaged VEP from one condition is concatenated with the other. The goal of the k-means cluster analysis is to determine the minimal number of different topographies that explain the maximal amount of the variance within this concatenated dataset while also preserving the information concerning when (i.e., at which points in time) each topography is present.
Step 2
Second, n data points (where the term data point refers to the VEP from all scalp electrodes at a given instant in time) from this concatenated dataset (hereafter, template maps) are randomly selected from the concatenated dataset. The number of data points can range from one to the number of total data points. On one extreme, if n is low, then the explained variance will remain low, and the dataset itself will be highly compressed, because it will now be represented by a small number of template maps. On the other extreme, if n is high, then the explained variance will also be high, but the dataset itself will not be compressed. Again, the goal is to determine a middle-ground between such extremes. This is accomplished by randomly taking different numbers of n template maps (see Step 5).
Step 3
Third, the spatial correlation (Eq. 1, below) between each of the n template maps and each time point of the concatenated dataset is calculated. This gives a spatial correlation value for each template map as a function of time, and for any given time point one of the n template maps yields the highest spatial correlation value. As alluded to above, what is empirically observed in VEP data is that a given template map will yield the highest spatial correlation for a sustained period of time after which another, different template map will yield the highest spatial correlation, and so on. From these spatial correlation values, the GEV of these template maps is calculated (Eq. 2, below). GEV gives a metric of how well these n template maps describe the whole dataset. Each of the n template maps is then redefined by averaging the maps from all time points when the nth template map yielded the highest spatial correlation versus all other template maps. Spatial correlation for each of these redefined template maps and the resultant GEV are recalculated as above. This procedure of averaging across time points to redefine each template map, recalculating the spatial correlation for each template map, and recalculating the GEV is repeated until the GEV becomes stable. In other words, a point is reached when a given set of n template maps cannot yield a higher GEV for the concatenated dataset.
Step 4
Because the selection of the n template maps is random, it is possible that neighboring time points were originally selected, which would result in a low GEV. To help ensure that this procedure obtains the highest GEV possible for a given number of n template maps, a new set of n template maps is randomly selected and the entire above procedure is repeated (i.e., Steps 1–3). It is important to note that the number of random selections is user-dependent and will simply increase computational time as the number of random selections increases. The set of n template maps that yields the highest GEV is retained.
Step 5
Finally, Steps 1–4 are now conducted for n + 1 template maps and can iterate until n equals the number of data points comprising the concatenated dataset.
Step 6
The above steps provide information on how well n, n + 1, n + 2 … etc. template maps describe the concatenated dataset. The next issue for this analysis is to determine what the optimal number of template maps is. In other words, the trade-off between increasing the GEV as one increases the number of template maps while simultaneously decreasing the degrees of freedom must be evaluated. The minimal number of template maps that yields the highest GEV is determined with a cross-validation criterion (see Materials and Methods for details). In addition to identifying how many maps account for the concatenated dataset, this entire procedure always retains information regarding the particular sequence (as a function of time) of template maps within each condition comprising the concatenated dataset. In short, this procedure is used as a hypothesis generation tool that describes both the timing of VEP components as well as topographic homogeneity between experimental conditions, which can in turn be statistically assessed using the fitting procedure described in Materials and Methods.
Equations
Spatial correlation
where
where n is the number of electrodes in the montage, ui is the potential measured from the ith electrode from a given experimental condition, and vi is the potential measured from the ith electrode from another experimental condition.
Global explained variance
where EV = C2.
Footnotes
-
This study was supported by National Institute of Mental Health Grants MH65350 and MH63434 (J.J.F.), the Swiss National Science Foundation Grant 3200BO-105680/1 and the Leenaards Foundation (M.M.M.), and National Eye Institute Grant EY-01472 (M.L.I.). We thank Denis Brunet for the development of Cartool EEG analysis software (http://brainmapping.unige.ch/cartool.php), as well as Beth Higgins and Deirdre Foxe for technical assistance. Data from this study are from M.L.I.'s doctoral thesis submitted in partial fulfillment of the requirements within the Department of Psychology at New York University. We also thank Drs. Robert Shapley, Daniel Senkowski, and Sophie Molholm for lively discussion of these data and insightful and critical comments on previous versions of this manuscript.
-
↵a It might additionally be claimed that IC and NC stimuli seem to exhibit luminance differences. However, it is important to note that any such difference is perceptual, rather than physical, in nature. This perceived brightness enhancement as well as the perceived object borders define modal completion, whereas the perception of object borders in the absence of a concomitant perceived brightness enhancement defines amodal completion (Michotte et al., 1964). Our previous work has demonstrated that modal and amodal completion use a common mechanism over the initial ∼150 ms poststimulus onset (Murray et al., 2004b), suggesting that VEP differences between IC and NC conditions prior to ∼150 ms do not follow from perceived brightness enhancement. Our previous work has similarly addressed the contribution of other low-level physical attributes to the IC effect, including contrast polarity and the local luminance differences resulting from the spatial distribution of inducers (Murray et al., 2002).
-
↵b Ringach and Shapley (1996) noted that subjects were very poor at discriminating NC-type stimuli, both when presented with four identical pacmen (i.e. at the same orientation, their “local orientation” condition) or when the four pacmen faced outwards (their “all-out” condition; the same as our NC condition). Subjects were quite poor at the NC task whether they were provided with local information alone or with local and global information together. Because we have chosen the more “global” method of presenting NCs, which preserves symmetry as well as orientation contrast in the stimuli, we believe this is the most analogous to the task subjects were performing with ICs and RCs. Thus, for all three contour types, subjects were making a forced-choice classification between two categories or types of shapes.
- Correspondence should be addressed to either of the following: Micah M. Murray, The Functional Electrical Neuroimaging Laboratory, Neuropsychology Division, Centre Hospitalier Universitaire Vaudois, Hôpital Nestlé, 5 Avenue Pierre Decker, 1011 Lausanne, Switzerland, micah.murray{at}chuv.ch; or John J. Foxe, The Cognitive Neurophysiology Laboratory, The Nathan S. Kline Institute for Psychiatric Research, 140 Old Orangeburg Road, Orangeburg, NY 10962, foxe{at}nki.rfmh.org
