Abstract
Because environmental information is often suboptimal, visual perception must frequently rely on the brain's reconstruction of contours absent from retinal images. Illusory contour (IC) stimuli have been used to investigate these “filling-in” processes. Intracranial recordings and neuroimaging studies show IC sensitivity in lower-tier area V2, and to a lesser extent V1. Some interpret these data as evidence for feedforward processing of IC stimuli, beginning at lower-tier visual areas. On the basis of lesion, visual evoked potentials (VEP), and neuroimaging evidence, others contend that IC sensitivity is a later, higher-order process. Whether IC sensitivity seen in lower-tier areas indexes feedforward or feedback processing remains unresolved. In a series of experiments, we addressed the spatiotemporal dynamics of IC processing. Centrally presented IC stimuli resulted in early VEP modulation (88–100 msec) over lateral–occipital (LOC) scalp—the IC effect. The IC effect followed visual response onset by 40 msec. Scalp current density topographic mapping, source analysis, and functional magnetic resonance imaging results all localized the IC effect to bilateral LOC areas. We propose that IC sensitivity described in V2 and V1 may reflect predominantly feedback modulation from higher-tier LOC areas, where IC sensitivity first occurs. Two additional observations further support this proposal. The latency of the IC effect shifted dramatically later (∼120 msec) when stimuli were laterally presented, indicating that retinotopic position alters IC processing. Immediately preceding the IC effect, the VEP modulated with inducer eccentricity—the configuration effect. We interpret this to represent contributions from global stimulus parameters to scene analysis. In contrast to the IC effect, the topography of the configuration effect was restricted to central parieto–occipital scalp.
Object recognition occurs despite ambiguous or incomplete information in the retinal image, as in situations of partial occlusion and poor lighting. The brain can therefore reconstruct contours absent from visual images (Petry and Meyer, 1987; Doniger et al., 2000, 2001). One method of investigating these processes involves illusory contour (IC) stimuli (Kanizsa, 1976), wherein subjects perceive an object's borders in the absence of luminance discontinuities. At present, the locus and timing of IC processing are unresolved.
Some report IC sensitivity at the lowest cortical processing stages. Animal intracranial studies have observed effects in area V2 (von der Heydt et al., 1984; Leventhal et al., 1998; Nieder and Wagner, 1999;Bakin et al., 2000) and sometimes also in area V1 (Redies et al., 1986;Grosof et al., 1993; Sheth et al., 1996; Lee and Nguyen, 2001;Ramsden et al., 2001). Human neuroimaging studies report similar effects (Hirsch et al., 1995; ffytche and Zeki, 1996; Larsson et al., 1999; Seghier et al., 2000). From these findings, some concluded that IC sensitivity is a feedforward bottom-up process (Grosof et al., 1993; ffytche and Zeki, 1996; Sheth et al., 1996;Leventhal et al., 1998; Albert and Hoffman, 2000).
Others propose that IC sensitivity occurs at higher processing stages. Behavioral investigations suggest critical roles for macaque V4 (De Weerd et al., 1996; Merigan, 1996) and inferotemporal (IT) areas (Huxlin and Merigan, 1998; Huxlin et al., 2000). In humans, visual evoked potentials (VEP) modulate relatively late (>150 msec after stimulus) to IC stimuli (Brandeis and Lehmann, 1989; Sugawara and Morotomi, 1991; Tallon-Baudry et al., 1996, 1997; Hermann et al., 1999;Korshunova, 1999; Csibra et al., 2000; Hermann and Bosch, 2001). Most VEP studies examined γ modulations (30–60 Hz) as an index of perceptual binding (Tallon-Baudry et al., 1996, 1997; Csibra et al., 2000) or target selection (Hermann et al., 1999; Hermann and Bosch, 2001), and some observed no wide-band effects before 200 msec (Tallon-Baudry et al., 1996, 1997). Nevertheless, the dynamics of IC processing (relative to cortical response onset) as well as its locus remain undetermined. Regarding the latter, one detailed functional magnetic resonance imaging (fMRI) study found differential activation to IC shapes of varying types, sizes, and gap-widths in higher-tier lateral–occipital (LOC) areas but not in lower-tier areas including V1 and V2 (Mendola et al., 1999).
Although all human neuroimaging studies observed effects in higher-tier cortical areas, studies in animals have neither investigated IC sensitivity in such areas nor addressed the timing of IC sensitivity relative to sensory response onset. We investigated these issues in a series of experiments, in which we combined high-density electrical mapping, source analysis, and fMRI to elucidate the mechanisms underlying IC processing in humans. The high temporal resolution of our electrophysiological technique permitted us to assess the relative timing of IC processes while recording simultaneously over the entire scalp. In experiment 1, stimuli were centrally presented while high-density event- related potential (ERP) recordings were made, to determine the timing and likely sources of VEP modulations specific to IC presence. In experiment 2, fMRI data were obtained from a second cohort of subjects, and their hemodynamic activations were used to constrain VEP source analyses of the electrophysiological data recorded from these same subjects. Experiment 3 examined whether the timing and locus of IC processing was affected by the contrast polarity of the display. Experiment 4 (a reanalysis of data from experiments 1 and 3) examined the effects of variations in the low-level features of the stimuli on the earliest VEP componency, allowing us to determine whether our approach was sensitive to modulations in lower-tier visual areas. Experiment 5 tested the effect of varying the retinotopic position of IC stimulus presentation on the timing and locus of IC processing. Collectively, our data support a model of IC processing wherein dorsal stream regions, which are initially insensitive to IC presence, coarsely demarcate the spatial extent of a given stimulus array and then input to ventral stream structures (e.g., the LOC) where IC sensitivity first occurs. Our data support the notion that IC effects observed previously in lower-tier areas are likely to be driven by feedback inputs from higher-tier areas.
Portions of this study have been published previously in abstract form (Murray et al., 2000).
MATERIALS AND METHODS
This study is composed of five separate experiments, the details of which are described in turn.
Experiment 1: central presentations of illusory contours
Subjects. Twenty-eight (10 female) neurologically normal, paid volunteers, aged 20–57 (mean ± SD = 33.4 ± 12.4) participated. All had normal or corrected-to-normal vision, and all but two of the subjects were right-handed (Oldfield, 1971). All subjects provided written informed consent, and the Institutional Review Board of the Nathan Kline Institute for Psychiatric Research approved all procedures.
Stimuli and task. Kanizsa-type (Kanizsa, 1976) illusory contour stimuli were presented to subjects on a computer monitor located 114 cm away while subjects fixated a central cross. These stimuli were constructed from “pacmen” inducers (Fig.1) that were oriented to either form or not form an illusory shape (“IC present” and “IC absent,” respectively). Five shapes were used: square, circle, triangle, pentagon, and five-pointed star. Inducers were circular, subtended 3° of visual angle in diameter, and appeared black on a gray background. To produce illusory shapes of the same maximal width and height (6° in either plane), the eccentricity (φ) of inducers varied across shapes. In the case of the square, inducers were centered at 4.25° eccentricity along 45° radii from central fixation. For the circle, they were located at 3° eccentricity along the horizontal and vertical meridians. For the triangle, the two lower field inducers were centered at 4.25° eccentricity, and the upper field inducer was centered at 3°. For both the pentagon and star, the two lower field inducers were centered at 3.5° eccentricity, the two lateral upper field inducers were centered at 3.1°, and the top inducer was centered at 3° along the vertical meridian. Likewise, different numbers of inducers were used to define these shapes (Table1). The support ratio (μ), defined as the percentage of the perimeter of the illusory shape revealed by the inducers (Ringach and Shapley, 1996), varied similarly, as did the surface area (λ) of the IC shapes (Table 1).
Stimuli and experimental paradigm.Left, Kanisza-type inducers were used in all experiments of the present study to define various geometric shapes (see Materials and Methods and Table 1 for details). Inducers were oriented to either form or not form an illusory shape. Right, The timing of stimulus presentations was such that inducers were presented for 500 msec duration (400 msec in experiment 5), followed by a blank screen for 1000 msec (450 msec in experiment 5). This was followed in turn by a Y/N prompt that remained on the screen until subjects made a forced-choice button-press response indicating whether an illusory shape was presented. A blank screen (1000 msec duration in experiments 1, 3, and 4; 700 msec duration in experiment 5) then preceded the next trial. Paradigmatic differences in experiment 2 are described in Materials and Methods.
IC stimulus characteristics (experiments 1–3)
After the presentation of each stimulus a “Y/N” cue appeared prompting a forced-choice response. Subjects pressed one button for a “No” response, indicating that they did not perceive an illusory shape, or a second button for a “Yes” response, indicating that they perceived such a shape. The entire experiment consisted of at least nine blocks (mean ± SD = 15.5 ± 6), each block containing 66 stimuli. IC present and IC absent inducer configurations were presented randomly and were equally probable. Subjects were encouraged to take breaks between blocks to maintain high concentration and prevent fatigue. The timing of presentations was such that each stimulus appeared for 500 msec, followed by a blank screen for 1000 msec. Then the Y/N response prompt appeared and remained on the screen until a response was made, allowing subjects to control stimulus delivery. A blank screen (1000 msec duration) followed responses. Use of the response prompt was motivated by the desire to diminish the impact of motor responses on the sensory VEP.
EEG data acquisition. Continuous EEG was acquired through Neuroscan Synamps from 64 scalp electrodes (impedances <5 kΩ), referenced to nose, bandpass filtered from 0.05 to 100 Hz, and digitized at 500 Hz (see Fig. 2, inset). Epochs of continuous EEG (−100 msec before stimulus onset to 500 msec after stimulus onset) were averaged from each subject separately for both the IC present and IC absent stimulus configurations to compute the VEP. Baseline was defined as the epoch from −100 msec to stimulus onset. Trials with blinks and eye movements were rejected off-line on the basis of horizontal and vertical electro-oculograms (resolution of ∼0.5°) (Murray et al., 2001). An artifact criterion of ±60 μV was used at all other scalp sites to reject trials with excessive EMG or other noise transients. The average (±SD) EEG epoch acceptance rate was 88% (±9). The average number of accepted sweeps per condition was 462 (±196), with the lowest for any subject being 248 and the highest being 975. Statistical and dipole source analyses as well as topographic mapping for all experiments in this study were performed on broad-band data, although filtered waveform data (40 Hz low-pass; 24 dB/octave) are displayed in the Figures.
Experiment 2: illusory contour processing assessed with combined EEG and fMRI
Subjects. Five subjects (three female; aged 24–33; mean ± SD = 29.0 ± 3.9) participated. Two of these subjects had also participated in experiment 1. All had normal or corrected-to-normal vision, and four of the five were right-handed. All subjects provided written informed consent, and the Institutional Review Board of the Nathan Kline Institute for Psychiatric Research approved all procedures.
Stimuli and task. Stimuli were physically identical to those of experiment 1; however, the task requirements and the timing parameters differed slightly in this experiment. Both the EEG and fMRI components required only passive viewing of the stimuli. In the case of the EEG portion, stimuli were presented for 300 msec duration [1–2 sec randomized interstimulus interval (ISI)], and both IC present and IC absent configurations appeared with equal probability from one trial to the next (randomized across trials). The entire EEG portion consisted of at least 10 stimulus blocks (mean ± SD = 10.8 ± 1.3), each containing 100 trials. Stimulation for the fMRI portion followed a “box-car” block design. Subjects viewed the following sequence: a blank gray screen (20 sec duration; hereafter, “rest”), a series of 20 IC shapes (300 msec duration; 700 msec ISI; randomized across the five shapes of experiment 1), rest for 20 sec, and a series of 20 IC absent configuration stimuli (300 msec duration; 700 msec ISI). In each set of functional images, this sequence was repeated three times. Likewise, the order of stimulus types was counterbalanced across subjects.
EEG data acquisition. Continuous EEG was acquired and the VEP (−100 msec before stimulus onset to 300 msec after stimulus onset) was calculated in an identical manner to experiment 1, except that data were collected from a 128-channel montage (∼2.4 cm inter-electrode spacing). The average EEG epoch acceptance rate was 84% (±15). The average number of accepted sweeps per condition was 464 (±107), with the lowest for any subject being 307 and the highest, 622. Electrophysiological results of this experiment are displayed on the three-dimensional (3-D) reconstruction of one subject's (B.H.) anatomical MRI, using the boundary element method (BEM) (Fuchs et al., 1998) as implemented in CURRY (Philips Research, Hamburg, Germany). Topographic mapping on this BEM reconstruction was based on the digitization of this same subject's fiduciary landmarks and electrode locations (Polhemus Fastrak and 3DspaceDX software, Neuroscan, Inc.). In addition to preserving the exact geometric relationships between electrodes, this technique facilitates the spatial co-registration of EEG and fMRI data by overlapping these fiduciary landmarks.
fMRI data acquisition and analysis. A 3 tesla SMIS (Marconi) system equipped with a head volume coil at the Center for Advanced Brain Imaging at the Nathan Kline Institute was used to acquire T2*-weighted echo-planar images (EPIs) (repetition time/echo time/flip angle = 3 sec/40 msec/90°; voxel size = 3.5 mm3; matrix size = 64×64). This sequence emphasizes the blood oxygenation level dependent (BOLD) response, which is an indirect index of local neural activity (Kwong et al., 1992; Ogawa et al., 1992; Logothetis et al., 2001). In each block of EPI scans, 90 volumes were acquired, each of which consisted of 31 contiguous slices, which covered the entire brain. The first five volumes were discarded to allow for stabilization of the BOLD signal. Visual stimulation was delivered through MR-compatible liquid crystal display goggles (Resonance Technology Inc., Northridge, CA). Head movement was minimized with the use of a custom-made head holder. In all five subjects, motion never exceeded 0.75 mm along any axis. All fMRI data analyses were conducted with SPM99 software (Welcome Department of Cognitive Neurology, London, UK). Images were realigned to the first included volume, normalized into Talairach coordinate space, and smoothed (7 mm3 full width at half maximum Gaussian kernel). We convolved the timing of the stimulation epochs (delayed box-car) with a canonical hemodynamic response function, and cross-correlated the resulting design matrix with the signal intensity changes observed in the EPI images. At test parametric map was generated using a height threshold of t ≥ 4.70 (p < 0.05, corrected for multiple comparisons). First, we rendered this activation map on a brain provided by SPM99 software. Then, we spatially co-registered this activation map with one subject's (B.H.) anatomical MRI data as well as with the group-averaged VEP data using CURRY software.
Experiment 3: the IC effect and contrast polarity
Subjects. Seventeen subjects (five female; aged 20–42; mean = 25.6 ± 6.5) from experiment 1 participated. All subjects, except one, were right-handed (Oldfield, 1971).
Stimuli and task. All experimental parameters were identical to experiment 1, except that the contrast polarity was inverted, such that inducers appeared gray on a black background. The entire experiment consisted of at least nine blocks (mean ± SD = 13.9 ± 4.3), each block containing 66 stimuli.
Data acquisition. Continuous EEG was acquired and the VEP was calculated in an identical manner to experiment 1. EEG epoch acceptance rate was 89.9% (±7.6). The average number of accepted sweeps per condition was 416 (±132), with the lowest for any subject being 251 and the highest, 750.
Experiment 4: stimulus features affecting scene analysis
Subjects. Data from the 17 subjects (5 female; aged 20–42; mean = 25.6 ± 6.5) who participated in both experiments 1 and 3 were included.
Stimuli and task. The stimuli were those of experiments 1 and 3. As described above, five IC shapes were defined by varying numbers of Kanizsa-type inducers. Additional stimulus parameters are listed in Table 1.
Data acquisition. For each subject, the EEG epochs from experiments 1 and 3 were pooled and averaged according to shape as well as IC presence versus absence to compute the VEP. This resulted in 10 VEPs for each subject (five shapes × two stimulus configurations). The average number of accepted sweeps per condition was 147 ± 44, with the lowest for any subject being 78 and the highest, 280.
Experiment 5: the IC effect and lateral presentations
Subjects. Twelve (seven female) neurologically normal, paid volunteers, aged 19–47 (mean ± SD = 26.3 ± 7.2) participated. Eleven of the 12 subjects were right-handed (Oldfield, 1971). Eight of these twelve subjects had also participated in experiment 1.
Stimuli and task. Stimulation was similar to experiments 1 and 3, except that the inducer array was presented entirely to the left or right of central fixation. As before, inducer stimuli were oriented either to form or not form an illusory shape and were presented to subjects on a computer monitor located 114 cm away as they centrally fixated. Contrast polarity was identical to experiment 1. All stimulus presentations were composed of five inducers, centered at identical locations for all shapes. Three of the five shapes from experiment 1 were used: the circle, pentagon, and five-pointed star. For each illusory shape there were three versions in which the inducers were not aligned, to make it less likely that subjects completed the task by remembering the orientation of a particular inducer. Inducers were circular and subtended 1.75° of visual angle in diameter. When present, illusory shapes maximally subtended 4° in both the horizontal and vertical planes. Stimuli were presented to either the left or right of a central fixation cross that remained on the screen (2° from the vertical meridian to the nearest edge of the stimulus). The entire stimulus subtended 5.5° in both horizontal and vertical planes.
The experiment consisted of at least 20 blocks (mean ± SD = 30 ± 9.7), each block containing 72 stimulus presentations. IC present and IC absent inducer configurations were equally probable, as was hemifield of presentation. The timing of stimulus presentations was such that each stimulus appeared for 400 msec, followed by a blank screen for 450 msec. After this, a Y/N cue appeared centrally, prompting a forced-choice response identical to that in experiment 1. The Y/N cue remained on the screen until a response was made, allowing subjects to control stimulus delivery. A blank screen (700 msec duration) followed responses.
Data acquisition. Continuous EEG was acquired in an identical manner to experiment 1. The average EEG epoch acceptance rate was 93.8% (±4.2). Epochs of continuous EEG (−100 msec before stimulus onset to 500 msec after stimulus onset) were averaged from each subject separately for both the IC present and IC absent stimulus configurations and each hemifield of presentation to compute the VEP. This resulted in four VEPs for each subject. Baseline was defined as the epoch from −100 msec to stimulus onset. The average number of accepted sweeps per condition was 336 (±75), with the lowest for any subject being 171 and the highest, 529.
Scalp current density and dipole source analyses
Information about the intracranial generators contributing to IC sensitivity was obtained using two methods. The first was topographic mapping based on scalp current density (SCD) analysis according to the methods of Huiskamp (1991) and as implemented in CURRY. SCD analysis takes advantage of the relationship between local current density and field potential defined by Laplace's equation, by calculating the second spatial derivative of the field potential, which is directly proportional to the current density. The SCD technique eliminates the contribution of the reference electrode and mathematically eliminates effects of volume conduction on the surface potential caused by tangential current flow within the scalp. The major strength of the SCD analysis is the possibility for improved visualization of approximate locations of intracranial generators through the reduced spatial superposition of gradients originating from multiple intracranial sources (Urbano et al., 1996).
The second method was dipole source analysis using electromagnetic source estimation as applied through CURRY software. This method assumes that there are a limited and distinct number of active brain regions over the VEP epoch, each of which can be approximated by an equivalent dipole. Dipole generators are placed within a three-shell spherical volume conductor model and overlaid on and adjusted to one of our cohort's (B.H.) BEM-segmented structural MRI. The forward solution to this dipole configuration is tested against the observed experimental data. When not fixed, the positions and orientations of the dipoles are iteratively adjusted to minimize the residual variance between the forward solution and the observed data. The upper bound of the number of modeled dipole sources is determined using a test dipole (Scherg and Picton, 1990). If the number of modeled sources,m, is adequate, then addition of another source (test dipole) and solving for m + 1 sources would not be expected to further reduce the residual variance, above that attributable to noise. For both SCD and dipole analyses, group-averaged VEP data were used to maintain the highest possible signal-to-noise ratio as well as to generalize our results across individuals.
One further note about the SCD topographic mapping method is warranted. An obvious constraint of the printed page is that only a limited number of discrete maps can be shown to represent a given topographic distribution, and such static maps fail to depict the full spatiotemporal dimensionality of the data. This can make it particularly difficult for the reader to assess both the stability of a given topography over time and the extent of contribution of background noise to the maps. In determining the display gain to be used for the maps in the current study, we followed the topography over its entire time course for each subject (through observing animated time-series). Observation of these maps in the baseline period (from −100 msec until the onset of significant activity at ∼50 msec) allowed us to determine the relative contribution of noise to the SCD topographic maps. The gain was then set so that background noise during this baseline period accounts for only one to two topographic lines of current density in a given SCD map. In all cases in which a topography is plotted for a single time point in this manuscript, we observed that topography to be stable over time (see Fig. 3C).
Statistical cluster plots
Because our primary interest was in determining the timing of IC sensitivity, we calculated point-wise paired t tests between VEP responses. By this method, we can identify the onset of differential responses between both the IC present versus IC absent conditions (hereafter, the IC effect) as well as VEP response onset versus the 0 μV baseline. For each electrode, the first time point where the t test exceeded the 0.05 α criterion for at least 11 consecutive data points (>20 msec at a 500 Hz digitization rate) was labeled as onset of either the IC effect or the VEP response (Guthrie and Buchwald, 1991).
The results of the point-wise t tests from 55 of the 64 electrodes are displayed as an intensity plot (see Fig. 11) to efficiently summarize and facilitate the comparison of the multiple data sets comprising this study. The x-, y-, andz-axes, respectively, represent time (post-stimulus onset), electrode location, and the t test result (indicated by a color value) at each data point. Posterior electrodes (from right to midline to left) include O2, PO8, P8, P6, PO6, PO4, P4, P2, Pz, POz, Oz, P1, P3, PO3, PO5, P5, P7, PO7, and O1. Central electrodes (from right to midline to left) include TP8, T8, C6, CP6, CP4, C4, C2, CP2, CPZ, CZ, C1, CP1, CP3, C3, C5, CP5, TP7, and T7. Frontal electrodes (from right to midline to left) include FT8, F8, F6, FC6, FC4, F4, F2, FC2, FCZ, FZ, F1, FC1, FC3, F3, F5, FC5, FT7, and F7. The nomenclature of these electrode sites is modified from the “International extended 10–20 system” (American Electroencephalographic Society, 1991).
RESULTS
Experiment 1: central presentations of illusory contours
Behavioral results
Subjects correctly indicated the presence or absence of IC shapes 94.6% (±8.3%) of the time.
Electrophysiological results
Inspection of group-averaged VEPs for the IC present and IC absent conditions revealed the traditional series of VEP components, including the P1, N1, and P2. These components were maximal over posterior scalp sites. Direct comparison of these conditions revealed a differential response to centrally presented IC shapes over posterior scalp sites (Fig. 2). The rising phase of the P1 component showed neither amplitude nor latency modulation with the presence versus absence of IC shapes. In contrast, a large difference in waveform morphology was apparent beginning at the falling phase of the P1 component.
Experiment 1 VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off). Data from a pair of frontal (F3/F4) and parieto–occipital (PO5/PO6) electrode sites are shown. Their locations on the scalp are indicated by large white discs on the 3-D reconstruction (BEM as implemented in CURRY) of one subject's (B.H.) anatomical MRI. Black tracesindicate the VEP response to the presence of an illusory contour (IC present), whereas light gray tracesindicate the corresponding VEP response to the non-inducing configurations (IC absent). Dark gray dashed traces represent the IC present minus IC absent difference.Black traces in the insets illustrate thep value of point-wise t tests between the IC present and IC absent conditions across the VEP epoch.
We were interested in determining the onset latency of the differential response to the presence versus absence of IC shapes. The point-wise paired t tests indicate that the earliest IC effect onset is at 88 msec after stimulus at scalp site P2. IC effect onset latencies at the remaining posterior scalp sites ranged between 88 and 102 msec, with near simultaneous onsets over both hemispheres (88 msec at site P2 vs 90 msec at site CP3; see Fig. 11, top left). The initial IC effect was largest at sites PO5 and PO6 (Fig. 2), peaked at ∼146 msec after stimulus onset, and persisted until ∼215 msec after stimulus onset (see Fig. 11). A second phase of the IC effect began at ∼230 msec and persisted until ∼320 msec. No IC effect was observed over frontal sites until ∼250 msec, persisting until ∼320 msec after stimulus. Figure 2 displays the VEP waveforms as well as the results of point-wise t tests from sites PO5 and PO6 in addition to two representative frontal sites.
For each subject and stimulus condition, we then calculated the area (vs the 0 μV baseline) for the 136–156 msec after stimulus window at sites PO5, PO6, P5, and P6 (sites of maximal IC effect) and submitted these area measures to a 2 × 2 × 2 repeated measures ANOVA. The within subjects factors were stimulus condition (IC absent vs IC present), hemisphere (left vs right), and electrode (two over each hemisphere). There was a main effect of stimulus condition (F(1,27) = 122.13; p< 0.001), which confirmed the results of the point-wise ttests. There was also a main effect of hemisphere (F(1,27) = 5.73; p < 0.03). Furthermore, there was a significant interaction between the factors of stimulus condition and hemisphere (F(1,27) = 4.52; p < 0.05), indicating a larger IC effect over the right versus left hemisphere.
A further emphasis of this study was the registration of the onset of the IC effect (as well as its time course) relative to onset of the initial visual cortical activation. That is, we sought to determine when the brain responded differentially to IC presence versus absence relative to when the brain responded to any visual stimulus (regardless of IC presence versus absence). We therefore determined the latency of the earliest deviation from baseline activity of the VEP response at each of the 64 scalp sites by calculating point-wise pairedt tests (two-tailed) between the 0 μV baseline and the collapsed data from the two stimulus conditions. The first time point forward from 30 msec after stimulus onset where the response exceeded the 0.05 α criterion for at least 11 consecutive data points was labeled as the VEP response onset at that site. The earliest response onset was 48 msec (e.g., sites P3, P4), consistent with other observations of what has been termed the C1 component (Clark et al., 1995; Murray et al., 2001; Foxe and Simpson, 2002; Molholm et al., 2002). From this average VEP response onset latency, we calculated a minimal lag of 40 msec until the onset of the IC effect.
SCD topographic maps of the group-averaged IC effect are shown in Figure 3, A and C, and illustrate the scalp topography of the IC effect at its peak (146 msec) (Fig. 3A), as well as the stability of the lateral–occipital distribution of the IC effect over time (Fig.3C). Although the point-wise t test analysis described above provides the latency of the IC effect, the scalp topography indicates that this effect occurs first over lateral–occipital areas and is larger over the right hemisphere. Moreover, no current density foci are observed over frontal scalp regions. In addition to the SCD maps, Figure 11 (top left panel) displays a statistical cluster plot of the results of point-wise t tests for the IC effect at 55 of the 64 electrodes over the 500 msec post-stimulus epoch. Two phases of the IC effect over lateral–posterior scalp sites, as well as the absence of a robust frontal effect, are readily apparent (∼100–300 msec and ∼325–425 msec).
SCD topographic maps of the IC effect for centrally presented stimuli. Maps in this and similar figures are displayed on the 3-D reconstruction (BEM as implemented in CURRY) of one subject's (B.H.) anatomical MRI data. These SCD foci are consistent with bilateral lateral–occipital generators, although they are more pronounced over the right hemisphere. Polarity of these maps is arbitrary, depending on the direction of the subtraction, and scales are shown. A, SCD topographic map (left-sided, back, and right-sided views) at 146 msec after stimulus onset depicting the IC effect in experiment 1 (inducers appeared black on agray background). B, SCD topographic map at 156 msec after stimulus onset depicting the IC effect in experiment 3 (inducers appeared gray on a black background; identical scale as in A).C, A series of maps (back view) depicting the stability of the SCD topography over the 88–168 msec post-stimulus epoch for the IC effect shown in A.
Source analyses were performed on the difference between the group-averaged VEP responses from the IC present minus IC absent conditions. We seeded the locations of two dipoles based on the average Talairach coordinates of the LOC areas described by Mendola et al. (1999) as responsive to the presence of Kanizsa-type IC shapes. The right hemisphere LOC dipole was located at 35, 85.4, 8 mm, and the left hemisphere LOC dipole was located at −35, 85.4, 8 mm. Initially, these two dipoles were fit to a single time point (146 msec) at the peak difference between the IC present and IC absent conditions. Only their locations were fixed, allowing free and independent dipole rotation until a minimal residual variance was achieved. We then fixed the orientation of both dipoles and widened the epoch to 116–156 msec, so as to encompass both the peak as well as the immediately preceding period. These two fixed dipoles explained on average 95.1% of the variance over this epoch (Fig. 4). The strength of each source over the 0–500 msec epoch indicates that both generators are activated simultaneously.
A, The positions and orientations of two fixed dipoles (cyan) are rendered in the 3-D reconstruction of one subject's (B.H.) anatomical MRI (BEM as implemented in CURRY; back and side views shown) at the peak of the IC effect (146 msec). B, On average, this pair of dipoles accounts for 95.1% of the variance between the observed data and the forward solution to these dipoles over the 116–156 msec post-stimulus epoch. C, The strength of these dipoles over the post-stimulus epoch indicates synchronous bilateral IC processing in the lateral–occipital areas.
Experiment 2: illusory contour processing assessed with combined EEG and fMRI
The results of experiment 1 provide compelling evidence that IC sensitivity follows the onset of the VEP response by a considerable lag (40 msec) and occurs first in higher-tier lateral–occipital brain areas. To interpret the IC effect observed in the VEP response and dipole source analysis in relation to previous hemodynamic imaging results more directly, we combined 128-channel EEG and fMRI data sets from a cohort (n = 5) of subjects (see Materials and Methods).
Electrophysiological results
VEP morphology was similar to that seen in experiment 1. The comparison of the IC present and IC absent conditions revealed a similar IC effect, beginning during the falling phase of the P1 and persisting through the peak of the N1 component (Fig.5A). Visual inspection of the difference waveforms between IC present and IC absent stimulus conditions indicated that in these subjects the IC effect peaked at ∼126 msec. We tested for an IC effect with a 2 × 2 × 3 repeated measures ANOVA using area measures (vs the 0 μV baseline) from each subject and stimulus condition over the 106–146 msec post-stimulus epoch at six parieto–occipital sites. The within subjects factors were stimulus condition (IC absent vs IC present), hemisphere (left vs right), and electrode (three electrodes over each hemisphere). There was a main effect of stimulus condition (F(1,4) = 233.66; p < 0.0001), confirming the IC effect. No other effects or interactions reached the 0.05 significance threshold.
VEP and fMRI results from experiment 2.A, VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off; identical color scheme as in Fig. 2) from two representative scalp sites illustrate the IC effect under passive viewing conditions. Electrode locations are indicated with large white discson the 3-D scalp reconstruction of one subject's anatomical MRI shown in the insets. B, The locations of fMRI results are shown on axial slices of a standard brain supplied with SPM99 software. White pixels indicate areas of significant BOLD signal increase for the IC present versus IC absent conditions (p ≤ 0.05; corrected for multiple comparisons across the entire image volume).
fMRI results
Activation maps of the IC effect (IC present vs IC absent) localize IC processing to the LOC areas bilaterally. This localization was independent of any a priori hypotheses in our analysis, because no regions of interest were specified, and we corrected for multiple comparisons across the entire image volume. Figure 5Bdisplays the BOLD response activation maps for the IC present versus IC absent comparison as rendered on axial slices of the standard brain supplied by SPM99 software. The locations (Talairach coordinates) and extent of these activation clusters as well as the results of the statistical tests are listed in Table 2. On the basis of the coordinates reported in previous studies of IC processing (Mendola et al., 1999) and object recognition processes (Malach et al., 1995; Grill-Spector et al., 1998a,b), we interpret the location of the two posterior clusters as within the LOC. It should be noted that the LOC region is considered to be composed of several areas that branch off dorsally and ventrally from a posterior–lateral vertex (Malach et al., 1995; Grill-Spector et al., 1998a,b). Consequently, there is often variation in coordinates ascribed to LOC area activation both within and across studies. Moreover, additional analyses using a region of interest (uncorrected threshold of p < 0.001 within a 10 mm sphere) approach failed to reveal significant clusters of activation in lower-tier areas V1 or V2 (data not shown), using the average Talairach coordinates for these areas described by Mendola et al. (1999). Activations in these lower-tier areas were observed, however, in the comparison of either stimulus condition versus rest. These collective fMRI results provide strong support for the proposal that the strongest IC processing occurs in higher-tier visual areas.
Volume summary of fMRI results with p values corrected for multiple comparisons within the entire volume
In addition to the LOC clusters, we also observed significant activation in the right parietal cortex. On the basis of previous findings (Grill-Spector et al., 1998a,b; Mendola et al., 1999), this cluster is likely situated in and just superior to V3A. Interestingly, in the across subjects analysis of Mendola et al. (1999), this area showed a small but statistically significant BOLD signal modulation to IC presence versus absence. Responses in V3A have likewise been reported in a recent study of cue-invariant object recognition (Grill-Spector et al., 1998a), implicating a role for dorsal visual areas in object processing (Sugio et al., 1999) (results of experiment 4, below). The present results may be detecting a similar modulation in parietal areas. However, the SCD topography and source analysis results of experiment 1 would argue against a prominent role of parietal areas in the earliest phase of the IC effect.
EEG/fMRI co-registration and source analysis
To better interpret the IC effect observed in the VEP, we co-registered both the EEG and fMRI data sets from the same five subjects into the 3-D reconstruction of one subject's (B.H.) anatomical MRI and constrained the location of dipole sources to the fMRI activation clusters. By this approach we were able to visualize the results from both neuroimaging techniques concurrently in the same coordinate space. This representation of the data emphasizes both the high temporal resolution of the electrophysiological technique as well as the high spatial resolution of fMRI. As in experiment 1, source analyses were performed on the difference between the group-averaged (n = 5) VEP responses from the IC present minus IC absent conditions. We fixed the locations and orientations of three dipoles to the Talairach coordinates of the fMRI clusters listed in Table 2 (Fig. 6A). Figure 6B displays the percentage of the variance explained (across all recording channels) by these three dipoles over a ±20 msec window surrounding the peak (∼120 msec) of the earliest IC sensitivity. On average, these dipoles explained 87.7% of the variance over the 106–146 msec epoch. Figure 6C plots the strength of each source over the epoch after onset of IC sensitivity (80–300 msec). It is important to emphasize that the fMRI clusters seen in this experiment almost certainly represent the activity of more than a single functional area within the LOC, just as the electrophysiological effect represents co-coordinated activity within a cluster of LO regions rather than a discrete activation of a single region. Thus, fitting each cluster with a single equivalent current dipole clearly represents an overly simplified modeling of the activity within this complex. As such, these dipoles should be considered to represent a “center of gravity” rather than a discrete neural locus.
VEP/fMRI co-registration and source analysis.A, The locations of the fMRI activation clusters (yellow) and VEP dipoles (blue), both from analyses of the group (n = 5) data of experiment 2, are shown spatially co-registered on the 3-D reconstruction of one subject's anatomical MRI. B, On average, these dipoles account for 87.7% of the variance between the observed data and the forward solution to these dipoles over the 106–146 msec post-stimulus epoch. C, The strength of these dipoles over the 80–300 msec post-stimulus epoch reveals the relative contribution of each source over time.
The co-registration of electrophysiological data (ERP) and hemodynamic data (fMRI - BOLD) from the same subjects (so-called multimodal imaging) has been widely acclaimed as a major advance in our abilities to detail the spatiotemporal dynamics of brain function, and such methods have been applied extensively in recent research (Simpson et al., 1995a,b; Martinez et al., 1999; Bonmassar et al., 2001;Dale and Halgren, 2001). However, the co-registration of these separate data sets is based on the premise that the same neural events are represented in both, and therefore it is critical to consider the relationship or coupling between the physiological phenomena underlying the different signal modalities. Within a single neuron, the current flow generated when postsynaptic receptors are bound by their respective neurotransmitter(s) results in both EPSPs and IPSPs, the balance and spatial distribution of which determine whether the neuron will generate an action potential. Synchronous transmembrane current flow in a population of neurons produces a macroscopic version of the PSP, the local field potential (LFP). The LFP distribution in the extracellular medium has a lawful spatial relationship to the net direction (i.e., inward or outward) and macroscopic anatomical profile of the current flow pattern across the active cellular elements (Freeman and Nicholson, 1975; Schroeder et al., 1995). A portion of these LFPs volume conduct to the scalp surface, and it is these and not the action potentials that are recorded during ERP studies (Mitzdorf, 1991; Schroeder et al., 1995). In fact, in most cases, transmembrane current flow does not lead to action potentials because the net combination of EPSP and IPSP results in either subthreshold activation or inhibition. In such cases, action potential measures might suggest low activity in a region, although a significant LFP is being generated. Clearly, the LFP would be a likelier index of metabolic activity in these cases and would be a likelier correlate of the BOLD response. Furthermore, transmembrane current flow has a clearer a priori relationship to hemodynamic responses than do action potentials. The reason is that most of the cortical energy production of a neuron, and therefore its metabolic load, supports functional (synaptic) glutamatergic neuronal activity (Sibson et al., 1998). A measure of support for these contentions is found in a recent study in which intracranial electrophysiologic recordings were compared with fMRI data concurrently recorded from the same monkeys (Logothetis et al., 2001). These authors found that the LFP (measured as activity between 10 and 130 Hz) was a better estimate of the BOLD response than multiunit activity (300–3000 Hz). They interpreted their data, which showed a strong correlation between the LFP and the BOLD response, as strong evidence that both were indexing the same physiological events in the brain.
Experiment 3: the IC effect and contrast polarity
Several sources of evidence indicate that the processing of illusory contours and figure-ground segregation are independent of the contrast polarity of the inducers. Intracranial studies of macaques reveal that figure-ground responses (Baumann et al., 1997; Zhou et al., 2000; Peterhans and Heitger, 2001) and illusory contour presence versus absence (Heider et al., 2000) are signaled in lower-tier areas independent of contrast polarity. Psychophysical studies demonstrate that illusory contour strength persists during variation in the contrast polarity across inducers (Prazdny, 1983, 1986; Shapley and Gordon, 1985; Dresp and Grossberg, 1997; Victor and Conte, 1998;Spehar, 2000) and subjective brightness (Ware, 1981) of the induced shape. Similarly, recent fMRI results from humans demonstrate contrast polarity independence for the differential activation to IC presence versus absence in LOC regions (Mendola et al., 1999). We therefore sought to determine whether the timing and locus of the IC effect was independent of the contrast polarity between inducers and the background.
Behavioral results
Subjects correctly indicated the presence or absence of IC shapes 96.3% (±3.2%) of the time. There was no difference in performance across contrast polarities (t16 = 1.48; p = 0.16) from the data of those subjects who participated in both experiments 1 and 3.
Electrophysiological results
VEP morphology was similar to that seen in experiment 1, and direct comparison of the IC present and IC absent conditions revealed a similar IC effect (Figs. 3B,7). As before, the rising phase of the P1 component showed neither amplitude nor latency modulation with the presence versus absence of IC shapes. In agreement with the results of experiment 1, a large difference in waveform morphology was apparent during the falling phase of the P1 component.
Experiment 3 VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off). Data are shown in an identical manner as in Figure 2 from two representative electrode sites (PO5and PO6).
Point-wise paired t tests between the VEP for the IC present versus IC absent condition were calculated to determine the onset latency of the IC effect (identical criterion as in experiment 1). The earliest IC effect onset was at 104 msec after stimulus at scalp site P3. IC effect onset latencies at the remaining posterior scalp sites ranged from 106 to 112 msec, with near simultaneous onset over both hemispheres (110 msec at site PO4). The IC effect was largest at sites PO5 and PO6, peaking at ∼162 msec after stimulus onset. No IC effect was observed over frontal sites. Figure 7 displays the VEP waveforms as well as the results of point-wise t tests from sites PO5 and PO6.
For each subject and stimulus condition, we then calculated the area (vs the 0 μV baseline) for the 152–172 msec post-stimulus window at sites PO5, PO6, P5, and P6 and submitted these area measures to a 2 × 2 × 2 repeated measures ANOVA. The within subjects factors were stimulus condition (IC absent vs IC present), hemisphere (left vs right), and electrode (two over each hemisphere). There was a main effect of stimulus condition (F(1,16) = 37.51; p < 0.001), which confirmed the results of the point-wise ttests. There was no main effect of hemisphere (F(1,16) = 2.30; p = 0.15). However, there was a significant interaction between the factors of stimulus condition and hemisphere (F(1,16) = 4.45; p < 0.05), indicating a larger IC effect over the right versus left hemisphere.
As before, we were interested in registering the latency of the IC effect within the framework of visual cortical response onset. We determined the onset latency of the earliest activity at each of the 64 scalp sites by calculating point-wise paired t tests (two-tailed) between the 0 μV baseline and the VEP response collapsed across IC presence and absence. The first time point forward from 30 msec post-stimulus onset where the response exceeded the 0.05 α criterion for at least 11 consecutive data points was labeled as the VEP response onset at that site. The earliest VEP response onset latency was 52 msec (sites P3 and P4), resulting in a lag of 52 msec until the onset of the IC effect.
SCD topographic maps of the group-averaged IC present minus IC absent difference are shown in Figure 3B. As in experiment 1, scalp topography of the IC effect is consistent with lateral–occipital areas and is larger over the right hemisphere. Moreover, no current density foci are observed over frontal scalp regions. In addition to the SCD maps, Figure 11 (top right panel) displays a statistical cluster plot of the results of point-wise ttests for the IC effect at 55 of the 64 electrodes over the 500 msec post-stimulus epoch. Two phases of the IC effect over lateral–posterior scalp sites, as well as the absence of a robust frontal effect, are again readily apparent (∼100–250 and ∼325–425 msec).
Experiment 4: stimulus features affecting scene analysis
Because a potential criticism of the findings of experiments 1–3 might be that the present VEP measurements are insensitive to earlier modulations, originating in lower-tier visual areas, we performed a further analysis of these data. Variation of low-level stimulus features (such as those listed in Table 1) is one means of optimizing the ability to detect modulations in lower-tier visual areas. For example, early modulation of the VEP (∼60 msec after stimulus onset) was observed for Kanizsa-type stimulus configurations consisting of four versus three inducers, independent of IC presence versus absence (Hermann et al., 1999). This modulation of the P1 component was interpreted to reflect an overall brightness difference between stimulus configurations with more versus fewer inducers and demonstrates the sensitivity of the VEP to changes in low-level stimulus features. Our question then is whether the low-level features that varied across the stimulus set used in experiments 1 and 3 produce modulations of the VEP responses before the IC effect. Such a result would indicate that our methodological approach is sensitive to modulation of activity in lower-tier visual areas. Such a finding would bolster our contention that the IC effect described in experiments 1–3 does indeed index the earliest IC sensitivity in cortex.
We used the stimuli from experiments 1 and 3 to address this question of scene analysis as well as to optimize the likelihood of observing modulations in lower-tier visual areas. We separately averaged the VEP responses to each of the IC present and corresponding IC absent inducer configurations for each of the five shapes (circle, pentagon, star, square, and triangle) used in experiments 1 and 3. Because the results of experiment 3 indicated that the IC effect is insensitive to contrast polarity, we collapsed the data sets from the 17 subjects who participated in both experiments 1 and 3. Recall that across these five shapes, several stimulus parameters varied: (1) the number of inducers, (2) support ratio, (3) surface area of the induced IC shapes, and (4) farthest eccentricity of any inducer (Table 1).
Electrophysiological results
Consistent with our description of the timing of IC sensitivity, we find no evidence for an IC effect before 88 msec after stimulus onset for any individual shape (data not shown). Rather, there was early (∼66 msec) modulation of the VEP responses to the square and triangle (both IC present and absent configurations) versus each of the other three stimulus shapes. In further contrast to the IC effect, this modulation began over posterior midline scalp and remained restricted primarily to central parieto–occipital scalp with some later but weak modulation over lateral parieto–occipital scalp (Fig.8). The mainly central occipital distribution of this effect is consistent with modulations of lower-tier areas. To assess which stimulus parameters influence this modulation, we calculated the area (vs the 0 μV baseline) over the 66–86 msec post-stimulus epoch for each shape and stimulus configuration and submitted these values to a 5 × 2 × 3 repeated measures ANOVA. The within subjects factors were shape (circle, pentagon, star, square, and triangle), stimulus configuration (IC present and IC absent), and scalp site (P7, Pz, and P8). There was a main effect of shape (F(4,13) = 8.69; p = 0.001), but no main effect of either configuration (F(1,16) = 0.32;p = 0.58) or scalp site (F(2,15) = 0.56; p = 0.58). Only the interaction between the factors of shape and scalp site was significant (F(8,9) = 9.85;p < 0.001), indicating that the effect of shape varied across scalp sites. A series of post hoc t tests (Table3) using these area measures collapsed across IC present and IC absent stimulus configurations reveal a graded amplitude decrease across shapes over posterior midline scalp (Fig.8A). An overlay of the shapes used in this experiment reveals that this amplitude decrease corresponds with the maximal eccentricity of inducer elements (Fig. 8B). Next, we collapsed the VEP responses (both IC present and IC absent) from the triangle and square and contrasted them against the corresponding collapsed data from all other shapes and stimulus configurations (hereafter “wide” and “narrow” extent, respectively) (Fig.8C). The SCD topography of this wide minus narrow difference is displayed in Figure 8D at 70 msec after stimulus onset and illustrates its central parieto–occipital distribution. Hereafter, we refer to this wide minus narrow difference as the configuration effect. In addition to its earlier timing, this configuration effect contrasts with the lateral–occipital distribution that characterizes the IC effect (Fig. 8C, compare with Fig.3).
Shape-wise analyses demonstrating the configuration effect. A, Bar graphs display the mean area (66–86 msec after stimulus) of the VEP response (sitesP7, Pz, and P8) to each shape independent of IC presence versus absence. A step function in mean area is observed over midline but not lateral scalp sites. Asingle asterisk indicates a significant difference (p < 0.01; paired t test) in mean area between both the response to the square and triangle versus a particular shape. The double asterisk indicates a significant difference (p < 0.01; pairedt test) in mean area between the circle and each of the other shapes. Mean area would appear to follow the eccentricity of inducers. B, A diagram of each of the IC shapes (thick black lines) indicates the variation in the farthest eccentricity of any inducer across shapes. C, VEP waveforms from sites P7, Pz, andP8. These data have been collapsed acrosswide shapes (square and triangle) andnarrow shapes (star, pentagon, and circle).D, SCD topographic maps of the wide − narrow difference at 76 msec after stimulus onset.
Shape-wise t tests (experiment 4)
The configuration effect reveals stages of scene analysis preceding IC sensitivity that appear to modulate according to the spatial distribution of inducers rather than to several other low-level stimulus parameters (Table 1). We posit that the maximal eccentricity of inducer elements (Table 1, φ) demarcates a coarse extent of visual feature space and is the determining parameter in this early modulation. Several findings from this analysis support this hypothesis. In contrast to Hermann et al. (1999), we observe no difference in the VEP responses to the triangle versus square (Table3), arguing against a difference in overall brightness (caused by the number of inducers) as an explanation of this early modulation. Rather, there was a significant amplitude difference between the responses to the square and circle, both of which are defined by four inducers. Likewise, there were smaller amplitude responses from stimuli defined by five inducers, relative to those defined by fewer inducers. This pattern also is in contrast to the prediction based on a strict interpretation of a “spotlight” of spatial attention (Eriksen and Yeh, 1985; Stöffer, 1994), because stimulus displays containing more local elements (inducers) should have yielded a larger response. The induced surface area (λ) of IC shapes is highly unlikely to be the mediating parameter, because the configuration effect occurs when IC shapes are both present and absent. Likewise, similar responses are observed across shapes with large surface area differences (Table 3, pentagon vs star). Variation in support ratio (μ) likewise cannot account for this modulation. The pattern of VEP response amplitudes across shapes does not follow the prediction of a linearly increasing variation across support ratios (Gegenfurtner et al., 1997), because by this reasoning the pentagon would be expected to yield the largest early VEP response.
Experiment 5: the IC effect and lateral presentations
It is also possible that our results from experiments 1–3 were biased toward the involvement of higher-tier visual areas, because centrally presented IC shapes were relatively large (maximally 6°) and the inducers were quite eccentric. Reducing the gap between inducing elements has been used experimentally to bias IC processing toward lower-tier visual areas. However, IC processing appears to be size invariant and did not produce modulation in lower-tier areas despite robust modulation in the LOC area (Mendola et al., 1999). It is important to note, however, that changes in gap width when inducers still straddle the vertical meridian limit the possible biasing of IC processes into lower-tier areas because of the anatomical and physiological properties of lower-tier visual areas. That is, when inducers straddle the vertical and horizontal meridians, IC borders must be established between anatomically separated cortical representations.
One physiological implication is that the formation of IC borders, even for IC shapes induced over small gaps, would be preceded by interhemispheric and intrahemispheric neural response interactions when stimuli are spatially dispersed across the horizontal and vertical meridians (Murray et al., 2001). Such interactions are mediated by horizontal and callosal connections. Candidate areas mediating the earliest IC sensitivity may therefore be restricted to those with sufficient callosal and horizontal connectivity (Van Essen et al., 1982; Tootell et al., 1988, 1998; Clarke and Miklossy, 1990). In agreement with this possibility, a recent VEP study from our laboratory has shown that these interactions are delayed relative to the initial cortical response onset by ∼25 msec (Murray et al., 2001).
A further limitation is based on the observation that the induction of IC sensitivity requires a minimal stimulus extent that is larger than the classical receptive fields of neurons in area V1 or V2 (von der Heydt et al., 1984; Peterhans and von der Heydt, 1989; von der Heydt and Peterhans, 1989). One implication of this observation is that IC sensitivity relies on contextual modulations (Lamme and Spekreijse, 2000). Thus, at least two possible mechanisms of IC sensitivity present themselves. Either long-range horizontal connections within a visual area (intrinsic and callosal) or feedback projections from higher-tier visual areas with sufficiently large receptive fields to span the inducing elements would be required to process IC stimuli (Rockland and Pandya, 1979; Gilbert, 1983; Maunsell and Van Essen, 1983; Van Essen et al., 1994; Gilbert et al., 1996; Spillmann and Werner, 1996). Evidence for top-down (i.e., feedback) modulation of activity in lower-tier visual areas has been shown in macaque V1 during figure-ground segregation (Kapadia et al., 1995, 1999; Lamme, 1995; Zipser et al., 1996; Hupé et al., 1998; Lamme et al., 1998a,b, 1999;Lamme and Spekreijse, 2000). These modulations occurred ∼80–100 msec after stimulus onset, considerably later than the initial onset of activity in the same V1 neurons (Zipser et al., 1996; Lamme and Spekreijse, 2000), and are suppressed by anesthesia (Lamme et al., 1998b), consistent with reentrant, feedback modulations from higher-tier areas (Lamme and Roelfsema, 2000). This time frame for contextual modulations in area V1 corresponds well to the results of a recent intracranial investigation that focused on the latency of IC sensitivity across the cortical layers of areas V2 and V1 (Lee and Nguyen, 2001). In this study, the earliest differential response to Kanizsa-type IC stimuli onset at 70 msec was in superficial layers of V2, with an effect in both deep layers of V2 as well as superficial layers of V1 lagging by ∼25–30 msec (Lee and Nguyen, 2001).
An alternative means of biasing illusory contour processing may be to vary the retinotopic position of inducers such that interhemispheric interactions (and potentially the involvement of higher-tier areas with large bilateral receptive fields) would not be required for IC processing. Lateral presentations should distinguish between two possible hypotheses of IC processing. The first contends that if IC sensitivity is a low-level and bottom-up process, then confining inducers to a single visual field should result in earlier modulation of the VEP than when centrally presented, because this would potentially be mediated by long-range local connectivity. Likewise a change in the scalp topography of the IC effect would be expected (relative to central presentations) that should be consistent with lower-tier visual areas. A second hypothesis contends that object recognition of centrally presented stimuli would be faster than when stimuli are laterally presented, in part because of the overrepresentation of central vision in cortex (Popovic and Sjostrand, 2001). Several additional lines of evidence support this hypothesis. Previous electrophysiological studies of laterally presented IC shapes did not observe differential modulation of the VEP response until ∼224 msec after stimulus onset (Brandeis and Lehmann, 1989), which is considerably later than our IC effect with centrally presented stimuli. Psychophysical studies involving the discrimination of illusory shapes of contours reveal that performance is best when stimuli appear centrally rather than laterally or either above or below fixation (Rubin et al., 1996) [see also Juttner and Rentschler (2000) for demonstrations with object categorization]. Still others demonstrate that performance on shape discrimination tasks is improved when stimuli are distributed between the left and right visual hemifields (Banich and Belger, 1990; Mohr et al., 1994). The prediction from these collective data would therefore be that the IC effect for laterally presented stimuli would shift later relative to central presentations. In experiment 5, we presented IC stimuli laterally to one or the other visual hemifield to distinguish between these two hypotheses.
Behavioral results
Subjects correctly indicated the presence or absence of IC shapes in the left visual field 97.8% (±2.3%) of the time and in the right visual field 97.5% (±2.2%) of the time. There was no difference in performance across visual fields (t11= 0.96; p = 0.36). For the eight subjects who also participated in experiment 1, there was no difference in the accuracy of performance when stimuli were centrally presented versus when they were presented to the left (t7 = 1.06; p = 0.32) or right (t7 = 0.72; p = 0.49) visual field. However, it is important to recall that subjects were able to self-pace stimulus delivery and that accuracy was emphasized over speed. Subjective reports on debriefing of subjects who had participated in experiment 1 indicated that the IC shapes did not “pop out” with the same saliency as when they were presented centrally.
Electrophysiological results
As in experiments 1 and 3, inspection of group-averaged VEPs for the IC present and IC absent conditions revealed the traditional series of VEP components, including P1, N1, and P2. These components were maximal over posterior scalp sites. Direct comparison of these conditions revealed a differential response to laterally presented IC shapes over posterior scalp sites (Fig.9). The P1 and N1 components showed neither amplitude nor latency modulation with the presence versus absence of IC shapes. In contrast, two large, successive differences in waveform morphology were apparent after the P2 component.
Experiment 5 VEP waveforms in response to left and right visual field presentations (40 Hz low-pass filter; 24 dB/octave roll-off). Data are shown in an identical manner as in Figure 2 from one contralateral representative electrode site for left visual field (PO6) and right visual field (PO5) stimulus presentations.
As before, to determine the onset latency of the IC effect with lateralized stimuli, we calculated point-wise paired t tests between the VEP for the IC present versus IC absent condition. For each electrode, the first time point where this comparison exceeded the 0.05 α criterion for at least 11 consecutive data points was labeled as the onset of the IC effect. For left visual field presentations, the earliest IC effect onset at 218 msec after stimulus was at scalp site PO4, and at 220 msec it was at site PO3. For right visual field presentations, the earliest IC effect onset at 202 msec after stimulus was at scalp site PO3, and at 214 msec after stimulus onset it was at site PO4. For both left and right visual field stimuli, the IC effect was largest at sites PO5 and PO6, peaking at 242 msec after stimulus onset over the hemisphere contralateral to the stimulated visual field and at 252 msec over the ipsilateral hemisphere (Fig. 9). No IC effect was observed over frontal sites in response to either visual field presentation. Figure 9 displays the VEP waveforms, as well as the results of point-wise t tests, for both left (PO6) and right (PO5) visual field presentations.
For each subject and stimulus condition, we then calculated the area (vs 0 μV baseline) for the 236–256 msec window at sites PO5, PO6, P5, and P6. For each visual field of presentation, we then submitted these area measures to a 2 × 2 × 2 repeated measures ANOVA. For each of the two ANOVAs, the within subjects factors were stimulus condition (IC absent vs IC present), hemisphere (left vs right), and electrode (two over each hemisphere). For left visual field presentations, there was a main effect of stimulus condition (F(1,11) = 13.82; p < 0.003), which confirmed the results of the point-wise ttests. There was no main effect of hemisphere (F(1,11) = 0.33; p = 0.58). Furthermore, there was no interaction between the factors of stimulus condition and hemisphere (F(1,11) = 1.88; p = 0.20), indicating an equivalent IC effect over both hemispheres. None of the other tests reached significance. For right visual field presentations, there was a main effect of stimulus condition (F(1,11) = 22.12; p < 0.001), which confirmed the results of the point-wise ttests. There was no main effect of hemisphere (F(1,11) = 1.49; p = 0.25); however, the interaction between the factors of stimulus condition and hemisphere approached the significance criterion (F(1,11) = 4.53; p = 0.06), suggestive of a larger IC effect over the left (contralateral) hemisphere.
It may be contended that the later IC effect latency in this experiment reflects a more general lag in visual cortical processing for lateral versus central presentations. We assessed this possibility by registering this IC effect within the framework of the visual cortical response by determining the onset latency of the earliest activity at each scalp site. For each of the 64 scalp sites, point-wise pairedt tests (two-tailed) were conducted between the 0 μV baseline and the collapsed response across stimulus conditions. The first time point forward from 30 msec post-stimulus onset where the response exceeded the 0.05 α criterion for at least 11 consecutive data points was labeled as the VEP response onset at that site. This latency was 62 msec for left and 64 msec for right visual field stimuli, ∼10 msec later than when stimuli were centrally presented. From this VEP response onset latency, we calculated a lag of ∼120 msec until the onset of the IC effect for laterally presented stimuli.
It is noteworthy that despite the dramatic latency shift in the IC effect, we observed no drop in performance accuracy between central and lateral presentations. The most parsimonious explanation is that we are observing a ceiling effect, because the determination of IC presence versus absence is a relatively trivial task as evidenced by >94% accuracy in all conditions. One possibility is that performance differences might have been observed had we asked our subjects to make a more fine-grained discrimination (e.g., discriminate between shapes). Likewise, this pattern of results would predict that reaction times, in correspondence with the latency of the IC effect, would be slower for lateral versus central presentations. The design of the present study, however, did not include collection of reaction times, because subjects responded after a response prompt to minimize the overlap of motor potentials with the early VEP components. Therefore, this prediction will be tested in future studies.
SCD topographic maps of the group-averaged IC present minus IC absent difference are shown in Figure 10. As with centrally presented stimuli, the scalp topography of the IC effect indicates that IC sensitivity occurs first over lateral–occipital areas of the hemisphere contralateral to stimulus presentation (although also involving the ipsilateral hemisphere). No current density foci are observed frontally. This is visualized more extensively in Figure 11 (bottom two panels), which displays statistical cluster plots of the results of point-wise t tests for the IC effect at 55 of the 64 electrodes over the 500 msec post-stimulus epoch. In sharp contrast to the results of experiments 1–3, the IC effect for laterally presented stimuli onsets is much later and is composed of a single phase (∼200–275 msec).
SCD topographic maps (left-sided, back, and right-sided views) of the IC effect for laterally presented stimuli of experiment 5. Top, SCD topographic maps at 236 msec after stimulus onset depicting the IC effect in response to left visual field stimuli. Bottom, SCD topographic maps at 236 msec after stimulus onset depicting the IC effect in response to right visual field stimuli. These SCD foci are consistent with bilateral lateral–occipital generators, although more pronounced over the contralateral hemisphere. Polarity of these maps is arbitrary, depending on the direction of the subtraction, and scales are shown.
Statistical cluster plots. Color values indicate the result of point-wise t tests evaluating the IC effect across post-stimulus time (x-axis) and electrode positions (y-axis) for 55 of the 64 electrodes (see Materials and Methods for details of electrode locations) used in experiments 1, 3, and 5. For clarity, only pvalues < 0.01 are color encoded. Top, Results from experiments 1 and 3 using centrally presented inducers (left, gray background;right, black background) indicate a biphasic IC effect over posterior scalp sites. No robust IC effect is observed over frontal sites. Bottom, Results from experiment 5 using laterally presented inducers indicate that the IC effect observed with centrally presented inducers shifts ∼120 msec later. As with centrally presented stimuli, no IC effect is observed over frontal sites. The lag in onset of the IC effect over the direct (contralateral) and indirect (ipsilateral) hemispheres can be readily seen for both left and right visual field presentations.
The IC effect for laterally presented stimuli is shifted ∼120 msec later relative to when stimuli are presented centrally. Importantly, for both central and lateral presentations, the latency of VEP response onset is indistinguishable, and the scalp topography of the IC effect is highly similar. A comparison of Figures 3 and 10 shows this general topographic similarity, suggesting the activation of a similar complex of LOC generators in both cases; however, some clear variations are apparent. Given the relative crudeness of the ERP topographic technique for spatially localizing intracranial generators, it would be premature to interpret these differences to any great extent. Nonetheless, it is reasonable to predict that although central and lateral stimuli both result in LOC activation, there is likely to be some variation in the relative contributions to this effect from the sub-areas that comprise the LOC region. Clearly, further investigation will be required to test this prediction.
More importantly, this pattern of results is not consistent with the prediction that IC processing is a strictly feedforward, bottom-up phenomenon. Likewise, this shift in the latency of the IC effect contrasts with previous VEP studies in which similar latencies (∼200 msec) for the IC effect have been observed for both central (Sugawara and Morotomi, 1991; Tallon-Baudry et al., 1996, 1997) and lateral presentations (Brandeis and Lehmann, 1989). That is, the onset of our IC effect for central presentations is earlier than in previous studies, whereas the timing of our IC effect for lateral presentations is in agreement with the results of Brandeis and Lehmann (1989).
DISCUSSION
High-density electrical mapping, source analysis, and fMRI were combined to elucidate mechanisms underlying IC processing in humans. We present four main findings. (1) The VEP modulated with IC presence versus absence—the IC effect. This effect onset was at 88 msec, it followed cortical response onset by 40 msec, and it was robust across contrast polarities. (2) Lateral presentations shifted IC effect onset 120 msec later. From these latencies, we propose that the earliest IC sensitivity is unlikely to occur during the initial phase of activation in lower-tier hierarchical areas. (3) Strongly supporting this hypothesis are combined VEP and fMRI results localizing the earliest IC effect to the LOC, regardless of the shape or the retinotopic position of the IC. (4) Visual feature analyses preceded IC sensitivity over central parieto–occipital scalp—the configuration effect. Collectively, our data support a model of IC processing wherein dorsal stream regions, which are initially insensitive to IC presence, coarsely demarcate the spatial extent of a given stimulus array and then input to ventral stream structures (e.g., the LOC area), where IC sensitivity first occurs. Our data support the notion that IC effects previously observed in lower-tier areas are likely to be driven by feedback (perhaps in combination with horizontal) inputs from higher-tier areas.
Timing of IC processing
The earliest VEP modulation to IC shape presence versus absence was onset at 88 msec. In contrast, the initial visual cortical response was onset at 48 msec. Thus, there is a 40 msec lag between cortical response onset and IC effect onset, which is sufficient for signal transmission to the LOC area. Response onset in human LOC areas can be predicted from intracranial recordings detailing the timing of activation across cortical regions (Schroeder et al., 1998) in awake monkeys. Onset latencies are ∼25 msec in V1 (Maunsell and Gibson, 1992; Givre et al., 1994; Schroeder et al., 1998, 2001) and ∼50 msec in IT (Schroeder et al., 1998, 2001; Mehta et al., 2000a,b). By applying a 3:5 ratio to draw correspondence between monkeys and humans (Schroeder et al., 1995, 1998, 2001), a conservative estimate of human V1 onset is 40–50 msec (consistent with the current findings) and 80–100 msec in LOC areas. From these estimates, the IC effect after central presentations is likely to commence with or shortly after initial activity in higher-tier ventral stream areas.
These timing data can be applied similarly to the interpretation of a recent investigation of IC processing dynamics in monkeys (Lee and Nguyen, 2001). The earliest IC response occurred in superficial V2 (70 msec), followed by deep V2 (95 msec) and superficial V1 (100 msec). Unfortunately, activation onset latencies were not reported. However, on the basis of known anatomical connections (Felleman and Van Essen, 1991) and previous studies in monkeys (Givre et al., 1994; Schroeder et al., 1998, 2001; Mehta et al., 2000a,b), this laminar activation profile is characteristic of a feedback input pattern. Likewise, mean latencies in V1 and V2 of the awake monkey are 24 and 32 msec, respectively (Schroeder et al., 1998). Therefore, the findings of Lee and Nguyen (2001) are unlikely to reflect modulation of the initial afferent volley to V1 or V2. This hypothesis is further supported by the impaired discrimination of IC shapes after lesions in V4 (De Weerd et al., 1996) or IT cortex (Huxlin and Merigan, 1998; Huxlin et al., 2000) and studies of contextual modulation (Hupé et al., 1998;Lamme and Spekreijse, 2000), as well as by the present SCD topography, dipole analysis, and fMRI results that all localized the IC effect to the LOC area.
Lateral versus central presentations reveal modes of object processing
Several lines of evidence support the interpretation that the temporal disparity in the IC effect for central versus lateral presentations likely represents successive stages of differential activation in the LOC area. Previous studies of object recognition from our laboratory have revealed a series of VEP modulations that were interpreted to reflect a change in the mode of object recognition (Doniger et al., 2000, 2001). We observed a late modulation, termed Ncl (∼290 msec peak) associated with perceptual closure processes leading to object identification of fragmented line drawings (Doniger et al., 2000, 2001). Identification under these conditions was considered to be effortful and to rely on semantic/episodic memory representations. An earlier modulation of the N1 component was observed for object recognition as a result of repetition priming, which was interpreted as an instance of automatic recognition based on a sensory trace (Doniger et al., 2001). Similar to the N1 and Ncl modulations of Doniger et al. (2000, 2001), the latency shift in the present IC effects for central and lateral presentations may be reflecting a shift in object recognition mode from automatic and “perceptual” to effortful and “conceptual” (Ritter et al., 1982; Tulving and Schacter, 1990;Humphreys et al., 2000). The implication is that these lateral presentations rely more on perceptual closure processes than on rapid object recognition. That is, the IC effect after lateral presentations resulted in modulation of the VEP very similar to the Ncl observed during perceptual closure paradigms, whereas the corresponding IC effect after presentations of stimuli centrally yielded modulation of the N1 as well as the Ncl componency. These early (in the case of central presentations) and late modulations (in the cases of both central and lateral presentations) can be readily visualized in Figure11.
Locus of IC processing
Our combined VEP and fMRI results localized the IC effect to the LOC area, which has been implicated repeatedly in object recognition (Malach et al., 1995; Moscovitch et al., 1995; Vanni et al., 1996;Grill-Spector et al., 1998a,b; Ishai et al., 1999; Kourtzi and Kanwisher, 2000), perceptual closure (Doniger et al., 2000, 2001), and IC sensitivity (Hirsch et al., 1995; ffytche and Zeki, 1996; Larsson et al., 1999; Mendola et al., 1999; Seghier et al., 2000). The LOC area is considered the homolog of macaque IT cortex (Gross et al., 1972; Sary et al., 1993; Kobatake and Tanaka, 1994) and has large, bilateral receptive fields (Kobatake and Tanaka, 1994; Tootell et al., 1998). Critically, areas V1 and V2 have small receptive fields and lack substantial callosal inputs, making it unlikely for these areas to mediate IC effects with large, centrally presented stimuli used in the present study (although these areas are likely to be involved in IC processing using feedback from areas like the LOC area). Object recognition in higher-tier areas has further been shown to be size, position, and cue invariant (Sary et al., 1993; Lueschow et al., 1994;Ito et al., 1995; Grill-Spector et al., 1998a). Some of these properties have recently been generalized to include IC processing (Mendola et al., 1999). Here, we replicate the localization of IC processing to the LOC area of Mendola et al. (1999) and further extend this finding to include both centrally and laterally presented IC shapes, demonstrating the position invariance of IC sensitivity in the LOC area. Although the lateral displacement of the inducer stimuli appears not to alter the locus of IC sensitivity, it would appear to shift its mode from automatic to effortful. Although it may be contended that Kanizsa-type stimuli biased IC processing toward higher-tier regions and that other IC varieties would have better detected IC processing in lower-tier areas, recent fMRI results suggest a central role for the LOC area in the processing of multiple IC varieties (Mendola et al., 1999). Nonetheless, one question for future research is the extent to which the dynamics of the present results generalize across IC varieties.
Hemispheric asymmetry of IC processing
This study found that although both hemispheres are involved in IC processing, the IC effect is larger over the right versus left hemisphere after central presentations, but not after lateral presentations. In the case of lateral presentations, the onset of the IC effect occurs first over the contralateral hemisphere with activation over the ipsilateral hemisphere lagging by ∼10 msec (consistent with estimates of interhemispheric transfer time) (Murray et al., 2001). However, the amplitude (measured from a 20 msec epoch centered on the peak of the IC effect over the contralateral hemisphere) did not differ over the two hemispheres in the case of lateral presentations. Further experiments will be required to resolve any hemispheric asymmetry of illusory contour processing.
Nonetheless, one hypothesis would maintain that the right hemisphere is specialized for global form processes, whereas the left hemisphere is specialized for analytic local processes (Robertson et al., 1988; Atchley and Atchley, 1998; Corballis et al., 1999; Han et al., 2000a,b). Psychophysical investigations of IC perception, however, provide no clear consensus regarding hemispheric dominance (Mattingley et al., 1997; Rasmjou et al., 1999), whereas a previous VEP study of attention and IC processes revealed simultaneous and symmetric effects for both left and right visual field presentations (Brandeis and Lehmann, 1989). The asymmetric IC effect after central presentations may reflect a relatively greater contribution of global processing in perceiving Kanizsa-type IC shapes. Recent reports describe a similar asymmetry during perceptual closure of fragmented images (Wasserstein et al., 1984; Doniger et al., 2001) as well as when selective attention to global versus local features of a stimulus array was manipulated directly (Fink et al., 1997; Heinze et al., 1998; Han et al., 2000a,b). It may instead be the case that global processes predominate in early perceptual processing preceding IC sensitivity (Navon, 1977; Sekuler, 1994). The configuration effect described in experiment 4 supports this postulation in so far as this early VEP modulation is sensitive to the overall, global distribution of inducer elements in the visual field, rather than to their number or IC presence (Fig. 8, Table 1). It is critical to note that despite its magnitude asymmetry, the onset of the IC effect occurs simultaneously over left and right hemispheres (when stimuli appear centrally), suggesting that both hemispheres perform IC completion processes in parallel, even if potentially performing different types of scene analysis.
Models of IC processing
Although an extensive review of existing models is beyond the scope of this study (Mendola, 2002), the present study endorses some modifications and highlights the importance of temporal information in modeling brain processes (Schroeder et al., 1998, 2001; Martinez et al., 1999). This study temporally and spatially dissociated several VEP modulations in response to Kanizsa-type IC shapes. At 66–86 msec the configuration effect appears to track the extent of inducers in the visual field. This modulation, focused over central parieto–occipital scalp, is insensitive to IC presence. In contrast, the IC effect (88–100 msec), focused over lateral–occipital scalp sites, is sensitive to IC presence independently of inducer extent. However, this later modulation is dependent on the retinotopic position of IC shapes, and its onset shifts 120 msec later with inducers confined within a hemifield. The timing of these modulations with respect to the onset of the visual cortical response supports the view that IC sensitivity as described previously in areas V2 and V1 may reflect modulation caused by feedback inputs from higher-tier visual areas. The present data thus support a three-phase model of IC processing wherein dorsal stream regions, mediating the configuration effect, establish a coarse global representation of object space that guides the subsequent IC effect in LOC areas of the ventral stream (Schroeder et al., 1998; Vidyasagar, 1999), in turn provoking feedback to lower-tier areas (e.g., V2 and V1).
Although this study highlights the importance of feedback modulations for IC sensitivity in lower-tier areas, the role of horizontal inputs should not be disregarded (Das and Gilbert, 1999; Mendola, 2002). In fact, our proposition of dorsal stream guidance of subsequent ventral stream IC processing is supported by the finding that the initial activity in ventral stream areas is consistent with lateral (vs feedforward) inputs, perhaps from dorsal stream areas (Schroeder et al., 1998). Likewise, several laboratories have reported dorsal versus ventral stream latency advantages in both monkeys (Nowak and Bullier, 1997; Schmolesky et al., 1998; Schroeder et al., 1998; Mehta et al., 2000a) and humans (Foxe and Simpson, 2002).
Existing models provide a context for the scene analyses indexed by the configuration effect by proposing that interpolation/perceptual organization processes precede IC formation (Grossberg and Mingolla, 1985; Dresp and Bonnet, 1993; Heitger and von der Heydt, 1993;Gegenfurtner et al., 1997). Such processes may highlight regions of the visual field and guide, from a coarse to fine scale, subsequent object recognition processes (Marr, 1982; Grossberg and Mingolla, 1985; Dresp, 1993; Dresp and Bonnet, 1993; Vidyasagar, 1999). The spatial extent of the inducers, regardless of whether an IC shape is ultimately perceived, would demarcate a subregion where objects may be present. Models of perceptual grouping and selective attention similarly propose that visual space is preattentively segmented according to Gestalt principles and subsequently analyzed under the direction of focal attention (Neisser, 1967; Treisman and Gelade, 1980; Treisman, 1982;Treisman and Schmidt, 1982). Studies of attention (Foxe et al., 1998;Heinze et al., 1998; Worden et al., 2000) and patients with neglect or extinction (for review, see Driver and Vuilleumier, 2001) suggest that dorsal stream regions may serve such a visuospatial segmentation function, although such regions may not be critical for IC sensitivity (Vuilleumier and Landis, 1998; Olk et al., 2001;Vuilleumier et al., 2001). The configuration effect and IC effect of this study are consistent with such models. In addition, the EEG/fMRI co-registration results of experiment 2 would indicate that parietal areas play a role in efficient IC processing (Grill-Spector et al., 1998a,b; Mendola et al., 1999; Sugio et al., 1999).
The IC effect and feature binding
Several models of object recognition and figure/ground segregation include synchronized neuronal activity as a mechanism for feature binding (for review, see Tallon-Baudry and Bertrand, 1999; Engel and Singer, 2001). An increase in induced γ band (30–50 Hz) activity is generally observed when comparing IC presence versus absence (Tallon-Baudry et al., 1996, 1997; Hermann et al., 1999; Csibra et al., 2000; Hermann and Bosch, 2001). Some interpret this and similar results as evidence for bottom-up binding of coherent visual features (Tallon-Baudry et al., 1996, 1997; Elliott and Müller, 1998; Tallon-Baudry and Bertrand, 1999; Elliott et al., 2000). Others suggest that such oscillations index higher-order processes such as attention (Pulvermüller et al., 1997; Csibra et al., 2000) or target selection (Hermann et al., 1999). Likewise, there is little consensus on either the timing or source(s) of these oscillations. Some report only late (∼280 msec) induced oscillations (Tallon-Baudry et al., 1996, 1997; Tallon-Baudry and Bertrand, 1999; Csibra et al., 2000), whereas others also observed earlier (∼150 msec) phase-locked effects (Hermann et al., 1999). Moreover, in the studies ofTallon-Baudry et al. (1996, 1997) and Tallon-Baudry and Bertrand (1999), effects were focused over central–posterior scalp sites, whereas Hermann et al. (1999) and Csibra et al. (2000) observed their effects frontally.
The timing of the present IC effect challenges previous claims that γ oscillations represent bottom-up feature binding (Tallon-Baudry et al., 1996, 1997). In these earlier studies, no differential IC response in the early (<200 msec) VEP was reported, whereas we observed early broadband VEP modulations over the lateral–occipital scalp considerably earlier. The IC effect not only precedes induced γ-band activity described in previous studies by ∼200 msec (Tallon-Baudry et al., 1996, 1997), but it is also consistent with our previous results describing the time course of visuospatial neural response interactions between stimuli in different quadrants (Murray et al., 2001). Resolving the timing and functional role of γ oscillations with regard to object representation and visual scene analysis will require further experimentation. However, the current data set makes it clear that broadband VEP modulations precede oscillatory effects, suggesting that these oscillations index a later processing stage.
Footnotes
This work was supported by National Institutes of Health Grants MH63434 (J.J.F.) and MH49334 (D.C.J.) and the Burroughs Wellcome Fund. Data from this study are from a thesis submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Sue Golding Graduate Division of Medical Sciences, Albert Einstein College of Medicine, Yeshiva University. We thank Dr. Glen Doniger for comments on previous versions of this manuscript and Deirdre Foxe for technical assistance. Special thanks are extended to Dr. Kevin Knuth, Dr. David Guilfoyle, and Raj Sangoi of the Center for Advanced Brain Imaging at The Nathan S. Kline Institute for Psychiatric Research for technical expertise in fMRI data acquisition. Sincere appreciation also goes to two anonymous reviewers for their detailed constructive comments.
Correspondence should be addressed to Dr. John J. Foxe, The Cognitive Neurophysiology Laboratory, Program in Cognitive Neuroscience and Schizophrenia, Nathan S. Kline Institute for Psychiatry Research, 140 Old Orangeburg Road, Orangeburg, NY 10962. E-mail: foxe{at}nki.rfmh.org.
M. M. Murray's present address: Functional Brain Mapping Laboratory, Department of Neurology, University Hospital of Geneva, 24 rue Micheli-du-Crest, CH-1211 Geneva, Switzerland.
