Abstract
Psychophysical thresholds and neuronal responses for isolated stimuli are strongly modified by nearby stimuli in the visual field. We studied the orientation and position specificity of these contextual interactions using a dual-frequency visual-evoked potential technique in developing human infants and adults. One set of small, oriented stimulus elements (targets) was tagged with a temporal frequency f1 of 4.52 Hz. The addition of an abutting second set of similar patches (flankers) tagged at f2 = 2.58 Hz had three effects: (1) The flankers reduced the second and fourth harmonic responses to the targets. This reduction was independent of flanker orientation or position and age. (2) The response to the combination of targets and flankers also contained nonlinear interaction terms (1f1 ± 1f2) that were tuned for flanker orientation and position in adults, but only for flanker orientation in infants 8-31 weeks of age. (3) Nonlinear interaction terms recorded at 2f1 ± 2f2 were large and untuned for flanker orientation and position in adults but were nearly absent in the youngest infants. The three forms of nonlinear interaction, thus, have differences in sensitivity to flanker orientation and position and differential growth trends, indicating that they are generated by different mechanisms. These three forms of interaction could serve different functional roles. The first process provides a nonselective gain control that is fully functional in early infancy. The second process, which develops slowly, is selective for the specific form of the stimuli. The third process, which is also immature, pools across orientation.
Introduction
The responses of striate cortical neurons to stimuli presented at one location in the visual field are often modulated by other stimuli presented in their immediate vicinity. These lateral interactions depend on the relative orientation (Blakemore and Tobin, 1972; Maffei and Fiorentini, 1976; Nelson and Frost, 1985; Li and Li, 1994; Kapadia et al., 1995; Levitt and Lund, 1997), contrast (Levitt and Lund, 1997; Polat et al., 1998; Sengpiel et al., 1998; Kapadia et al., 1999; Chen et al., 2001), and spatial configuration (Kapadia et al., 1995; Walker et al., 1999, 2000) of the interacting elements. Several studies have shown that stimuli that are of the same orientation (iso-oriented) interact most strongly or specifically (Li and Li, 1994; Kapadia et al., 1995; Polat et al., 1998; Walker et al., 1999). Psychophysical thresholds are also strongly modulated by the presence of additional stimuli in the visual field. Adjacent, high-contrast elements are known, for example, to interfere with letter recognition (Stuart and Burian, 1962; Flom et al., 1963; Hess and Jacobs, 1977) and vernier acuity (Westheimer, 1975; Levi and Klein, 1985; Levi et al., 1985), effects referred to as “crowding.” Contrast detection thresholds can be either facilitated or suppressed by surrounding stimuli Polat and Sagi, 1993, 1994a,b; Polat et al., 1997), as can the perceived contrast of suprathreshold targets (Ejima and Takahashi, 1985; Chubb et al., 1989; Cannon and Fullenkamp, 1991, 1993, 1996; Xing and Heeger, 2000, 2001).
Given the ubiquity and functional significance of lateral interaction in the visual system, it is surprising how little is known about its development. Green et al. (1996) studied end-zone and side-zone inhibition in 4-week-old kittens. They noted that both were present and that their development coincided with the development of the excitatory properties such as orientation tuning. Sokol et al. (1992) and Grose-Fifer et al. (1994) studied lateral interactions in human infants using a wind-mill/dartboard stimulus to elicit visual-evoked potentials (VEPs). Their stimulus comprised alternating segments of dynamic and static elements. They found that the response to the dynamic elements was suppressed by the static elements and that the extent of suppression was the same for infants and adults. A second type of nonlinear interaction was recorded at a frequency equal to the difference/sum of the two input frequencies. This “intermodulation” response was found to be immature up to at least 20 weeks.
Here, we extend Sokol et al. (1992) and Grose-Fifer et al. (1994) by using spatially localized stimuli that allow for a more detailed analysis of the spatial organization and orientation tuning of lateral interactions. Both the local orientations and the relative positions of interacting elements may be of functional significance (Polat and Sagi, 1993, 1994b; Polat and Norcia, 1996, 1998; Kapadia et al., 2000). We have found two distinct forms of interaction that are insensitive to relative orientation and position in both infants and adults, and a third form of interaction that is selective for both relative orientation and position in adults but only partially selective in infants.
Materials and Methods
Observers. A total of 56 healthy full-term (±3 weeks gestation) infants and 18 adults participated. All adult observers had Snellen acuity correctable to 6/6 or better in each eye and no prior history of strabismus or amblyopia. The research protocol was approved by the Institutional Review Board of the California Pacific Medical Center and conformed to the tenets of the Declaration of Helsinki. Written informed consent was obtained from the parents of the infants and the adult observers, after the VEP recording procedure was explained.
Stimuli and apparatus. Stimulus generation and signal analysis were performed by in-house software running on separate Power Macintosh G3 computers. Stimuli were presented on a multi-synch video monitor (800 × 600 pixels; vertical refresh, 72 Hz; video bandwith, 150 MHz; MRHB2000; Richardson Electronics, Inc.) that was positioned at 100 cm for both infants and adults, generating visual fields of 18° × 14°. The mean luminance was 130 cd/m2.
We studied interactions between one set of small grating patches (the target) and a second set of patches (the flanker). Both target and flanker patches were pattern reversed using a sinusoidal modulation profile. By tagging the target and flanker patches with different temporal frequencies, we could separate the responses from the target, which occurred at the harmonics of 4.52 Hz, from those of the flanker, which occurred at the harmonics of 2.58 Hz (Regan and Cartwright, 1970; Regan and Regan, 1987; Victor and Conte, 2000). Grating contrast was 80% for both the target and flanker patches that were square, and of side length 0.6°. The gratings were in spatial sine-phase with respect to the aperture and had a spatial frequency of 1 c/deg. The individual patches were, thus, composed of a bright-dark edge. In the different experiments described below, we varied the relative orientation and placement of the target and flanker patches (Fig. 1).
Left, Geometric definition of relative orientation. Target and flanker elements (oblique lines) form angles, ΘT and ΘF, with respect to the line drawn between their centers. The orientation convention is mirror symmetric, and the separation is indicated by s. Right, Schematic illustrations of a subset of the stimulus configurations. Only a portion of the actual display is represented. a, Target-alone condition: 59 grating patches that reversed in contrast at 4.52 Hz. b, Collinear condition: paired grating patches, with target reversing at 4.52 Hz and collinear flanker reversing at 2.58 Hz. The relative orientation is (0,0). The orientation axis of the individual pairs was random. c, Parallel condition: tests (4.52 Hz) and flankers (2.58 Hz) were of the same orientation within a pair, but were not collinear (90,90). The orientation axis running through the centers of the individual pairs was random. d, Orthogonal end-zone: the orientation of tests and flankers within a pair differed by 90°. The flankers were in the end-zone of the tests. Relative orientation is (0,90). e, Orthogonal side-zone: the orientation of tests and flankers within a pair differed by 90°. The flankers were in the side-zone of the tests. Relative orientation is (90,0).
Figure 1 (left) shows the notational convention we will use to describe the stimuli in terms of their relative orientation. We parameterize relative orientation (s,θT,θF) with three parameters, where s is the center-to-center spacing (0.6°) and θT and θF are the orientations of the target and flanker, each relative to the line connecting the centers. The orientations of the targets and flankers lie in the range 0 ≤ θi < 180 [Geisler et al. (2001) for a slightly different parameterization]. We do not specify the orientation of the line connecting the centers, because this additional parameter was distributed randomly across the individual target-flanker doublets.
There were 59 target patches and a matching number of flankers, when these were present. The 59 target patches were oriented randomly and were separated from their neighbors by 0.7°, edge to edge. The centers of the patches were placed on a 1.8° spacing hexagonal grid. We used many small patches both to increase the amplitude of the evoked response and to obviate the need for strict fixation on a single pair of patches.
Figure 1, a-d, shows schematic examples of several of the stimulus configurations used in the experiments. Figure 1a shows the targets-alone condition that served as a baseline condition for most of the experiments. Because there is only a single oriented element and not both within a pair, relative orientation is undefined. Figure 1b shows the collinear configuration in which each patch abuts its neighbor's end-zone. Its relative orientation is (0,0). Note that collinearity is defined within a pair of patches, not over the entire array of patches. Figure 1c shows the parallel configuration, which is an arrangement of targets abutting the side-zone of the flankers, with both target and flankers having the same orientation within a pair. Its relative orientation is (90,90). Figure 1d shows the target with an orthogonally oriented flanker in its end-zone (0,90), and Figure 1e shows the flanker in the side-zone (90,0). Additional stimulus conditions were presented to adults and will be described below.
VEP recording and procedure. Infants were seated in their parent's lap in front of the monitor. The experimenter attracted the infant's attention to the stimulus with a small toy centered on the monitor's display. Recordings were interrupted when the infant was judged not to be attending. Adults were instructed to simply fixate the toy, mimicking the fixation conditions imposed on the infants. Pilot experiments with the target-alone and collinear flanker stimuli indicated that the fixation toy caused small, statistically significant reductions in the amplitude of the adult's response, but these reductions affected all response components equally. Recording sessions consisted of 4-8 trials per condition for infants and 10 trials per condition for adults. The trials were randomly interleaved across conditions in blocks of two trials for infants and five trials for adults. Viewing was binocular.
Signal acquisition and data analysis. Three electrodes were placed over the occipital pole at O1, OZ, and O2 of the International 10-20 system. The reference and ground electrodes were placed at CZ and PZ, respectively. The EEG was amplified at a gain of 20,000 for infants and 50,000 for adults, with amplitude band-pass-filter settings of 0.3-100 Hz at -6 dB (model 12 A5; Grass Instruments, Quincy, MA). The EEG was digitized to a nominal 16 bits accuracy at 432 Hz (National Instruments PCI-MIO-16XE-50). The horizontal synch signal from the video card was conditioned and used to clock the analog-to-digital converter (six samples per video frame). The display was updated during the vertical blanking interval, and the vertical synch signal was provided via a digital input line to the data acquisition routine for exact synchronization of the data acquisition to the display. If the experimenter interrupted the display with a mouse input, both display and data acquisition program loops were reset to a previous point in the display that was at least 1 sec before the mouse press.
Temporal frequency spectra. A Discrete Fourier Transform was used in preliminary analyses to calculate complete spectra at 0.64 Hz resolution (Fig. 2 only). First, spectra from each observer in a given stimulus condition were obtained from time-averages computed over a 1.55 sec epoch length using all data from all trials in a given condition. This epoch length is the shortest one that contains integer numbers of cycles of both target and flanker frequencies. The individual observer spectra were then coherently averaged in the frequency domain. Each individual, thus, contributed equally to the grand averages. Coherent averaging uses both phase and amplitude information and emphasizes response components that are consistent in phase across observers. This method is, thus, conservative in identifying the presence of response components.
Temporal frequency spectra for collinear and target-alone conditions obtained by coherently averaging the records of 11 adult observers. The two spectra are plotted with a mirror reflection about the common x-axis. The target-alone condition contains responses at twice and four times the target stimulus frequency of 4.52 Hz (2f1 = 9.02 Hz; 4f1 = 18.08 Hz). The spectrum from the collinear condition also contains these components, but at reduced amplitude, an effect referred to in the text as suppression. In addition, responses at 2f2 and 4f2 generated by the 2.58 Hz flankers are also present, as are second-order (1f1 ± 1f2) and fourth-order (2f1 ± 2f2) interaction terms.
Adaptive filtering. A recursive least square (RLS) adaptive filter (Tang and Norcia, 1995) was used to determine VEP amplitude and phase for the second-order terms (2f1,2f2, f1 + f2 and f1 - f2) and the fourth-order terms (4f1, 4f2, 2f1 +2 f2 and 2 f1 -2 f2), where f1 was 4.52 Hz and f2 was 2.58 Hz. The target and flanker temporal frequencies were integer submultiples of the vertical frame rate (16 and 28) and were nearly incommensurate. The memory length of the filter was set to 1.55 sec, yielding a spectral resolution of ∼0.6 Hz, which was sufficient to maintain all terms up to fourth-order distinct from one another in the analysis. Within a trial, the sine and cosine weights of the filter were coherently averaged to yield a single amplitude and phase for each trial and response frequency. These complex amplitudes were then coherently averaged across all trials in a given stimulus condition for each subject. Average amplitudes for group data (e.g., all figures after Fig. 2) were computed without consideration of response phase (amplitude only) to enable us to use conventional multivariate statistics to assess differences between groups and conditions.
Statistical analyses. Differences between conditions were assessed using a multivariate approach to repeated measures (multivariate analysis of variance or MANOVA) that takes into account the correlated nature of repeated measures data (for review, see Keselman et al., 2001).
Results
The scalp-recorded VEP to counterphase reversal of simple stimuli, such as the target-alone configuration (Fig. 1a), is composed of responses at even harmonics (multiples) of the input temporal frequency f, with the second harmonic 2f being dominant. This is shown for 11 adults in the bottom spectrum of Figure 2. The strong second harmonic (and fourth harmonic) responses and the lack of a first harmonic response for counterphase stimuli is commonly attributed to an effective full-wave rectification of the response across the population of responding neurons. In a mass response such as the surface-recorded VEP, complex cells, which are full-wave rectifiers, would be expected to generate only even-order response components to counterphase modulation. Simple cells, in contrast, generate both even-order and odd-order responses. The odd-order responses vary with the phase relationship between the stimulus and the receptive field, dropping to near zero at a null phase (Movshon et al., 1978; Reid et al., 1987; Jagadeesh et al., 1997). In the volume-conducted mass response, the responses of simple cells with all relative spatial phase relationships will combine in complementary pairs to form even-order responses only; the odd-order components will cancel.
Signatures of lateral interaction
When collinear flankers are added to the target display (Fig. 1b) the amplitude of the second and fourth harmonic responses generated by the targets at 2f1 = 9.04 Hz and 4f1 = 18.08 Hz are reduced (Fig. 2, top spectrum). We will refer to this form of lateral interaction as “suppression.” The spectrum for the targets plus flankers stimulus also contains even harmonic responses generated by the flankers at 2f2 = 5.16 Hz and at 4f2 = 10.32 Hz. In addition to the response components that are directly related to the two input frequencies, additional responses at frequencies equal to the sum and difference of the two input frequencies (1f1 - 1f2 = 1.94 Hz; 1f1 + 1f2 = 7.1 Hz) are observed as well as responses at frequencies equal to 2f1 ± 2f2 (3.88 and 14.2 Hz). The responses at 1f1 ± 1f2 are second-order with respect to the input and the responses at 2f1 ± 2f2 are fourth-order with respect to the input. Responses at frequencies that are combinations of low-order multiples of the input frequencies indicate that the two inputs have passed through a common nonlinear site, such as the spiking threshold of cortical neurons. These components are referred to as “intermodulation” components in the engineering literature and are direct indicators of neural convergence in that they reflect the precise temporal structure of the inputs. The intermodulation components are another indicator of lateral interaction, given that the two inputs were presented at laterally separated locations in the visual field. The particular intermodulation frequencies that are present in the output are determined by the form and order of the nonlinearity at the convergence site (Regan and Regan, 1988; Victor and Conte, 2000).
Orientation and configuration specificity of lateral interactions
Orientation tuning cannot be measured for single stimuli in the VEP, but orientation-dependent lateral interactions can be measured between stimuli using either suppression or intermodulation measures. Lateral interactions can depend on the separation of elements, on their local orientations, and on their relative positions. If an interaction is specific for both local orientation and relative position, the interaction is selective for relative orientation. In the first experiment, we demonstrate a disassociation in the relative orientation tuning of suppression and intermodulation interactions. The stimuli are illustrated schematically in Figure 3A. We varied the relative orientation of gratings comprising the target and flanker over a wide range, keeping their separation constant. The gratings were successively rotated by equal amounts about their centers within the square apertures. We used (θT,θF) of (0,0), (15,15), (30, 30), (45,45), (60,60), (75,75), and (90,90), omitting the constant distance parameter. Stimuli at the extremes of the range had the same local orientations and were either collinear (0,0) or parallel but not collinear (90,90). The remaining stimuli comprise a set of L-junctions of varying angle. The response to the target patches presented alone was also measured as a control.
A, Schematic illustration of the stimuli used to study relative orientation tuning in adults. In each configuration, individual patches were rotated about their centers by equal amounts. Only the grating component was rotated, and the envelope function remained constant. The separation s was fixed at 0.6°. The stimuli were parameterized by two angles ΘT and ΘF, indicating the rotation angle of the target and flanker with respect to the line connecting the two centers. B, Relative orientation tuning for pooled self-terms (2f1, 4f1). The x-axis indicates the value of ΘT/F. The response to the targets presented without the flankers is shown as the single circle. Suppression is untuned and averages ∼50%. Errors are SEs of the mean of the pooled amplitudes across subjects. C, Pooled second-order intermodulation component amplitudes. Plotting symbols are as in B. There is a linear trend in orientation tuning, with maximal interaction for collinear stimuli. The filled circle indicates the experimental noise level, as measured at the same frequencies in the target-alone condition. D, Pooled fourth-order intermodulation component amplitudes. Note that the response is significantly above the noise level (circle) for all conditions and shows little specificity for relative orientation. E, Direct comparison of second- and fourth-order tuning functions (filled vs open symbols, respectively). There is a trend toward more tuning in the second-order response. Noise levels are replotted from C and D.
Nonspecific suppression effects on 2F and 4F responses to the targets
We first asked whether the strength of suppression depends on relative orientation. As an index of the total response to the target, we added the amplitudes of the second and fourth harmonics in quadrature (i.e., for each observer, in each condition, we calculated the square root of the sum of the individual observer's squared second and fourth harmonic amplitudes). Figure 3B shows these pooled second and fourth harmonic responses plotted as a slice through the relative orientation parameter space. Errors in this figure are SEs of the mean of the pooled amplitude values computed across subjects in each stimulus condition. Response amplitudes were largest for the Oz-Cz derivation, and there were no interactions across electrodes, so the results are presented from this derivation. The single point (filled circle) indicates the pooled response amplitude for the target-alone condition in which no suppression can occur.
The addition of flankers of any orientation produces a uniform amplitude reduction (suppression effect) in which the target response was reduced to ∼50% of the level obtained in the target alone condition.
Specificity of the intermodulation terms
When the abutted targets and flankers are of the same orientation, they are expected to activate many neurons in common, and one would naively think that the strength of intermodulation would simply be a function of the orientation difference between the targets and the flankers. The (0,0)° and (90,90)° conditions would be expected to produce the largest output at the intermodulation frequencies, and the (45,45)° case the minimum. This pattern is not observed: the amplitude of the intermodulation terms was instead a linear rather than U-shaped function (Fig. 3C,D). The experimental noise level is indicated by the open and filled circles in Figure 3, C-E. These measurements were obtained using data collected in the target-alone condition, in which no intermodulation is expected.
The linear trend was highly significant for the second-order terms (F(1,10) = 12.46; p = 0.005; Fig. 3C, squares), but was marginally nonsignificant (F(1,10) = 3.69; p = 0.08) for the fourth-order terms (Fig. 3D, open squares). The second-order and fourth-order response data are replotted together in Figure 3E for comparison. A direct comparison failed to detect a significant difference in tuning (F(6,60) = 0.67; p = 0.68), which may be attributable to a lack of power, given our relatively small sample size.
Selective and nonselective lateral interactions
In this experiment, we obtained suppression of the target signal for lateral configurations, as well as intermodulation between targets and flankers. These effects replicate previous work in adults that compared lateral and overlapping maskers (Ratliff and Zemon, 1982; Zemon and Ratliff, 1982). The results extend these previous results by showing that the lateral suppression between abutting, high-contrast stimuli is independent of the relative orientation between stimulus elements, but that the intermodulation between stimulus components is, at the same time, tuned for relative orientation. The fact that suppression of the self-term components is untuned, whereas the intermodulation components are tuned, indicates that the two effects are not mediated by a single common nonlinearity such as a sigmoidal nonlinearity. A sigmoidal nonlinearity is capable of generating both suppression and intermodulation (Regan and Regan, 1988; Candy et al., 2001), as is the class of power law rectifiers with exponents between 1 and 2 (Regan and Regan, 1988). However, if only a single nonlinear mechanism were operative, the orientation tuning of the suppression and intermodulation data should be the same, whereas they are not. Our data, thus, suggest that separate mechanisms are operative.
Development of iso-orientation interactions
It would be difficult, if not impossible, to perform detailed parametric studies of relative orientation tuning using within-subjects designs in infants because of their very short periods of quiet attentiveness. Nonetheless, by a combination of between-subjects designs and the selection of key stimulus contrasts, we can reduce the individual experiments to manageable proportions for use with infants. The data obtained from adults suggests that suppression was independent of relative orientation, but that the second-order intermodulation terms showed small but measurable tuning, especially if one contrasts the two extreme conditions: collinear (0,0) versus iso-oriented, but not collinear (90,90). The purpose of the first developmental experiment was, thus, to probe the spatial organization of iso-orientation interactions using both suppression and intermodulation measures.
Thirty-two infants, 8-31 weeks of age, and 18 adults between 18 and 59 years of age, participated in this experiment. Eleven of the 18 adult observers participated in the previous experiment. For purposes of analysis, the infants were broken into two age groups: 8-15 weeks (n = 18) and 16-31 weeks of age (n = 14). We compared responses for the target-alone configuration (4.52 Hz) to a condition in which the 2.58 Hz flankers were abutted and collinear (Fig. 1b) and one in which the flankers were parallel but not collinear (Fig. 1c). We obtained the largest responses from OZ in both infants and adults. Response amplitude fell off more quickly from Oz to O1 or O2 in infant than in adults. We, thus, selected Oz for further analysis for all observers.
Suppression is adult-like in infants
Response amplitudes for the youngest age group were almost a factor of 10 larger than adults' (Fig. 4). We, thus, chose to quantify suppression as the fractional reduction of the target response components in the two test conditions compared with the target response measured when the target was presented alone. We performed a log transform on each observer's response amplitude, so that, when averaging across observers, similar percentage decreases would result in the same effect size, independent of absolute amplitude. This also has the benefit of removing age interaction effects that are solely caused by overall amplitude differences.
A, Target suppression results for iso-oriented stimuli for 8- to 15-week-old infants (top), 16- to 31-week-old infants (middle), and adults (bottom). The response to the targets-alone condition is shown by the stippled bars, the response to the collinear (0,0) condition by the vertical striped bars, and the response to the parallel (90,90) condition is shown by the horizontal striped bars. Icons indicate the configurations schematically. Responses recorded at 2f1 are shown on the left, and responses recorded at 4F1 are shown on the right. Infants and adults showed comparable amounts of suppression of the target-alone response, independent of the placement of the flanker. The infants' response contains less fourth harmonic as indicated by the ratios plotted in the top right corner of each subpanel. Errors are SEs of the mean for unpooled amplitudes at each harmonic. B, Intermodulation component amplitudes. Pooled second-order amplitudes are shown on the left, and pooled fourth-order amplitudes are shown on the right. The labeling convention is as above. See Results for details.
At all ages, the presence of the flankers reduced the target response by ∼40%, independent of whether the flankers are collinear or not (Fig. 4A). The results of the experiment were analyzed using the MANOVA approach to repeated measures. The design tested the between-subject factor age (three levels) and the within-subject factors of response-order (second vs fourth harmonic of F1) and configuration (three levels; targets-alone vs with each of the two flankers). The effect of age was highly significant (F(2,62) = 34.33; p < 0.001) because of large overall amplitude differences. Age did not interact with configuration (F(4,124) = 1.36; p = 0.252), indicating that the overall pattern of suppression was adult like. However, there were significant interactions between age and order (F(2,62) = 22.08; p < 0.001) and age, order, and configuration (F(4,124) = 4.18; p = 0.003). Infants showed relatively smaller fourth harmonic response compared with their second harmonic responses, and infants 16-31 weeks of age showed less suppression of their fourth harmonics than did the adults or younger infants. The ratios of second harmonic to fourth harmonic response amplitudes were 5.7 and 5.8 for infants 8-15 and 16-31 weeks of age, respectively, compared with 2.2 for the adults. Although the infant fourth harmonics compose a smaller fraction of the infants total response, they are well above the experimental noise level. The suppressed infant fourth-order responses (Fig. 4A) are approximately six times the noise floor for the youngest age group, approximately three times the noise floor in the older age group, and about twice the noise floor in adults. The noise level was measured as the average amplitude during the experimental trials at two nonstimulus-related frequencies in the EEG, one immediately above 4F1 and one immediately below 4F1. The infant fourth harmonic responses are, thus, a smaller part of the total response in infants, but they are well above the noise floor. The same is not true of the fourth-order intermodulation terms (compare Fig. 4B), suggesting that different mechanisms may be involved.
We tested the equality of suppression for the collinear and parallel flankers for infants and adults separately. The two age groups each showed equal suppression for the two configurations: infants (F(1,62) = 0.05; p = 0.821) and adults (F(1,62) = 0.03; p = 0.874).
As a test of the stability of the data, we repeated this experiment for seven of the first infants tested. The means for all components were nonsignificantly different across the two sessions, which were recorded within 1-2 weeks of each other.
Immature intermodulation components for iso-oriented stimuli
We first established that all the measured intermodulation terms were above the experimental noise level, so that the following amplitude comparisons were not limited by the evoked responses being at or below the noise level. To do this, we compared second- and fourth-order intermodulation amplitudes to the experimental noise levels established in the target-alone condition. We pooled infant data across age and pooled across the two flanker types. Infant second-order intermodulation was well above the noise level (F(1,62) = 32.8; p < 0.001), and their fourth-order intermodulation was just measurably above it (F(1,62) = 4.4; p = 0.04). Both second- and fourth-order terms were far above the noise level for the adults (second-order: F(1,63) = 188.58, p < 0.001; fourth-order: F(1,63) = 293.25, p < 0.001).
We then asked specifically whether the infant second-order intermodulation terms were larger for the collinear configuration than for the parallel, but parallel condition as they were in adults in the first experiment. This analysis indicated that the infant second-order intermodulation terms were not spatially selective (F(1,62) = 0.91; p = 0.34), but that the adults' intermodulation terms were (F(1,62) = 6.5; p = 0.013). Neither adult or infant fourth-order intermodulation were spatially selective (adults: F(1,62) = 0.56, p = 0.46; infants: F(1,62) = 3.10, p = 0.083).
In a final analysis, we divided the infants into two age groups and tested the effects of age, configuration, and order in a single MANOVA that controlled significance levels for all comparisons and took into account the covariance structure in the data. The data entered into this analysis are shown in Figure 4B. There was a highly significant main effect of age associated with large overall amplitude differences across age (F(2,62) = 169.27; p < 0.001). There was a significant effect of order, with second-order amplitudes being larger overall than fourth-order amplitudes (F(1,62) = 46.77; p < 0.001). There was a significant effect of configuration, with the responses for the flankers-present conditions being larger than in the control condition (F(2,124) = 47.18; p < 0.001). There was a configuration by age interaction because of changing overall levels of intermodulation across age (F(4,124) = 4.56;p = 0.002).
Finally, and importantly, in terms of the interpretation of the lower-order statistical interactions, there was a significant three-way interaction between age, order, and configuration. The difference between second- and fourth-order interaction depended on age (F(4,124) = 6.64; p < 0.001): intermodulation was spatially tuned only at second-order and only in adults. In adults, collinear stimuli (0,0) produced more second-order intermodulation than did parallel targets and flankers of the same orientation (90,90).
The lack of spatial selectivity of the infant second-order intermodulation components could be caused by infants lacking an extra-level of lateral interaction for collinear stimuli that adults have, or alternatively, infants could have an excess of lateral input from parallel stimuli. Without some other form of absolute reference, it is not possible to tell which of these developmental scenarios is more likely. One way of obtaining a reference for the relative strength of the lateral interaction is to normalize the intermodulation responses by the amplitude of the target-alone response. This normalization removes the large amplitude differences between infants and adults and yields a measure of how strong the interaction terms are relative to the response to a single input. The single-input response in this analysis is assumed to represent an independent estimate of the transfer “strength” of the nonlinear mechanism that is generating the intermodulation responses. The amplitude of the second harmonic recorded in the target-alone condition was used as the normalization factor.
Figure 5 plots the normalized amplitude of the second- and fourth-order intermodulation terms as a function of age and flanker configuration. The normalization is specified as an intermodulation ratio between the log values, with a value of 0 indicating equal amplitudes and 0.3 indicating a factor of 2. The fourth-order terms (Fig. 5, open markers) show a substantial increase between 8-15 weeks and 16-31 weeks, at which time they are still smaller than those of the adults. The second-order terms (Fig. 5, filled markers) are constant over this time period. Throughout the age range we have examined, the second-order components were larger than the fourth-order components, whereas the opposite is true in adults. The differential developmental pattern for second- and fourth-order interaction terms suggest that they are generated by separate mechanisms.
Normalized intermodulation response amplitudes. The data from Figure 4B have been replotted by expressing the intermodulation component amplitudes as a fraction of the second-order self-term amplitudes (2f1) for each observer. Infants show relatively more side-zone interaction than adults. The adult specialization for collinearity is, thus, caused by less input from the side-zone relative to the end-zone. Fourth-order intermodulation is weak in the youngest age group and develops more rapidly than does second-order interaction over the range of ages tested.
Figure 5 shows that second-order intermodulation levels of the infants are equal to or larger than those for the collinear arrangement in the adults and are larger than the adult levels for the parallel arrangement. This pattern of results suggests that the infants' lack of spatial specificity is caused by an excess of lateral inputs from parallel stimuli, rather than a relative deficiency of collinear interactions.
Infants showed adult-like lateral suppression of the target response for both collinear and parallel maskers (second and fourth harmonic data). These results replicate those of Sokol et al. (1992) and Gross-Fifer et al. (1994), who also found adult-like suppression with windmill dartboard stimuli in very young infants (2-8 weeks of age). Our results extend these previous results to indicate that these suppressive interactions do not depend on the relative spatial position of the stimuli. The windmill dartboard stimulus probes lateral interaction along the orientation axis (end-zone), whereas we have examined both this direction and the perpendicular direction (side-zone).
Sokol et al. (1992) and Gross-Fifer et al. (1994) suggested that the lateral interaction indexed by the intermodulation components was later developing than the interaction indexed by the suppression of their self-term, based on developmental differences in the relative amplitudes of the intermodulation and self-terms (Sokol et al., 1992), differences in temporal tuning of the intermodulation terms (Gross-Fifer et al., 1994) and in the response phase between infants and adults (Sokol et al., 1992; Grose-Fifer et al., 1994). In our recordings, the fourth-order intermodulation terms are particularly small relative to the overall response amplitude. The infants' second-order terms are not particularly small, but they are spatially nonselective.
Partial relative orientation specificity of lateral interactions in infants
In our first experiment with adults, we found that suppression was independent of the relative orientation of the stimuli, but that second-order intermodulation declined as θT and θF increased. Infants in the second experiment did not show differential intermodulation for the extremes of this continuum: (0,0) and (90,90). The results of the first experiment were surprising in that target and flankers with an orientation difference of 90° (-45,45) produced more intermodulation than stimuli that had no orientation difference (90,90). A suggestion as to why this may be comes from the psychophysical work of Kellman and Shipley (1991), who have suggested that spatial integration is stronger for contours that are relatable. Relatable contours are those that can be connected by a smooth curve (differentiable at least once) and monotonic (the interpolated edges do not bend through >90°). An L-configuration of (45,45) is relatable, but T-configurations of (0,90) and (90,0) are not, even though each has the same 90° orientation difference between target and flanker carriers. If relatability is an important factor in lateral interactions, flankers that have a 90° orientation difference forming T-junctions rather than L-junctions should fail to produce intermodulation. In the following experiment, we find this to be true in both adults and infants. We used two T-junction configurations with relative orientations of (0,90) and (90,0) and two control conditions (0,0) and (90,90) in which the carrier orientations were the same within a pair. The stimuli are shown schematically in Figure 6. The use of these stimulus pairs allows us to compare configural specificity off both the end-zone and side-zone of the target.
Orientation tuning of lateral interactions in the end-zone (left) and side-zone (right) of the target. Top, Pooled self-term amplitudes (2f1, 4f1) for the target-alone condition (stippled bars) and collinear (vertical hatching), parallel (horizontal hatching), and orthogonal end/side-zone (diagonal hatching) configurations. Masking is equally strong for collinear, parallel, and orthogonal configurations in both infants and adults in the end-zone (left) and side-zone (right). Pooled second-order (f1 ± f2) amplitudes (middle) and pooled fourth-order (2f2 ± 2f2) amplitudes (bottom) for end-zone (left) and side-zone (right) configurations. Hatching indicates configurations as in A. See Results for details.
In this experiment, 24 infants not tested in the previous experiment participated. The infants ranged in age from 8 to 24 weeks. Fourteen adults, most of whom had participated in the previous experiment, also contributed data. The adults ranged in age from 18 to 59 years old. We compared lateral interactions with the flankers placed in the end-zone of the targets (0,90) in one group of infants (n = 12) and in the side-zone (90,0) in another group of infants (n = 12), because in the previous experiment the iso-oriented flankers were also in the end-zone and side-zone. Each of these groups included six infants 8-15 weeks old and six infants 16-24 weeks old. The target-alone condition was presented as a control. Adult observers completed the entire design.
The statistical analysis for this experiment tested the following two-level factors: zone (end, side) and orientation (iso, orthogonal). Self-terms and intermodulation terms were analyzed separately for adults and infants. The analyses compared the amplitudes in the different flanker present conditions.
Suppression is independent of relative orientation
Figure 6 shows the suppression effects measured from the pooled second and fourth harmonics of the target for the end-zone configurations (left) and side-zone configurations (right). Within each of these panels, we plot the infant data on the left and the adult data on the right. Responses from the target-alone condition are indicated by stippled bars. Responses from the collinear configuration (0,0) are shown by vertically hatched bars, and those from the orthogonal-orientation conditions (0,90) and (90,0) are shown by the diagonally hatched bars. In the adults, suppression was equally strong for iso-oriented and orthogonally oriented flankers (orientation: F(1,13) = 1.57; p = 0.23) and for the end-zone and side-zone (zone: F(1,13) = 0.05; p = 0.83). There was no orientation by zone interaction (F(1,13) = 0.05; p = 0.83). Suppression was, thus, completely independent of flanker orientation or placement.
Suppression in the infants was similarly nonselective: suppression was equal for both orientations (orientation: F(1,22) = 1.54; p = 0.23) and for both zones (zone: F(1,22) = 0.003; p = 0.96). There was a marginally significant orientation by zone interaction (F(1,22) = 4.44; p = 0.05). Follow-up univariate comparisons indicated that iso-orientation and orthogonal-orientation amplitudes differed in the side zone (F(1,22) = 5.61; p = 0.03), but not the end zone (F(1,22) = 0.38; p = 0.55). We do not attribute much significance to this interaction, because the effect-size is small and because the zone effect was measured across subjects.
Infant intermodulation is orientation tuned but not spatially specific
The second-order intermodulation data of both adults and infants was, in contrast, orientation specific [intermodulation was present for (0,0) and (90,90) iso-oriented flankers, but not for (0,90) or (90,0) orthogonally oriented flankers]. Fourth-order intermodulation amplitudes were independent of relative orientation. The amplitude values for second- and fourth-order intermodulation terms on which this conclusion is based are shown in Figure 6. The middle two panels plot the pooled amplitudes of the second-order terms, and the bottom two panels plot the pooled amplitudes of the fourth-order terms. All plotting conventions are those described above for Figure 6.
The design of the statistical analysis was the same as that for the self-terms: a complete 2 × 2 within-subjects design for the adults and mixed design in the infants. In addition to the effects of orientation and zone, this design included the factor order composed of two levels, second and fourth, corresponding to second- and fourth-order intermodulation terms. In the adults, there was a significant order by zone by orientation interaction (F(1,13) = 4.63; p = 0.05): the level of intermodulation depended on each of the three factors. The interpretation of the three-way interaction is relatively straightforward. Second-order amplitudes were smaller than fourth-order amplitudes [order: F(1,13) = 14.35; p = 0.002]. Iso-orientation amplitudes were larger than orthogonal-orientation amplitudes (orientation: F(1,13) = 38.85; p < 0.001) mainly because iso-amplitudes and orthogonal-amplitudes differed for the second-order terms (F(1,13) = 57.30; p < 0.001), but not for the fourth-order terms (orientation by-order: F(1,13) = 0.05; p = 0.83]. A zone by orientation interaction (F(1,13) = 7.55; p = 0.02) was caused by the second-order amplitudes being larger for the collinear configuration compared with the iso-orientation parallel configuration (F(1,13) = 18.19; p < 0.001) on a univariate follow-up test), whereas the cross-oriented tests and flankers produced equal second-order amplitudes in the end-zone and side-zone (F(1,13) = 1.28; p = 0.28), also on a univariate follow-up test).
There was, thus, a greater degree of orientation tuning in the end-zone compared with the side-zone for the second-order terms (the difference between iso and orthogonal orientations was larger in the end-zone). In contrast, fourth-order amplitudes were constant across orientation and zone. Fourth-order amplitudes in the adults were well above the noise level for all conditions (as indicate by the lower target-alone amplitudes), whereas second-order amplitudes dropped to the noise level for orthogonally oriented configurations.
In the infants, intermodulation interactions depended on both order and orientation (order by orientation: F(1,22) = 7.06; p = 0.01). Separate tests indicated that second-order intermodulation was orientation tuned (F(1,22) = 9.19; p = 0.006), but that fourth-order intermodulation was not (F(1,22) = 0.64, p = 0.43). The orientation effect was marginally significant (F(1,22) = 3.95; p = 0.059). There were no effects or interactions involving zone, consistent with the results shown in Figure 4 for iso-orientation configurations. There was an effect of order (F(1,22) = 9.95; p = 0.005) caused by second-order amplitudes being larger than fourth-order amplitudes, the opposite of what was observed in adults. This effect was also seen in the data of Figures 4B and 5.
Lateral suppression was not selective for flanker orientation and placement in adults, replicating the effects seen in Figures 3 and 4. Infants showed an adult-like pattern of suppression with respect to orientation tuning and the same lack of spatial selectivity seen in Figure 4 for iso-oriented targets and flanker. However, both infants and adults showed orientation-selective second-order intermodulation terms and nontuned fourth-order terms. Infants, thus, show partial selectivity for second-order orientation in their second-order intermodulation responses. The fourth-order terms were small in infants, as in the previous experiment (Figs. 4 and 5, compare with Fig. 6). This effect is especially strong, in that the fourth-order amplitudes were smaller than second-order responses in infants, but they were larger than second-order responses in adults.
We also performed a direct comparison in adults between intermodulation levels for T-junction flankers of (0,90) and (90,0) and L-junctions with the same 90° orientation difference (45,45) by pooling data from the 10 observers who participated in both experiments. The data are plotted in Figure 7. Second-order intermodulation is well above the noise level (target-alone, stippled bar) for the (45, 45) L-junction condition, but not for the (0,90) or (90,0) T-junctions (bars with oblique hatching). Fourth-order intermodulation, in contrast, was well above the noise level and was untuned for all configurations (right). This result reinforces the notion that the second-order intermodulation terms are specific for relative orientation and not for simple differences in local orientation. Fourth-order intermodulation, in contrast was independent of relative orientation. Fourth-order intermodulation terms have also been identified by Victor and Conte (2000), who studied the effects of gaps and lateral displacements using two-frequency stimuli similar to the ones we have used. Their focus was on very small offsets from collinearity that were in the hyperacuity range, but they also studied larger displacements. Their lateral displacement manipulation changed stimulus relatability, and they, like us, found second-order terms were diminished by nonrelatable stimuli but that fourth-order terms were not, if the degree of nonrelatability was large enough.
Relative orientation tuning for stimuli with 90° orientation differences between target and flanker. Top, Pooled second-order intermodulation amplitudes are shown on the left, and pooled fourth-order intermodulation amplitudes are shown on the right. The target-alone configuration (stippled bars) indicates the experimental noise level. Responses for (0,90), (90,0), and (45,45) configurations are indicated by hatched bars. L-junction stimuli (45,45) show larger second-order intermodulation responses than do T-junctions in either the end-zone (0,90) or side-zone (90,0). The two T-junction stimuli lead to second-order intermodulation responses that are at the noise level. Fourth-order intermodulation responses, in contrast, are well above the noise level and are not selective for stimulus configuration (right).
Discussion
The present experiments provide additional evidence for the two types of nonlinear interaction between abutting high-contrast stimuli first identified by Ratliff and Zemon (1982) and Zemon and Ratliff (1982, 1984). One is the suppression of response amplitude caused by the presence of an additional stimulus, and the second is the generation of intermodulation. On the basis of the fact that strength of the intermodulation depends strongly on element separation, but that suppression does not, they inferred that two separate interactions, referred to as short and long range were operational. Our analysis also suggests that these two types of interaction reflect different functional processes. The nonselective interaction involved in suppression appears to be the result of an intensive nonlinearity such as contrast gain control, whereas the more specific interaction indexed by the second-order intermodulation terms is consistent with a form-processing mechanism. The observation of spatial phase sensitivity in the intermodulation response by Ratliff and Zemon (1982) and Zemon and Ratliff (1982) is also consistent with this interpretation. In addition, we have identified a third process, represented in the fourth-order intermodulation terms that is nonselective over the range of configurations we have tested. Our data indicate that fourth-order intermodulation is not generated by the suppression interaction, because fourth-order interaction is strongly reduced in young infants who have an adult-like suppression effect.
The configural specificity we have observed in the second-order intermodulation response is quite similar to that observed by Kapadia et al. (1995) in V1 of alert behaving monkeys. They found that a collinear flanking bar placed outside the classical receptive field of some V1 cells facilitated the response to a bar placed inside the receptive field. In the cells that showed this facilitation, making the target and flanker laterally offset (nonrelatable) reduced the facilitation, as did the introduction of large but technically relatable orientation differences or nonrelatable T-junctions. Flanker suppression also depends on the position of either single-patch (Walker et al., 1999) or double-patch flankers (Cavanaugh et al., 2002). These and all other studies of nonclassical surround interaction have used the failure of additivity of center and surround responses as an indicator of surround interactions. This method lumps together suppression and intermodulation interactions because the targets and flankers have the same time course and both types of effect are, thus, projected onto the same response frequencies. This may, in part, explain the wide diversity of interactions observed in V1, given that suppression may be untuned for relative orientation at this level and some, but not all intermodulation components may be highly tuned.
Our results, while broadly consistent with the original nonlinear model proposed by Zemon and Ratliff (1984), force a rejection of this model. Zemon and Ratliff (1984) pointed out that a memory-less point-nonlinearity sandwiched between two linear filters would be expected to generate intermodulation components. They used a squaring nonlinearity and computed its output spectrum analytically as being composed solely of second-order components. This previous work (Ratliff and Zemon, 1982; Zemon and Ratliff, 1982, 1984) typically used input temporal frequencies of 0 and 3 Hz that resulted in overlaps between “self”and intermodulation terms (static and 3 Hz inputs yield second harmonics that are the same frequency as the fourth-order intermodulation terms). In this situation, fourth-order intermodulation responses would be mixed with second harmonic, self-terms. In the cases where distinct frequencies were used, only the second-order difference frequency was analyzed. Because of these two limitations, they were unable to identify the prominent fourth-order responses we have observed. Later work (Victor and Conte, 2000) was able to identify fourth-order terms using laterally displaced by stimuli.
A modified version of the sandwich model with a single, higher-order nonlinearity could generate fourth-order, as well as second-order, responses (Fig. 8A). This model, however, fails to explain our data in several respects. First, a single nonlinearity should have the same configuration tuning for both second- and fourth-order terms. This was not found. There is a nearly complete absence of second-order intermodulation with T-junction configurations that support robust fourth-order intermodulation in adults (see Figs. 3, 6, 7), and the second-order terms are tuned for the placement of the flanker. Second, the developmental sequences are very different (fourth-order intermodulation increased more rapidly with development than does second-order intermodulation; Fig. 5). In the context of a sandwich model, developmental changes in the tuning of the output filter could contribute to this apparent dissociation. If, for example, the output filter increased its high-frequency cutoff, more fourth-order sum frequency response would be passed by the system. However, we find that the fourth-order difference term also shows a similar immaturity (data not shown). Because this component is lower in frequency than the second-order sum term, a simple change in the temporal bandwidth of the output filter is an unlikely explanation.
Phenomenological models of lateral interaction. A, Single high-order nonlinearity NL of fourth-order sandwiched between two linear filters, LF. The system output contains even harmonics of the two input frequencies as well as second- and fourth-order intermodulation terms. All terms at the output should share common tuning properties. B, Two-stage nonlinear cascade. Lateral inputs are combined only at the second stage, yielding only fourth-order intermodulation terms. This model does not account for second-order intermodulation. C, Two-stage nonlinear cascade with nonlinear interaction at both stages. The output of the first stage produces second-order terms that serve as the input to the second stage. The output of the second stage contains fourth-order self-term and intermodulation term. To be consistent with the data, the first stage must be tuned for relative orientation, but the second stage is not.
Another deficiency of the single squaring nonlinearity is that it cannot capture the self-term suppression effect. Candy et al. (2001) showed that a sigmoidal nonlinearity of the form r = cm/(k + cn) does predict suppression in two-input experiments. Depending on the specific choice of parameters and range of input intensities, a nonlinearity of the form r = cn/(k + cn) produces both suppression effects and fourth-order terms. Regan and Regan (1988) also showed that sigmoids and certain power law rectifiers will produce suppression and intermodulation. However, all single nonlinearity models will fail, because of the observed difference in orientation tuning and spatial selectivity of the second- and fourth-order intermodulation terms.
Cascade nonlinear model
An alternative model structure involves the cascade of two low-order nonlinearities. This general model structure has been proposed in the context of motion integration (Simoncelli and Heeger, 1998), texture discrimination (Victor and Conte, 1991, 1996), and vernier acuity (Victor and Conte, 2000). Higher-order response terms in this type of model are expected because of the cascaded nonlinearities. To take a specific example, the first stage in the Simoncelli and Heeger (1998) model is a sigmoidal nonlinearity. With exponents of 2 and a suitable value for k, the model is dominantly second-order. If these outputs are then passed to another second-order stage, fourth-order intermodulation will be generated, as well as fourth harmonics of the input frequencies.
The present data offer several constraints on possible versions of this model. One of these constraints is the configural specificity of the second-order intermodulation components in the absence of strong tuning of the fourth-order intermodulation response. This eliminates a version of the model in which the lateral interaction occurs only at the second site (Fig. 8B), because there would be no second-order intermodulation terms in this model. A more plausible model has an orientation and position tuned second-order nonlinear interaction at the first stage (Fig. 8C), followed by a second stage that pools across orientation and position.
The model structure and orientation tuning of Figure 8C is similar to that proposed by Simoncelli and Heeger (1998) for the integration of motion signals. In their model, the first nonlinear stage is direction selective and, by implication, orientation selective, whereas the second nonlinear stage pools across directions (e.g., orientation). Whether our stimuli are tapping the specific motion integration pathway modeled by Simoncelli and Heeger (1998) or the texture processing network of Victor and Conte (1996) cannot be determined at this point.
It must be noted, however, that completely characterizing a fourth-order model structure would require the use of four or more input temporal frequencies rather than only two, as we have used (Chua and Liao, 1991). Whereas we can state with certainty that the system we are probing has strong fourth-order structure, there are several different versions of a two-stage model that we could not discriminate, nor can we specify the details of each nonlinearity.
Development of spatial selectivity
Developmentally, it appears that both stages of the model are immature: the notional first stage is spatially selective in adults (less second-order interaction in the side-zones than in the end-zone) but not in infants where the iso-orientation interactions are equally strong in the end-zone and side-zone. A common feature of synaptogenesis in cortex is the overproduction of synapses during early development and their subsequent elimination after the initial phase of exuberance is completed. In human visual cortex, synaptic density reaches its peak at ∼8-11 months of age, with infants having approximately twice the synaptic density of adults (Huttenlocher et al., 1982; Huttenlocher and de Courten, 1987). During the initial phase of development, synaptic contacts are apparently made at random (Garber et al., 1980). It is, thus, possible that the infant's lack of spatially selective interaction is caused by excess synaptic input from the side-zone, which is eliminated later in development. Lateral inputs do not appear to be present in either infants or adults from orthogonal orientations in the end-zone or side-zone.
In the context of a two-stage model, the relative lack of fourth-order interaction terms could be caused by a relative immaturity in the second stage. Neurons that combine stimuli of different orientation, such as plaids, have been found in extrastriate cortical areas MT and V3 (Movshon et al., 1985; Rodman and Albright, 1989; Stoner and Albright, 1992; Gegenfurtner et al., 1997; Dobkins et al., 1998). It is thus possible that the second-stage is located in extra-striate cortex and that development in extra-striate areas lags that of striate cortex.
Footnotes
This work was supported by National Institutes of Health Grant EY-06579 to A.M.N. and a Rachel C. Atkinson Fellowship to C.H.
Correspondence should be addressed to Dr. Anthony M. Norcia, Smith-Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, CA 94115. E-mail: amn{at}ski.org.
Copyright © 2003 Society for Neuroscience 0270-6474/03/238630-11$15.00/0
