 |
||||
|
||||
|
||||
|
||||
Previous Article | Next Article
The Journal of Neuroscience, September 1, 2000, 20(17):6594-6611
Coding of Border Ownership in Monkey Visual Cortex
Hong Zhou1, Howard S. Friedman1, 3, and Rüdiger von der Heydt1, 2 1 Krieger Mind/Brain Institute, 2 Department of Neuroscience, and 3 Department of Biomedical Engineering, Johns Hopkins University, Baltimore, Maryland 21218
ABSTRACT
TOP
ABSTRACT
INTRODUCTION
Areas V1 and V2 of the visual cortex have traditionally been conceived as stages of local feature representations. We investigated whether neural responses carry information about how local features belong to objects. Single-cell activity was recorded in areas V1, V2, and V4 of awake behaving monkeys. Displays were used in which the same local feature (contrast edge or line) could be presented as part of different figures. For example, the same light-dark edge could be the left side of a dark square or the right side of a light square. Each display was also presented with reversed contrast.
We found significant modulation of responses as a function of the side of the figure in >50% of neurons of V2 and V4 and in 18% of neurons of the top layers of V1. Thus, besides the local contrast border information, neurons were found to encode the side to which the border belongs ("border ownership coding"). A majority of these neurons coded border ownership and the local polarity of luminance-chromaticity contrast. The others were insensitive to contrast polarity. Another 20% of the neurons of V2 and V4, and 48% of top layer V1, coded local contrast polarity, but not border ownership. The border ownership-related response differences emerged soon (<25 msec) after the response onset. In V2 and V4, the differences were found to be nearly independent of figure size up to the limit set by the size of our display (21°). Displays that differed only far outside the conventional receptive field could produce markedly different responses. When tested with more complex displays in which figure-ground cues were varied, some neurons produced invariant border ownership signals, others failed to signal border ownership for some of the displays, but neurons that reversed signals were rare.
The influence of visual stimulation far from the receptive field center indicates mechanisms of global context integration. The short latencies and incomplete cue invariance suggest that the border-ownership effect is generated within the visual cortex rather than projected down from higher levels.
Key words: primate visual cortex; visual perception; perceptual organization; figure-ground segregation; awake macaque monkey; single-unit activity; nonclassical receptive fields; area V1; area V2; area V4
INTRODUCTION
When neural function in the monkey visual cortex was first analyzed, it was concluded that the initial stages represent visual information in terms of local features, each neuron analyzing the small area of the retinal image covered by its receptive field, which occupies only a tiny fraction of the whole visual field (Hubel and Wiesel 1968
, 1977
Lamme et al. (Lamme, 1995
In the experiments to be described we have studied the context dependence of contrast border responses using displays in which the same local feature (contrast edge or line) was presented as part of a figure either on one or the other side of the receptive field. The Gestalt psychologists have pointed out that perception tends to "assign" contrast borders to objects (Koffka, 1935
View larger version (34K):[in this window]
[in a new window]
Figure 1. Perception of border ownership. A, Rubin's vase (Rubin, 1915). This well known ambiguous figure demonstrates the tendency of the visual system to interpret contrast borders as occluding contours and to assign them to one of the adjacent regions. In this example, figure-ground cues have been carefully balanced, but the black and white regions are generally not perceived as adjacent; instead, perception switches back and forth, and the borders belong either to the vase or to the faces. B, Isolated regions of contrast are generally perceived as "figures", that is, objects seen against a background. C, This display is generally perceived as two overlapping rectangles rather than a rectangle adjacent to an L-shaped object.
MATERIALS AND METHODS
Single neurons were recorded from areas V1, V2, and V4 in eight hemispheres of four alert, behaving monkeys (Macaca mulatta). The animals were prepared by attaching a peg for head fixation and two recording chambers (over the left and right visual cortex) to the skull with bone cement and surgical screws. The surgery was done under aseptic conditions under pentobarbital anesthesia induced with ketamine; buprenorphine was used for postoperative analgesia. All procedures conformed to the principles regarding the care and use of animals adopted by the American Physiological Society and the Society for Neuroscience, as verified by the Animal Care and Use Committee of the Johns Hopkins University.
Recording
The methods of recording were essentially the same as in von der Heydt and Peterhans (1989)1 week, another hole was drilled, and recording resumed for up to five holes in each chamber. Electrodes with fine tips were used that easily isolate single cells (platinum-iridium, 0.1 mm diameter, taper 0.07-0.1 glass-coated, impedance 3-15 M
at 1 kHz; von der Heydt et al. 2000
Anatomical methods
After the recordings were completed, the animal was anesthetized, and thin, sharply pointed marker pins were inserted in parallel tracks at known positions around the recording regions with the positioning device used for recording. The animal was then given an overdose of pentobarbital, and the brain was perfused with buffered 4% formaldehyde. The pins were removed, the tissue was blocked and soaked in 30% sucrose, and 50 µm frozen sections were cut at right angles to the orientation of the pins (tangential sections). The sections were stained for cytochrome oxidase. The positions of the recording tracks were determined from the electrode positioning coordinates by interpolating between the positions of the marker pins. For one animal (M12) the recording sites were reconstructed by tracing the outlines, layers, and pin holes with a computer-controlled microscope (Neurolucida) and plotting the tracings together with the positions of the recording tracks. The depths were determined by aligning the depth records in the track charts (see above) with the corresponding anatomical landmarks. This method generally confirmed our previous identification of cortical layers according to physiological criteria. In the other three animals, only the locations of the tracks were determined to verify the cortical areas. In this case, the layer assignment of V1 was based only on the track charts.Visual stimulation and behavioral paradigm
Two experimental setups were used, setup 1 for animals M12 and M15 and setup 2 for animals M13 and M16 (in the results to be presented below, the first digits of the neuron identification numbers indicate the animal). In setup 1, visual stimuli were generated by an Omnicomp GDS 2000 processor controlled by a personal computer and displayed on a Hitachi HM4119 color monitor with a 60 Hz refresh rate. Fixation target and test stimuli were viewed through a mirror stereoscope at a distance of 51 cm. The visual field measured 11.5° square for each eye, with a resolution of 400 × 400 pixels. In setup 2, visual stimuli were generated by a Silicon Graphics Indigo2 workstation and displayed on a BARCO CCID 121 FS color monitor with a resolution of 1280 × 1024 pixels and 72 Hz refresh rate. This display was viewed directly with both eyes at a distance of 93 cm and subtended 21° by 17° visual angle. The stimuli were colored or gray rectangles presented on a neutral gray background, as specified in Table 1. Eye movements were monitored by means of video-based infrared pupil tracking (Iscan) with 0.15° horizontal and 0.28° vertical resolution.
The animals were trained to fixate their gaze by requiring them to respond to an orientation change that could only be resolved in foveal vision. The fixation target was a 7 arc min white square divided by a thin gray line whose change from vertical to horizontal had to be detected. The target was centered on a 19 arc min black square to facilitate fixation. The general trial sequence was as follows: target onset, monkey responds by pulling a lever and begins to fixate, 0.5-5 sec variable interval (fixation period), target rotates, monkey responds by releasing lever, 1-2 sec variable interval (monkey usually looks away from target), new trial begins with target onset, etc. The hit rate during recording sessions was ~95% on average. Two of the monkeys (M13 and M16) served also in a study on perceptual filling in, in which they were trained to respond to a color change of a peripherally viewed disk-ring stimulus. There was no difference between these monkeys and the other two monkeys in the results of the present experiments. Procedure. To study a representative sample of cells, an exhaustive analysis was attempted. We did our best to study every cell that was isolated and not to skip "difficult cells." After isolation of a cell, the receptive field was examined with rectangular bars, and the optimal stimulus parameters were determined by varying the length, width, color, orientation, and binocular disparity (in setup 1). Using this optimal stimulus, we then determined the "minimum response field", which was defined as the minimum visual field region outside which the stimulus did not evoke a response (Barlow et al., 1967
View this table: [in this window]
[in a new window]
Table 1. CIE (1931) coordinates of the stimuli typically used for testing color selectivity Routside)/(Redge
Overlapping figures. In this test, the border between a square and an L-shaped region was centered on the receptive field as illustrated in Figure 3. Human observers generally perceive such displays as two overlapping figures. Again, the contrast borders in the receptive field are locally identical in A and B, but belong perceptually to different figures. The same is true for C and D, with figure colors reversed. Most of the display remained unchanged between A and B (or C and D), as indicated by hatching in Figure 3E.
View larger version (66K):[in this window]
[in a new window]
Figure 2. The standard test for determining the effect of border ownership on edge responses. In A and B, identical contrast edges are presented in the receptive field (ellipses), but in A, the edge is the right side of a dark square, in B, it is the left side of a light square. The relation is analogous between C and D, with reversed contrasts. E, The hatched region indicates the neighborhood of the receptive field in which displays A and B (or C and D) are identical. The preferred color of the cell (including black, white, and gray) and a light gray were used as the colors in these displays.
View larger version (70K):[in this window]
[in a new window]
Figure 3. Overlapping figure test. In each of these displays two regions of approximately the same area are presented on either side of the receptive field (ellipses). As in Figure 2, the contrast edges in the receptive field were identical in A and B and in C and D but belonged perceptually to different figures. In E, the hatched area indicates the region of identical stimulation.
Data collection and data analysis
The signal from the microelectrode was passed through adjustable 24 db/octave high-pass and low-pass filters and a window amplitude discriminator. Spike events were recorded with 0.1 msec resolution, and those during the fixation period were analyzed. To determine the time course of responses, the spike trains of each cell were convolved with a Gaussian, averaged over repetitions, and normalized to the mean firing rate of the cell during the period of analysis (1 sec). For the curves of Figure 20 the normalized responses were averaged across cells for each cortical area (convolution with= 16 msec for V1, and
RESULTS
Because the representation of contrast borders was the focus of this study, only results from edge-selective neurons are reported in this paper. This means that all neurons included in this study (1) responded to lines or edges much longer than the receptive field (neurons with strong end stopping were excluded), and (2) did not respond, or responded much less, when a large uniform stimulus was centered on the receptive field. For neurons of V1 and V2, edge selectivity was determined from the responses to a 4° (occasionally 6°) square, and the criterion was a surface-to-edge response ratio <0.25 (see Materials and Methods), which means that the cell responded at least four times more vigorously to an optimally oriented edge than to the center of the square. In V2 and the top layers of V1, ~85% of the neurons are edge-selective by this criterion (H. Zhou, H. S. Friedman, and R. von der Heydt, unpublished observations). In V4, edge selectivity was assessed with larger figures because the minimum response fields of most V4 cells were much larger than those of V1 and V2 cells (see below). For many V4 cells the minimum response field could not be determined because part of its boundary was outside the display field. Therefore, for V4 cells, the most sensitive position ("hot spot"), as determined with a bar or edge, was usually taken as the center of receptive field for the subsequent tests. Only cells with clear edge selectivity were studied. These were <50% of the V4 cells that we attempted to study. Our sample also includes a few cells that responded to thin bars, but to neither surface nor borders of uniform color squares. These cells were studied with outlined figures. The vast majority of the cells included in this study were orientation-selective (see below).
Border-ownership coding was studied in 206 cells, 63 of V1, 91 of V2, 45 of V4, and seven from the V1-V2 and V2-V3 border regions. The cells of V1 were recorded in layers 2 and 3 of the cortex. Fifty-six cells (30 of V1 and 26 of V2) were studied using the small-field, stereoscopic setup, and 143 cells (33 of V1, 65 of V2, and 45 of V4) were studied using the large-field, direct-view setup. Thirty-seven cells (20 of V1, 17 of V2) were studied with static displays, the rest with switching displays (see Materials and Methods). Solid squares as shown in Figure 2 were used if a cell responded to edges, which was generally the case (187 cells). If a cell responded only to lines and thin bars, outlined squares were used (19 cells; 2of V1, 6 of V2, 11 of V4). (A number of cells were tested with both solid and outlined squares and with other types of figure-ground displays.)
The receptive fields of the cells of V1 and V2 were located in the lower contralateral visual field at eccentricities between 0.6 and 6° (median, 1.5°) for V1 and between 0.2 and 7.4° (median, 2.0°) for V2. The standard test was performed with squares of 4°, or sometimes 6° size. This is much larger than the typical size of the minimum response fields of V1 and V2 at those eccentricities. In V1, the length of the minimum response fields varied between 0.2° and 1.2° (median, 0.5°), and the width varied between 0.1° and 1.2° (median, 0.5°). In V2, the lengths were 0.2-3.0° (median, 0.7°), and the widths were 0.1-2.7° (median, 0.4°). For 13 cells (2 of V1 and 11 of V2) the "length" of the minimum response fields was negative (see Materials and Methods). Most of the cells tested in V1 and V2 responded best to long edges or bars; only 15 of 161 cells exhibited moderate degrees of end stopping (responses to short bars 2.1 times stronger than responses to long bars on average). In V4, the eccentricities of receptive fields ranged from 0.3 to 11° (median, 6.6°). In 10 of the V4 cells the standard test was performed with square sizes of 4-8° and in 24 cells with square sizes of 10-17°. One edge of the square was centered on the hot spot, while the other edge was either outside the response field or cutoff by the display. The lengths of the minimum response fields ranged from 1.7 to 12° (median, 3.6°), and the widths ranged from 0.1 to 5° (median, 2.4°).
In the following Section 1, we will present the results on the frequency and strength of border-ownership signals and contrast polarity signals, as assessed with the standard test, in the three cortical areas. We will discuss the range of spatial integration and the time course of the border-ownership signals and try to relate these findings to the conventional receptive field properties. In Section 2, we will present results of experiments in which we varied the stimulus configurations to gain insight into the mechanisms of border-ownership coding.
Section 1: results obtained with the standard test
Each neuron was tested with the four kinds of displays shown in Figure 2 (see Materials and Methods). If the activity of a neuron was determined by local features, it would respond equally to A and B, and equally to C and D, because these pairs of stimuli are locally identical. However, we found that many neurons responded to the same local edge differently, depending on the side to which the edge belonged. Based on the results of the standard test, we distinguished four types of results: (1) cells coding border ownership, (2) cells coding the polarity of edge contrast, (3) cells coding border ownership and polarity of edge contrast, and (4) cells coding neither border ownership nor polarity of contrast. We will first present examples of these four types of results and describe some control experiments, and then we will discuss the classification of cells and their reliability in signaling border ownership and contrast polarity.
Type 1: border ownership
Figure 4 shows the responses of a cell from V2. The stimulus was a green square surrounded by gray in A and D, and a gray square surrounded by green in B and C (the cell was not particularly color-selective, but green produced the largest response). The ellipses indicate the minimum response field of the cell, and the crosses mark the position of the fixation target. The raster plots at the bottom show the responses to repeated random presentations of the four stimuli. (Each row of small lines represents the activity during one fixation period; for each condition, responses have been sorted by the length of the fixation period.) It can be seen that the cell preferred a square on the left side (A and C) regardless of the figure contrast. Figure 5 shows the mean strengths of responses with SEs for the conditions of Figure 4 (4° squares) and for square sizes of 10 and 15°. Displays that were identical in the response field of the cell have been juxtaposed in rows A and B, and the bars below represent the corresponding responses. In each case, the cell responded more strongly when the square was on the left side (row A), and this preference was independent of the contrast polarity. Cells with this type of behavior were found in all three cortical areas, but more often in V2 and V4 than in V1.
View larger version (52K):[in this window]
[in a new window]
Figure 4. Example of border-ownership coding in a cell of area V2. The stimuli are shown at the top, and event plots of the corresponding responses are shown at the bottom. The ellipses indicate the location and orientation of the receptive field, and the crosses show the position of the fixation target. In the event plots, small vertical lines represent the times of action potentials, relative to the moment of lever pulling (which generally indicated the beginning of fixation). Small squares indicate the times of target flip (end of fixation). Each row represents a trial. Several repetitions are shown for each condition, sorted according to the length of the fixation period. A, B, The cell responded better to the edge of a green square on the left side than to the edge of gray square on the right side of the receptive field, although both stimuli were locally identical (green depicted here as light gray). C, D, When the colors were reversed, the cell again responded better to an edge that belonged to a square on the left than a square on the right. Square size, 4°; length of minimum response field, 0.4°; location in visual field (0.0°,
View larger version (57K):[in this window]
[in a new window]
Figure 5. Size invariance of border-ownership coding. The same V2 cell as in Figure 4. Rows A and B show the stimuli, with pairs of locally identical stimuli juxtaposed. Conventions as in Figure 4. Bar graphs below show mean firing rates and SEs of the corresponding responses. Square sizes: 1 and 2, 4°; 3 and 4, 10°; 5 and 6, 15°. For each size, and for either contrast polarity, the responses were stronger when the square was located on the left side of the receptive field.
Type 2: edge-contrast polarity
Cells that were selective for edge-contrast polarity were also observed in all three cortical areas. Figure 6 shows an example of a cell recorded in layer 2/3 of V1. Yellow squares on light gray background (A, D) were compared with light gray squares on yellow background (B, C). The cell responded more strongly in A and B than in C and D, but whether the square was on the left or the right side made no difference. Thus, the local edge contrast determined the responses of this cell.
View larger version (53K):[in this window]
[in a new window]
Figure 6. Selectivity for local contrast polarity. Cell of layer 2/3 of V1. The cell responded more strongly to light-dark edges (A, B) than to dark-light edges (C, D) irrespective of the position of the figure. The colors were yellow (depicted as light gray) and gray. Location of receptive field (
Type 3: border ownership and edge-contrast polarity
Figure 7 shows the responses of a cell of V2 that was selective for side of figure and local contrast polarity. This cell was color-selective, preferring dark reddish colors, as illustrated in Figure 8. Brown and gray were used for this test, as shown. The cell responded to the top edge, but not the bottom edge of the brown square and barely at all to the edges of the gray square. The differences between C and A and between D and B indicate that the cell was selective for edge-contrast polarity, which is typical for many V1 simple cells (Schiller et al., 1976
View larger version (51K):[in this window]
[in a new window]
Figure 7. Example of simultaneous coding of border-ownership and edge-contrast polarity. This cell of area V2 was color-selective with a preference for dark, reddish colors (see Fig. 8). Brown and gray were used for the test. Conventions are the same as for Figure 4. The cell responded to the top edge of a brown square (C), but hardly at all to the bottom edge of a gray square (D), although in both cases the same gray-brown color boundary was presented in the receptive field. The cell did not respond at all to edges of the reversed contrast (A, B). Square size, 4°; length of minimum response field, 1.4°; location in visual field (1.4°,
View larger version (20K):[in this window]
[in a new window]
Figure 8. The color selectivity of the cell of Figure 7. Bars of 15 colors (Table 1) were flashed in the receptive field for 500 msec with intervals of 500 msec. The graph represents mean firing rates with SEs. Activity during the On phases is plotted upward, and activity during the Off phases is plotted downward. It can be seen that the cell responded to reddish hues better than to greenish hues, with a preference for the darker representative of each hue (brown > red, beige > yellow, black > white, etc.). Whereas, in most cells, the demonstration of border-ownership coding did not require chromatic stimuli, cells with striking chromatic selectivity were also common.
View larger version (56K):[in this window]
[in a new window]
Figure 9. Size invariance of border-ownership coding in the color-selective cell of Figures 7 and 8. Side of ownership produced response differences for 4 and 6° squares, but not for squares of 11° size. Human observers also find the distinction of figure and ground weak for the largest size. Colors were brown and gray, here depicted as dark and light gray.
Type 0: neither border ownership nor contrast polarity
Many cells responded approximately equally to all four conditions of Figure 2. These were edge-selective and generally orientation-selective cells with no preference for local contrast polarity or figure-ground direction.Controls
Given the generally small size of the minimum response fields of the cells, we wondered whether variations in retinal stimulus position caused by fixational eye movements could have affected the responses. Random variations would only add noise to the data, but if changes in fixation were related to the figure position, a systematic variation of responses could result. We have analyzed the eye movements quantitatively in one of our monkeys, M16. The eye movement displays and performance in the fixation task of the other animals indicated that their fixation behavior was similar. The following data were obtained during the 3 d after the neural data collection was completed. The same fixation task was used, and the four displays of the standard test were presented in counterbalanced order, exactly as during the neural data collection. Eye movements were recorded for ~4000 fixation periods, each 0.5-5 sec in duration. As always, only trials in which the animal performed the fixation task correctly were analyzed. Figure 10 shows the results. Plots a-c show data from three successive blocks of recording, each consisting of ~320 trials, with figure orientations of 45, 135, and 45°, respectively. Each plot represents the means and SDs of the eye position signal for the four stimulus conditions labeled A-D, as in Figure 2. It can be seen that the differences between the means were small compared to the overall variation. The relevant measure is of course the variation of gaze position perpendicular to the edge of the figure, that is, along the 135° diagonal for plots a and c and along the 45° diagonal for plot d. Thus, the question is whether means A and C were separated from means B and D along these axes. No such separation was apparent in these samples. To determine the statistical distribution of gaze shifts that might have influenced our analysis of neural responses, we have segmented the eye movement recordings into blocks of 40 trials, which was the typical length of blocks used to assess border-ownership selectivity. The component of eye movements in the critical direction was computed, and a two-way ANOVA with the factors side-of-figure and contrast polarity was performed for each block. The histogram in Figure 10d shows the distribution of the effects of the factor side-of-figure. One can see that the gaze shifts were generally small (absolute size <2.8 arc min in 50%, <7 arc min in 95% of the cases) compared to the size of the minimum response fields of the cells that we have studied (median widths of 30, 24, and 144 arc min for V1, V2, and V4). Thus, the effects of variations in gaze position on the magnitude of the neural responses were probably insignificant.
View larger version (39K):[in this window]
[in a new window]
Figure 10. Eye movements during fixation. Recordings from monkey M16 during the standard test displays with 4° squares. a-c, Means and SDs of horizontal and vertical positions of gaze, grouped according to display type (Fig. 2, A-D). Each plot represents ~320 trials with stimulus orientations of 135° (a), 45° (b), and 135° (c) recorded in succession. If the side of the square had influenced gaze position systematically, means A and C would appear displaced relative to means B and D in the direction perpendicular to stimulus orientation. No systematic displacements were apparent. d, Histogram shows the distribution of the effects of figure position on eye movements perpendicular to stimulus orientation, as determined by ANOVA, in 101 blocks of 40 trials each. Positive values designate eye movements that would have moved the receptive field toward the center of the figure. Binwidth, 3 arc min. The eye movements were small and not systematically related to the side of the figure.
Because the retinal image is never stationary under normal conditions, border ownership signals should not depend critically on stimulus position if they were to serve a function in vision. Figures 11 and 12 show examples of recordings in which we have studied the effect of varying the stimulus position. The cell of Figure 11 was recorded in V2 and had a small response field. The cell of Figure 12 was from V4 and had a large response field. For each cell, an edge was positioned in the receptive field center at the preferred orientation and presented either as part of a dark square or as part of a light square, as shown in the insets. Position was then varied along an axis orthogonal to the edge. The responses are plotted as a function of edge location relative to the center position. Open circles represent the responses obtained with the light square (A), filled circles represent the responses obtained with the dark square (B). It can be seen that the cell of Figure 11 responded better to the top left edge of the light square than to the bottom right edge of the dark square at any position, and the cell of Figure 12 responded to the top edge of the light square, but not the bottom edge of the dark square, at any position. These results show that border ownership signals are robust and do not depend on the exact positioning.
View larger version (48K):[in this window]
[in a new window]
Figure 11. Position invariance of border ownership coding in a cell of V2. The top left edge of a light square (A), and the bottom right edge of a dark square (B), were centered on the receptive field, and position of the squares was then varied. Mean firing rates and SEs are plotted as a function of edge location relative to receptive field center. Open circles represent responses to edge of light square (A), and filled circles represent responses to edge of dark square (B). In either case, the maximum response was obtained when the edge was centered on the receptive field, but the responses were stronger for A than for B at any position. SA, Level of spontaneous activity. Ellipse, Minimum response field. Cross, Fixation target. Line indicates range of variation of edge position, positive toward bottom right. Colors were aqua and gray; size of square, 4°; receptive field location (0.4°,
View larger version (43K):[in this window]
[in a new window]
Figure 12. Position invariance of border ownership coding in a cell of V4. The cell responded to the top edge of the light figure (A), but not the bottom edge of the dark figure (B) at any position. SA, Spontaneous activity. Straight line indicates range of variation of edge position, positive downward. Ellipse delineates the most sensitive region of the receptive field (approximate contour of half-maximal response), the gray line the more uncertain total extent (note that the short axis of the ellipse corresponds to the preferred orientation of the cell). Colors, yellow and light gray; width of figure, 10°; receptive field location (4.6°,
As pointed out above, the size of the square determines the area of identical stimulation in the standard test. In other words, a difference in response indicates an influence from outside the region occupied by the squares in the two positions (Fig. 2, hatched area). This area was generally much larger than the minimum response field of the cell. It is important to note that in cells of V1 and V2 and also in many cells of V4, the minimum response field is sharply defined, and no response can be evoked with bars outside this field. Figure 13 illustrates an example of a cell of V2 with a small receptive field near the fovea. A shows position response functions obtained with an optimally oriented bar of 1° length. The insets illustrate bars and receptive field. It can be seen that the responses drop to zero when the bar leaves the small region marked by the ellipse. This region measures <1° along the preferred orientation. The limits of the response field were confirmed with a square of 8° size (B). In the standard test, performed with squares of the same size, the cell responded better when the figures were on the bottom left side of the receptive field than on the other side (C). Thus, although contours presented as close as 1° to the receptive field center produced zero responses, the activity evoked by a contour in the response field was strongly modulated by the image context at a distance of >4°. Note also that the cell was not end-stopped (strong response to the center of the 8° long edge).
View larger version (28K):[in this window]
[in a new window]
Figure 13. Comparison of conventional receptive field size and extent of image context integration. Responses of a cell of V2. A, Firing rate as a function of the position of a 0.2° wide, 1° long static white bar on gray background. The bar was presented at the preferred orientation of the cell, and position was varied along that orientation (right) and along the orthogonal axis (left). Insets show the bar at positions +1 and
Quantitative classification
We found no differences regarding the coding of border ownership between the data recorded with the two setups. Specifically, the proportions of cells that showed an influence of side-of-ownership were similar. Also, tests with displays that remained constant throughout the trial and displays in which the figures were switched on after the beginning of fixation produced essentially the same results (Fig. 14). The data from the two setups and the results obtained with static and switching displays were therefore pooled for the statistical analysis.
View larger version (17K):[in this window]
[in a new window]
Figure 14. Static and switching displays produced similar results. Responses from a cell of V2 that was selective for border ownership and contrast polarity. The labels refer to the displays of Figure 2. Colors red and gray were used.
Figure 15 shows, for the three cortical areas, the distributions of the magnitude of the effect of side-of-ownership, expressed as the ratio between the responses to the nonpreferred side and the preferred side. For cells tested with the standard solid squares (n = 180) the responses for the two contrast polarities were added, for example, (B + D )/(A + C) for the cell of Figure 4. Another 19 cells that were tested with outlined squares (in which figure and background had the same color) are included in this figure. It can be seen that response ratios <0.5 were common in V2 and V4. For cell 13id4 of Figure 4 the ratio was 0.35, for cell 12ij2 of Figure 7 it was 0.11. Cells with such low ratios were also found in V1, but less frequently. The statistical significance of the effects of border ownership and contrast polarity was determined by performing a three-factor ANOVA on the data of the standard test for each cell, the factors being side-of-ownership (Fig. 2, A and C vs B and D), local contrast polarity (A and B vs C and D), and time after lever pulling (which indicates the beginning of fixation). Only the spikes during the fixation period were included, and data were sampled in 1 sec bins. The factor time was included to account for response variations during the fixation periods. A significance level of 0.01 was chosen. The distributions of the four response types in the three cortical areas are summarized in Figure 16. In V2 and V4, more than half of the cells showed significant border ownership modulation (type 1 and type 3: V2, 59%; V4, 53%), compared to 18% of the cells of V1. A majority of these cells were also affected by the polarity of edge contrast (type 3). Cells that were influenced by border ownership irrespective of the contrast polarity (type 1) were rare in V1, but made up 15% of the cells in V2 and V4. The p values for side-of-ownership differed significantly between the areas (p < 10
4; Kruskal-Wallis); the differences V1-V2 and V1-V4 were significant (p < 0.005), but the difference V2-V4 was not (p > 0.75). There was no difference between areas in the p values for local contrast polarity (p > 0.17).
View larger version (16K):[in this window]
[in a new window]
Figure 15. The distributions of the magnitude of the border-ownership effect in the three cortical areas V1, V2, and V4. The response ratio is the ratio of the mean response to the nonpreferred side over the mean response to the preferred side.
View larger version (44K):[in this window]
[in a new window]
Figure 16. The distributions of the types of contour responses found in cortical areas V1, V2, and V4. Classification based on two-factor ANOVA. Ownership, Responses modulated according to side of ownership; contrast, responses modulated according to local contrast polarity; ownership & contrast, modulation by either factor; none, no modulation. In V2 and V4, more than half of the cells showed border-ownership modulation. Note that there are fewer cells in this figure than in Figure 15 because cells tested only with outlined figures are not included here.
Only the main effects of side-of-ownership and local contrast polarity were represented Figure 16. In many cells (58 of 180) the analysis indicated significant interaction of these factors. Most of these cells (38 of 58) were of type 3 (significant effects of side-of-ownership and local contrast polarity). To understand this result, consider the four displays of Figure 2. An absence of interaction implies that the sum of the responses to A and D equals the sum of the responses to B and C. For the cell of Figure 4, this was approximately true (interaction
The cell of Figure 7 responded to the edge of a brown figure embedded in gray (C) but not to the edge of a gray figure embedded in brown (D). Brown was the preferred color of the cell, as determined with flashing bars (Fig. 8). The question is of interest whether type 3 cells in general preferred displays in which the preferred color was figure over displays in which the preferred color was ground. This was in fact the case in 31 of 34 color or luminance-selective type 3 cells, while three cells preferred the displays in which the preferred color was ground.
Significant interaction between side of ownership and local contrast polarity was also found in 20 cells in which only one of the main effects, or none of them was significant (side of ownership, 6; local contrast polarity, 8; none, 6). This behavior is also interesting because these cells were edge-selective, but preferred figures of one color, no matter on which side the figure was presented. In the extreme, such a cell would respond, for example, to A and D of Figure 2, but not to B and C, although exactly the same edges were presented. What mattered was which color was figure and which was ground. Five cells of this kind were recorded in V1, eight in V2, and seven in V4.
The time after beginning of fixation had a significant effect only in a minority of cells (V1, 11 of 61; V2, 12 of 85; V4, 11 of 34). Thus, the mean firing rates of most cells were rather constant throughout the fixation period. Furthermore, there was generally no interaction between side-of-ownership and time (significant interactions were found in one cell of V1, two cells of V2, and three cells of V4). This means that, generally, the border-ownership signal neither strengthened nor weakened during the fixation period. Indeed, a two-factor analysis of the effects of border-ownership and contrast polarity, ignoring the time variable, produced similar results as the ones shown in Figure 16. Also the three-way interaction, which would indicate a time dependence of figure-color preference, was rarely significant (one cell of V1, two cells of V2, three cells of V4; three of these cells were among the aforementioned group with interaction between side-of-ownership and time). This analysis was done at a coarse scale, the exact time course of the response onset will be discussed below.
Reliability of signals
The above classification gives the answer to the question: was there a significant influence of the factors side of ownership and contrast polarity? A somewhat different question is: how well would the neural signals discriminate side of ownership and/or contrast polarity? This is the question of how reliable the information is that a group of cells can provide to other stages of processing. The first question examines the strength of the given data (and the answers depend on the amount of data sampled from the individual cells), the second question concerns the reliability of the neural signals (and the answer should not dependent on sample sizes, except for an error of estimate).
The reliability of the neural border ownership signals was assessed by determining the proportion of consistent responses of a simple decision model, as shown in Figure 17. We assume that for each cell A recorded in a given area there exist three sister cells B-D, whose receptive fields are identical except that those of A and D are mirror images of each other, and so are those of B and C and that local contrast selectivity is reversed between A and C and between B and D. Note that we do not make assumptions about the actual selectivity of these cells. We only assume that for each receptive field structure that is realized there is also the mirror-image structure, and for each pattern of contrast polarity-sensitive inputs (e.g., ON- and OFF-types) there is also a pattern of reversed polarity. Because the cells have otherwise identical properties, recording the responses of one cell to the four stimuli of Figure 2 tells us how each of the four cells would respond to these stimuli. Our model figure-ground mechanism combines the responses of the four cells in the form (A
View larger version (10K):[in this window]
[in a new window]
Figure 17. A decision model used to estimate the reliability of border ownership signals. The model assumes four neurons A-D with identical receptive fields except for reversals of side preference and contrast polarity preference, as indicated by the symbols (tabs for side preference, fill pattern for contrast polarity preference). The decision was based on the responses of these four neurons combined in the form (A
Reliability estimates were obtained for each cell using the first second of responses. We calculated the output of the decision model for all combinations of the available responses to stimuli A-D of Figure 2 (on the order of 104 combinations). The proportion of cases in which the output indicated side of figure (or edge-contrast polarity) correctly was then determined ("correct" being defined by the mean responses to the four stimuli). This proportion estimates the probability that a correct decision would be made from 1 sec of response of the quadruple of cells to a random presentation of one of the stimuli of Figure 2 (assuming that the signals of the four cells are statistically independent). The resulting values are distributed between 0.5 (random) and 1 (perfectly consistent). To illustrate the results: the reliability of the cell of Figure 4 (type 1) was 97% for side of ownership and 82% for local contrast polarity; the cell of Figure 6 (type 2) was only 77% reliable for side of ownership, but 99% for local contrast polarity; the cell of Figure 7 (type 3) was 100% reliable for side of ownership, and 100% for local contrast polarity.
Figure 18 shows the joint distributions of the reliability estimates for border ownership and local contrast polarity discrimination for areas V1, V2, and V4. Each point represents an individual cell. Histograms of the integrated distributions are shown on the margins. The plots show that cells with perfect border-ownership coding (data points near the right margin) were common in V2 and V4, but rather rare in V1, where only four cells came close to 1. Nevertheless, the existence of such signals in the primary visual cortex is remarkable. The histograms at the top suggest that the distributions for V2 and V4 might be bimodal. Comparing medians, significant differences were found between V1 and V2 and between V1 and V4 (p < 0.003), but not between V2 and V4 (p = 0.87; Kruskal-Wallis). On the other hand, it can be seen that coding of edge-contrast polarity was excellent in many cells of V1 (data points near the top margin). Contrast polarity-selective cells were also found in the other two areas in similar proportions, as indicated by the histograms on the right margins. The median reliability in contrast polarity coding was not different between the three areas (p > 0.5). It can be seen also that some cells of V2 and V4 performed well in both dimensions (points in the top right corner).
View larger version (19K):[in this window]
[in a new window]
Figure 18. The reliability of neural responses in signaling border-ownership and local contrast polarity. Each dot represents the reliability estimates for one neuron derived from 1 sec samples of activity as explained in Results (0.5 = random, 1.0 = perfectly consistent). The histograms show distributions of reliability estimates for each dimension. Cells that reliably signaled edge-contrast polarity were common in V1 (dots at top of scatter plot). Their relative frequency was similar in the extrastriate areas, as shown by histograms on right margins. Cells that signaled border ownership were rare in V1, but common in V2 and V4 (dots near right margins). Substantial fractions of cells signaled both border ownership and polarity of contrast (dots in top right corners).
Size invariance
The results of Figures 5 and 9 showed that neurons were border ownership-selective over a range of figure sizes. Because the size of the figure determined the area of the visual field that received identical stimulation in the displays to be compared (Fig. 2), the maximum size at which a reliable difference is still obtained indicates the extent of visual context integration and the extent of neural convergence which is necessary to achieve such integration. We have examined 26 cells (three of V1, 17 of V2, and six of V4) with the standard test using different square sizes. Figure 19 plots the reliability of border-ownership coding as a function of size. Lines connect the data points of individual cells. The plots include results for displays in which large parts of the squares were cut off by the screen margins as, for example, in Figure 5. However, data for perceptually ambiguous displays have been excluded (that is, displays divided in two by a straight border, the displays on right of Fig. 9, and displays in which only one corner of the square was visible). Surprisingly, in most cases the reliability of border-ownership coding diminished only slightly with increasing figure size.
View larger version (20K):[in this window]
[in a new window]
Figure 19. The effect of figure size on border ownership discrimination in cortical areas V1, V2, and V4. Each point represents a reliability estimate based on a test with one size; points for the same cell are connected by lines. The figure size generally had little effect.
Time course
Response histograms were computed for the four conditions for each cell, normalized, and then averaged over cells (see Materials and Methods). Only cells with significant border-ownership modulation were included. The mean responses for the preferred and nonpreferred sides of the square (average of the two contrast conditions) are plotted in Figure 20 as functions of time after stimulus onset. It can be seen that the responses to the preferred and non-preferred sides diverged almost from the beginning in all three cortical areas. To quantify the latencies, we have determined the points of half-maximal signal for the sum and the difference of the responses for the two sides. The latencies of the summed responses were 57 msec in V1, 43 msec in V2, and 63 msec in V4, and the corresponding latencies of the differences were 69, 68, and 73 msec. (The reason why the summed response of V2 has a shorter latency than that of V1 might be the higher relative weight of magnocellular input in the sample of V2 compared to that of top layer V1) (cf. Bullier and Nowak, 1995
View larger version (20K):[in this window]
[in a new window]
Figure 20. The time course of border-ownership modulation. The figure shows the average responses of all neurons with significant border-ownership modulation in the three areas. The responses of each cell were normalized to its mean firing rate during the fixation period and averaged. Zero on the time scale refers to the onset of the figure-ground display. Thick and thin lines represent responses to preferred and nonpreferred sides, averaged over both contrast polarities. A differentiation was evident shortly after the response onset for cells in all three cortical areas.
We wondered if other signals existed in the cortex that could provide the peripheral information earlier. Using the same method, we have analyzed the responses of the cells that did not differentiate side of ownership and obtained latencies of 54, 74, and 69 msec for the three cortical areas, which is similar to the latencies of the border-ownership cells. Taken together, these results indicate that the cortical processing that leads to border-ownership discrimination requires no more than ~25 msec.
Relation to conventional receptive field properties
Because the signals of some cells carry border-ownership information, whereas others do not, the question arises if this phenomenon can be related to any of the conventional receptive field properties. We have analyzed orientation, color, and disparity selectivity, and the property of end stopping. Orientation selectivity was quantified by the orientation modulation index:
where Rmax and Rmin are the maximum and minimum responses of the orientation tuning. As mentioned above, most cells included in this study were orientation-selective (OMI > 0.6; V1, 89%; V2, 78%; V4, 82%). The orientation modulation index was not different<
