 |
|||||
|
|||||
|
|||||
|
|||||
|
|||||
Previous Article | Next Article
The Journal of Neuroscience, December 1, 2001, 21(23):9419-9429
Macaque Inferior Temporal Neurons Are Selective for Three-Dimensional Boundaries and Surfaces
Peter Janssen, Rufin Vogels, Yan Liu, and Guy A. Orban Laboratorium voor Neuro-en Psychofysiologie, KU Leuven Medical School, B-3000 Leuven, Belgium
ABSTRACT
TOP
ABSTRACT
INTRODUCTION
The lower bank of the superior temporal sulcus (TEs), part of the inferior temporal cortex, contains neurons selective for disparity-defined three-dimensional (3-D) shape. The large majority of these TEs neurons respond to the spatial variation of disparity, i.e., are higher-order disparity selective. To determine whether curved boundaries or curved surfaces by themselves are sufficient to elicit 3-D shape selectivity, we recorded the responses of single higher-order disparity-selective TEs neurons to concave and convex 3-D shapes in which the disparity varied either along the boundary of the shape, or only along its surface. For a majority of neurons, a 3-D boundary was sufficient for 3-D shape selectivity. At least as many neurons responded selectively to 3-D surfaces, and a number of neurons exhibited both surface and boundary selectivity. The second aim of this study was to determine whether TEs neurons can represent differences in second-order disparities along the horizontal axis. The results revealed that TEs neurons can also be selective for horizontal 3-D shapes and can code the direction of curvature (vertical or horizontal). Thus, TEs neurons represent both boundaries and surfaces curved in depth and can signal the direction of curvature along a surface. These results show that TEs neurons use not only boundary but also surface information to encode 3-D shape properties.
Key words: macaque; vision; extrastriate; inferior temporal; binocular disparity; object
INTRODUCTION
The lower bank of the rostral superior temporal sulcus (STS), TEs, contains neurons selective for disparity-defined three-dimensional (3-D) shapes (Janssen et al., 1999b
, 2000a
The stimuli in our previous experiments were disparity-defined 3-D shapes filled with a texture of random dots and having both the surface and boundary curved in depth along the vertical axis. With two-dimensional (2-D) stimuli, IT neurons have been shown to respond selectively to either boundary (shape) or to surface characteristics (texture or color), or to combinations of the two (Desimone et al., 1984
In the previous studies, the stimuli were curved only along the vertical axis. The second aim of the present study was therefore to investigate whether TEs neurons can also represent differences in second order disparities along the horizontal axis, such as that in a vertically oriented cylinder. In real-world objects, the (local) disparity can vary along either the vertical or the horizontal axis, or along both (Koenderink, 1990
MATERIALS AND METHODS
Subjects. Recordings were made in three hemispheres of two juvenile rhesus monkeys (monkey J. and monkey C.). Both subjects showed stereopsis, as demonstrated by means of visual evoked potentials (Janssen et al., 1999a
Recording sites. Standard extracellular recordings were made with tungsten microelectrodes (Frederick Haer Co., Bowdoinham, ME) in the rostral lower bank of the STS (targeted Horsley-Clark coordinates: 16 mm anterior and 22 mm lateral). Before surgery, an anatomical MRI was obtained. The recording positions were verified using CT scan (slice thickness 1 mm) with the guiding tube in situ (Janssen et al., 1999b
Stimuli. The stimuli were disparity-defined, concave and convex 3-D shapes. Because our previous study showed that most 3-D shape-selective TEs neurons respond selectively to convex and concave 3-D shapes (Janssen et al., 2000b
minimum luminance)/minimum luminance] = 4). No cross-talk between the monocular images was measured using a photomultiplier 475R (Brandenburg, Surrey, UK) equipped with a Hamamatsu (Hamamatsu City, Japan) R453 tube. The average vertical and horizontal diameter of the stimulus was 6.2 and 6.3°, respectively. A fixation target was superimposed on the stimulus. The fixation distance was 86 cm.
View larger version (31K):[in this window]
[in a new window]
Figure 1. Stimuli. A, Two-dimensional shapes used to derive the 3-D stimuli of the search test. B, Vertical 3-D shape. The monocular images are shown for the correlated (first row), the decorrelated (second row), the solid 3-D shape (third row), and the 3-D rim (fourth row). The icons on the right schematically illustrate the perceived 3-D structure when the images are crossed fused. In all four vertical 3-D shapes, the outer contours were identical and curved in depth along the vertical axis. The surface of the correlated vertical 3-D shape was also curved in depth, as indicated by the luminance gradient in the icon of the correlated vertical 3-D shape. The surface of the 3-D boundary stimuli was uninformative about depth, indicated by the homogeneously dark (decorrelation), gray (solid), or blank (rim) surface in the icons on the right. C, Surface stimuli. The monocular images are shown for the restricted surface (top row) and the large surface (bottom row) stimuli, together with schematic illustrations of the perceived 3-D structure (right). The disparity varied along the surface of the shape in the vertical and in the horizontal direction. Both scale bars on the left of the monocular images indicate 2°. The scale bar for the restricted surface also applies to B and D. Note that the icons on the right only represent the 3-D percept, whereas the actual 2-D shape of the stimulus can be seen in the monocular images. Thus, only the large surface was square-shaped. D, Horizontal 3-D shape. The disparity varied only along the horizontal axis on both surface and boundary.
We used three sets of stimuli: (1) 3-D shapes in which both surface and boundary were curved in depth, (2) stimuli in which only the boundary was curved in depth, and (3) stimuli that only contained a disparity variation along the surface of the shape. The magnitude of the disparity variation equaled 0.26°, and the dot size equaled 0.032° for all 3-D stimuli. In the "correlated vertical 3-D shape", the disparity varied only along the vertical axis of both surface and boundary, as in the previous experiments (Janssen et al., 1999b
Two types of stimuli contained no disparities along the boundary. In the "restricted surface stimulus", disparity varied along both the vertical and the horizontal axis in such a way as to produce zero disparity at the boundary (Fig. 1C). The Gaussian disparity gradient along the vertical axis was multiplied by a second Gaussian function that varied from 0 to 1 along the horizontal axis. The resulting disparity was maximal in the center of the shape and smoothly approached zero toward the boundary. The "large surface stimulus" was identical to the restricted surface in its central part, but extended over a 13 × 13° square area at zero disparity. Thus, the large surface differed from the restricted surface in two respects: the textured surface was two times larger than the restricted surface and the 2-D shape visible in the monocular images was removed (Fig. 1C).
The correlated horizontal 3-D shape only contained a Gaussian disparity variation along the horizontal axis within both the surface and the boundary of the shape (Fig. 1D). In the decorrelated horizontal 3-D shape, the dots were uncorrelated between the left and right eyes, again eliminating the disparity information in the surface but preserving the disparity information along the boundary. Note, however, that the amount of boundary disparity for a horizontal gradient is much smaller than for a vertical gradient and depends on the 2-D shape.
Disparity gradients along the horizontal axis result in texture density cues at each point where the disparity value changes, e.g., at the change from 0 to 0.032°, from 0.032 to 0.064°, etc. (Cobo-Lewis, 1996
Testing procedure. We searched for responsive neurons using a set of 30 stimuli (three 3-D profiles combined with ten 2-D shapes). One depth profile consisted of a concave, vertical 3-D shape, the other two were concave and convex restricted surfaces. All stimuli were presented foveally while the monkeys performed a fixation task. Responsive neurons were then tested further with 3-D stimuli derived from a single 2-D shape. Neurons were tested with concave and convex vertical 3-D shapes and restricted surfaces, combined with monocular presentations of these stimuli to verify the stereoscopic nature of the response. Next, 3-D shape-selective neurons were tested with preferred and nonpreferred 3-D shapes (either the vertical 3-D shape or the restricted surface) presented at three different positions in depth ("position-in-depth test"). The disparity difference between the near and the far positions equaled 0.39°. On the basis of the position-in-depth test, neurons were classified as either zero-order disparity selective (responsive to position-in-depth) or higher-order disparity selective, i.e., responsive to the spatial variation of disparity (Janssen et al., 2000b
The first purpose of the boundary-surface test was to determine the contribution of surface and boundary information to the response difference between concave and convex 3-D shapes. To test whether a 3-D boundary was sufficient for 3-D shape-selectivity, we presented concave and convex decorrelated vertical 3-D shapes and solid vertical 3-D shapes. The boundary-surface test also included a vertical correlated 3-D shape pair with boundary and surface in depth. To determine whether a 3-D surface is sufficient for 3-D shape-selectivity, we presented stimuli in which the disparity varied only along the surface, i.e., the restricted and the large surface stimuli. The boundary-surface test also included a concave and a convex correlated horizontal 3-D shape to determine the selectivity for disparity variations along the horizontal axis, the second purpose of the test.
Neurons that appeared to be selective for either the decorrelated or the solid shape were additionally tested with a concave and convex vertical 3-D rim, together with the preferred and nonpreferred correlated vertical, decorrelated, and solid 3-D shape ("rim test").
The stimuli in the boundary-surface test differed from the stimuli used in our previous studies with regard to dot size and disparity magnitude (Janssen et al., 2000a
To determine to what extent the 3-D surface-selectivity can also be observed for different random dot textures, we tested neurons that had responded selectively to the restricted surface in the boundary-surface test by using concave and convex surfaces with 3 different dot sizes: 0.032°, 0.065°, and 0.13° ("dot size test"). The latter stimuli were composed of white and black dots on a gray background (mean luminance, 1.08 cd/m2; dot density, 25%; Michelson contrast, 67%).
Finally, some neurons selective for the horizontal 3-D shape were tested with monocular controls and horizontal decorrelated 3-D shapes.
Data analysis. Net neural responses were computed trialwise by subtracting the number of spikes counted in a 400 msec interval immediately preceding stimulus onset from the number of spikes in a 400 msec interval starting 80 msec after stimulus onset. The significance of 3-D shape-selectivity was assessed using ANOVA (p < 0.05). The selectivity was judged not to arise from purely monocular mechanisms if the difference in response between the dichoptic presentations was at least three times larger than the difference between the sum of the responses to the two monocular presentations (Janssen et al., 1999b
In the position-in-depth test, the neurons were tested with either the correlated vertical or the restricted surface stimulus. A neuron was classified as responsive to the spatial variation of disparity if at no position in depth did the response to the nonpreferred 3-D shape significantly exceed any response to the preferred 3-D shape, as assessed by a post hoc least significant difference (LSD) test (p < 0.05; Janssen et al., 2000b
To quantify the degree of selectivity within a pair of 3-D shapes, we computed a selectivity index (SI), defined as SI = [(net response to the preferred 3-D shape
RESULTS
We recorded the responses of 196 3-D shape-selective neurons in area TEs. Of these neurons, 120 were initially tested with two pairs of 3-D shapes and monocular presentations of these shapes, and were subsequently studied using the position-in-depth test. On the basis of this test, 16 neurons were classified as responsive to zero-order disparities and were excluded from the analysis. The 104 higher-order disparity-selective neurons entered the boundary-surface test. The remaining 76 neurons were studied in the outline control test (see Materials and Methods). Testing of a subset of 42 neurons suggested that the large majority (36 of 42) of these 76 neurons were higher-order disparity selective.
Position-in-depth test
To decide between zero- and higher-order disparity selectivity, each neuron was tested with concave and convex 3-D shapes at three different positions in depth before the boundary-surface test. Because of the wide range of disparities used, this test also allows us to evaluate the possible influence of vergence eye movements. Figure 2A shows the responses of a single 3-D shape-selective neuron in the position-in-depth test. The icons above the peristimulus time histograms (PSTHs) illustrate the position in depth of the stimulus with respect to the plane of fixation (indicated by the fixation target). This neuron preserved its 3-D shape preference for all three positions tested, implying selectivity for the spatial variation of disparity contained in the stimulus, i.e., higher-order disparity selectivity (Janssen et al., 1999b
View larger version (14K):[in this window]
[in a new window]
Figure 2. Position-in-depth test. A, Example neuron. The icons above the peristimulus time histograms depict the position-in-depth of the stimulus for an observer viewing from the left (left column, near; right column, far). The horizontal scale bar (top right) indicates 0.25° disparity. Below the PSTHs, the mean position of left (red) and right eye (blue) are shown. The vertical bar on the left of the eye position traces indicates 1°, and each horizontal bar indicates the duration of stimulus presentation (600 msec). The green lines are plotted as a reference at 200 msec after stimulus onset. The large scale bar on the right indicates 72 spikes/sec. The neuron fired strongly and selectively to the convex 3-D shape at every position in depth, and the selectivity was present in the initial part of the response. B, Population PSTH of all neurons that preferred the convex 3-D shape (N = 52). Same conventions as in A. The 3-D shape-preference is preserved at every position in depth. C, Population PSTH for all neurons that preferred the convex 3-D shape and for which binocular eye movements were recorded (N = 12). The mean positions of left (red) and right eye (blue) are plotted below the histograms. Same conventions as in A. Again, the response difference between convex and concave can be observed in the initial part of the response at every position in depth and is not influenced by the small (0.1°) vergence eye movement. D, Population PSTH of all neurons that preferred the concave 3-D shape (N = 52). Same conventions as in A.
Figure 2B shows population PSTHs, normalized to the highest bin count in the PSTHs of each neuron, of the responses of the 52 of 104 neurons for which the preferred stimulus was a convex 3-D shape. The 3-D shape preference is preserved at every position in depth. We obtained binocular eye position traces for 28 higher-order disparity-selective neurons, 12 of which preferred the convex 3-D shape. The population PSTH for these 12 neurons is shown in Figure 2C. Below each histogram, the mean horizontal positions of the left and right eyes are plotted. Again, the neuronal selectivity was present in the initial part of the response and was preserved over all positions tested. As for the neuron in Figure 2A, the difference in response was largely unaffected by the change in eye position. Figure 2D shows the population PSTH for the neurons that preferred the concave 3-D shape (N = 52), which is very similar to that for the convex cells (Fig. 2B). Consistent with our previous studies (Janssen et al., 1999b
Selectivity for 3-D boundaries in the boundary-surface test
To determine whether 3-D boundaries can be sufficient for 3-D shape selectivity, we tested neurons selective for the correlated vertical 3-D shape pair with stimuli containing only boundary disparities. Two example neurons are shown in Figure 3, A and B. The neuron in Figure 3A responded selectively to the concave-convex vertical 3-D shape pair (first column), but was even more selective for the decorrelation and the solid shape pair, in which only the boundary was curved in depth (second and third column). The restricted and the large surface pairs did not evoke significant response differences (last two columns). Clearly, the mere presence of a disparity variation along the boundary of the shape was sufficient for 3-D shape selectivity in this neuron. The neuron in Figure 3B, however, showed the opposite response behavior: strong selectivity for the correlated vertical 3-D shape pair, but no significant response difference for the decorrelated or the solid shape. The apparent surface-based selectivity of this neuron was consistent with the significant response difference for the restricted surface pair.
View larger version (29K):[in this window]
[in a new window]
Figure 3. Selectivity for 3-D boundaries. A, Neuron showing selectivity for the correlated vertical, the decorrelated, and the solid shape pair but no selectivity for the 3-D surfaces. B, Neuron showing selectivity for the correlated vertical 3-D shape and for the restricted surface but no significant response differences for the decorrelated, the solid 3-D shape, or the large surface stimulus. The responses to the preferred 3-D shape (either concave or convex) are plotted in the top row (below the icons), the responses to the nonpreferred 3-D shape in the bottom row. The scale bars in A and B indicate 65 spikes/sec.
Of 104 neurons, a total of 61 showed selectivity for the correlated vertical 3-D shape pair. [The remaining 43 neurons were only selective for the restricted surface stimulus (see below).] Thirty-seven neurons (61%) were selective for either the decorrelated or the solid shape pair. Because a 3-D boundary alone was sufficient to evoke significant response differences, they were termed "boundary neurons." Figure 4A shows the normalized population PSTH for these 37 boundary neurons, for the correlated vertical 3-D shape, the decorrelated, and the solid shape. The graph illustrates the substantial selectivity for 3-D boundaries in this population of cells. The median selectivity index (SI) equaled 0.88, 0.80, and 0.70 for correlated, decorrelated, and solid shape pair, respectively (Wilcoxon matched pairs tests, NS). The slightly higher selectivity for the decorrelated 3-D shape was reflected in the larger number of neurons selective for the decor-related (29) compared with the solid shape pair (23). Furthermore, the boundary selectivity emerged rapidly after stimulus onset: response differences were already significant by 100-120 msec after stimulus onset in each of the three conditions (Fig. 4A, arrow). In Figure 4B, the population responses are plotted for those boundary neurons for which binocular eye movements were recorded (N = 12). The higher number of neurons selective for the solid (10) compared with the decorrelated shape pair (7) produced the relatively larger response difference for the solid shape pair compared with the decorrelated shape pair in this sample of neurons. Figure 4C shows the mean difference in the horizontal eye position for concave and convex 3-D shapes in the correlated vertical, decorrelated, and solid shape conditions, for the same neurons as in the second row (N = 12). As was the case in the position-in-depth test, the small vergence responses (0.1°) could not account for the observed 3-D shape selectivity.
View larger version (12K):[in this window]
[in a new window]
Figure 4. Time course of 3-D boundary selectivity. A, Normalized population PSTH for all boundary neurons (N = 37), showing the average normalized response to the preferred (red) and nonpreferred (blue) correlated vertical (full line), the decorrelated (dashed line), and the solid shape pair (dotted line). B, Normalized population PSTH for all boundary neurons for which binocular eye movements were recorded (N = 12). Same conventions as in A. C, Mean difference in horizontal position between the left and the right eye, for the correlated vertical (first row), the decorrelated (second row), and the solid shape pair (bottom row), for convex (green) and concave (black) 3-D shapes. The vertical calibration bar on the left indicates 1°. The pink vertical line is a reference positioned at 200 msec after stimulus onset.
Although boundary neurons responded selectively to at least one of the 3-D boundaries, neurons selective for the decorrelated 3-D shape frequently showed a significant effect of decorrelation on either response strength or degree of selectivity. A 2 × 2 ANOVA on the net responses to the correlated vertical 3-D shape pair and the decorrelated shape pair revealed either reduction in the net response (significant main effect) and/or reduced difference in response between the convex and concave decorrelated shapes (significant interaction) in 18 of 29 neurons (62%). Thus, even this population of boundary neurons was frequently affected by the removal of surface information.
The stimuli in the boundary-surface test differed from the stimuli used in our previous studies in both dot size and disparity magnitude (see Materials and Methods). To investigate to what extent 3-D boundary selectivity can be observed for larger dot sizes and larger disparity magnitudes, we tested 76 3-D shape-selective TEs neurons with the correlated vertical 3-D shapes used in Janssen et al. (2000a
View this table: [in this window]
[in a new window]
Table 1. Comparison between the results of the boundary-surface and the outline control test Rim test
The most reduced version of a 3-D-boundary stimulus consisted of a 3-D rim, which contained only the outer contours of the original correlated 3-D shape. In the rim test, 20 neurons showing boundary selectivity in the boundary-surface test were studied further with the 3-D rim pair, the correlated, decorrelated, and solid vertical shape pairs. Figure 5A shows the responses of a boundary neuron that is equally selective for the 3-D rim than for the other boundary stimuli (ANOVA, interaction between 3-D structure and boundary stimulus; F(3,64) = 0.56; NS). Overall, 11 of 20 neurons tested (55%) displayed a significant selectivity to the 3-D rim pair. The population PSTH of those 11 rim neurons is shown in Figure 5B. The median SI equaled 0.77 for the rim pair, which did not differ significantly from that for the correlated 3-D shape (0.83). For at least this population of rim neurons, the removal of all surface information did not result in a reduction of the 3-D shape selectivity compared with a stimulus in which both boundary and surface were curved in depth. In the remaining nine boundary neurons, however, no selectivity was present for the 3-D rim pair, indicating that even in boundary neurons the mere presence of a surface was required for 3-D shape selectivity.
View larger version (23K):[in this window]
[in a new window]
Figure 5. Rim test. A, Example neuron showing selectivity for all 3-D boundaries tested. The calibration bar on the right indicates 67 spikes/sec. B, Normalized population PSTH for all neurons selective for the 3-D rim (N = 11). Same conventions as in Figure 3.
To compare both selectivity and response strength in the population of boundary neurons for the correlated vertical, decorrelated, solid shape, and rim, we plotted the NRD for the 3-D boundary stimuli as a function of the NRD for the correlated vertical 3-D shape pair in Figure 6. The arrows indicate the respective median NRDs for the four stimuli containing boundary disparities. Most data points are located below the diagonal, but a substantial number of points lie near or even above the diagonal. The decorrelated and the solid shape evoked on average lower response rates than the correlated 3-D shape (Fig. 4A). Therefore, the NRD for the correlated vertical 3-D shape (0.76) was significantly larger than for the decorrelated (0.31), solid (0.48), and rim stimulus (0.16; Wilcoxon matched pairs test; p < 0.05). Although large differences in the degree of selectivity for the different 3-D boundaries can be observed for single neurons, the clouds of dots largely overlap (Wilcoxon matched pairs tests; NS), demonstrating the equivalence of all boundary stimuli tested at the population level. Overall, the data clearly demonstrate that at least for a subpopulation of neurons in TEs, curved boundaries in depth can be sufficient for 3-D shape selectivity.
View larger version (19K):[in this window]
[in a new window]
Figure 6. Comparison of selectivity for correlated vertical and boundary stimuli. The NRDs for the decorrelated 3-D shape (triangles), the solid shape (asterisks), and the 3-D rim (squares) are plotted as a function of the NRD for the correlated vertical 3-D shape, for all boundary neurons (N = 37). The arrows on the vertical axis indicate the median NRDs for each boundary stimulus, and the arrow on the horizontal axis indicates the median NRD for the correlated vertical 3-D shape.
Selectivity for 3-D surfaces
We wanted to determine to what extent surface disparities are sufficient for 3-D shape selectivity. In other words, can we observe response differences between concave and convex stereoscopic surfaces? Figure 7A illustrates a neuron selective for a surface curved in depth. Whereas the vertical 3-D shapes with boundaries in depth elicited only weak responses, the neuron fired strongly and selectively to the restricted surface (post hoc LSD test; p < 0.005). Note that this response difference
as for all 104 neurons tested
View larger version (23K):[in this window]
[in a new window]
Figure 7. Selectivity for 3-D surfaces. A, Example neuron showing strong selectivity for the restricted surface and a weaker selectivity for the large surface stimulus. B, Example neuron showing equally strong selectivity for the restricted and the large surface stimulus, together with a weak selectivity for the vertical correlated 3-D shape. Same conventions as in Figure 3. The scale bars on the right indicate 65 spikes/sec in A and 75 spikes/sec in B.
Overall, 73 neurons in our sample (70%) showed significant response differences for the restricted surface (median SI, 0.77). Twenty-six of the neurons selective for the restricted surface (36%) were also selective for the large surface (median SI, 0.83). Only six neurons showed significant selectivity for the large surface (median SI, 0.69) but not for the restricted surface (median SI, 0.49). Thus, the number of surface-selective neurons totaled 79. Table 2 shows the relative numbers of neurons selective for the surface and for the correlated vertical 3-D shape, as well as the relative numbers of boundary neurons for each of these cell classes. For those neurons selective for both the vertical correlated 3-D shape and the surface, but not for any of the boundary stimuli (N = 12), the presence of a curved surface was both sufficient and necessary for 3-D shape selectivity. Conversely, 13 neurons showed boundary selectivity but no selectivity for any of the 3-D surfaces. However, the restricted surface was curved along both vertical and horizontal axes, and all boundary stimuli were curved only along the vertical axis. Because these 13 neurons may not be selective for doubly curved surfaces, it cannot be concluded that the boundary in depth was necessary for 3-D shape selectivity in those neurons.
View this table: [in this window]
[in a new window]
Table 2. Relative numbers of neurons selective for the correlated vertical 3-D shape compared with the number of surface-selective neurons Figure 8A shows the normalized population PSTH for the population of surface-selective neurons. As was the case for the 3-D boundaries (Fig. 4), the selectivity for the restricted surface was substantial and emerged rapidly after stimulus onset. The large surface elicited much smaller response differences, on average, than did the restricted surface. The eye position traces for the surface stimuli were very similar to those of the vertical correlated 3-D shape (data not shown). Figure 8B plots the NRDs for the restricted and large surface, making a distinction between neurons that were selective for the large surface (circles) and neurons that were not (squares). Most data points are located below the diagonal, but a sizable proportion of the neurons (14 of 78; 18%) lies at or above the diagonal, indicating that in some cases, the monocular contours were not necessary for 3-D shape selectivity. On average, however, the response differences were significantly higher for the restricted surface (median NRD, 0.75) than for the large surface (median NRD, 0.2; Wilcoxon matched pairs test; p < 0.001).
View larger version (18K):[in this window]
[in a new window]
Figure 8. Surface selectivity of the population. A, Normalized population PSTH for all surface-selective neurons (N = 79). The average response is plotted to the preferred (black) and nonpreferred (gray) restricted surface (full line) and large surface (dotted line). B, Scatterplot of the NRD for the large surface stimulus as a function of the NRD for the restricted surface stimulus, plotted separately for neurons selective (circles) or not selective (squares) for the large surface. The arrows indicate the median NRDs for large and restricted surface.
We determined to what extent the selectivity for the restricted surface could also be observed using different dot sizes or a different dot density in 34 of 73 neurons selective for the restricted surface (dot size test). Twenty-six of those neurons (76%) were selective for at least one of the dot sizes at the 25% dot density, and 14 neurons (41%) preferred the same 3-D shape for all three dot sizes in the test (data not shown). Thus, the selectivity for a 3-D surface can be invariant for the texture pattern carrying the surface information.
Mixed selectivity
The neuron in Figure 9 displayed robust selectivity for the correlated vertical 3-D shape, as well as a large response difference for the decorrelated and the solid shape. The restricted surface, however, also evoked a significant response difference (post hoc LSD test; p < 0.001). Because both a curved boundary and a curved surface were sufficient for 3-D shape selectivity, this neuron represents an example of a mixed selectivity for 3-D boundaries and 3-D surfaces. Overall, 24 neurons in our sample (Table 2) showed a mixed selectivity for both boundary stimuli and either the restricted or the large surface.
View larger version (20K):[in this window]
[in a new window]
Figure 9. Mixed selectivity for 3-D boundaries and surfaces. Example neuron showing robust selectivity for the correlated vertical, decorrelated and solid 3-D shape, as well as for the restricted surface. Same conventions as in Figure 3. The vertical calibration bar on the right indicates 150 spikes/sec.
Selectivity for horizontal compared with vertical 3-D shapes
The second goal of the present study was to determine whether TEs neurons are selective for second-order disparities along the horizontal axis. We tested all 104 3-D shape-selective neurons with a concave and a convex horizontal 3-D shape, in which both surface and boundary were curved in depth along the horizontal axis.
Figure 10A shows four example neurons illustrating all possible combinations of vertical and horizontal selectivity. The first neuron on the left (a) is selective for vertical but not for horizontal 3-D shapes, the second (b) is selective for both directions of curvature, the third (c) was selective for horizontal but not for vertical. Finally, column d shows a neuron for which the selectivity was inverted for horizontal (concave) compared with vertical (convex). Table 3 shows the relative numbers of neurons selective for vertical and horizontal 3-D shape. Note that because the horizontal 3-D shape was not included in the search test, the numbers of neurons mentioned do not necessarily provide a reliable estimation of the actual proportions of neurons in TEs. A total of 29 neurons responded selectively to the horizontal 3-D shape pair. Of 16 neurons selective for the vertical and the horizontal 3-D shape, nine were selective in the same direction (as in Fig. 10A, column b), and seven showed the inverted selectivity (Fig. 10A, column d). The NRD for the vertical 3-D shape pair is plotted against the NRD for the horizontal 3-D shape pair in Figure 10B, for all neurons selective for either the vertical or the horizontal 3-D shape pair (N = 74). Most neurons lie either below the diagonal (a and d) or in the top left quadrant (c). Thirty neurons in our sample showed no significant response differences for either of the two directions of curvature alone (Table 3), but were selective for the restricted surface. The latter neurons seemed to require the presence of two disparity gradients in the stimulus.
View larger version (30K):[in this window]
[in a new window]
Figure 10. Selectivity for vertical and horizontal 3-D shapes of the population. A, Four example neurons showing selectivity for the vertical but not for the horizontal 3-D shape (a), for both vertical and horizontal 3-D shape (b), for horizontal but not for vertical 3-D shape (c), and for vertical but the inverted selectivity for horizontal (d). Same conventions as in Figure 3. B, Scatterplot of the NRD for the horizontal 3-D shape plotted as a function of the NRD for the vertical correlated 3-D shape, separately for the neurons selective (circles) or not selective (squares) for the horizontal 3-D shape. The letters indicate the subsets illustrated in A.
View this table: [in this window]
[in a new window]
Table 3. Relative numbers of neurons selective for the correlated vertical 3-D shape compared with the number of neurons selective for the horizontal 3-D shape We tested whether TEs neurons can signal the direction of curvature for their preferred 3-D structure, e.g., can signal whether a convex 3-D shape is vertical or horizontal. We compared the largest response of the neuron (either to the vertical or to the horizontal 3-D shape) to the response evoked by the same disparity variation along the orthogonal axis. The large majority of the neurons (53 of 74; 72%) showed significant differences between the responses to vertical and horizontal 3-D shapes (t test; p < 0.05). Note that 58 neurons were selective for one direction of curvature but not for the orthogonal direction (Table 3) and that most of these neurons differentiated between the two directions of curvature.
Figure 11 shows the normalized population PSTH for all neurons selective for the horizontal 3-D shape pair (N = 29). As for the boundary and the surface-selective neurons, the response was already selective in its initial part. The median SI for the horizontal 3-D shape in this population of neurons equaled 0.74, which is comparable to the SI for the correlated vertical 3-D shape (0.75) and for the restricted surface stimulus (0.77). The bottom graph plots the mean eye position difference for concave and convex horizontal 3-D shape for those neurons for which binocular eye movements were recorded (N = 7). A small vergence response was detected, but the neuronal selectivity was already present before the change in eye position could have any effect. Taken together, our results clearly show that the basic selectivity for horizontal 3-D shapes is present in TEs.
View larger version (13K):[in this window]
[in a new window]
Figure 11. Vertical and horizontal selectivity. A, B, Normalized population PSTHs for preferred (black) and nonpreferred (gray) horizontal 3-D shape, for all neurons selective for the horizontal 3-D shape (N = 29; A), and for those horizontal neurons for which binocular eye movements were recorded (N = 7; B). C, The mean difference in horizontal eye position for concave (black) and convex (gray) horizontal 3-D shape. The vertical gray line is a reference at 200 msec after stimulus onset. The vertical scale bar indicates 1°.
To determine whether the disparity variation along the horizontal boundary was sufficient for 3-D shape selectivity, we tested seven neurons with correlated and decorrelated horizontal 3-D shapes, as well as the monocular presentations of these stimuli. Only one cell showed a significant difference in response to the decorrelated 3-D shape pair (post hoc LSD test; p < 0.025). Moreover, the selectivity for the correlated 3-D shapes never reflected a selectivity for the monocular images per se (data not shown). Thus, the selectivity for the horizontal 3-D shape is based mainly on a selectivity for the disparity variation along the surface of the shape.
DISCUSSION
We studied the 3-D shape selectivity of TEs neurons by presenting 3-D shapes in which either the boundary or the surface were curved in depth, together with 3-D shapes in which both boundary and surface were curved in depth. The results revealed a substantial selectivity for 3-D boundaries in a subpopulation of TEs neurons. In the same recording area, neurons could also be selective for stimuli in which the disparity varied only within the surface of the shape (surface selectivity). Surface- and boundary selectivity could even be observed within single neurons (mixed selectivity). Additionally, we demonstrated that TEs neurons can also be selective for horizontal 3-D shapes and can code the direction of curvature (vertical or horizontal).
Zero-order disparity (position-in-depth) selectivity based on the boundary has been demonstrated in striate and extrastriate visual areas (Burkhalter and Van Essen, 1986
IT neurons respond selectively to 2-D shapes (Gross et al., 1972
The large surface was included in the test to determine the extent to which the monocular contours of the stimulus were necessary for 3-D shape selectivity. A sizable proportion of the neurons responded as selectively to the large surface as to the restricted surface, indicating that the monocular contours were not critical. This conclusion was further confirmed by the response behavior in the search test, in that some neurons showed similar 3-D shape-selectivity for every 2-D shape in the test. This subpopulation of neurons therefore exhibits strong 3-D shape-selectivity but weak or absent 2-D shape-selectivity. Such neurons could encode differences in 3-D structure within planar surfaces, e.g., a groove or protrusion on a wall. Most TEs neurons, however, responded less strongly to the large surface than to the restricted surface. In the search test, these neurons displayed at least some selectivity for 2-D shape, which, on average, was not significantly different from lateral TE (Janssen et al., 2000a
In most real-world objects, boundary and surface have the same 3-D structure, but some objects are only sparsely textured. The recognition of such objects could be facilitated by extracting disparity variations along their boundaries. Ramachandran et al. (1973)
Koenderink (1990)
We have shown that both boundaries and surfaces curved in depth are represented by TEs neurons, and that these neurons can signal the direction of curvature along a surface. Future research will systematically investigate the coding of 3-D surfaces in a parametric shape space.
FOOTNOTES Received July 26, 2001; revised Sept. 18, 2001; accepted Sept. 19, 2001.
This work was supported by the Fonds voor Wetenschappelijk Onderzoek (FWO) Grant G.0142.00, GOA 2000/11, and Geneeskundige Stichting Koningin Elisabeth. P.J. is a research fellow of the FWO. We thank M. Depaep, P. Kayenbergh, G. Meulemans, and G. Vanparrys for technical assistance, S. Raiguel for critical reading of this manuscript, and the Division of Radiology of the KU Leuven Medical School for help with the CT and MRI scans.
Correspondence should be addressed to Rufin Vogels, Laboratorium voor Neuro-en Psychofysiologie, Herestraat 49, B-3000 Leuven, Belgium. E-mail: Rufin.Vogels{at}med.kuleuven.ac.be.
REFERENCES
- Burkhalter A, Van Essen DC (1986) Processing of color, form and disparity information in visual areas VP and V2 of ventral extrastriate cortex in the macaque monkey. J Neurosci 6:2327-2351[Abstract].
- Cobo-Lewis AB (1996) Monocular dot-density cues in random-dot stereograms. Vision Res 36:345-350[Medline].
- Dean P (1976) Effects of inferotemporal cortex lesions on the behaviour of monkeys. Psychol Bull 83:41-71[Web of Science][Medline].
- Desimone R, Albright TD, Gross CG, Bruce CJ (1984) Stimulus-selective properties of inferior temporal neurons in the macaque. J Neurosci 4:2051-2062[Abstract].
- Felleman DJ, Van Essen DC (1987) Receptive field properties of neurons in area V3 of macaque monkey extrastriate cortex. J Neurophysiol 57:889-920
[Abstract/Free Full Text] .- Gross CG, Rocha-Miranda CE, Bender DB (1972) Visual properties of neurons in inferotemporal cortex of the macaque. J Neurophysiol 35:96-111
[Free Full Text] .- Howard IP, Rogers BJ (1995) In: Binocular vision and stereopsis. Oxford: Oxford UP.
- Janssen P, Vogels R, Orban GA (1999a) Assessment of stereopsis in rhesus monkeys using Visual Evoked Potentials. Doc Ophthalmol 95:247-255.
- Janssen P, Vogels R, Orban GA (1999b) Macaque inferior temporal neurons are selective for disparity-defined 3-D shape. Proc Natl Acad Sci USA 96:8217-8222
[Abstract/Free Full Text] .- Janssen P, Vogels R, Orban GA (2000a) Selectivity for three-dimensional shape that reveals distinct areas in macaque inferior temporal cortex. Science 288:2054-2056
[Abstract/Free Full Text] .- Janssen P, Vogels R, Orban GA (2000b) Three-dimensional shape coding in macaque inferior temporal cortex. Neuron 27:385-397[Web of Science][Medline].
- Judge SJ, Richmond BJ, Chu FC (1980) Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20:535-538[Web of Science][Medline].
- Koenderink JJ (1990) In: Solid shape. Cambridge, MA: MIT.
- Komatsu H, Ideura Y (1993) Relationships between color, shape, and pattern selectivities of neurons in the inferior temporal cortex of the monkey. J Neurosci 70:677-694.
- Komatsu H, Ideura Y, Kaji S, Yamane S (1992) Color selectivity of neurons in the inferior temporal cortex of the awake macaque monkey. J Neurosci 12:408-424[Abstract].
- Kovacs G, Vogels R, Orban GA (1995) Selectivity of macaque inferior temporal neurons for partially occluded shapes. J Neurosci 15:1984-1997[Abstract].
- Logothetis NK, Sheinberg DL (1996) Visual object recognition. Annu Rev Neurosci 19:577-621[Web of Science][Medline].
- Mayhew JEW, Frisby JP (1976) Rivalrous texture stereograms. Nature 264:53-56[Medline].
- Mayhew JEW, Frisby JP (1980) The computation of binocular edges. Perception 9:69-86[Medline].
- Missal M, Vogels R, Orban GA (1997) Responses of macaque inferior temporal neurons to overlapping shapes. Cereb Cortex 7:758-767
[Abstract/Free Full Text] .- Poggio GF, Gonzalez F, Krause F (1988) Stereoscopic mechanisms in monkey visual cortex: binocular correlation and disparity selectivity. J Neurosci 8:4531-4550[Abstract].
- Ramachandran VS, Madhusudhan Rao V, Vidyasagar TR (1973) The role of contours in stereopsis. Nature 242:412-414[Medline].
- Roy JP, Komatsu H, Wurtz RH (1992) Disparity sensitivity of neurons in monkey extrastriate area MST. J Neurosci 12:2478-2492[Abstract].
- Sary G, Vogels R, Orban GA (1993) Cue-invariant shape selectivity in inferior temporal cortex. Science 260:995-997
[Abstract/Free Full Text] .- Tanaka K (1996) Inferotemporal cortex and object vision. Annu Rev Neurosci 19:109-139[Web of Science][Medline].
- Tanaka K, Fukuda HK, Moriya Y (1991) Coding visual images of objects in the inferotemporal cortex of the macaque monkey. J Neurophysiol 66:170-189
[Abstract/Free Full Text] .- Tanaka H, Uka T, Yoshiyama K, Kato M, Fujita I (2001) Processing of shape defined by disparity in monkey inferior temporal cortex. J Neurophysiol 85:735-744
[Abstract/Free Full Text] .- Taira M, Tsutsui KI, Jiang M, Yara K, Sakata H (2000) Parietal neurons represent surface orientation from the gradient of binocular disparity. J Neurophysiol 83:3140-3146
[Abstract/Free Full Text] .- Uka T, Tanaka H, Yoshima K, Kato M, Fujita I (2000) Disparity selectivity of neurons in monkey inferior temporal cortex. J Neurophysiol 84:120-132
[Abstract/Free Full Text] .- Ungerleider LG, Mishkin M (1982) Two cortical visual systems. In: Analysis of visual behavior (Ingle DJ, Goodall MA, Mansfield RJ, eds), pp 549-586. Cambridge, MA: MIT.
- Vreven D, McKee SP, Verghese P (2001) Disparity curvature interferes with stereo acuity. Invest Ophthalmol Vis Sci 42:S939.
- Wilcox LM, Chodirker L, Bray B (2000) Stereoscopic surface interpolation. Invest Ophthalmol Vis Sci 41:S735.
Copyright © 2001 Society for Neuroscience 0270-6474/01/21239419-11$05.00/0
This article has been cited by other articles:
M. Gerbella, A. Belmalih, E. Borra, S. Rozzi, and G. Luppino
Cortical Connections of the Macaque Caudal Ventrolateral Prefrontal Areas 45A and 45B
Cereb Cortex, April 30, 2009; (2009) bhp087v1.
[Abstract] [Full Text] [PDF]
S. Georgieva, R. Peeters, H. Kolster, J. T. Todd, and G. A. Orban
The Processing of Three-Dimensional Shape from Disparity in the Human Brain
J. Neurosci., January 21, 2009; 29(3): 727 - 742.
[Abstract] [Full Text] [PDF]
S. Tanabe and B. G. Cumming
Mechanisms Underlying the Transformation of Disparity Signals from V1 to V2 in the Macaque
J. Neurosci., October 29, 2008; 28(44): 11304 - 11314.
[Abstract] [Full Text] [PDF]
E. Borra, A. Belmalih, R. Calzavara, M. Gerbella, A. Murata, S. Rozzi, and G. Luppino
Cortical Connections of the Macaque Anterior Intraparietal (AIP) Area
Cereb Cortex, May 1, 2008; 18(5): 1094 - 1111.
[Abstract] [Full Text] [PDF]
G. A. Orban
Higher Order Visual Processing in Macaque Extrastriate Cortex
Physiol Rev, January 1, 2008; 88(1): 59 - 89.
[Abstract] [Full Text] [PDF]
C. Chandrasekaran, V. Canon, J. C. Dahmen, Z. Kourtzi, and A. E. Welchman
Neural Correlates of Disparity-Defined Shape Discrimination in the Human Brain
J Neurophysiol, February 1, 2007; 97(2): 1553 - 1565.
[Abstract] [Full Text] [PDF]
S. Tanabe, T. Doi, K. Umeda, and I. Fujita
Disparity-Tuning Characteristics of Neuronal Responses to Dynamic Random-Dot Stereograms in Macaque Visual Area V4
J Neurophysiol, October 1, 2005; 94(4): 2683 - 2699.
[Abstract] [Full Text] [PDF]
J. Hegde and D. C. Van Essen
Role of Primate Visual Area V4 in the Processing of 3-D Shape Characteristics Defined by Disparity
J Neurophysiol, October 1, 2005; 94(4): 2856 - 2866.
[Abstract] [Full Text] [PDF]
K. C. Anderson and R. M. Siegel
Three-dimensional Structure-from-motion Selectivity in the Anterior Superior Temporal Polysensory Area, STPa, of the Behaving Monkey
Cereb Cortex, September 1, 2005; 15(9): 1299 - 1307.
[Abstract] [Full Text] [PDF]
Y. Liu, R. Vogels, and G. A. Orban
Convergence of Depth from Texture and Depth from Disparity in Macaque Inferior Temporal Cortex
J. Neurosci., April 14, 2004; 24(15): 3795 - 3800.
[Abstract] [Full Text] [PDF]
Z. Kourtzi, M. Erb, W. Grodd, and H. H. Bulthoff
Representation of the Perceived 3-D Object Shape in the Human Lateral Occipital Complex
Cereb Cortex, September 1, 2003; 13(9): 911 - 920.
[Abstract] [Full Text] [PDF]
J. D. Nguyenkim and G. C. DeAngelis
Disparity-Based Coding of Three-Dimensional Surface Orientation by Macaque Middle Temporal Neurons
J. Neurosci., August 6, 2003; 23(18): 7117 - 7128.
[Abstract] [Full Text] [PDF]
This Article Abstract 
Full Text (PDF) Submit an eLetter Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in Web of Science Similar articles in PubMed Alert me to new issues of the journal Download to citation manager 
Citing Articles Citing Articles via HighWire Citing Articles via Web of Science (32) Citing Articles via Google Scholar Google Scholar Articles by Janssen, P. Articles by Orban, G. A. Search for Related Content PubMed PubMed Citation Articles by Janssen, P. Articles by Orban, G. A.
|
|
|
|
 |
|