Neural Selectivity and Representation of Gloss in the Monkey Inferior Temporal Cortex

Recording sites. A, Schematic illustration showing the recording site within a lateral view of the monkey cerebral cortex (in red) and the approximate position of the recording chamber. B, An MRI image of a coronal section of the brain of monkey AQ positioned 8 mm anterior to the interaural line. A guide tube made of gold 650 μm in diameter is inserted targeting the lower bank of the STS in the right hemisphere. C, Top view of the areas of electrode penetration in the lower bank of the STS in three hemispheres are indicated by colored contours (red, monkey AQ right hemisphere; blue, AQ left; green, TV left) with stereotaxic coordinates. A thick black line indicates the lip of the STS, and a thin gray line indicates the fundus of the STS in one hemisphere (AQ right). The positions of the lip and fundus of the STS in the other two hemispheres were very similar to those shown here. The circles indicate the positions of grid holes where electrodes were penetrated, and the gray circles indicate the positions where neurons responsive to the gloss stimulus set were obtained. A colored dot indicates the position where a gloss-selective neuron was recorded.

Examples of stimuli. A, Examples of stimuli with 10 different shapes and 5 different surface reflectances rendered under default illumination (Eucalyptus Glove). B, Examples of shuffled stimuli (shape 3) with five different surface reflectances. C, Examples of stimuli (shape 3) with five different surface reflectances rendered under illumination 2 (Campus at Sunset).

Responses to gloss stimulus set. A, Responses of an example neuron (cell 1) to the gloss stimulus set. The responses are depicted as raster plots and poststimulus time histograms (PSTHs). The horizontal bars under the PSTHs indicate the stimulus presentation period. B, Response magnitude of cell 1 to each stimulus in the gloss stimulus set represented by the size of the object image. This neuron strongly responded to stimuli with sharp highlights and did not respond to stimuli with weak glossiness. C, Response magnitude of cell 1 to each stimulus in the gloss stimulus set represented as the diameter of a circle and plotted at the corresponding position in the gloss stimulus space. D, E, Responses of another neuron (cell 2) plotted using the same format as in B and C, respectively. This neuron selectively responded to shiny objects with blurred highlights due to large specular reflectance and roughness. F, G, Responses of a third neuron (cell 3) plotted using the same format as in B and C, respectively. This neuron strongly responded to matte stimuli without clear highlights and to those with small specular reflectance and large roughness.

Effects of a change in object shape and pixel shuffling on the activity of the neurons depicted in Figure 4. A, Responses of cell 1 (the same neuron depicted in Fig. 4A–C) sorted according to the rank order of its response magnitude when the optimal shape was used. The horizontal axis indicates rank order for the optimal shape (shape 3); the vertical axis indicates response magnitude (with SEM). The red line depicts the responses to the optimal shape; the blue line, those to the nonoptimal shape (shape 2); and the black line, those to the shuffled stimuli. Object images with the optimal shape are shown at the top in rank order. The inset shows the relationship between the responses to each stimulus in the gloss stimulus set for the optimal (horizontal axis) and nonoptimal (vertical axis) shapes. B, C, Responses of cells 2 (the same neuron depicted in Fig. 4D,E) and 3 (the same neuron depicted in Fig. 4F,G), respectively. The optimal and nonoptimal shapes were shapes 3 and 9 for cell 2 and shapes 8 and 4 for cell 3. The conventions are as in A.

Effects of shape change and pixel shuffling: population analysis. In the scatter plot, horizontal axis indicates correlation coefficient between the responses to the optimal and nonoptimal shapes, and vertical axis that between the responses to the optimal shape and shuffled stimuli. If a neuron did not exhibit significant response to the nonoptimal shape or shuffled stimuli, they are plotted on the horizontal or vertical axis, respectively. We defined “gloss-selective” neurons using two criteria: (1) They should be responsive to a nonoptimal shape, and there should be significant correlation between the patterns of stimulus selectivity between the optimal and nonoptimal shapes (p < 0.05). And (2) they should not show significant response to shuffled stimuli (<10 spikes/s and/or p > 0.05, t test), or the correlation for the stimulus selectivity between the optimal shape and shuffled stimuli should not be significant. The red circles represent gloss-selective neurons that satisfied these two criteria. The blue circles represent cells that exhibited significant correlation between the responses to the optimal and nonoptimal shapes, as well as between the responses to the optimal shape and shuffled stimuli. The histogram at the top depicts the distribution of the correlation coefficients between the responses to the optimal shape and shuffled stimuli. The histogram at the right depicts the distribution of correlation coefficients between the responses to the optimal and nonoptimal shapes. In the histograms, the solid bars represent cells in which both correlation coefficients were obtained, and the open bars represent cells in which only one of the correlation coefficients was obtained.

Separability index for a change in shape. A, Distribution of the separability index for a change in object shape. The horizontal axis indicates the separability index, the vertical axis the number of cells. B, Distribution of r² between the actual and predicted responses computed from only the second principal component. The filled and open bars indicate significant and nonsignificant cells, respectively, based on the permutation test. C, Raw interaction plot connecting the responses to the gloss stimulus set for the optimal shape (left) and the nonoptimal shape (right) for four representative example neurons. The left two panels are for cell 1 and cell 2 depicted in Figures 4 and 5. Separability index for each neuron is 0.92, 0.93, 0.96, and 0.98, respectively.

Distribution of the selectivity and sparseness indices among gloss-selective neurons. A, Distribution of the selectivity indices of 57 gloss-selective neurons. The horizontal axis indicates the selectivity index, and the height of each bar indicates the number of cells (left vertical axis). The rightmost bar indicates cells with a selectivity index over 1.2. The black line indicates the cumulative percentage of indices (right vertical axis). B, Distribution of the sparseness indices of the 57 gloss-selective neurons. The horizontal axis indicates the sparseness index. Other conventions are the same as in A.

Rank order of the responses to the gloss stimulus set: population average. A, Average of the responses of 57 gloss-selective neurons (Fig. 6, red circles) to stimuli with the optimal shape (red line), a nonoptimal shape (blue line), and shuffled stimuli (black line), sorted according to the rank order of the responses to the optimal shape for each neuron. B, Average of the responses of 43 neurons that showed significant correlation between the responses to the optimal shape and shuffled stimuli (Fig. 6, blue circles). Other conventions are the same as in A.

Stimulus preference of gloss-selective neurons. A, C, Responses of a gloss-selective neuron that was selectively responsive to shiny objects with clear highlights (cell 4). B, D, Responses of another gloss-selective neuron that was selectively responsive to matte objects (cell 5). These are examples of gloss-selective neurons recorded from monkey TV, while those shown in Figure 4 (cell 1 and cell 2) are from monkey AQ. Conventions are the same as in Figure 4, B and C. E, Population average of the normalized responses of 57 gloss-selective neurons to each stimulus in the gloss stimulus set. F, The numbers of gloss-selective neurons that showed a peak response to each stimulus in the gloss stimulus set.

Effects of illumination change. A, Responses of cell 1 (the neuron depicted in Fig. 4A–C) sorted according to the rank order of the response magnitudes under the default illumination (illumination 1, Eucalyptus Grove). The horizontal axis indicates the rank order of the responses, and the vertical axis indicates the response magnitude (with SEM). The red line depicts the responses under illumination 1; the blue line depicts those under different illumination (illumination 2, Campus at Sunset). Object images rendered with illumination 1 are shown at the top. The inset shows the relationship between the responses to each stimulus in the gloss stimulus set under illuminations 1 (horizontal axis) and 2 (vertical axis). B, Summary of the effects of the illumination and shape in 48 gloss-selective neurons tested under both illuminations. In the scatter plot, the horizontal axis indicates the correlation coefficient between the responses under the two different illumination conditions, and the vertical axis indicates the correlation coefficient between the responses to the optimal and nonoptimal shapes. Given our definition of gloss-selective neurons, all of these neurons showed significant correlation between their responses to the optimal and nonoptimal shapes. The red circles represent neurons that showed significant correlation between the responses elicited under the two illumination conditions. The histogram at the top depicts the distribution of the correlation coefficients between the responses under the two illumination conditions. The black bars represent neurons that exhibited significant correlation, and the gray bars, nonsignificant correlation. The histogram at the right depicts the distribution of correlation coefficients between the responses to the optimal and nonoptimal shapes.

Effect of illumination on responses among gloss-selective neurons. A, Distribution of separability indices for the change in illumination. The filled and open bars indicate significant and nonsignificant cells, respectively, based on the permutation test. B, Distribution of r² between the actual and predicted responses computed from only the second principal component. C, Average of the responses of 57 gloss-selective neurons to stimuli with the optimal shape rendered under default illumination (illumination 1, Eucalyptus Grove; red line) and another illumination (illumination 2, Campus at Sunset; blue line) sorted according to the rank order of the responses under illumination 1.