We used fMRI on neurologically intact humans to investigate whether or not there are different neural substrates for the different kinds of information that a visual surface signals (shape from texture vs material properties from texture). Participants attended to differences in the shape (flat/convex), texture and color (wood/rock), or material properties (soft/hard) of a set of circular surfaces. Attending to shape activated the contour-sensitive lateral occipital (LO) area, and attending to texture activated a region of the collateral sulcus (CoS) that overlaps with the parahippocampal place area (PPA). Interestingly, attending to material properties activated the same texture-sensitive region in the CoS. These results demonstrate the existence of different neural substrates for the different types of information that a visual surface signals. With regard to object shape, the organization of the LO area may be complex, with neurons tuned not only to the outline shape of objects, but also to their surface curvature independent of contour. Moreover, to our knowledge, this is the first study to demonstrate that processing surface texture, which occurs within the scene-sensitive PPA, is a route to accessing knowledge about an object's material properties. With this in mind, we propose that models of visual cortical organization should focus not only on the particular stimulus category to which a region maximally responds (e.g., objects, scenes), but also on the stimulus attributes that best support the processing of that category (e.g., shape, texture, material properties).
Using fMRI, we recently demonstrated that separate regions of the ventral stream are sensitive to processing shape and surface properties (Cant and Goodale, 2007; Cant et al., 2009). Specifically, the lateral occipital (LO) area, which has been implicated in object recognition (Malach et al., 1995), is sensitive to shape, whereas a region of the collateral sulcus (CoS) that overlaps with the parahippocampal place area (PPA) (Epstein and Kanwisher, 1998) is sensitive to the surface properties of objects (particularly texture). We also found a “transition zone” between the LO area and the PPA, which overlapped with the fusiform face area (FFA) (Kanwisher et al., 1997) and showed sensitivity to both shape and surface properties.
But demonstrating that separate brain regions process shape and surface properties does not necessarily mean that these regions function independently. Thus, in a series of behavioral studies we showed that the regions recruited during the perception of shape and surface properties can indeed function independently—at least when perceiving surface properties means perceiving the material from which objects are made (Cant et al., 2008). In fact, the textures on the objects used in these behavioral and fMRI studies have always been clearly linked with the material from which those objects are made (e.g., marble, brick, wood). But while surface properties are commonly associated with particular materials, they can also provide cues to an object's shape (e.g., shape-from-texture and shape-from-shading) (Ramachandran, 1988a,b; Humphrey et al., 1997; Georgieva et al., 2008). Thus, using the same behavioral paradigm but with stimuli where surface properties contributed directly to the perception of 3-D shape (flat or convex surfaces), we now found that shape and surface properties are not processed independently, and instead share common processing resources (Cant and Goodale, 2009).
Together, these neuroimaging and behavioral studies suggest that surface cues are processed by two separate networks: one where surface cues reveal an object's shape and another where surface cues signal its material properties. The present experiments were designed to test this idea. In experiment 1, participants attended to the shape of the surface (flat/convex) or to the material signaled indirectly by the texture and color of that surface (wood/rock) in a set of stimuli with exactly the same circular contour (see Fig. 1). We hypothesized that LO would show more activation when participants attended to surface shape, and that the PPA would show more activation when participants attended to surface texture (and color), which would replicate our previous findings of texture sensitivity in this region (Cant and Goodale, 2007). In experiment 2, we investigated whether or not the processing of surface cues in the PPA is directly linked to the actual material properties of objects. Participants used surface cues to attend to the shape (flat/convex) or to the compliance of the material (hard/soft) of a set of circular surfaces. We predicted that attending to surface shape would again elicit greater activation in the LO area, but attending to more direct material properties (i.e., compliance) would result in greater activation in the PPA.
Materials and Methods
Fifteen healthy participants (8 male, 7 female) took part in both experiments 1 and 2. All participants (mean age, 27.00 years; age range, 21–33 years), except one, were right-handed, and all reported normal or corrected-to-normal visual acuity, gave their informed consent to participate in the study in accordance with the Declaration of Helsinki, and had no history of neurological disorders. The participants were selected from graduate students and postdoctoral fellows studying psychology, neuroscience, or biomedical physics at the University of Western Ontario. The procedures and protocols for both experiments were approved by Review Board for Health Sciences Research Involving Human Participants for the University of Western Ontario and the Robarts Research Institute.
Face-place-object localizer stimuli
Stimuli used to localize face-, place-, and object (FPO)-sensitive areas consisted of grayscale photographs of faces, various place images (furnished rooms, buildings, city landscapes, and natural landscapes such as forests, deserts, and beaches), and both living and nonliving objects. Scrambled versions of each image were also presented to participants. All categories of objects, including scrambled images, were 250 × 250 pixels in size.
Stimuli used in both experiments 1 and 2 consisted of full-color photographs of common objects or materials that were cropped to fit within a circular contour and were then presented on a white background (Fig. 1; all images were 200 × 200 pixels in size). In experiment 1, two different surface textures/colors (rock and wood) and two different surface shapes were used (flat and convex), yielding four different stimulus types (flat rock, flat wood, convex rock, and convex wood, each of which contained 20 different exemplars for a total of 80 different stimuli used in experiment 1). In experiment 2, two different surface material properties were used (hard and soft), and two different surface shapes were used (flat and convex), again yielding four different stimulus types (flat hard, flat soft, convex hard, and convex soft, each of which contained 20 different exemplars for a total of 80 different stimuli used in experiment 2). The definitions of “hard” and “soft” materials used in experiment 2 were based on the material compliance of the objects. A hard material was one that could not be crushed in the palm of the hand or deformed with a poke of the finger (e.g., a golf ball), and, in contrast, a soft material was one that could easily be crushed or deformed (e.g., a plush soccer ball) (Fig. 1). A fixation cross was placed at the center of each object to ensure participants maintained fixation during the experimental runs. In experiment 1, shape was not crossed with texture (i.e., the same texture was not presented as both a flat and a convex surface), because we did not generate these images using computer software but simply used existing images that were freely available online. We should note, however, that there were some similarities across the flat and convex surfaces, simply because the natural variation inherent in wood and rock surfaces affords some repetition and redundancy when collecting 40 exemplars of each type of texture. Surface shape and material properties were also not crossed in experiment 2, and, in fact, performing this manipulation would have made the images look unrealistic (e.g., a convex vinyl record or a flat bowling ball). But importantly, all stimuli were used equally in the shape and texture/material tasks within each experiment (i.e., a given flat wooden surface was presented equally often in the shape and texture tasks of experiment 1).
Stimulus presentation was controlled by Superlab Pro version 2.0.4 (Cedrus Corporation). Each image was rear projected via an LCD projector (Silent Vision 6011, screen resolution of 1024 × 768, Avotec) onto a screen mounted behind the participant as he or she lay in the bore of the magnet. The participant viewed the images through a mirror, which was mounted to the head coil and was placed directly above the eyes. The distance from the participant's eyes, via the mirror, to the screen was ∼60 cm. A response pad was placed in the participant's right hand, and behavioral reaction time (RT) measures were recorded from a computer in the control room when the participant was engaged in the 1-back task of the experimental runs (described below).
The FPO localizer was used in both experiments (the runs for the FPO localizer, as well as the runs for both experiments, were all conducted during the same scanning session for each participant) and was designed to identify face, place, and object areas in each participant. A single run of the FPO localizer consisted of randomly presented blocks of intact face, place, or object stimuli (four blocks of each), interleaved with blocks of scrambled images from each category. Two separate runs were performed for each participant, one at the beginning of the session and one half-way through the session. These two runs were used to define functional regions of interest (ROIs) in both experiments (i.e., the PPA used in experiment 1 was the same as the PPA used in experiment 2). Each run had a unique order of block presentation, and the run orders were counterbalanced across participants. Each run lasted 6.44 min, starting and ending with the presentation of epochs of scrambled images. Participants were instructed to maintain central fixation while passively viewing the images. The number and timing of the images presented were consistent across the different categories of stimulus blocks. That is, each face, place, object, and scrambled block contained 32 images, each image was presented for 400 ms, and each image was followed by a 50 ms interstimulus interval, yielding 14.4 s stimulus blocks. No images were repeated within or across blocks.
Participants were shown all 20 examples of each type of stimulus (flat rock, flat wood, convex rock, and convex wood) before entering the magnet. This was done to ensure that all participants could readily distinguish between the rock and wood textures, and between the flat and convex surfaces used in experiment 1.
In each experimental run, presentations of 16 s experimental blocks were interleaved with 12 s fixation blocks (where no stimuli were presented, and participants were required to fixate the cross that remained on the center of the screen) to allow the BOLD response to return to baseline levels. Immediately after each fixation period, a 4 s instructional period was presented, wherein a cue was given to participants, informing them explicitly to attend to a particular stimulus feature in the ensuing experimental block (e.g., the word “SHAPE” or “TEXTURE” appeared centrally, instructing the participant to attend to that particular feature of the forthcoming stimuli).
To ensure that participants paid attention to the correct stimulus cue, a trial-by-trial 1-back task (adapted from Corbetta et al., 1990) was used, where participants were instructed to press a button if, in a pair of stimuli, the second image contained the same surface shape or surface texture as the first (depending of what they were instructed to attend to). In a single trial, the first object was presented for 600 ms, was followed by a briefly flashed (200 ms) blank screen, then the second image was presented (also for 600 ms), and the trial ended with 600 ms of blank screen (to provide adequate time to prepare for the next trial). Participants were instructed to respond as soon as the second image appeared. Thus, each trial in an experimental block lasted for a duration of 2 s, and there were eight trials in total (four “same” and four “different” trials), yielding 16 images presented during a 16 s long experimental block. The number of trials in an entire run was balanced so that there were approximately equal numbers of trials where 0, 1, or both stimulus features changed upon presentation of the second image. Blocks of each experimental task (i.e., attention to surface shape or surface texture/color) were randomly presented six times throughout each run, and there were a total of five unique run orders used (one order for each functional run undertaken; each run lasted 6.44 min). Presentation of all five run orders was counterbalanced across participants. Importantly, throughout all five functional runs, the visual input across the two experimental tasks was identical (save for the order of presentation across runs). The only manipulation was the instruction to attend to a particular stimulus feature, a procedure that has been shown to reliably increase the neural response of cortical regions that process the attended feature (Corbetta et al., 1990; Murray and Wojciulik, 2004).
Before beginning the experiment, participants were shown all 20 examples of each type of stimulus (flat hard, flat soft, convex hard, and convex soft). As in experiment 1, this was done to ensure that all participants could readily distinguish between the flat and convex surfaces, and between the hard and soft materials used in experiment 2. With respect to the hard and soft materials, participants were asked to name each stimulus to determine whether or not they recognized each object (if the object was not recognized, the experimenter provided the name, but it should be noted that each participant recognized at least 95% of the stimuli), and upon successful recognition (or recognition after the experimenter provided the name of the stimulus), the experimenters verified that the participants perceived the correct material compliance of each stimulus (i.e., that soft materials were indeed perceived as soft and hard materials were perceived as hard). The procedures for the functional runs in experiment 2 were identical to those of experiment 1 in all respects (save for the instructional cue “MATERIAL” being used instead of “TEXTURE”, and the order of experimental blocks within each run).
This experiment was performed with a 3.0 tesla Siemens MAGNETOM Tim Trio whole-body imaging MRI system at the Robarts Research Institute, using a Siemens radio-frequency 32-channel head coil to collect BOLD weighted images (Ogawa et al., 1992). A series of sagittal T1-weighted test images were collected for each participant to select 38 contiguous, 3.5-mm-thick functional slices of quasi-axial orientation aligned parallel to a line connecting the anterior and posterior commissures, sampling all occipital, temporal (excluding the temporal poles), and parietal cortices. Functional volumes were collected using a T2*-weighted, navigator echo-corrected, EPI pulse sequence [volume acquisition time = 2 s; 200 volumes collected/imaging run; repetition time (TR) = 2000 ms; 64 × 64 matrix size; flip angle (FA) = 90°; echo time (TE) = 30 ms; FOV = 21.1 cm; 3.3 × 3.3 × 3.5 mm voxel size]. In between the functional scans of experiments 1 and 2, T1-weighted anatomical images of the whole brain were collected with axial slice orientation (3-D MPRAGE with TI = 900 ms; TE = 2.98 ms; TR = 2300 ms; FA = 9°; 240 × 256 matrix size; 192 slices; 1.0 × 1.0 × 1.0 mm voxel size).
Data analyses were performed using the Brain Voyager QX software package (Brain Innovation). Imaging data were preprocessed by applying algorithms to conduct slice scan time correction, temporal (but not spatial) smoothing (a linear trend removal, and a temporal high-pass filter to remove frequencies in the data below 3 cycles per run), and motion correction (see below), and the resulting functional data were superimposed onto anatomical brain images that had been transformed into a common stereotaxic space using the Talairach procedure (Talairach and Tournoux, 1988). To ensure that head motion or scanner artifacts did not contaminate the functional data we collected, we viewed time course movies of each functional run from each participant before any preprocessing was conducted. We also evaluated head motion and scanner artifacts by applying the motion-correction algorithm of Brain Voyager software. Based on the agreement between the output from the time course movies and the motion-correction algorithm, we eliminated any functional runs where we observed head motion in excess of 1 mm in translation and/or 1° of rotation, and we also eliminated runs where there was evidence of paradigm-correlated motion.
Data from the FPO localizer and the experimental runs for both experiments were analyzed using a general linear model (GLM) approach, accounting for hemodynamic lag (Friston et al., 1995). Predictor variables were created for each condition in the localizer and experimental scans (FPO: faces, places, objects; experiment 1: shape and texture; experiment 2: shape and material). For both the FPO localizer and experimental scans, activated voxels were identified by means of a t test contrasting the predictors in the regression equation against a fixed baseline level of activation (scrambled images for the FPO localizer, fixation epochs for the experimental task, significance thresholds varied across individuals; see below). Using this method of analysis, we identified significantly active face (faces vs place and object images), place (places vs face and object images), and object (objects vs scrambled images) areas of cortex from the FPO localizer scans. Surface shape (activation to shape vs material), surface texture (texture vs shape), and surface material-property (material vs shape) selective regions were identified from the functional scans of experiments 1 and 2.
For our main method of analysis, we used an ROI (Saxe et al., 2006) approach, and used the FPO localizer to identify face, place, and object areas in each participant separately. Activation maps for all statistical contrasts were corrected for multiple comparisons by using the false discovery rate of Brain Voyager software, set at the level of q < 0.05 for each region in all participants (p values varied across regions and participants; see below). Significant regions of interest in each individual were defined using a combination of anatomical and functional criteria (e.g., the PPA was defined as all contiguous voxels along the collateral sulcus in parahippocampal cortex for the contrast of places vs faces and objects). Event-related averages from the experimental runs (experiment 1: the activation for surface shape and surface texture; experiment 2: the activation for surface shape and surface material properties; calculated for a single participant and averaged across all experimental runs separately in each experiment) were then extracted from each region of interest. The activation levels for each condition in both experiments were measured as the percentage BOLD signal change from baseline, which was defined as the activation in a 4 s window before the onset of the instructional cue. This 4 s window corresponded to the activity that was present in the previous fixation block. These event-related averaging time courses were then subjected to paired-sample t tests, performed separately on each hemisphere on a region-by-region basis (SPSS software package). This was done to assess whether the level of activation from one experimental condition was significantly different from that of the other (e.g., in the first experiment, whether the activation resulting from attention to surface shape differed from the activation for surface texture in the independently identified object-sensitive region from the FPO localizer). Using this method, activation from the experimental conditions was independent of the statistical test used to identify each category-sensitive region of cortex.
In addition to using the FPO localizer, we also used the group data from experiments 1 and 2 as separate localizers. The motivation for performing these additional analyses was to uncover additional shape-, texture-, and material-sensitive regions that could not have been discovered by using the FPO localizer (since the FPO localizer focused solely on category-selective object, scene, and face regions).
We conducted a random-effects analysis with the group data from experiment 1 to localize regions selective to processing surface shape (activation to shape vs texture) and surface texture (texture vs shape; regions of interest were defined using the same criteria described above). We then extracted the event-related averages corresponding to the shape and material conditions in experiment 2 (done separately for each participant) from these independently localized regions, and, as we described above, these data were then subjected to paired-samples t tests. As was the case with the FPO localizer, using this method ensured that each participant's activation in the shape and material conditions from experiment 2 was independent of the statistical tests used to identify shape- and texture-sensitive regions (which were defined using the group data from experiment 1). Similarly, we conducted a random-effects analysis with the group data from experiment 2 to independently localize shape- and material-selective regions, which were further probed with the individual participant data from experiment 1.
Behavioral data from both experiments were analyzed by importing RT measures recorded by the Superlab Pro software package into Microsoft Excel. Trials where a response was initiated <100 ms after the onset of the second image in a trial were discarded, and the RT data for the shape (experiments 1 and 2) and texture (experiment 1)/material (experiment 2) conditions for correct trials only were then compared using a paired-samples t test (SPSS software package). The number of misses and false positives were analyzed in this manner as well.
ROI analysis: shape and texture processing in face, place, and object areas
We used the FPO localizer to independently localize face-, place-, and object-sensitive cortical regions in each participant. These regions were identified by means of t tests contrasting various predictors in the regression equation for the FPO localizer. The probability threshold for significant activation varied across both cortical regions and participants, and ranged from p < 0.02 to p < 0.0003 (corrected for multiple comparisons by using the false discovery rate of Brain Voyager software set at the level of q < 0.05 for each region in every participant). Event-related time courses corresponding to activity from the task in experiment 1 were then extracted from each brain region (done separately for each participant), and the integrated area under the curve of each of these time courses was calculated (to account for hemodynamic lag, the first three data points of each waveform were not included in the calculation of the area under the curve). The resulting measures were then subjected to a region [LO, PPA, FFA, occipital face area (OFA)], by hemisphere (right, left), by condition (surface shape, surface texture) repeated-measures ANOVA, and post hoc paired-samples t tests were used to further evaluate significant interactions (at the level of p < 0.05).
Regions that were independently localized by the FPO localizer included the PPA (places vs objects and faces), area LO (objects vs scrambled images), the FFA, and the OFA (faces vs objects and places for both). We were not able to identify all of these regions bilaterally in each participant (i.e., 13 of 15 individuals exhibited activity in the left PPA and OFA, and only 10 participants demonstrated activity in the left FFA), and this posed a statistical problem due to the issue of unequal sample sizes inherent in repeated-measures designs. Thus, for the sake of clarity and completeness, we circumvented this problem by substituting the missing cells in the full ANOVA with the group mean of the participants. It should be noted as well that the same pattern of results was obtained in an analysis that was conducted in each region and hemisphere separately without filling in these empty cells. Figure 2a shows examples of the four ROIs identified in individual brains, and Figure 2b summarizes the differences in activation for the two experimental conditions in each ROI.
The overall region-by-hemisphere-by-condition interaction was significant (F(3,42) = 10.87, p < 0.001). Since the patterns of activation for the processing of surface shape and surface texture did not differ across the right and left hemispheres of the PPA (F(1,14) = 0.83, ns) and area LO (F(1,14) = 0.23, ns), the data we report below for these regions are collapsed across hemispheres (but the activation patterns and statistical comparisons for both hemispheres are displayed in Fig. 2b). The PPA showed significantly higher activation for processing surface texture (i.e., wood and rock surfaces) than it did for processing surface shape (i.e., flat and convex surfaces; t(14) = 4.24, p < 0.001). In contrast, the activation for surface shape was significantly greater than the activation for texture in area LO (t(14) = 3.62, p < 0.005). In contrast to the PPA and area LO, significantly different patterns of activation were observed across the right and left hemispheres of the FFA (F(1,14) = 10.27, p < 0.01) and the OFA (F(1,14) = 43.14, p < 0.001). Specifically, in the FFA we observed significantly greater activation for surface shape compared with surface texture in the right hemisphere (t(14) = 3.16, p < 0.01), but the activations in these conditions were not significantly different in the left hemisphere (t(14) = 0.58, ns). Finally, different patterns of activation were observed across the right and left OFA, as the right OFA showed significantly higher activation when participants attended to surface shape (t(14) = 3.38, p < 0.005), whereas the left OFA showed significantly higher activation for surface texture (t(14) = 3.57, p < 0.005). In summary, the PPA shows more sensitivity to processing surface texture compared with surface shape, area LO and the FFA show more sensitivity to processing surface shape, and the OFA shows sensitivity to processing both of these stimulus features.
To determine whether or not the patterns of activation for surface shape and surface texture were significantly different across these four cortical regions, we conducted separate region-by-condition repeated-measures ANOVAs for all possible pairs of regions (as with the analyses above, data from the PPA and area LO were collapsed across hemispheres, but data from the FFA and OFA were not). For the following analyses, a significant region-by-condition interaction for a particular pair of regions would indicate that these regions show significantly different patterns of activation for the processing of surface shape and surface texture.
The results of these analyses were clear. The patterns of activation in the texture-sensitive PPA were significantly different from those observed in the more shape-sensitive area LO, the right and left FFA, and the right OFA (LO: F(1,14) = 38.26, p < 0.001; right FFA: F(1,14) = 27.92, p < 0.001; left FFA: F(1,14) = 7.83, p < 0.05; right OFA: F(1,14) = 28.95, p < 0.001), but were not different from those observed in the left OFA (F(1,14) = 0.10, ns). The patterns of activation in the shape-sensitive area LO did not differ from those in the right FFA (F(1,14) = 1.41, ns) and right OFA (F(1,14) = 0.27, ns), but they did differ from the patterns of shape and texture processing in the left FFA (F(1,14) = 10.14, p < 0.01) and left OFA (F(1,14) = 43.17, p < 0.001). Finally, the patterns of activity in the FFA and OFA did not differ in the right hemisphere (F(1,14) = 0.55, ns) but they did differ significantly in the left hemisphere (F(1,14) = 5.48, p < 0.05). Together, these results extend upon the findings from each region individually and demonstrate that attending to either the surface shape or surface texture of stimuli elicits significantly different patterns of activation across different regions of occipitotemporal cortex. Specifically, and in agreement with our previous studies (Cant and Goodale, 2007; Cant et al., 2009), attending to surface shape activates regions that are more posterior and lateral, whereas attending to surface texture (and color) activates regions that are more medial and anterior.
ROI analysis: additional shape- and material/texture-sensitive cortical regions
We conducted a group random-effects GLM using the data from experiment 2 (averaged across all 15 participants) to independently localize shape- (shape vs material) and material-property-sensitive regions (material vs shape) that would not have been uncovered using the FPO localizer (because the FPO localizer focused solely on category-selective object, scene, and face regions). We then extracted the event-related time courses corresponding to the data from experiment 1 (shape and texture conditions) from these independently localized regions (using the same procedure as the FPO localizer described above). This procedure ensured that the statistical analyses contrasting the activations in the shape and texture conditions from experiment 1 were independent of the statistical contrasts used to define each region (which were conducted with the data from experiment 2). For illustrative purposes, the group data are displayed on a single participant's anatomical brain scan (this method of illustration, of course, does not account for the individual differences in the sulcal and gyral patterns across participants). [For a summary of the Talairach coordinates and the cluster sizes of each ROI uncovered in this analysis, see Table 1.]
As illustrated in Figure 3a, the group random-effects GLM uncovered four regions of cortex, two of which were found bilaterally, and two of which were localized unilaterally to the left hemisphere (all regions: t(14) = 3.01, p < 0.009). The contrast used to find shape-sensitive regions uncovered bilateral activity in a region of cortex that seemed to coincide with area LO, and bilateral activity along the intraparietal sulcus (IPS) in parietal cortex. The contrast used to find material-property-sensitive regions uncovered unilateral activity in a posterior region of the left CoS (pCoS), and unilateral activity in a lateral region of the left superior parietal lobule (lSPL).
Since the patterns of activation in the right and left hemispheres of area LO and the IPS were not significantly different from each other (hemisphere-by-condition interaction for area LO: F(1,14) = 0.09, ns; IPS: F(1,14) = 3.39, ns), we combined the data from both hemispheres for each of these regions, and, in combination with the data from the left pCoS and the left lSPL, conducted a four (region: LO, IPS, pCoS, and lSPL) by two (condition: shape and texture) repeated-measures ANOVA to examine the patterns of activation in each region in greater detail. This analysis revealed a significant main effect of region (F(3,42) = 21.99, p < 0.001), a nonsignificant main effect of condition (F(1,14) = 0.01, ns), and a significant region-by-condition interaction (F(3,42) = 10.65, p < 0.001). We conducted post hoc t tests to further investigate this significant interaction, and these analyses revealed that area LO showed significantly greater levels of activation when participants attended surface shape compared with surface texture (t(14) = 4.42, p < 0.001; data collapsed across both hemispheres, but the data from the left and right hemispheres individually showed the same pattern of results and are illustrated in Fig. 3b), but the exact opposite pattern of results was observed in the pCoS (t(14) = 4.61, p < 0.001) (Fig. 3b). Finally, the amount of activation in response to attending surface shape and surface texture did not differ significantly in either the IPS (t(14) = 1.32, ns; again, data collapsed across both hemispheres, but the left and right IPS showed the same null result, as illustrated in Fig. 3b) or the lSPL (t(14) = 1.02, ns).
To follow up on the significant results observed in area LO and the pCoS, we conducted the same type of pairwise region-by-region comparison that we used in the FPO localizer analysis, and found that the region-by-condition interaction between area LO and the pCoS was significant (F(1,14) = 90.38, p < 0.001), demonstrating that these regions process surface shape and surface texture in significantly different ways.
In summary, the results of these analyses parallel those observed in the FPO localizer analysis, in that sensitivity to processing surface shape was observed in more lateral regions of occipitotemporal cortex, and sensitivity to processing surface texture was observed in more medial regions (and regions in the parietal cortex showed no differential sensitivity to processing either stimulus attribute, a finding that tentatively suggests that the deployment of attention was equated across the two experimental conditions; for a further discussion of this issue, see Discussion). We should note, however, that the texture-sensitive region along the CoS discovered in this analysis is more posterior than the texture-sensitive region uncovered using the FPO localizer, which was functionally defined as the PPA (a more detailed discussion of the spatial correspondence of the regions uncovered using the different localizers is given below).
Behavioral analyses were conducted on the RT data, the number of misses, and the number of false positives from the experimental task averaged across all 15 participants. Each category of data was subjected to a paired-samples t test (α = 0.05; experimental condition: shape and texture; dependent measure: RT, number of misses, and number of false positives, respectively). In the RT data analysis, trials where the participant's response was either three SDs above or below the mean response were excluded from analysis (this was done individually for each participant), as were trials where the participant responded <100 ms after the presentation of the second stimulus in a trial (i.e., an anticipatory response). Accuracy on the experimental task ranged from 87.08 to 98.33% correct. Accuracy averaged across all 15 participants was 92.97% correct.
Participants responded significantly faster when attending to an object's surface shape (mean = 478.37, SEM = 7.28) compared with its surface texture (mean = 492.49, SEM = 5.74; t(14) = 3.72, p < 0.005), and they also missed significantly fewer responses in the surface-shape task (mean = 6.20, SEM = 0.85) compared with the surface-texture task (mean = 10.33, SEM = 1.20), t(14) = 3.52, p < 0.005). Finally, the difference between the number of false positives committed on shape (mean = 7.80, SEM = 1.60) and texture (mean = 10.80, SEM = 1.98) trials approached, but did not reach, significance (t(14) = 2.08, p = 0.057).
We believe one way to interpret these behavioral results is to posit that the differences between the shape and texture conditions arose as a result of a few confusing stimuli, where rock surfaces were confused for wood surfaces, and vice versa. To investigate this possibility, we examined the stimuli in all shape and texture trials where participants missed a response or committed a false positive. From this investigation, we tallied, for every stimulus individually, the number of times a missed response or false positive was made when each stimulus was presented as the first or the second image in a trial. We then converted each of these numbers into a proportion of the total number of missed responses and false positives for each stimulus type, as a function of when it was presented in a trial (e.g., the proportion of times the 10th exemplar of convex rock led to a false positive when it was presented as the second image in a trial). From these proportions, we flagged any stimulus that was responsible for at least 20% of the missed responses and false positives in both image-presentation positions. We then tallied common stimulus pairings that led to missed responses and false positives, and cross-referenced the most common pairings with the proportions described above (i.e., were any of the two stimuli in the most common pairings responsible for at least 20% of the missed responses or false positives committed for each stimulus type in both the shape and texture conditions?). This procedure gave us separate lists of perceptually confusing stimuli for the missed responses and false positives, and we derived our final list of confusing images by selecting the stimuli that appeared in both of these lists. We then eliminated the missed-response trials and the false positives that contained these stimuli, and reanalyzed the behavioral data. We should note that this procedure was conservative, in that it flagged only three different stimuli for exclusion. Nevertheless, the reanalysis demonstrated that the difference between the number of missed responses in the shape (mean = 5.67, SEM = 0.83) and texture (mean = 4.80, SEM = 0.80) conditions was not significant (t(14) = 1.05, ns), nor was the difference in the number of false positives committed by participants, which only approached, but did not reach, significance in the original analysis (shape: mean = 5.53, SEM = 1.35; texture: mean = 7.20, SEM = 1.73; t(14) = 1.41, ns).
ROI analysis: shape and material processing in face, place, and object areas
The statistical procedures for the FPO localizer analysis of experiment 2 were identical to those of experiment 1, with the only difference being that event-related data from experiment 2 were extracted from the same four category-selective regions identified previously (i.e., area LO, the PPA, the FFA, and the OFA) (Fig. 4a).
Similar to the results in experiment 1, the overall region-by-hemisphere-by-condition ANOVA was significant (F(3,42) = 5.56, p < 0.005). We collapsed the data across hemispheres for the PPA, the FFA, and the OFA, since the patterns of activation for shape and material properties did not differ across the left and right hemispheres in each of these regions (PPA: F(1,14) = 0.003, ns; FFA: F(1,14) = 3.08, ns; OFA: F(1,14) = 0.04, ns). In all three of these regions, we found that the activation was highest when participants attended material properties (i.e., hard or soft surfaces) compared with surface shape (PPA: t(14) = 6.23, p < 0.001; FFA: t(14) = 2.37, p < 0.05; OFA: t(14) = 2.64, p < 0.05; results and statistical comparisons for each hemisphere are presented separately in Figure 4b). In area LO, however, the patterns of activation did differ across hemispheres (F(1,14) = 19.15, p < 0.001). Specifically, the right LO exhibited greater activation for surface shape compared with material properties (replicating the shape sensitivity found in experiment 1; t(14) = 2.24, p < 0.05), but surprisingly, the processing of these two stimulus attributes did not differ in the left LO (t(14) = 0.48, ns). We found this null result surprising, as area LO is known to be highly selective to processing shape. The perception of the material properties of the stimuli used in experiment 2 (hard and soft surfaces) is likely associated with a visual analysis of surface texture and a haptic analysis of material compliance. Since there is a region in the vicinity of the left LO that is known to respond to both visual and tactile input [the LO tactile-visual area (LOtv)] (Amedi et al., 2001, 2002; Allen and Humphreys, 2009), it is plausible that partially sampling from LOtv would give rise to the null result we observed between the shape and material conditions in the left LO. This is speculation of course, and future experiments should test this prediction by directly comparing the processing of shape and material compliance in the left LO and LOtv by using independent localizers to define each region.
We conducted pairwise region-by-region ANOVAs to assess whether or not the patterns of activation across any two regions were significantly different. Interestingly, this analysis revealed a distinctly different pattern of results compared with the findings in experiment 1. Specifically, the patterns of activation for processing surface shape and material properties in the shape-sensitive right LO were significantly different from those observed in the PPA (F(1,14) = 22.13, p < 0.001), the FFA (F(1,14) = 36.67, p < 0.001), and the OFA (F(1,14) = 17.89, p < 0.001). In contrast, the comparisons between the left LO and these three regions did not reach significance (PPA: F(1,14) = 3.42, p = 0.086; FFA: F(1,14) = 2.97, ns; OFA: F(1,14) = 3.13, p = 0.099), likely because the difference in activation between the processing of shape and material properties did not reach significance in the left LO. Finally, the PPA, FFA, and OFA demonstrated more sensitivity to processing material properties compared with surface shape, a pattern of activation that was not significantly different across all three regions (all Fs < 0.33).
In summary, we present evidence that the processing of shape and the processing of material properties engages different regions of occipitotemporal cortex. Specifically, a region that is involved in the perception of shape is localized laterally in the ventral stream (i.e., the right LO), and a region that is involved in the perception of surface material properties is localized more medially and anteriorly (i.e., the CoS in the PPA). Interestingly, the left LO showed no differential sensitivity to processing either surface or material properties, and two regions that showed sensitivity to processing surface shape in experiment 1 (i.e., the right FFA and the right OFA) showed more sensitivity to processing material properties in experiment 2.
ROI analysis: additional shape- and texture/material-sensitive cortical regions
We took the same approach to uncovering additional shape- and material-property-sensitive regions in experiment 2, but this time swapped how the data from experiments 1 and 2 were used (i.e., this time we used the group data from experiment 1 to localize regions, and then extracted each participant's event-related data from experiment 2 from these independently localized regions). [See Table 2 for a summary of the Talairach coordinates and cluster sizes of each ROI uncovered in this analysis].
The contrast used to find shape-sensitive regions with the group random-effects GLM from experiment 1 uncovered three regions of cortex, two of which were found bilaterally (area LO and the IPS), and the third was localized unilaterally to the left hemisphere [a region along the ventral aspect of the CoS (vCoS); all regions: t(14) = 3.68, p < 0.003] (Fig. 5a). The contrast used to find regions more sensitive to processing surface texture uncovered one region, which was localized unilaterally to the left pCoS (t(14) = 2.95, p < 0.01).
The data from the left and right hemispheres of area LO and the IPS were not significantly different from each other (LO: F(1,14) = 4.39, ns; IPS: F(1,14) = 0.19, ns), so we collapsed the data from both hemispheres for each of these regions and conducted a four (region: LO, IPS, vCoS, pCoS) by two (condition: shape and material) repeated-measures ANOVA to explore the processing of shape and material properties in each region in greater detail. As we observed in experiment 1, the main effect of region was significant (F(3,42) = 13.82, p < 0.001), the main effect of condition was not significant (F(1,14) = 3.05, ns), and, importantly, the region-by-condition interaction was significant (F(3,42) = 12.71, p < 0.001). Post hoc t tests revealed that area LO was more sensitive to processing surface shape than material properties (t(14) = 3.07, p < 0.01; data collapsed across both hemispheres, but the data from the left and right hemispheres individually showed the same pattern of results and are illustrated in Fig. 5b). In contrast, the pCoS (t(14) = 2.81, p < 0.05) and vCoS (t(14) = 5.28, p < 0.001) were both more sensitive to processing material properties than surface shape (interestingly, the vCoS was functionally localized as a region sensitive to processing surface shape, yet the data extracted from this region demonstrated greater sensitivity to processing material properties). Finally, the processing of surface shape and material properties did not differ significantly in the IPS (t(14) = 1.13, ns; data collapsed across both hemispheres, but the left and right IPS showed the same pattern of null result, as illustrated in Fig. 5b).
Pairwise region-by-region comparisons revealed that the shape-sensitive LO showed significantly different patterns of activation compared with the pCoS and vCoS, which were more sensitive to processing material properties (LO vs pCoS: F(1,14) = 22.77, p < 0.001; LO vs vCoS: F(1,14) = 49.22, p < 0.001). The patterns of activation in these latter two material-sensitive regions were not significantly different (F(1,14) = 0.07, ns). In other words, regions that are sensitive to shape and material properties engage in significantly different types of perceptual processing on the same visual object.
In summary, the results of this analysis are in agreement with all previous analyses, insofar as regions sensitive to processing surface shape are located laterally in occipitotemporal cortex (i.e., area LO), and, like regions that are sensitive to processing surface texture, regions sensitive to processing the material properties of a visual surface are located medially in occipitotemporal cortex (i.e., the pCoS and vCoS). Similar to the results of experiment 1, a region in the parietal cortex (i.e., the IPS) showed no preference for processing surface shape over material properties, or vice versa.
The behavioral analyses conducted in experiment 2 followed the same procedures as those conducted in experiment 1. Accuracy on the experimental task ranged from 76.67 to 99.58% correct. Accuracy averaged across all 15 participants was 88.17% correct. Participants' response latencies in the surface shape (mean = 492.08, SEM = 6.76) and material-property (mean = 480.01, SEM = 9.47) conditions did not differ significantly (t(14) = 1.27, ns), but there were significantly fewer missed responses when participants attended to the surface shape of the stimuli (mean = 6.33, SEM = 1.22) compared with material properties (mean = 22.07, SEM = 2.47; t(14) = 7.79, p < 0.001), and there were also significantly fewer false positives committed in the surface-shape condition (mean = 4.53, SEM = 0.83) compared with material-property condition (mean = 17.80, SEM = 2.37; t(14) = 6.97, p < 0.001).
To further investigate the differences in the number of missed responses and false positives in the shape and material conditions, we used the same procedure for flagging potentially confusing stimuli that we used in experiment 1. The results of this reanalysis did not differ from the original analysis, as participants still missed significantly fewer responses in the shape condition (mean = 5.47, SEM = 1.13) compared with the material-property condition (mean = 21.00, SEM = 2.38; t(14) = 7.80, p < 0.001), and the same trend was observed in the number of false positives committed by participants (shape: mean = 3.87, SEM = 0.70; material properties: mean = 16.40, SEM = 2.23; t(14) = 6.40, p < 0.001).
Spatial correspondence of regions localized using the FPO and experimental localizers
In this study, we used three separate functional localizers to examine the processing of surface shape, surface texture, and surface material properties in the brain. We used the FPO localizer to examine the processing of these stimulus attributes in category-selective face-, place-, and object-sensitive cortical regions, and we used the data from experiments 1 and 2, when participants were attending to differences in surface shape, surface texture, and material properties directly, to discover additional shape- and texture/material-sensitive cortical regions that would not have been uncovered using the FPO localizer. We were encouraged to find that, in general, the results of these three localizers were in agreement with the main finding from our previous studies (Cant and Goodale, 2007; Cant et al., 2009). That is, lateral regions of occipitotemporal cortex are more sensitive to processing object shape (area LO), whereas medial regions are more sensitive to processing surface texture/material properties (the PPA and the pCoS).
To examine the spatial correspondence between the regions in the shape-sensitive lateral pathway and the texture/material-sensitive medial pathway, we overlaid the main ROIs uncovered using the FPO (i.e., area LO and the PPA) and experimental (i.e., area LO and the pCoS) localizers onto the brain of a single participant (for illustrative purposes only; all regions thresholded at p < 0.01 using separate random-effects analyses on the group data from each localizer; group data are used for the FPO localizer to facilitate the comparison with the ROIs from each experimental localizer, which were defined using group data). We chose to focus on these three ROIs specifically because the common shape-sensitive region across all three localizers was area LO, and the common texture/material-sensitive regions across all localizers resided along the CoS.
With respect to the shape-sensitive lateral pathway, there was a good spatial correspondence between area LO defined using both experimental localizers (using the contrast of shape > texture in experiment 1, and shape > material in experiment 2) and area LO defined using the FPO localizer (using the contrast of objects > scrambled images) (Fig. 6a). In contrast, we were intrigued to find that the spatial distributions of the regions within the texture/material-sensitive medial pathway were not entirely consistent across the FPO and experimental localizers. As can be seen in Figure 6b, we found slightly overlapping and adjacent texture/material-property-sensitive regions in the pCoS using the data from experiments 1 and 2 (with the contrasts of texture > shape and material > shape, respectively), but both of these regions were more posterior to the texture/material-property region found in the CoS using the FPO localizer (i.e., the PPA, uncovered with the contrast of scenes > faces and objects). [We should also note that we uncovered a region in a ventral portion of the CoS using the data from experiment 1 as a localizer (using the contrast of shape > texture). This region showed sensitivity to both shape and material properties, and it did not overlap with either of the texture/material-sensitive regions uncovered along the CoS (the pCoS from the experimental localizers and the PPA from the FPO localizer)].
Together, these results tentatively suggest that a lateral region of the ventral stream (i.e., area LO) is involved in multiple aspects of shape processing (e.g., processing outline shape and surface curvature). Moreover, these results also suggest the existence of a functional network of regions sensitive to processing both surface texture and material properties in medial areas of occipitotemporal cortex, along the CoS (i.e., the pCoS and the PPA). Importantly, these results reveal the benefits of using multiple functional localizers to examine the correspondence between category selectivity (e.g., face, place, and object perception) and the processing of different stimulus features (e.g., surface shape, surface texture, and surface material-property perception).
Our findings demonstrate that attending to the shape of a surface (flat/convex) activates LO, an object-sensitive region (Malach et al., 1995; Grill-Spector and Malach, 2001), whereas attending to the texture (and color) of the same surface activates the CoS, within the scene-sensitive PPA (Epstein and Kanwisher, 1998), as well as a more posterior region, the pCoS. These findings replicate our previous studies (Cant and Goodale, 2007; Cant et al., 2009) but also extend them by demonstrating that attending to different aspects of the same surface activates different regions of the visual system—even when there are no differences in the outline contours of the stimuli. The finding that LO responds not only to shape defined by orientation and depth discontinuities, but also to shape defined by surface cues such as texture gradients and specular highlights is in agreement with recent neuroimaging work (e.g., Georgieva et al., 2008), and shows that two aspects of shape (outline contour and surface curvature) are both processed by LO. Clearly, shape is more complicated than mere differences in 2-D contour (e.g., line drawings), and future neuroimaging studies of shape representation should directly compare the processing of traditional shape with that of surface curvature.
In experiment 2, instead of attending to indirect material cues such as texture, participants attended to material properties directly by focusing on the compliance of the surfaces (hard/soft). The same texture/color-sensitive regions in the CoS were again activated, demonstrating a link between the processing of texture and material properties. But why would the association of texture/color and material properties occur within the scene-sensitive PPA? Interestingly, a number of studies have demonstrated that surface properties such as color and texture (i.e., coarse spatial frequency) play an important role in scene processing (Gegenfurtner and Rieger, 2000; Goffaux et al., 2005; Oliva, 2005). The parahippocampal cortex has also been implicated in the processing of contextual associations (Aminoff et al., 2007; Bar et al., 2008a,b), which might explain why this region is activated by surface cues that signal an object's material properties. After all, material properties (signaled by vision) can be accessed only via stored associations between the visual appearance of an object and earlier haptic experience with that object. The materials used in this study are clearly associated with individual objects, and not scenes per se. Thus, an interesting question concerns whether or not the activation in the PPA changes as one moves from object-based to scene-based material representations. At present, the scene-based and contextual association-based accounts of the parahippocampal cortex have not been reconciled. Perhaps further research into the coding of surfaces and materials in this region will connect these two opposing ideas. But this is speculation, and future studies should examine the effects of manipulating both the surface/material properties and contextual associations of objects on multiple aspects of scene processing (e.g., perception, recognition, and categorization).
The most complete overlap between surface texture and material-property regions across both experiments was observed in the pCoS. Interestingly, Cavina-Pratesi et al. (2010a,b) have found evidence of texture sensitivity in a region of the CoS that is very similar in location to the one we report here. They suggested that this region processes 3-D aspects of texture that can be appreciated visually and haptically, as opposed to surface pattern, which they interpret as variation in color or luminance across an object's surface. While this is an intriguing possibility, our earlier work showing texture sensitivity in anterior regions of the CoS when variations in color were held constant complicates this interpretation (Cant et al., 2009). But together, it is clear that medial regions of the ventral stream are sensitive to processing the surface and material properties of objects (see also Peuskens et al., 2004).
Similar to our previous work (Cant and Goodale, 2007; Cant et al., 2009), we found that the right FFA was responsive to more than one feature (i.e., shape and material). These results are consistent with electrophysiological and human neuroimaging studies that have demonstrated sensitivity to shape (Walsh et al., 1992; Merigan, 1996), color (Zeki, 1973, McKeefry and Zeki, 1997; Hadjikhani et al., 1998; Conway et al., 2007), and texture (Merigan, 2000) in regions along the fusiform gyrus. Based on these findings, we claim that the FFA can be thought of as a “transition zone,” situated functionally and anatomically in between the prominence of shape processing laterally and the prominence of texture and material-property processing medially and anteriorly.
The face-sensitive OFA was also sensitive to processing shape and material (and texture). Together, these results suggest that multiple features contribute to face perception. This is not surprising, as numerous investigations of face processing have demonstrated the utility of using both shape- and surface-based cues (Wilson et al., 2002; Yip and Sinha, 2002; Sadr et al., 2003; Vuong et al., 2005; Russell and Sinha, 2007; Russell et al., 2007). The aspect of shape that we used was surface curvature, which is a feature that varies across individual faces and across individual facial features. Moreover, while people do not possess wooden or rocky faces (save for a statue of Sylvester Stallone), they certainly possess features that are comparatively harder (e.g., chins) or softer (e.g., lips) than others. Perhaps this is why we found more sensitivity for shape compared with texture in face-sensitive regions in experiment 1, and found sensitivity to processing material compliance in these regions in experiment 2. But this is speculative, and future studies should directly examine the contribution of shape, texture, and material properties to various aspects of face processing to better understand the functional variation observed here.
We do not believe that our neuroimaging findings can be explained by differences in the difficulty, and hence the deployment of attention, across our behavioral tasks. When we reanalyzed the behavioral data, we found that the differences in experiment 1 likely reflected the influence of a few confusing stimuli (see Results). Moreover, there were no differences in activation between conditions in either experiment in parietal areas that have been implicated in covert shifts of attention (Culham and Kanwisher, 2001). Thus, we believe that our neuroimaging results reflect feature-specific perceptual processing rather than differential deployments of attention related to unmatched task difficulty.
In both experiments, we manipulated participants' attention to different features of the same stimuli, a manipulation we deemed successful because overall performance across experiments was very good (91%). We believe this attentional switching is related to the differential patterns of activation we observed in the PPA and LO. Specifically, attending to the texture or material of a surface increased activity in the PPA, whereas attending to the shape of the same surface increased activity in LO. These findings are consistent with studies that demonstrate that attending to a particular stimulus feature increases the activation in regions that process the attended feature (Corbetta et al., 1990; Murray and Wojciulik, 2004) and establish a testable link between behavioral performance and neural activity.
While localizing regions that are sensitive to processing shape versus texture/material is an important step in understanding ventral stream cortical organization, this by itself does not reveal the underlying computations performed by these regions, nor does it reveal how these separable attributes are combined to provide a unified percept. At present, there is not a tractable solution to the “binding problem” in the field of object processing. Our own behavioral results suggest that the separable dimensions of shape and material properties may not be integrated at the level of perception (Cant et al., 2008), but whether this integration occurs in visual short-term or long-term memory is an open question.
In summary, our results suggest that there are different neural substrates in the ventral stream for the different kinds of information that a visual surface signals. Specifically, a lateral network involving LO uses surface cues (e.g., texture gradients, specular highlights) to process shape, whereas a medial network involving the CoS and PPA uses surface cues to process material properties. Moreover, the OFA and FFA process multiple visual features, and we think of these regions as transition zones from the prominence of shape processing to the prominence of texture/material processing. In light of these findings, we question the utility of labeling ventral-stream regions (e.g., PPA, FFA) based solely on the general stimulus category (object, face, or scene) that they are most selective too. To be clear, we are not questioning category selectivity as a potential organizing principle in the visual system but are noting that category-selective regions process more than just a single type of image (e.g., our stimuli were not related to scenes or faces in any obvious way, yet they still drove activation in the PPA and FFA). Thus, we suggest that models of visual cortical organization should focus not only on the stimulus category to which a region maximally responds (e.g., objects, faces, or scenes), but also on the stimulus features that best support the processing of that category (e.g., shape, texture, color, or material properties).
This research was supported by the Canadian Institutes of Health Research (M.A.G.), the Canada Research Chairs Program (M.A.G.), and a postdoctoral Fellowship from the Natural Sciences and Engineering Research Council of Canada (J.S.C.). We thank the three anonymous reviewers for their helpful comments on an earlier version of this manuscript.
- Correspondence should be address to Jonathan S. Cant, 33 Kirkland Street, William James Hall, Room 744, Cambridge, MA 02138.