Abstract
Fifteen years ago, an intriguing area was found in human visual cortex. This area (the parahippocampal place area [PPA]) was initially interpreted as responding selectively to images of places. However, subsequent studies reported that PPA also responds strongly to a much wider range of image categories, including inanimate objects, tools, spatial context, landmarks, objectively large objects, indoor scenes, and/or isolated buildings. Here, we hypothesized that PPA responds selectively to a lower-level stimulus property (rectilinear features), which are common to many of the above higher-order categories. Using a novel wavelet image filter, we first demonstrated that rectangular features are common in these diverse stimulus categories. Then we tested whether PPA is selectively activated by rectangular features in six independent fMRI experiments using progressively simplified stimuli, from complex real-world images, through 3D/2D computer-generated shapes, through simple line stimuli. We found that PPA was consistently activated by rectilinear features, compared with curved and nonrectangular features. This rectilinear preference was (1) comparable in amplitude and selectivity, relative to the preference for category (scenes vs faces), (2) independent of known biases for specific orientations and spatial frequency, and (3) not predictable from V1 activity. Two additional scene-responsive areas were sensitive to a subset of rectilinear features. Thus, rectilinear selectivity may serve as a crucial building block for category-selective responses in PPA and functionally related areas.
Introduction
Many neurons in primary visual cortex (V1) respond preferentially to specific visual features that vary along one dimension, such as orientation. By comparison, numerous higher visual cortical areas respond best to images from specific categories, which can embody an enormous range of image features. Because almost all of visual cortex ultimately derives its input from V1, this has raised a longstanding puzzle: how is the simple stimulus selectivity in V1 (e.g., for orientation, contrast) combined to ultimately process more complex stimuli at higher cortical levels?
This question is especially relevant in the parahippocampal place area (PPA), which responds strongly to images of places or scenes (Aguirre et al., 1998; Epstein and Kanwisher, 1998). Since that initial functional characterization of PPA, it has been reported that PPA also responds more strongly to isolated buildings (Aguirre et al., 1998; Levy et al., 2004), inanimate objects (Ewbank et al., 2005), tools (Chao et al., 1999), and objectively large objects (Konkle and Oliva, 2012). The PPA response also includes information about the spatial layout and objects within it (Kim and Biederman, 2011; Harel et al., 2013).
Which stimulus features, shared by the above stimuli, are “preferred” by PPA, thus contributing to such a diverse range of stimulus selectivity? Previous studies showed that PPA responds selectively to specific first-order (1d-varying) visual features, including higher spatial frequencies (Rajimehr et al., 2011; Zeidman et al., 2012) and cardinal orientations (Nasr and Tootell, 2012a). Here we hypothesized that PPA is also activated selectively by a second-order stimulus feature (rectilinear angles), which could serve as a higher-stage functional building block across multiple image categories. This rectilinear hypothesis could clarify why PPA is strongly activated by many man-made objects and tools (Chao et al., 1999), as well as scenes, especially indoor scenes (Henderson et al., 2007), that are typically “carpentered environments” (Switkes et al., 1978). In general, this rectilinear hypothesis is consistent with previous behavioral and electrophysiological studies demonstrating that lower-level features play an important role in the generation of category-selective responses (Nosofsky, 1986; Anderson, 1991, Schyns and Rodet, 1997; Sigala and Logothetis, 2002; Ohayon et al., 2012).
Although the current experiments were mainly designed to clarify processing in PPA, they might also differentiate the role of PPA from that of two additional cortical areas that have also been interpreted as scene-selective: transverse occipital sulcus (TOS) (Grill-Spector, 2003) and retrosplenial cortex (RSC) (Maguire, 2001). Compared with PPA, TOS is thought to be more involved in lower-level aspects of scene processing (Hasson et al., 2003), whereas RSC shows greater activity during spatial navigation (Epstein et al., 2007; Park and Chun, 2009) and layout encoding (Wolbers et al., 2011; Harel et al., 2013).
Materials and Methods
Statistics of natural scenes and wavelet filtering: image sets
We used 820 images from 13 different image sets, as listed below: A total of 92 grayscale images (512 × 512 pixels) were used in the scene versus nonscene contrast. The 46 scenes included both indoor and outdoor environments. Nonscene images included 23 images of faces (cropped frontal-view photographs of a single face) and 23 images of inanimate objects (cropped photographs of individual objects), both presented against a gray background. This stimulus set was used previously to localize scene-selective areas (Nasr et al., 2011; Rajimehr et al., 2011).
For the face versus scene contrast, a total of 16 grayscale images (1024 × 768 pixels) were used. Faces included 8 group photographs (i.e., multiple faces, in frontal view). Scenes were 8 photographs of indoor scenes. These scenes and faces were matched for visual field area (i.e., 2D size). This image set served as the PPA localizer in the current experiment, and in previous studies (Nasr et al., 2011; Rajimehr et al., 2011; Nasr and Tootell, 2012a, b).
For the contrast of face versus indoor scenes versus outdoor scenes, a total of 69 grayscale images (512 × 512 pixels) were used (23 images each). These images were also used previously to localize PPA (Nasr et al., 2011; Rajimehr et al., 2011).
For the contrast of tools versus real-world objects versus faces, a total of 111 images (250 × 250 pixels) were used. The tools consisted of 37 images selected from the BOSS database (Brodeur et al., 2010), each showing a single tool on a white background. The real-world objects included 37 images selected from the same database, each showing a single object on a white background. The faces consisted of 37 images selected from the Tarr Laboratory Face-Place stimulus depository (http://www.tarrlab.org). Each image contained a cropped frontal-view photograph of a single face on a spatially uniform background. All images were converted to grayscale.
For the contrast of small versus large real-world objects, a total of 400 images (200 images for each type) were obtained from a published stimulus set (Konkle and Oliva, 2012) and converted to grayscale.
For the contrast of weak versus strong spatial context, a total of 178 images (256 × 256 pixel) (89 images for each type) were obtained from a published stimulus set and converted to grayscale (Bar and Aminoff, 2003).
Wavelet filters
Gabors and wavelet banks.
Wavelet filtering was based on an algorithm described previously (Krüger et al., 1996), except as follows. Instead of using a square root function that results in wavelets with a curved shape (banana filters), we used an absolute value function, which gives the wavelets an angular configuration (Fig. 1). The resulting Gabors could be varied independently in orientation and spatial scale, and the angle between the two arms of the Gabor could be independently varied to test different angles.
Illustration of different Gabors filters used to quantify the strength of the representation of different angles. A, Variations in orientation. B, Variations in spatial scale. C, Gabors used to detect the range of test angles.
Convolution of a single Gabor with a given image yielded a value indexing the presence of the angle depicted by the Gabor (see Fig. 2). To make this analysis invariant for the size and rotation for each angle of interest (i.e., 30°, 60°, 90°, 120°, 150°, and 180°), a wavelet bank was created consisting of 4 different spatial scales (1/5, 1/9, 1/15, and 1/27 cycles per pixel) and 16 different orientations (22.5°–360° in 22.5° steps).
Preprocessing and filtering.
Before applying the filters, all images were rendered in grayscale. From each image, an edge map was then constructed by applying a Canny edge detector (Canny, 1986). Implementation of this high-frequency edge detection step is consistent with a previous finding that PPA responds selectively to higher (compared with lower) spatial frequency power (Rajimehr et al., 2011). For each specific angle and spatial scale, this edge map was convolved with Gabors of different orientations, and the output was averaged. The resulting matrix provided the orientation-invariant wavelet coefficients for the angle corresponding to that of the Gabor at different points on the edge map. For each image, this matrix was averaged over edge points, and then the mean wavelet coefficient values for each angle were normalized separately across the image set. This process was repeated for different spatial scales. Results from each image were averaged across spatial filters to provide a value that indexed the strength of representation of a given angle within that image. This index was also used for further comparisons throughout this study.
Subjects
For each experiment, human subjects were selected randomly from a pool of 26 subjects (14 females), 20–36 years of age. Table 1 shows the experiments in which each of these subjects participated. All subjects had normal or corrected-to-normal visual acuity and radiologically normal brains, without history of neuropsychological disorder. All experimental procedures conformed to National Institutes of Health guidelines and were approved by Massachusetts General Hospital protocols. Written informed consent was obtained from all subjects.
Subjects used for each experiment from a pool of 26 subjects
Stimuli and procedure
Experiment 1.
Two stimulus sets were presented in this experiment (see Fig. 3A). Each set contained 22 images, including multiple natural objects of either rectilinear or round shape. Here and throughout all experiments, images were 768 × 768 pixels, except as noted. All images were presented within a circular aperture (diameter = 20°). Here and in all experiments below, display resolution was 38.4 pixels per degree. The location of illuminant source varied randomly between images without any apparent difference between the two experimental conditions (see also Yue et al., 2013). Comparisons of root mean square (RMS) contrast between experimental conditions did not show any significant difference across conditions (F(1,43) = 0.01; p = 0.94).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 12 runs and each run included 4 blocks.
Experiment 2.
Two stimulus sets were presented in this experiment (see Fig. 3C). Each set included 16 images (537 × 537 pixels), each image showing a rectilinear or round everyday real-world object, digitally cropped from the background. The average retinal size (based on total surface area) was equivalent (diameter = 14°) for each set of objects. There was no significant difference between RMS contrasts across the experimental conditions (F(1,31) = 0.23; p = 0.63).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 12 runs, and each run included 4 blocks.
Experiment 3.
Stimuli were regular arrays of computer-generated 3D shapes (see Fig. 7A). Four shapes were tested in independent conditions: spheres, cones, cubes, and 4-sided pyramids. Each shape condition consisted of 16 images, within a square aperture (20° × 20°). All shapes were aligned along a central plane. The location of illuminant source was varied systematically in semirandom order, equivalently for all shape types. There was no significant difference between the RMS contrasts across experimental conditions (F(3,60) = 1.89; p = 0.14).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subjects participated in 15 runs, and each run included 8 blocks.
For the test of shape versus category selectivity, two stimulus sets were presented and each set contained 8 images of individual faces or scenes. Stimulus size was adjusted between faces and scenes, and all images were presented within a circular aperture (diameter = 20°). Trials were blocked according to the stimulus type (16 images/block; duration 1 s/image). Each subject participated in 12 runs, and each run included 4 blocks.
Experiment 4.
Computer-generated arrays of 2D squares were presented, either at cardinal (20 images) or oblique (20 images) orientation. Analogous arrays of circles (20 images) were also presented (see Fig. 10A). All stimuli were presented within a circular aperture (20° diameter). The size of each individual square/circle varied randomly between 0.3–3°. Each image contained 100 overlapping elements located randomly within the array. There was no significant difference between the RMS contrasts across experimental conditions (F(2,63) < 0.01; p = 0.99).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 12 scan runs, and each run contained 6 blocks.
Experiment 5A.
Stimuli were arrays of triangles (internal corners = 60°), squares (90° corners), hexagons (120° corners), dodecagons (150° corners), and circles (see Fig. 11A). The location of individual shapes within an image was selected pseudo-randomly, without overlap between shapes. According to this method, 240 stimuli per condition were generated offline. The number of shapes was equated between five experimental conditions. Each array included 40 shapes within a virtually circular region (diameter = 20°). The perimeter sum across elements and line thickness were kept constant while the side length of each element was allowed to vary across conditions. Although we found a significant difference between the RMS contrasts across experimental condition (F(4,1199) = 60.93; p < 0.01), the highest contrast belonged to the arrays of circles (circles > dodecagons > hexagons > triangles > squares), which was opposite to the direction in our fMRI results in PPA and TOS (see Fig. 11).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 12 runs. Each run contained 16 blocks.
Experiment 5B.
As a control manipulation in Experiment 5A, an additional two experimental conditions equated the number of angles for triangles and squares, resulting in arrays of 80 triangles and 60 squares. Because the contrast of squares versus circles was the main contrast of interest across experiments, we also generated an independent array of circles corresponding to the square array, in which the number of circles was equal to the number of squares. As in Experiment 5A, the location of individual shapes within an image was selected pseudo-randomly without any overlap between shapes, and 240 stimuli per condition were generated offline. Although the perimeter sum across elements and line thickness were equated between conditions and also between Experiments 5A and 5B, the side length of individual shapes was necessarily smaller in Experiment 5B compared with Experiment 5A. Here again, we found a significant difference between the RMS contrasts across experimental condition (F(2,719) = 63.12; p < 0.01), but again, the highest contrast belonged to the arrays of circles (circles > triangles > squares).
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 12 runs. Each run contained 16 blocks.
Experiment 6.
Stimuli (672 total images; 96 images/condition) were arrays of angles which varied systematically between 30° through 180°, in 30° steps (see Fig. 14A). Each angle subtended 1.5 ° × 1.5°. Each array contained 120 nonoverlapping angles, distributed otherwise-randomly across the display. Otherwise-identical arrays of semicircular arcs were presented as a control. Angles/semicircles were presented against a spatially uniform background within a virtual circular limit (diameter = 20°). Angle orientation was varied semirandomly in each trial to minimize possible biases to cardinal orientations in PPA (Nasr and Tootell, 2012a). Line thickness and total line length were equal across all conditions. We found a significant difference between the RMS contrasts across experimental conditions (F(6,671) = 199.28; p < 0.01), but the highest contrast belonged to the arrays of 120°, and the pattern of RMS contrast variation (i.e., 120° > 150° > 90° > 60° > 30° > 180° > half-circles) differed from the PPA response pattern.
In each scan session, trials were blocked according to the stimulus shape (16 images/block; duration 1 s/image). Each subject participated in 8 runs with 14 blocks/run.
Localizers.
To localize scene-selective areas, subjects viewed an independent set of stimuli contrasting images of real-world faces versus scenes. In independent scans, a foveally centered checkerboard disk (radius = 1.5° or 58 pixels) and a peripheral annulus (5°-10° radius) were presented to localize corresponding “foveal” and “peripheral” retinotopic subdivisions of V1. Further details of the stimuli and analysis were published previously (Nasr et al., 2011).
In Experiment 1, we also localized PPA based on another independent set of stimuli. During those localizers, subjects viewed images of scenes (22 images other than those used in the other localizer and Experiment 3) and real-world objects (44 images other than those used in the main test). Stimulus size was equated between scenes and objects (768 × 768 pixels), and all images were presented within a circular aperture (diameter = 20°). Trials were blocked according to the stimulus type (16 images/block; duration 1 s/image). Each subject participated in 12 runs of 4 blocks/run.
In an additional analysis (see Fig. 5), we localized PPA according to the group-constrained subject specific method proposed earlier (Fedorenko et al., 2010; Julian et al., 2012). Briefly, in this method, scene-selective areas were previously localized in 30 subjects (none of whom participated in our tests) using the activity contrast evoked by scene versus objects in movie clips (threshold p < 10−4). Next, a probabilistic map was generated from that data, by overlaying the results from individual subjects in common stereotaxic (MNI) space. This overlaid map was then smoothed with a Gaussian filter (FWHM = 6 mm) and thresholded to include only those voxels that had at least 10% overlap across subjects (for further details, see Julian et al., 2012). The resulting map was then transformed from MNI space to the group-averaged brain using Freesurfer (Fischl et al., 1999; Fischl, 2012).
Spatial frequency measurement.
As shown in Table 2, all stimuli were tested for a high (>5 cycles/deg) spatial frequency bias, to avoid possible confounds resulting from stronger high-frequency components (Rajimehr et al., 2011). Original images used in Experiments 1 and 6 did not show such a bias, so those stimuli were not spatially filtered. In Experiment 4, all images were spatially filtered to equalize power at high and low frequencies. In Experiments 2, 3, and 5, we found a significant difference between high spatial frequency power across conditions (Table 2), but because it was opposite to the expected PPA bias (based on the rectilinearity hypothesis), those images were not filtered.
High spatial frequency power of stimuli used in Experiments 1–6
Experimental stimuli and task
Stimuli were presented via LCD projector (Sharp XG-P25, 1024 × 768 pixels, 60 Hz refresh rate) onto a rear-projection screen. MATLAB, version 7.8 (MathWorks) and Psychophysics Toolbox (Brainard, 1997; Pelli, 19974) were used to control stimulus presentation.
Each run began and ended with an additional fixation-only block (16 s). All images were centered on the display screen against a spatially uniform gray background. In all experiments, subjects were instructed to maintain fixation on a very small (0.1°) central red square, during performance of a dummy attention task to stabilize attention on a task irrelevant to the sensory variables of main interest. Dummy task accuracy converged on 75% using a staircase method (Yue et al., 2011).
Imaging and data analysis
All subjects were scanned in a horizontal 3T scanner (Siemens Tim Trio). Gradient echo EPI sequences were used to acquire functional images (TR 2000 ms, TE 30 ms, flip angle 90°, 3.0 mm isotropic voxels, and 33 axial slices). In the fMRI scans, the field of view included the whole brain for all subjects. A 3D MP-RAGE sequence (1.0 mm isotropic) was also used to obtain high-resolution anatomical imaging from the same subjects. Functional and anatomical data were preprocessed and analyzed using FreeSurfer and FS-FAST (version 5.3; http://surfer.nmr.mgh.harvard.edu/) (Fischl, 2012).
For each subject, we reconstructed the cortical surface based on the high-resolution anatomical data (Fischl et al., 1999). All functional images were corrected for motion artifact and then spatially smoothed using a 3D Gaussian kernel (2.5 mm HWHM), and normalized across scans. To estimate the intensity of the hemodynamic response, a model based on a γ function was fit to the fMRI signal, and then the average signal intensity maps were calculated for each condition (Friston et al., 1999). Voxelwise statistical tests were conducted by computing contrasts based on a univariate general linear model. Finally, the significance levels were projected onto the inflated/flattened cortex after a rigid coregistration of functional and anatomical volumes. To generate group-averaged maps, functional maps were spatially normalized across sessions and across subjects using Freesurfer. Next, activity within each individual's brain was spatially transformed onto the averaged human brain using a spherical transformation (Fischl et al., 1999) and then averaged using both fixed and random effects models (Friston et al., 1999).
ROI analysis
For each subject, we defined ROIs for scene-selective areas PPA, TOS, and RSC, and retinotopic borders, based on independent localizing stimuli (see above) at a threshold level of p < 10−4. The cortical map of “foveal” and “peripheral” retinotopic eccentricity was based on additional scans using radial checkerboard patterns. Within different blocks, a foveal disk (r = 1.5°) and a peripheral annulus (5°-10° radius) were presented. The contrast between these “foveal” and “peripheral” blocks was used to localize the representation of the peripheral stimulus border (Nasr et al., 2011, 2012a).
In all analyses, fMRI activity for each condition was measured relative to the activity during presentation of a uniform gray stimulus (baseline). To test the effect of independent factors, we applied a repeated-measures ANOVA, with Greenhouse–Geisser correction whenever the sphericity assumption was violated. Subsequent comparisons between individual conditions were based on paired-sampled t tests.
To compare selectivity for category versus rectilinearity, we applied a test of ANOVA (rather than repeated-measures ANOVA) to the measured selective activity because 4 (of 12) subjects were not common between the two experiments. Because group-averaged activity maps showed that all effects were generated bilaterally (without any apparent difference between left and right hemispheres), activity from both hemispheres was averaged in all ROI analyses to strengthen the power of the statistical tests.
Center of activity test
To test for a shift in location of the center of activity across Experiments 1–6 (see Fig. 16), we defined the center of activity for each individual subject. This search for the center of activity extended throughout the fusiform gyrus and collateral sulcus, within which PPA is presumably located (Nasr et al., 2011). To reduce the intersubject variability in the location of PPA (and its influence on our results), the location of activity centers for each individual subject was transformed and mapped on a group-averaged brain, which was reconstructed based on a large and independent set of data (Freesurfer fsaverage) (Fischl et al., 1999). These locations were defined and tested in RAS coordinates (right vs left, anterior vs posterior, superior vs inferior). Independently, we tested whether the activity center shifted significantly in either anterior versus posterior or left versus right directions. Because different subjects participated in different tests, here we used a test of ANOVA rather than repeated-measures ANOVA.
Results
The hypothesis tested here was that rectilinear image features selectively activate the cortical area PPA plus perhaps the functionally similar area TOS, and to a lesser extent, RSC. The rectilinear hypothesis implies specific corollaries, which we also tested here. First, a rectilinear bias should increase responses in PPA to a wide range of rectangular stimuli, including (but not limited to) real-world images. Second, rectilinear shapes (either real-world or artificial) should activate PPA more strongly than round shapes, which lack both angles and straight lines. Third, isolated right angles should activate PPA more strongly compared with otherwise-identical nonright angles. Fourth, rectangles should activate PPA more than nonrectilinear polygons (e.g., triangles). Fifth, PPA should respond preferentially to rectilinear features at any orientation. Sixth, the response selectivity for rectilinear shapes may be comparable with that for visual category (although the direct comparison is complicated and stimulus-dependent). Seventh, if the rectilinear bias reflects initial cortical processing (building blocks), it may preferentially activate posterior (compared with anterior) portions of PPA. Our six independent fMRI experiments (plus additional preliminary experiments, not described here) confirmed all of these predictions.
Measurements of stimulus rectilinearity
Scenes versus nonscenes.
As one index of image rectilinearity, we first developed a wavelet filter and a corresponding 90° Gabor (Gabor, 1946; Krüger et al., 1996). Convolving this wavelet filter with each individual image provided a value indexing the presence of right angles independent of the size and orientation of contours in each image (see Materials and Methods). This “rectilinearity index” was first measured for two different image sets used previously (Nasr et al., 2011; Rajimehr et al., 2011; Nasr and Tootell, 2012a, b) to localize PPA, based on either scenes versus nonscenes (i.e., faces and nonface objects) or scene versus faces. Consistent with our hypothesis, we found that the rectilinearity index was significantly larger for scenes compared with nonscenes in general (Fig. 2A; F(1,92) = 9.20, p < 0.01), and to faces in particular (Fig. 2B; F(1,16) = 10.36, p < 10−4). Images in the second set were larger (in pixels) compared with images in the first set (see Materials and Methods), resulting in a generally larger rectilinearity index for the second image set (Fig. 2B) compared with the first one (Fig. 2A). However, such differences did not affect the main conclusion that the rectilinearity index was higher in scenes compared with nonscenes, including faces.
The value indexing the strength of representation of right angles (a defining feature of rectilinear shapes) was measured in a subset of stimulus sets used in previous studies to localize PPA and other scene-selective areas, based on (A) scenes versus nonscenes (Nasr et al., 2011) and (B) scenes versus faces (Nasr et al., 2011; Nasr and Tootell, 2012a, b). The same measurement was applied to stimuli from additional studies, which evoked higher fMRI activity in PPA, based on a preference for (C) indoor scenes versus outdoor scenes versus faces (Nasr et al., 2011), (D) tools versus other nonface objects versus faces (Chao et al., 1999), and (E) objectively large versus small objects (Konkle and Oliva, 2012). In all these results, stimuli that were reported to evoke a stronger response in PPA also showed a correspondingly higher rectilinearity index. F, Conversely, a stimulus comparison that activated only the anterior tip (but not most) of PPA did not show any significant difference between the index values (Bar and Aminoff, 2003). Error bars indicate 1 SE.
Next, we tested whether nonright angles could help differentiate scenes from isolated objects (Table 3). In these results, the above index was used to quantify the strength of 30° and 60° (i.e., acute) angles in our test images. This index showed no significant differences between scenes and nonscenes (F(1,92) < 0.24, p > 0.62). Similarly, the indexes of 120° and 150° (obtuse) angles were not significantly different from each other (F(1,92) = 0.03, p = 0.87, respectively), or higher for nonscenes rather than scenes (F(1,92) = 8.99, p < 0.01).
Discrimination based on value indexing the strength of the representation of different anglesa
An “angle” of 180° (i.e., a straight line) is a special case because it has no corner. Interestingly, the index for 180° angles tended to be larger for scenes compared with nonscenes. However, at this level of signal averaging, this difference was not quite significant (F(1,92) = 3.39, p = 0.07; but see below). Overall, only angles of 90° (and perhaps 180°) (i.e., the rectilinear angle(s)) consistently distinguished scenes from nonscenes in these measurements.
Additional stimulus categories
In addition to scenes, PPA reportedly responds selectively to images of: (1) tools and/or man-made objects vs faces (Chao et al., 1999); (2) objectively large versus small objects (Konkle and Oliva, 2012), and/or (3) indoor versus outdoor scenes (Henderson et al., 2007). Here we measured the rectilinearity index for each of these stimulus categories, based on either the original stimuli (when published) or analogous stimuli (see Materials and Methods). In all three comparisons, we found that rectilinear features were significantly more likely in those stimulus categories that have been reported to preferentially activate PPA (p < 0.01; Fig. 2C–E). Conversely, stimuli used in one additional study (Bar and Aminoff, 2003) showed neither a higher rectilinearity index (Fig. 2F; F(1,178) = 0.04, p = 0.85) nor significant fMRI activity in most of PPA (i.e., activity was limited to the anterior tip of PPA, and further anteriorly). Thus, this image analysis supported the rectilinear hypothesis, without ruling out possible additional influences (see Discussion).
Our results showed that 180° angles could discriminate between faces versus tools, and between indoor versus outdoor scenes, but not between objectively small versus large objects. Other angles showed significantly less discriminability (Table 3). Thus, sensitivity to straight lines may improve category discrimination, but that information was less diagnostic for proposed PPA category selectivity, compared with right angles.
In this limited meta-analysis, stimulus conditions within a study were equated for size and contrast, but these parameters varied to a great extent between studies (Materials and Methods). Thus, the within-study results were not confounded by variations in size and contrast, but the between-study variation was reflected in a large difference in the rectilinearity index across studies.
Experiment 1: real-world images, multiple objects
Below we describe six independent fMRI experiments designed to test the effect of images with rectilinear features, relative to control images. In our first fMRI experiment, we tested whether the rectilinear bias would be evident in the response to real-world images. In Experiment 1 (n = 15), stimuli included real-world images of multiple everyday inanimate objects of either round versus rectilinear shape (e.g., Fig. 3A). Stimuli were equated for aperture size and RMS contrast (see Materials and Methods). Application of a one-factor ANOVA to the rectilinearity index confirmed that the qualitatively chosen rectilinear versus round stimuli were also quantitatively differentiated by this index (F(1,44) = 24.34, p < 10−4).
Stimuli and results in Experiments 1 and 2. Stimuli were images of “rectilinear” or “round” multiple (A) or single (C) real-world objects, in Experiments 1 and 2, respectively. B, D, Corresponding group-averaged maps, based on random effect analysis, demonstrating that all three scene-selective areas (PPA, TOS, and RSC) responded preferentially to the rectilinear objects. No masking was used in any of the analyses in this study. In the group activity maps, the borders of PPA, TOS, and RSC are indicated using solid, dashed, and dotted lines, respectively, based on an independent set of localizing scans comparing faces versus scenes in the same group of subjects (e.g., Fig. 5). This stimulus contrast did not produce significant selectivity for rectilinear objects within visually stimulated V1 (or other early retinotopic areas), as defined based on retinotopic mapping in the same group of subjects (see Materials and Methods). This V1 border is indicated with dashed white lines (asterisk = foveal representation). Activity is shown in both the inflated (leftmost and rightmost) and flattened (middle) cortical surface formats. Error bars indicate 1 SE. R, Right hemisphere.
The resultant group activity map (Fig. 3B) showed that PPA (plus TOS and RSC) responded preferentially to rectilinear objects, compared with round objects. This result was also evident in the great majority of our individual subjects (Fig. 4A). ROI analysis confirmed this effect in PPA (paired t test; t(14) = 6.10, p < 10−4), TOS (t(14) = 3.59, p < 0.01), and RSC (t(14) = 2.15, p = 0.05). Area V1 showed a quite different response, actually responding significantly less to images of rectilinear compared with round objects (t(14) = −10.92, p < 10−7).
Localization of the rectilinearity-related activity evoked in response to the contrast of rectilinear versus round multiple real-world objects, for all 15 individual subjects that participated in Experiment 1. A, Ventral (and slightly medial) view of the averaged brain. The yellow rectangle represents the brain region magnified in B. B, Activity resulting from rectilinear versus round stimuli, and the borders of PPA based on the scenes versus faces localizer, for each individual subject (black lines). To facilitate the comparison between subjects, each individual subject's activity was overlaid on the averaged brain. In all panels, the anterior versus posterior axis is oriented upward versus downward, respectively.
We also compared the location of this rectilinear-selective activity relative with the borders of PPA, as defined using an independent set of scenes versus objects rather than scenes versus faces. In all cases, these localizing images were independent of the images used in the main experiment. Consistent with our previous findings (Nasr et al., 2011), we found that the location of PPA remained stable across these different localizers. Furthermore, the rectilinear-selective activity was also located within PPA (Fig. 5). This colocalization of PPA remained closely overlapping even when PPA was localized based on a group-constrained subject specific method from a different laboratory (Julian et al., 2012), in which the PPA localizer was based on quite different (e.g., moving rather than stationary) stimuli.
Localization of the group-averaged rectilinearity-related activity in Experiment 1, relative to the border of PPA defined using the contrast of either scenes versus faces, or scenes versus objects, based on either fixed or random effect analyses. Additional borders of PPA (black lines) were defined based on an independent study (and a nonoverlapping group of subjects and a different scanning site, compared with that used in Experiment 1) using the group-constrained subject specific method (Julian et al., 2012) (see Materials and Methods). The location and surface area of PPA were similar across these conditions. The highest rectilinearity-related activity was located within all four borders of PPA as defined in the current study, and mostly within PPA based on the independent study.
Experiment 2: real-world images, single objects
In Experiment 2 (n = 11), each stimulus image included a single rectilinear or round object that was digitally extracted from surrounding scenes (Fig. 3C). Stimuli were adjusted for size, RMS contrast, and spatial frequency (see Materials and Methods). Again, the rectilinearity index was significantly higher for the rectilinear rather than the round objects (F(1,31) = 9.99, p < 0.01).
As in Experiment 1, the resultant group-averaged activity map showed that PPA and TOS responded preferentially to the rectilinear objects compared with the round ones (Fig. 3D). Again, the rectilinear selectivity was evident in almost all of the individual hemispheres tested (Fig. 6A). ROI analysis confirmed this effect in PPA (t(10) = 6.79, p < 10−4) and TOS (t(10) = 3.59, p < 10−6). In response to these stimuli, area RSC did not show significant differential activity (see Discussion). In contrast to PPA and TOS, the V1 response level did not vary significantly between round and rectilinear stimuli (t(10) = 0.66, p = 0.52).
Localization of the rectilinearity-related activity evoked in response to images of single real-world rectilinear versus round objects, relative to the borders of PPA (defined based on an independent scenes vs faces contrast) for all 11 individual subjects that participated in Experiment 2. Other details are as in Figure 4.
Experiment 3: computer-generated 3D shapes
To further confirm and characterize this rectilinear bias, we next tested whether it would be evoked in response to computer-generated 3D shape arrays, including cubes versus spheres (Fig. 7A) (n = 12). Here and in all experiments below, stimuli were arranged in an array to increase the aggregate fMRI responses. As a control test of intermediate shapes, we also presented cones and four-sided pyramids (i.e., two types of right conic solid, with circular or square bases, respectively) (Fig. 7; see Materials and Methods). The rectilinear index successfully differentiated these four shape categories (cubes (mean ± SD of the rectilinearity index; 0.78 ± 0.04) > pyramids (0.61 ± 0.10) > spheres (0.04 ± 0.00) > cones (0.01 ± 0.00); F(3,64) = 908.19, p < 10−49) (F(3,64) = 908.19, p < 10−49).
Stimuli and results in Experiment 3. A, Stimuli included arrays of cubes and pyramids (i.e., “rectilinear” shapes; top) compared with arrays of cones and spheres (i.e., “rounded” shapes; bottom). The cones were oriented so that they showed only curved edges and surfaces, excepting the apical singularity. Thus, from this viewpoint, the cones were considered “round” in terms of 2D contours. The four-sided pyramids were considered “rectilinear” because they are essentially cubes seen from an atypical viewpoint. In these stimulus examples, the source location of the virtual illuminant was varied to illustrate the experimental range in shading. However, in the actual experiment, the illuminant source was varied systematically in semirandom order, equivalently for all shape types. B, Group-averaged activity map showed a significant preference for the arrays of rectilinear objects, relative to the round ones, largely limited to PPA and TOS. In this experiment, dorsal RSC also responded selectively to the rectilinear objects. As in Experiment 1, a significant rectilinear activity bias was not found within V1. C, Results of ROI analysis within these three scene-selective areas. Other details are similar to those in Figure 3.
Consistent with our hypothesis, we found that the most-rectilinear shapes (cubes and pyramids) activated all three areas (PPA, TOS, and RSC) more strongly, compared with the roundest shapes (i.e., spheres and cones). This rectilinear bias was robust in both the group-averaged maps (Fig. 7B), and in nearly all (∼22 of 24) of the subjects/hemispheres (Fig. 8).
Localization of the rectilinearity-related activity evoked in response to computer-generated rectilinear shapes (i.e., cubes and pyramids) versus round shapes (spheres and cones), relative to the borders of PPA (defined based on scenes vs faces contrast), for all 12 individual subjects that participated in Experiment 3. Other details are as in Figure 4.
Figure 7C shows the results of the corresponding ROI analysis. Application of a one-factor repeated-measures ANOVA(cubes vs pyramids vs cones vs spheres) to PPA activity showed a general effect of stimulus shape (F(3,33) = 35.80, p < 10−9), consistent with the rectilinear hypothesis. Consistent with the rectilinear filtering results, the fMRI difference between responses to the cubes versus spheres was larger than that between the intermediate shapes (pyramids vs cones) (t(11) = 2.84, p = 0.01).
In TOS, responses were also influenced by stimulus shape (F(3,33) = 53.06, p < 10−11), but the responses to cubes versus spheres did not differ significantly compared with pyramids versus cones (t(11) = 0.59, p = 0.56). As in PPA, RSC showed a significant effect of stimulus shape (F(3,33) = 6.92, p < 10−3) and a significant difference between cubes versus spheres compared with pyramids versus cones (t(11) = 2.11, p = 0.05). V1 activity was also influenced by the stimulus shapes (F(3,33) = 10.27, p < 10−4), but again quite differently compared with PPA/TOS: V1 responses were lower to rectilinear rather than round stimuli (t(11) = −6.07, p < 10−3).
Selectivity for category versus rectilinearity
These data raise a question about the underlying neural representations: does PPA (and perhaps TOS) respond only secondarily to rectilinear shapes, based on a fundamental primary response to a given category (e.g., scenes)? Alternatively, does PPA respond primarily to rectilinear shapes, and only secondarily to rectilinear features in scenes and other image categories?
Ultimately, such a comparison is challenging. For one thing, category differences cannot be easily isolated from shape differences, to the extent that (for instance) buildings or tools are intrinsically rectilinear. Moreover, it is difficult to quantify units of “place-ness” or “in/animacy,” etc. Even if such units can be defined, the shape versus category comparison may remain an apples versus oranges contrast, without a common scale.
Despite these caveats, it was of interest to compare these competing hypotheses by comparing PPA/TOS responses in our data with (1) spheres versus cubes (Experiment 3), relative to (2) an independent set of scenes (indoor and outdoor) versus individual faces, distinct from those used as a localizer (see Materials and Methods).
Ten subjects were used in this experiment; 8 subjects also participated in Experiment 3. We chose scenes versus faces as the category contrast because that contrast generates one of the largest activity differences reported in PPA and TOS (Epstein and Kanwisher, 1998; Schwarzlose et al., 2008; Mur et al., 2012; Vul et al., 2012), and that category-based contrast is often used as a conventional localizer for these areas (Aguirre et al., 1998; Epstein and Kanwisher, 1998; Rajimehr et al., 2011; Nasr et al., 2011; Nasr and Tootell, 2012a, b).
Figure 9 shows PPA and TOS responses relative to a uniform gray baseline stimulus. Consistent with the results from Experiment 3, we found that these computer-generated cubes consistently evoked stronger activity than spheres in both PPA (t(11) = 8.56, p < 10−5) and TOS (t(8) = 8.00; p < 10−4). Consistent with the nature of the localizer, both areas also showed selectivity for this independent set of scenes versus faces (p < 10−3). In PPA, response selectivity was statistically equivalent (F(1,21) = 0.48, p = 0.50) for shape (cubes–spheres) versus category (scenes–faces). In TOS, the selectivity for shape was even greater (F(1,21) = 13.05, p < 0.01) than that for category.
Comparison of fMRI responses with differences in stimulus shape (left) compared with differences in stimulus category (right), in areas PPA (A) and TOS (B). Stimulus shape differences were based on the responses to arrays of 3D spheres versus cubes, used in Experiment 3. Category differences were based on the contrast of faces versus scenes, a robust and common localizer for PPA and TOS (Nasr et al., 2011; Epstein and Kanwisher, 1998; Rajimehr et al., 2011; Nasr and Tootell, 2012a, b). In each panel, white data points indicate the activity for individual subjects measured relative to fixation-only blocks. Group-averaged means are shown in red and green, for cubes versus spheres and faces versus scenes, respectively.
Across subjects, mean activity was much less variable in response to scenes versus faces, compared with shapes. This difference in variability may reflect the fact the ROIs were defined by a scenes versus faces localizer, rather than a shape-based localizer.
Experiment 4: filled overlapping shapes
As a next step toward progressively simpler visual stimuli, Experiment 4 (n = 15) tested for a rectilinear bias using arrays of filled circles versus squares (Fig. 10A). As in the real world, such stimuli overlapped each other, thus incorporating foreground/background cues. As a secondary goal, this experiment explicitly tested whether the PPA preference for rectilinear stimuli remained, even when rectilinear stimuli were presented at oblique orientations (Nasr and Tootell, 2012a). Here, squares were presented at either cardinal (i.e., horizontal and vertical) or oblique orientations, within independent blocks. As expected, the rectilinear index successfully differentiated these arrays of squares from circles (F(1,43) = 133.58, p < 10−13).
Stimuli and results for Experiment 4. A, Examples of the complete stimuli: arrays of filled overlapping squares presented at either cardinal (left) or oblique (middle) orientations, and an otherwise identical array of circles (right). The group-averaged activity maps (B) showed a stronger response to arrays of squares (averaged over both orientations) compared with circles in PPA and TOS, but not in RSC. In addition, low-amplitude, patchy activity was also found in V1, but that activity was confined to the retinotopic representation at and beyond the peripheral borders of the stimuli (i.e., that activity was not driven by the experimental stimulus contrast). Dashed white lines indicate the borders of stimulus-activated V1 defined using a separate set of retinotopic scans for the same group of subjects (asterisk = foveal representation). Consistent with these results, an ROI analysis (C) showed stronger activity in PPA and TOS in response to squares compared with circles, even when the squares were presented at oblique orientations. Other details are as in Figure 3.
The map of brain activity showed a higher response in PPA and TOS (but not in RSC) to the squares, compared with circles (Fig. 10B,C). In PPA, this rectilinear-selective activity was centered within the posterior portion of the area borders, as localized by the scenes versus faces contrast. The ROI results also showed that PPA responses to the oblique orientations were significantly lower (t(14) = 2.31, p = 0.04) compared with the cardinal orientations, thus replicating the fMRI-based oblique effect. Importantly, these data showed further that PPA responses to the oblique squares remained significantly stronger compared with the circles (t(14) = 2.34, p = 0.03). Thus, both rectilinearity and cardinal orientations significantly enhance the PPA response.
As described previously (Nasr and Tootell, 2012a), here we again found no significant cardinal orientation bias in TOS (t(14) = 0.22, p = 0.83) or RSC (t(14) = 1.48, p = 0.16). Also, the ROI analysis did not reveal any significant differential activity in V1 between squares versus circles, at either cardinal (t(14) = −0.59, p = 0.56) or oblique (t(14) = 0.27, p = 0.79) orientations. Accordingly, the cortical map (Fig. 10B) showed no significant activity bias in the stimulus-driven representation in V1, although more peripheral representations showed minor patches of activity, as described previously (Nasr and Tootell, 2012a).
Experiment 5A: closed unfilled shapes, equated for shape number
The next stimulus reduction tested whether PPA preferred line drawings of squares (internal corners = 90°) compared with circles, and also compared with other isosceles polygons, including triangles (60° corners), hexagons (120° corners), and dodecagons (150° corners) (Fig. 11A; Materials and Methods). Again, the number of shapes was kept constant across the stimulus arrays. However, unlike the previous experiment, stimuli here were unfilled line drawings, and the shapes did not overlap. Overlap can change the background (but not the foreground) shapes. The rectilinear index successfully differentiated these five shape categories, with the squares showing the highest value (F(4,1199) = 955.66, p < 10−15°).
Stimuli and results in Experiment 5A. A, Stimulus examples. In the actual experiment, each stimulus array included 40 shapes, instead of the single shapes shown here. Group-averaged activity maps (B) showed a stronger response evoked by arrays of squares compared with circles, largely confined within PPA and TOS, but not within RSC or V1. C, Corresponding ROI analysis. In PPA (bottom left), squares evoked a stronger response compared with other shapes. The TOS response profile (bottom right) was similar but less differentiated across the simplest polygons. Details are otherwise as in Figure 3.
The results (n = 13) confirmed the rectilinear prediction in PPA. The group-averaged maps (Fig. 11B) showed a stronger response to squares compared with circles, in both PPA and TOS, but not in RSC (see Discussion). The rectilinear bias was relatively higher in the posterior portion of PPA, as defined by the independent scene versus face localizer. At lower threshold levels, we also found a significantly stronger response to squares compared with triangles and hexagons. However, this difference was mainly confined to patches in PPA; it did not extend to TOS (Fig. 12). This difference suggested a more specific selectivity function for PPA compared with TOS, a conclusion confirmed below. An ROI analysis showed a significant effect of stimulus shape in both PPA (F(4,48) = 14.98, p < 10−7) and TOS (F(4,48) = 19.50, p < 10−8). No shape evoked a larger response compared with squares, in either PPA or TOS.
Activity maps for the contrast of squares versus triangles (A) and squares versus hexagons (B) in Experiment 5. In both cases, we found a higher response to squares in PPA, tending toward the posterior part of that area. Because the results did not show any difference between the PPA response across the two different matching criteria (i.e., equated for shape vs angle number), in B, the activity maps were averaged over the two matching conditions to increase the overall signal-to-noise ratio.
Experiment 5B: closed unfilled shapes, equated for angle number
It could be argued that the above results were influenced by variation in the number of angles, which covaried with shape. To address this, we presented additional control conditions (independent blocks within the same scan session for Experiment 5) in which the most diagnostic polygons (i.e., triangles and squares) were equated for the number of angles rather than shape (see Materials and Methods). Because the square versus circle comparison was the main contrast of interest, we also generated another set of circle arrays in which the number of circles was equal to the number of squares. All arrays were equated for the perimeter sum across all elements.
When images were equated for the number of angles rather than shapes, TOS and PPA (but not RSC) again showed a preference for squares compared with circles (Fig. 8A,B). Again, the ROI analysis showed a significant effect of stimulus shape in PPA (F(2,24) = 35.27, p < 10−7) and TOS (F(2,24) = 23.18, p < 10−5), with squares evoking the highest response.
In addition, comparisons of responses to the two equating criteria (i.e., in Experiments 5A and 5B) revealed a novel (and computationally helpful) stimulus invariance that further distinguished PPA from TOS. In PPA, application of a two-factor repeated-measures ANOVA (shapes [circles vs triangles vs squares] and equating criteria [shape number vs angle number]) showed a significant effect of shape (F(2,24) = 56.91, p < 10−9), but no significant difference between the responses to the two equating criteria (F(1,12) < 0.01, p = 0.96), without interaction between the two factors (F(2,24) = 0.04, p = 0.96). However unlike PPA, TOS showed a significantly higher response when stimuli were equated for shape number (F(1,12) = 11.44, p < 0.01) plus a significant effect of stimulus type (F(2,24) = 46.52, p < 10−8) and a significant interaction between these factors (F(2,24) = 5.24, p = 0.01). Thus, PPA (but not TOS) showed invariance for specific features (i.e., the number of angles/shapes and/or side length). Contrary to what we found in PPA and TOS, V1 did not show any significant selectivity for rectilinear shapes (i.e., squares) during either Experiment 5A (Fig. 11) or Experiment 5B (Fig. 13). Instead, it showed higher amplitudes to dodecagons and circles compared with rectangles, when equated for either the number of shapes (F(4,48) = 10.82, p < 10−5) or the number of angles (F(2,24) = 10.39, p < 0.01).
Response comparison for stimulus pairs that were equated for the number of angles (Experiment 5B) instead of the number of shapes (Experiment 5A). Data were generated as a control manipulation, acquired in intermixed blocks in Experiment 5. A, When equated for the number of angles, the group-averaged activity maps showed a stronger response to the array of squares, compared with circles, that was located mainly within PPA and TOS (but not within RSC or V1), similar to the results in Figure 11. These results were also evident in the ROI analysis (B): in both PPA (bottom left) and TOS (bottom right), squares evoked a stronger response compared with circles, regardless of the two criteria by which the stimuli were equated. Dashed line indicates number of shapes; solid line indicates number of angles. Furthermore, comparison of the responses to these two stimuli showed that PPA responses remained statistically equal regardless of this stimulus variation, whereas TOS showed a relatively higher response to those stimuli equated for the number of objects. Other details are as in Figure 3.
Experiment 6: open-ended lines
Results of Experiment 5 raise the possibility that PPA responds in a predictable manner to even simpler components of rectangles, such as the corners and sides. Accordingly, Experiment 6 tested for a higher response in PPA to open-ended angles of 90° (and perhaps 180°), relative to other angles (tested in 30° steps), and to semicircular arcs (Fig. 14A). As in Experiment 5, all stimuli here were equated for the perimeter sum across elements and line width, and presented at systematically varied orientations, in 45° steps, to eliminate possible orientation biases (see Materials and Methods). The rectilinear index successfully differentiated these seven conditions from each other, and 90° angles showed the highest value (F(6,671) = 115.11, p < 10−99).
Stimuli and results for Experiment 6. A, Stimulus shapes. In the actual experiments, each stimulus array contained 120 angles or semicircles, rather than the four shown here. In group-averaged activity maps, arrays of both 90° (B) and 180° (C) angles evoked a stronger response compared with semicircles in PPA and TOS. Analogous biases were not found in RSC or early retinotopic areas (e.g., V1, dashed white lines). The ROI analysis (D) confirmed these results, showing further that PPA (bottom left) had a bimodal response profile (a preference for both 90° and 180° angles). However, the TOS response (bottom right) to 180° angles was significantly smaller than its response to 90° angles. Other details are as in Figure 3.
Consistent with our hypothesis and with the results in Experiment 5, the fMRI activity maps (n = 16) showed a distinctively higher response to angles of 90° (Fig. 14B) and 180° (Fig. 14C) in PPA, compared with semicircles. This rectilinear bias was concentrated in posterior (rather than anterior) PPA. In the 90° contrast, the activity also extended slightly posteriorly beyond PPA, toward areas PHC1/2, and/or areas VO1/2 (Brewer et al., 2005; Arcaro et al., 2009; Silver and Kastner, 2009; Witthoft et al., 2013). Area TOS also showed a prominently higher response to 90° angles and a weaker response to 180°. Area RSC was not activated differentially by these simple stimuli.
Figure 14D shows the ROI results. Consistent with the cortical maps (Fig. 14B,C), application of a one-factor repeated-measures ANOVA yielded a significant effect of angle size on fMRI activity in PPA (F(6,90) = 7.69, p < 10−5) and TOS (F(6,90) = 9.05, p < 10−7), but not in RSC (F(6,90) = 1.87, p = 0.10). PPA showed higher activity to both right angles and straight lines, relative to the semicircles, and relative to all the remaining nonrectilinear angles 30°, 60°, 120°, and 150° (t(15) > 3.30, p < 0.01). Responses to 90° and 180° were statistically similar to each other (t(15) = 0.44, p = 0.66), suggesting a bimodal response function in PPA.
Like PPA, TOS showed higher activity to angles of 90° (and unexpectedly, to 120°), compared with semicircles, and also compared with the remaining angles of 30°, 60°, and 150° (t(15) > 4.54, p < 10−3). However, unlike PPA, the TOS response to 180° (a straight line) was significantly smaller than its response to 90° angles (t(15) = 2.34, p = 0.03). Thus, TOS showed a relatively unimodal (rather than bimodal) response function. V1 showed a relatively monotonic decrease from curved and acute angles to increasingly obtuse angles, without any specificity for right angles (F(6,90) = 10.04, p < 10−7).
It is straightforward to imagine how outputs from V1 (or orientation-selective regions in V2 or V3) might be combined to “construct” a 2D rectilinear selectivity in PPA and/or TOS (Fig. 15). Similar models have been proposed for other cortical areas (e.g., Marr and Nishihara, 1978; Riesenhuber and Poggio, 1999). However a dominant preference for rectilinear features has not been demonstrated previously in any cortical area, to our knowledge.
Possible generation of rectilinear selectivity in PPA, based on inputs from a retinotopic and orientation-selective map of early visual cortex. This schematic is based on published maps from orientation-selective subregions in primate cortical areas V1 and V2. It not yet known whether PPA receives direct input from either of these areas, but slight modifications of this model could accommodate the interposition of additional cortical areas or processing stages between early visual areas (e.g., V1/V2) and PPA. Left, Map of preferred orientation (small white bars) and receptive field (retinotopic) location, in an early visual cortical area. Top versus bottom receptive field positions are indicated in red through black. Analogous left-versus-right visual field locations are indicated in blue through black. Right, In subsequent area(s) (e.g., PPA), a template-based sensitivity to right angles and lines can be generated by simply combining single-orientation information from the appropriate visual field position.
Distribution of activity within PPA
Previous evidence suggests that posterior versus anterior regions in PPA are driven more by (1) simple versus complex stimulus cues (Arcaro et al., 2009; Rajimehr et al., 2011; Baldassano et al., 2013) and/or (2) decreased versus increased spatial context (Bar and Aminoff, 2003), respectively. Our data (Fig. 16) supported this hypothesis. Simple line-based shapes (which also had weak spatial context) produced activity that was centered in posterior PPA. Conversely, the more complex and overlapping stimuli (including real-world objects, with stronger spatial context) activated posterior and anterior PPA more evenly, and the latter activity centers were located near the middle of PPA. Similarly, the center of activity for the real-world contrast of scenes versus faces remained essentially centered in PPA, for all experiments.
Detailed topography of activity in PPA in response to the rectilinear versus round shapes, across all six experiments. We found a general anterior-to-posterior shift in the center of activity, consistent with covariations in stimulus complexity. Our most complex stimuli (Experiments 1–4; multiple/single real-world objects, 3D shapes, and overlapping luminance-varying 2D shapes, respectively) produced activity that extended throughout much or all of localizer-defined PPA, thus centered (cyan circles) toward the middle of PPA. In contrast, our simplest stimuli (Experiments 5 and 6, based on line configurations) produced activity that was centered toward the posterior border of PPA. To emphasize the center of activity, here all activity maps were based on fixed rather than random effects, and thresholds were normalized across experiments. For comparison, the center of activity for the localizer itself (scenes vs faces) is indicated with a black asterisk in each panel; it did not shift significantly across the different experiments. In all panels, the PPA borders (black lines) indicate the area borders for the tested groups of subjects, based on independent stimuli and scans (e.g., Fig. 5).
To test whether this effect was statistically significant, we localized the center of activity for each individual subject, in response to the same stimulus contrasts used in Figure 16 (see also Materials and Methods). Application of a two-factor ANOVA (hemisphere × experiment) to the location of the center of activity in the anterior versus posterior direction (up vs down in Fig. 16) confirmed a significant effect of experiment (F(5,167) = 7.07, p < 10−5) but no significant effect of hemisphere (F(1,167) < 10−3, p = 0.99) and no significant interaction between the effects of the two factors (F(5,167) = 0.75, p = 0.59). Application of this same test to the location of activity centers in the left versus right direction showed an expected significant effect of hemisphere (F(1,167) = 10,577.59, p < 10−143) but no significant effect of experiment (F(5,167) = 1.40, p = 0.23) and no significant interaction between the effect of the two factors (F(5,167) = 1.60, p = 0.16). Thus, the shift in the center of activity in response to simple through complex stimuli was statistically significant, and not the result of between-subject variability.
Discussion
Relationship to prior studies
To what extent did unplanned variation in rectilinear features contribute to previous fMRI results? For instance, earlier studies reported higher PPA activity in response to images of (1) indoor versus outdoor scenes (Henderson et al., 2007), (2) tools versus faces (Chao et al., 1999), and (3) large versus small objects (Konkle and Oliva, 2012). Our rectilinear filtering of analogous stimuli (Fig. 2) suggests that those experimental stimuli (indoor scenes or tools or large objects) included more rectilinear features compared with the control stimuli (outdoor scenes or faces, respectively). Thus, the current results are empirically consistent with past results, despite possible differences in interpretation.
In this context, it is worth noting that scenes are comprised of objects: the concept of a scene without objects (e.g., a hypothetical deep space with no celestial bodies) is undefined (see also Henderson and Hollingworth, 1999; Epstein, 2005). Thus, our demonstrations of the effect of rectilinear bias in single and multiple natural objects may well generalize to correspondingly differing scenes and to specific categories of objects that include rectilinear features (e.g., inanimate objects, tools, isolated buildings). Supporting this idea, the medial portion of the fusiform gyrus (presumably including the lateral PPA) is activated by identification of objects, in addition to recognition of scenes (Grill-Spector et al., 2003).
Importantly, the demonstration of a rectilinear bias does not rule out previously reported category-selective differences. As described above, some categories (e.g., buildings) may simply have more rectilinear features on average, compared with other categories (e.g., faces). In other cases, stimulus rectilinearity may be an uncontrolled variable. For example, several studies have tested whether PPA responds preferentially to indoor versus outdoor scenes, without considering possible differences in rectilinearity. Such studies generated either a positive result (Henderson et al., 2007) or a negative result (Epstein and Kanwisher, 1998) or no significant differences (Bar and Aminoff, 2003).
Furthermore, additional lower-level features may influence PPA responses (Rajimehr et al., 2011; Nasr and Tootell, 2012a; Troiani et al., 2014), analogous to those that affect V1, V2, MT, and even higher-tier areas, such as the fusiform face area (George et al., 1999; Gilad et al., 2009; Ohayon et al., 2012; Yue et al., 2013). High-level processes and top-down control may also modulate PPA activity (Freedman and Miller, 2008; Cromer et al., 2010; Roy et al., 2010, but see Minamimoto et al., 2010), all independent of a rectilinear bias.
Additional scene-selective areas
In previous reports, two additional cortical areas (TOS and RSC) were often coactivated with PPA, based on selectivity for scenes (Nasr et al., 2011) or isolated buildings (Aguirre et al., 1998; Grill-Spector, 2003; Levy et al., 2004). Here, rectilinear features also activated TOS (in all experiments) and RSC (Experiments 1 and 3), compared with round/curved stimuli.
The consistent coactivation of TOS with PPA suggests a similar role for these two areas and/or strong connections between them. However, unlike PPA, TOS responded poorly to straight lines (Experiment 6). Thus, TOS functioned generally as a corner detector, with a preference for right (and perhaps near-right obtuse) angle corners. Also, unlike PPA, TOS activity apparently reflected the influence of additional lower-level factors (Experiment 5), consistent with a more “transitional” (lower-level) functional role for TOS compared with PPA (Hasson et al., 2003).
Interestingly in TOS, the cubes versus spheres comparison evoked a greater response difference compared with scenes versus faces (Fig. 9). This higher selectivity for shape rather than category again supports a lower-tier role of TOS compared with PPA (Hasson et al., 2003).
Unlike PPA and TOS, RSC was not significantly activated by simple stimuli lacking 3D cues (Experiments 4–6) or single objects (Experiment 2). However, RSC did show a rectilinear bias to multiple real-world objects and more complex shapes (Experiments 1 and 3, respectively). This evidence supports previous suggestions that RSC activation requires more complex features compared with PPA and TOS (Epstein et al., 2007; Park and Chun, 2009; Wolbers et al., 2011; Harel et al., 2013).
Anterior–posterior organization in PPA
Previous two-condition comparisons suggest that PPA can be subdivided into posterior and anterior subareas, such that more anterior subareas respond better to more complex objects (with stronger spatial context), and posterior areas respond better to simpler stimuli (with weaker spatial context) (Bar and Aminoff, 2003; Arcaro et al., 2009; Rajimehr et al., 2011; Baldassano et al., 2013). These anterior versus posterior subdivisions also differ in functional connectivity (Baldassano et al., 2013; Nasr et al., 2013). The systematic comparisons here, between six otherwise similar conditions, confirmed this idea (Fig. 16). Overall, this evidence supports the idea that simple rectilinear features serve as functional building blocks in posterior PPA, which are elaborated and integrated with more complex spatial properties in anterior PPA.
Teleology
As described above, the current results raise the question: does PPA respond fundamentally (primarily) to rectilinear features, and thus only secondarily to scenes, inanimate objects, and tools, to the extent that the corresponding category exemplars include rectilinear features? Alternatively, are the fundamental neural representations in PPA reversed, so that rectilinear features only remind subjects of scenes (and rectilinear visual objects), thus indirectly recruiting secondary activity in a PPA that fundamentally represents scenes? Or does the truth lie somewhere in between?
Answers to these questions will require additional experiments. In the meantime, the former hypothesis begs a question: why would the visual system emphasize the processing of rectilinear angles? The following speculations may be germane.
First, a right angle is unique among all other angles because a right angle defines paired angles that are both equal and mirror-symmetrical on a linear baseline. Indeed, the right angle is one of Euclid's elements. For several millennia in many cultures, the classic right angular “post and beam” structure (a literal building block) has been a main basis for load-bearing structures (e.g., houses) and objects (e.g., chairs and tables). One practical advantage is that right angles can be defined geometrically, without knowledge of arithmetic or complex measuring devices.
Right angles are formed by the intersection of the gravitational axis relative to the average angle formed by the horizon, to a first approximation. This property may well aid body and head orientation, especially for upright locomotion on two (rather than four) legs. It may be no accident that the term “right angle” arises from the Latin angulus rectus (literally, “upright angle”), alluding to a vertical line oriented perpendicular to a horizontal base.
Origins of rectilinear selectivity
More specifically, did this rectilinear bias develop gradually over millions of years of evolution (i.e., to exploit mild image biases in natural scenes), or more recently, as a specific sensitivity to carpentered environments and objects?
One challenge to answering this question is that the relevant scenes are unknown (and likely variable) throughout human evolution. Were such scenes dominated by sandy desert, mountains, forest, savannah, or jungle? Also, what was the relevant historical period, over the tens of millions of years of primate evolution? By comparison, carpentered environments have only been present for ∼10 millennia. However, other dramatic examples of human-pressured evolution have also occurred over similarly brief spans of time, including purpose-bred dog strains, and agricultural plants, such as corn. A third possibility is that rectilinear sensitivity develops even more quickly, within each person's lifetime. Fourth, our results could reflect a coexisting selectivity for both rectilinearity and category (e.g., scenes) at the level of fMRI. For instance, the current results could reflect a snapshot in a continuing evolution from a primitive sensitivity to a broad range of angles, to a more specific selectivity for right angles, arising from current carpentered environments.
In conclusion, several findings are worth reemphasizing. First, previously described shape-related variables in lower-level visual cortex are often one-dimensional in nature (e.g., orientation, spatial frequency, and planar motion). By comparison, rectilinearity is a 2D feature, supporting the idea that PPA/TOS reflects mid-level processing. This finding also reveals one way in which lower-level features are processed at a higher visual cortical stage. Second, we found that the selectivity for rectilinear features was consistent and strong in all six experiments, and the selective activity was largely confined to PPA/TOS at conventional thresholds. In PPA/TOS, even very abstract shape differences (cubes vs spheres) produced differences in fMRI amplitude comparable with those produced by a well-known category-based (faces vs scenes) contrast (Fig. 9). Third, the rectilinear hypothesis was quantifiable based on image filtering, without experimenter intervention or subjective judgments. Fourth, the rectilinear hypothesis is simple, which has advantages both intuitively (Occam's razor; Gauch, 2003) and formally (Solomonoff, 1964). Finally, the rectilinear hypothesis could potentially unify at least some of the wide divergence in current higher-order interpretations of PPA function (Aguirre et al., 1998; Epstein and Kanwisher, 1998; Chao et al., 1999; Levy et al., 2004; Ewbank et al., 2005; Konkle and Oliva, 2012). Even if/when the rectilinear hypothesis ultimately proves incomplete, it will likely have served to clarify the response selectivity in these intriguing cortical areas.
Footnotes
This work was supported by National Institutes of Health Grants R01 MH67529 and R01 EY017081 to R.B.H.T., the Martinos Center for Biomedical Imaging, the National Center for Research Resources, and the MIND Institute. We thank Ali Amin-Manosour for help with the data collection and Xiaomin Yue for making the stimuli in Experiment 3.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Shahin Nasr, Athinioula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, 149 13th Street, Charlestown, MA 02129. shahin{at}nmr.mgh.harvard.edu
This article is freely available online through the J Neurosci Author Open Choice option.
