Patterns of response to visual scenes are linked to the low-level properties of the image

Highlights

• Scene-selective regions play an important role in the way we navigate our environment.

• Previous studies have suggested that these regions are linked to scene category.

• We show that patterns of response to scenes are linked to the low-level image properties.

Abstract

Scene-selective regions in the brain play an important role in the way that we navigate through our visual environment. However, the principles that govern the organization of these regions are not fully understood. For example, it is not clear whether patterns of response in scene-selective regions are linked to high-level semantic category or to low-level spatial structure in scenes. To address this issue, we used multivariate pattern analysis with fMRI to compare patterns of response to different categories of scenes. Although we found distinct patterns of neural response to each category of scene, the magnitude of the within-category similarity varied across different scenes. To determine whether this variation in the categorical response to scenes could reflect variation in the low-level image properties, we measured the similarity of images from each category of scene. Although we found that the low-level properties of images from each category were more similar to each other than to other categories of scenes, we also found that the magnitude of the within-category similarity varied across different scenes. Finally, we compared variation in the neural response to different categories of scenes with corresponding variation in the low-level image properties. We found a strong positive correlation between the similarity in the patterns of neural response to different scenes and the similarity in the image properties. Together, these results suggest that categorical patterns of response to scenes are linked to the low-level properties of the images.

Introduction

The ability to perceive and recognize different visual scenes is essential for spatial navigation in the world. Although real-world scenes can be incredibly complex and heterogeneous, human observers are able to reliably recognize and categorize images of scenes even when the images are shown briefly (Greene and Oliva, 2009, Joubert et al., 2007, Potter, 1975). These studies have been taken to suggest that the initial perception of natural images is based on the global, visual properties – the gist – of the scene (Greene and Oliva, 2009, Oliva and Torralba, 2001).

Neuroimaging studies have found a number of regions of the human brain that respond selectively to visual scenes. Damage to these regions often leads to impairments that are specific to scene perception and spatial navigation (Aguirre and D'Esposito, 1999, Mendez and Cherrier, 2003). The parahippocampal place area (PPA) is a region of the posterior parahippocampal gyrus that displays preferential activity to images of scenes over and above images of objects and faces (Aguirre and D'Esposito, 1997, Epstein and Kanwisher, 1998). Other place selective regions include the retrosplenial complex (RSC) located immediately superior to the PPA and the transverse occipital sulcus (TOS) or occipital place area (OPA) on the lateral surface of the occipital lobe (Dilks et al., 2013, Epstein, 2008).

The spatial layout of different categories of scenes can vary quite considerably (Torralba and Oliva, 2003). Although neuroimaging studies using univariate analyses have reported comparable levels of response to scenes as diverse as natural landscapes, cityscapes and indoor scenes in scene-selective regions (Aguirre and D'Esposito, 1997, Epstein and Kanwisher, 1998), more recent studies using multivariate analyses have found distinct patterns of response in these regions to different categories of scene (Walther et al., 2009, Walther et al., 2011). Interestingly, these patterns of neural response have also been shown to correlate with patterns of behavioral response, but not with the low-level image properties of the images (Walther et al., 2009). This suggests that there is a dissociation between the perceptual categorization of scenes and their underlying image properties. However, this conclusion has been challenged by other studies that have suggested that the patterns of response in scene-selective regions are better explained by the spatial layout of the scene rather than by semantic category (Kravitz et al., 2011, Park et al., 2011). Although these studies are not explicit about how the image properties of the scene are linked to the patterns of neural response, work in computer vision indicates that semantically-distinct scene categories can be identified on the basis of their characteristic low-level image statistics. For example, the GIST descriptor can be used to accurately classify different scene categories and derive spatial properties such as openness (Torralba and Oliva, 2003).

Our aim was to determine whether categorical patterns of brain activity within scene-selective regions are linked to the low-level properties of the images from each category of scene. To address this issue, we measured the pattern of response to different categories of scenes using fMRI. Next, we asked how similar the low-level properties of images from each category were to each other. Finally, we asked whether differences in the categorical response to different visual scenes might be due to variation in low-level image properties. Our prediction was that, if low-level visual properties are linked to categorical patterns of response in these regions, then scene categories with similar image statistics should elicit correspondingly similar patterns of brain activity.