Abstract
We used functional magnetic resonance imaging (fMRI) to demonstrate the existence of a mechanism in the human lateral occipital (LO) cortex that supports recognition of real-world visual scenes through parallel analysis of within-scene objects. Neural activity was recorded while subjects viewed four categories of scenes and eight categories of 'signature' objects strongly associated with the scenes in three experiments. Multivoxel patterns evoked by scenes in the LO cortex were well predicted by the average of the patterns elicited by their signature objects. By contrast, there was no relationship between scene and object patterns in the parahippocampal place area (PPA), even though this region responds strongly to scenes and is believed to be crucial for scene identification. By combining information about multiple objects within a scene, the LO cortex may support an object-based channel for scene recognition that complements the processing of global scene properties in the PPA.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Potter, M.C. Meaning in visual search. Science 187, 965–966 (1975).
Biederman, I., Rabinowitz, J.C., Glass, A.L. & Stacy, E.W. On the Information extracted from a glance at a scene. J. Exp. Psychol. 103, 597–600 (1974).
Schyns, P.G. & Oliva, A. From blobs to boundary edges: Evidence for time- and spatial-scale-dependent scene recognition. Psychol. Sci. 5, 195–200 (1994).
Renninger, L.W. & Malik, J. When is scene identification just texture recognition? Vision Res. 44, 2301–2311 (2004).
Greene, M.R. & Oliva, A. Recognition of natural scenes from global properties: seeing the forest without representing the trees. Cognit. Psychol. 58, 137–176 (2009).
Epstein, R. & Kanwisher, N. A cortical representation of the local visual environment. Nature 392, 598–601 (1998).
Aguirre, G.K., Zarahn, E. & D'Esposito, M. An area within human ventral cortex sensitive to “building” stimuli: evidence and implications. Neuron 21, 373–383 (1998).
Park, S., Brady, T.F., Greene, M.R. & Oliva, A. Disentangling scene content from spatial boundary: complementary roles for the parahippocampal place area and lateral occipital complex in representing real-world scenes. J. Neurosci. 31, 1333–1340 (2011).
Malach, R. et al. Object-related activity revealed by functional magnetic-resonance-imaging in human occipital cortex. Proc. Natl. Acad. Sci. USA 92, 8135–8139 (1995).
Peelen, M.V., Fei-Fei, L. & Kastner, S. Neural mechanisms of rapid natural scene categorization in human visual cortex. Nature 460, 94–97 (2009).
Epstein, R., Graham, K.S. & Downing, P.E. Viewpoint-specific scene representations in human parahippocampal cortex. Neuron 37, 865–876 (2003).
Haxby, J.V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).
Walther, D.B., Caddigan, E., Fei-Fei, L. & Beck, D.M. Natural scene categories revealed in distributed patterns of activity in the human brain. J. Neurosci. 29, 10573–10581 (2009).
Drucker, D.M. & Aguirre, G.K. Different spatial scales of shape similarity representation in lateral and ventral LOC. Cereb. Cortex 19, 2269–2280 (2009).
Haushofer, J., Livingstone, M.S. & Kanwisher, N. Multivariate patterns in object-selective cortex dissociate perceptual and physical shape similarity. PLoS Biol. 6, e187 (2008).
MacEvoy, S.P. & Epstein, R.A. Decoding the representation of multiple simultaneous objects in human occipitotemporal cortex. Curr. Biol. 19, 943–947 (2009).
Diana, R.A., Yonelinas, A.P. & Ranganath, C. High-resolution multi-voxel pattern analysis of category selectivity in the medial temporal lobes. Hippocampus 18, 536–541 (2008).
Bar, M. & Aminoff, E. Cortical analysis of visual context. Neuron 38, 347–358 (2003).
Potter, M.C., Staub, A. & O'Connor, D.H. Pictorial and conceptual representation of glimpsed pictures. J. Exp. Psychol. Hum. Percept. Perform. 30, 478–489 (2004).
Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proc. Natl. Acad. Sci. USA 103, 3863–3868 (2006).
Rogosa, D. Comparing nonparallel regression lines. Psychol. Bull. 88, 307–321 (1980).
Naselaris, T., Prenger, R.J., Kay, K.N., Oliver, M. & Gallant, J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 63, 902–915 (2009).
Kay, K.N., Naselaris, T., Prenger, R.J. & Gallant, J.L. Identifying natural images from human brain activity. Nature 452, 352–355 (2008).
Reddy, L., Kanwisher, N.G. & VanRullen, R. Attention and biased competition in multi-voxel object representations. Proc. Natl. Acad. Sci. USA 106, 21447–21452 (2009).
Zoccolan, D., Cox, D.D. & DiCarlo, J.J. Multiple object response normalization in monkey inferotemporal cortex. J. Neurosci. 25, 8150–8164 (2005).
Moran, J. & Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782–784 (1985).
MacEvoy, S.P., Tucker, T.R. & Fitzpatrick, D. A precise form of divisive suppression supports population coding in the primary visual cortex. Nat. Neurosci. 12, 637–645 (2009).
Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).
Reynolds, J.H. & Heeger, D.J. The normalization model of attention. Neuron 61, 168–185 (2009).
Beck, D.M. & Kastner, S. Stimulus similarity modulates competitive interactions in human visual cortex. J. Vis. 7 (19), 11–12 (2007).
Beck, D.M. & Kastner, S. Stimulus context modulates competition in human extrastriate cortex. Nat. Neurosci. 8, 1110–1116 (2005).
Fei-Fei, L., Iyer, A., Koch, C. & Perona, P. What do we perceive in a glance of a real-world scene? J. Vis. 7 (1), 10 (2007).
Schwarzlose, R.F., Swisher, J.D., Dang, S. & Kanwisher, N. The distribution of category and location information across object-selective regions in human visual cortex. Proc. Natl. Acad. Sci. USA 105, 4447–4452 (2008).
Sayres, R. & Grill-Spector, K. Relating retinotopic and object-selective responses in human lateral occipital cortex. J. Neurophysiol. 100, 249–267 (2008).
Kravitz, D.J., Kriegeskorte, N. & Baker, C.I. High-level visual object representations are constrained by position. Cereb. Cortex 20, 2916–2925 (2010).
Andresen, D.R., Vinberg, J. & Grill-Spector, K. The representation of object viewpoint in human visual cortex. Neuroimage 45, 522–536 (2009).
Eger, E., Kell, C.A. & Kleinschmidt, A. Graded size sensitivity of object-exemplar-evoked activity patterns within human LOC subregions. J. Neurophysiol. 100, 2038–2047 (2008).
Epstein, R.A. Parahippocampal and retrosplenial contributions to human spatial navigation. Trends Cogn. Sci. 12, 388–396 (2008).
Kravitz, D.J., Peng, C.S. & Baker, C.I. Real-world scene representations in high-level visual cortex: it's the spaces more than the places. J. Neurosci. 31, 7322–7333 (2011).
Spiridon, M. & Kanwisher, N. How distributed is visual category information in human occipito-temporal cortex? An fMRI study. Neuron 35, 1157–1165 (2002).
Habib, M. & Sirigu, A. Pure topographical disorientation: a definition and anatomical basis. Cortex 23, 73–85 (1987).
Aguirre, G.K. & D'Esposito, M. Topographical disorientation: a synthesis and taxonomy. Brain 122, 1613–1628 (1999).
Aguirre, G.K. Continuous carry-over designs for fMRI. Neuroimage 35, 1480–1494 (2007).
Epstein, R.A. & Higgins, J.S. Differential parahippocampal and retrosplenial involvement in three types of visual scene recognition. Cereb. Cortex 17, 1680–1693 (2007).
Epstein, R.A., Parker, W.E. & Feiler, A.M. Where am I now? Distinct roles for parahippocampal and retrosplenial cortices in place recognition. J. Neurosci. 27, 6141–6149 (2007).
Acknowledgements
The authors wish to thank E. Ward, A. Stigliani and Z. Yang for assistance with data collection. This work was supported by US National Eye Institute grant EY-016464 to R.A.E.
Author information
Authors and Affiliations
Contributions
S.P.M. and R.A.E. designed the experiments. S.P.M. collected fMRI data and R.A.E. collected behavioral data. S.P.M. analyzed data with input from R.A.E. S.P.M. and R.A.E. wrote the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–7, Supplementary Results (PDF 380 kb)
Rights and permissions
About this article
Cite this article
MacEvoy, S., Epstein, R. Constructing scenes from objects in human occipitotemporal cortex. Nat Neurosci 14, 1323–1329 (2011). https://doi.org/10.1038/nn.2903
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.2903
This article is cited by
-
Disentangling diagnostic object properties for human scene categorization
Scientific Reports (2023)
-
Multiple visual objects are represented differently in the human brain and convolutional neural networks
Scientific Reports (2023)
-
Predictive processing of scenes and objects
Nature Reviews Psychology (2023)
-
Brain-inspired models for visual object recognition: an overview
Artificial Intelligence Review (2022)
-
Object representations in the human brain reflect the co-occurrence statistics of vision and language
Nature Communications (2021)