Elsevier

NeuroImage

Volume 29, Issue 2, 15 January 2006, Pages 605-618
NeuroImage

Spatiotemporal dynamics of human object recognition processing: An integrated high-density electrical mapping and functional imaging study of “closure” processes

https://doi.org/10.1016/j.neuroimage.2005.07.049Get rights and content

Abstract

Humans are capable of recognizing objects, often despite highly adverse viewing conditions (e.g., occlusion). The term “perceptual closure” has been used to refer to the neural processes responsible for “filling-in” missing information in the visual image under such conditions. Closure phenomena have been linked to a group of object recognition areas, the so-called lateral–occipital complex (LOC). Here, we investigated the spatiotemporal dynamics of perceptual closure processes by coregistering data from high-density electrical recordings (ERPs) and functional magnetic resonance imaging (fMRI) while subjects participated in a perceptual closure task. Subjects were presented with highly fragmented images and control scrambled images. Fragmented images were calibrated to be ‘just’ recognizable as objects (that is, perceptual closure was necessary), whereas the scrambled images were unrecognizable. Comparison of responses to these two stimulus classes revealed the neural processes underlying perceptual closure. fMRI revealed an object recognition system that mediates these closure processes, the core of which consists of the LOC regions. ERP recordings resulted in the well-characterized NCL component (for negativity associated with closure), a robust relative negativity over bilateral occipito-temporal scalp that occurs in the 230–400 ms timeframe. Our investigations further revealed an extended network of dorsal and frontal regions, also involved in perceptual closure processes. Inverse source analysis showed that the major generators of NCL localized to the identical regions within LOC revealed by the fMRI recordings and detailed the temporal dynamics across these LOC regions including interactions between LOC and these other nodes of the object recognition circuit.

Introduction

One critical and highly adaptive aspect of human object recognition processes is our seemingly effortless ability to identify objects even when only partial, and often very sparse visual information is presented to the observer. The neural processes responsible for filling-in of missing information that enables eventual object recognition under partial viewing conditions (e.g., fog, partial occlusion, camouflage, poor lighting) have come to be referred to as “perceptual closure” processes (e.g., Bartlett, 1916, Snodgrass and Feenan, 1990, Foley et al., 1997). The present study uses the excellent temporal resolution of high-density electrical recordings in combination with the precise spatial localization abilities of functional imaging to investigate the spatiotemporal dynamics of these perceptual closure processes.

In the past decade, both functional imaging and electrophysiological studies have been used separately to investigate the neural processes responsible for basic object recognition. Functional imaging studies have revealed a cluster of regions within the so-called ventral visual stream that play an important role in cue-invariant object recognition (e.g., Malach et al., 1995, Kanwisher et al., 1997, Grill-Spector et al., 1998, Haxby et al., 1999). This cluster of regions is known as the lateral–occipital complex (LOC). The LOC is situated at the lateral and ventral aspects of the occipital lobe, which includes dorsal–lateral–occipital lobe close to area MT/V5 and ventral fusiform (Malach et al., 1995, Lerner et al., 2001). To our knowledge, only a single functional imaging study, which used positron emission tomography (PET), has directly investigated perceptual closure processes, and this study suggested that closure processes were also carried out within regions of the LOC (Gerlach et al., 2002). Two other related studies (Op de et al., 2000, Denys et al., 2004) have also shown activation in the regions of LOC while presenting degraded and scrambled shapes, presumably resulting from the activity of closure processes, although neither study was designed to expressly investigate these processes.

Closure processes have been mostly studied using electrophysiological techniques. In a series of studies from this laboratory (Doniger et al., 2000, Foxe et al., 2001, Doniger et al., 2001a, Doniger et al., 2002), line drawings of common objects, which were systematically fragmented to varying degrees, were used to study the timing of the neural processes responsible for perceptual closure. Our paradigm involved the presentation of sequences of fragmented pictures, whereby we began at very high levels of fragmentation and then incrementally increased the amount of information present in a given image until just enough of the object was present for subjects to “close” the picture (Snodgrass and Corwin, 1988, Doniger et al., 2000). Our studies revealed a robust event-related potential component termed the NCL (for negativity associated with closure) that tracked the neural processes related to perceptual closure. This component was manifest as a relative negativity over bilateral occipito-temporal scalp and occurred in the 230–400 ms timeframe (Doniger et al., 2000), typically peaking at 290–300 ms. The scalp topography of this component suggested that it largely reflected neural activity from the LOC or nearby structures. In addition, based on our previous studies (Doniger et al., 2001a), we also hypothesized the existence of other generators outside the LOC (e.g., frontal regions) involved in closure. While topographic mapping has suggested the involvement of LOC regions in the generation of the NCL, this has not been directly assessed. In the present study, we employed an integrative neuroimaging approach, combining the spatial accuracy of fMRI with the temporal resolution that ERP offers, to study the spatiotemporal dynamics of closure processes. We conducted a modified version of the perceptual closure experiment used in our previous work, while collecting both imaging and electrophysiological measures on the same subjects. This allowed us to coregister hemodynamic responses with electrical dynamics through the application of source analysis methods (Scherg and Picton, 1991, Grave de Peralta et al., 2001). Our data show that the bulk of NCL activity is indeed generated within regions of the LOC. Furthermore, the timing of these processes, which can be derived from the electrical recordings, challenges models of perceptual closure that posit simple feed-forward mechanisms. Rather, our data suggest that closure processes occur relatively late in processing and are likely due to recursive processing within these structures, subsequent to an initial relatively automatic phase of object processing. Furthermore, through the application of source analysis, we were able to elucidate the time course of activity within the various generators involved in closure, allowing us to speak to temporal inter-relationships between these nodes of the object recognition circuit.

Section snippets

Participants

Seven (4 female) neurologically normal paid volunteers, aged 21–35 (mean = 26), participated in the ERP study. Of these, six also participated in the fMRI section of the study in a counterbalanced order. Due to a mild bout of claustrophobia in the scanner, one of the seven ERP subjects had to be excluded from this portion of the study. All subjects provided written informed consent, and the procedures were approved by the Institutional Review Board of the Nathan Kline Institute. All subjects

fMRI data acquisition

A 3 T SMIS (Marconi) system equipped with a head volume coil at the Center for Advanced Brain Imaging (CABI) at the Nathan Kline Institute was used to acquire T2*-weighted echo-planar functional images (EPIs) (TR = 2000, FA = 90, matrix size = 64 × 64, FOV = 224 mm, voxel size = 3.5 mm3) emphasizing the blood oxygenation level dependent (BOLD) response. In each run, 230 volumes (20 contiguous functional slices) were localized in an oblique-coronal orientation that covered the occipital lobes,

fMRI results

The P values of the statistical maps obtained for each condition were Bonferroni corrected for multiple comparisons. Effects were accepted as significant only when P (corrected) < 0.001. Both experimental conditions (unscrambled and scrambled) activated widespread and substantially overlapping cortical networks. To characterize the cortical activation associated with closure, we contrasted the relative activation to “unscrambled” pictures versus “scrambled” pictures. This revealed significantly

Discussion

The high spatial resolution of fMRI was married with the millisecond temporal resolution of ERPs to assess the generators of perceptual closure processes, as indexed by the NCL component of the visual evoked potential. A major role for regions of bilateral LOC and frontal cortex in these processes was confirmed, and a concomitant role for dorsal visual regions was suggested. Here, we discuss the spatiotemporal dynamics of these object recognition processes in the context of the timing and

Conclusion

High-density electrical mapping, source analysis, and functional neuroimaging were used to map the spatiotemporal dynamics of perceptual closure processes. The data strongly support a model in which object recognition is achieved through interplay between the LOC and frontal cortical regions. The current data, together with findings from previous studies, indicate that there are dorsal and ventral contributions to object recognition and that frontal regions are activated for explicit object

Acknowledgments

This work was supported by grants from the National Institute of Mental Health (MH65350 to JJF and MH49334 to DCJ) and a grant from the National Institute on Aging (AG22696 to JJF). SM is supported by a Kirschstein National Research Service Award (NRSA) post-doctoral fellowship (MH068174). The authors would like to express their sincere appreciation to Dr. Vance Zemon for his insights. Deirdre Foxe and Beth Higgins provided invaluable technical help with data collection. We would also like to

References (90)

  • J.V. Haxby et al.

    The effect of face inversion on activity in human neural systems for face and object perception

    Neuron

    (1999)
  • S.M. Kosslyn et al.

    The cognitive neuroscience of mental imagery

    Neuropsychologia

    (1995)
  • C.M. Michel et al.

    Electric source imaging of human brain functions

    Brain Res. Brain Res. Rev.

    (2001)
  • Y. Miyashita et al.

    Neural representation of visual objects: encoding and top–down activation

    Curr. Opin. Neurobiol.

    (2000)
  • M.M. Murray et al.

    Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study

    Neuropsychologia

    (2001)
  • F. Perrin et al.

    Mapping of scalp potentials by surface spline interpolation

    Electroencephalogr. Clin. Neurophysiol.

    (1987)
  • G.A. Rousselet et al.

    Processing of one, two or four natural scenes in humans: the limits of parallelism

    Vision Res.

    (2004)
  • H.E. Schendan et al.

    Neurophysiological evidence for two processing times for visual object identification

    Neuropsychologia

    (2002)
  • M. Scherg et al.

    Two bilateral sources of the late AEP as identified by a spatio-temporal dipole model

    Electroencephalogr. Clin. Neurophysiol.

    (1985)
  • S.L. Thompson-Schill et al.

    Effects of repetition and competition on activity in left prefrontal cortex during word generation

    Neuron

    (1999)
  • A.D. Wagner et al.

    Recovering meaning: left prefrontal cortex guides controlled semantic retrieval

    Neuron

    (2001)
  • J. Warnking et al.

    fMRI retinotopic mapping—Step by step

    NeuroImage

    (2002)
  • M. Bar

    A cortical mechanism for triggering top–down facilitation in visual object recognition

    J. Cogn. Neurosci.

    (2003)
  • F.C. Bartlett

    An experimental study of some problems of perceiving and imagining

    Br. J. Psychol.

    (1916)
  • I. Biederman

    Recognition-by-components: a theory of human image understanding

    Psychol. Rev.

    (1987)
  • G.M. Boynton et al.

    Linear systems analysis of functional magnetic resonance imaging in human V1

    J. Neurosci.

    (1996)
  • C. Cavada et al.

    Posterior parietal cortex in rhesus monkey: II. Evidence for segregated corticocortical networks linking sensory and limbic areas with the frontal lobe

    J. Comp. Neurol.

    (1989)
  • C. Cavada et al.

    The anatomical connections of the macaque monkey orbitofrontal cortex. A review

    Cereb. Cortex

    (2000)
  • K. Denys et al.

    The processing of visual shape in the cerebral cortex of human and nonhuman primates: a functional magnetic resonance imaging study

    J. Neurosci.

    (2004)
  • F. Di Russo et al.

    Cortical sources of the early components of the visual evoked potential

    Hum. Brain Mapp.

    (2001)
  • E.C. Dias et al.

    Changing plans: a high density electrical mapping study of cortical control

    Cereb. Cortex

    (2003)
  • G.M. Doniger et al.

    Activation timecourse of ventral visual stream object-recognition areas: high density electrical mapping of perceptual closure processes

    J. Cogn. Neurosci.

    (2000)
  • G.M. Doniger et al.

    Impaired sensory processing as a basis for object-recognition deficits in schizophrenia

    Am. J. Psychiatry

    (2001)
  • G.M. Doniger et al.

    Impaired visual object recognition and dorsal/ventral stream interaction in schizophrenia

    Arch. Gen. Psychiatry

    (2002)
  • M. Eimer

    The face-specific N170 component reflects late stages in the structural encoding of faces

    NeuroReport

    (2000)
  • A.K. Engel et al.

    Dynamic predictions: oscillations and synchrony in top–down processing

    Nat. Rev., Neurosci.

    (2001)
  • D.J. Felleman et al.

    Distributed hierarchical processing in the primate cerebral cortex

    Cereb. Cortex

    (1991)
  • M.A. Foley et al.

    Investigations of closure processes: what source-monitoring judgments suggest about what is “closing”

    Mem. Cogn.

    (1997)
  • J.J. Foxe et al.

    Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing

    Exp. Brain Res.

    (2002)
  • J.J. Foxe et al.

    Early visual processing deficits in schizophrenia: impaired P1 generation revealed by high-density electrical mapping

    Neuroreport

    (2001)
  • J.J. Foxe et al.

    Filling-in in schizophrenia: a high-density electrical mapping and source-analysis investigation of illusory contour processing

    Cereb. Cortex

    (2005)
  • K.J. Friston et al.

    The relationship between global and local changes in PET scans

    J. Cereb. Blood Flow Metab.

    (1990)
  • K.J. Friston et al.

    Statistical parametric maps in functional imaging: a general linear approach

    Hum. Brain Mapp.

    (1995)
  • P. Girard et al.

    Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities

    J. Neurophysiol.

    (2001)
  • R. Goebel et al.

    Functional imaging of mirror and inverse reading reveals separate coactivated networks for oculomotion and spatial transformations

    NeuroReport

    (1998)
  • Cited by (113)

    • Memory influences visual cognition across multiple functional states of interactive cortical dynamics

      2019, Psychology of Learning and Motivation - Advances in Research and Theory
      Citation Excerpt :

      The N3 complex is the first ERP to differ between perceptually-matched pictures of objects that have been successfully categorized at the entry level relative to those that were unidentified (Doniger et al., 2000; Schendan & Kutas, 2002), decreasing in amplitude with better categorization and naming success. In one approach to timing of visual knowledge, the N3 complex was shown to be smaller for categorized real objects compared to “pseudo” (unknown, unreal) object versions (Kroll & Potter, 1984) with matched visual features (Schendan et al., 1998), or other novel, visual structures (Daffner, Mesulam, Scinto, Acar, et al., 2000; Folstein & Van Petten, 2008; Folstein, Van Petten, & Rose, 2008; Gruber & Muller, 2006; Gruber, Trujillo-Barreto, Giabbiconi, Valdes-Sosa, & Muller, 2006; Holcomb & McPherson, 1994; McPherson & Holcomb, 1999; Sehatpour et al., 2006). Amplitude of the N3 increases as the categorization task becomes more specific (Schendan et al., 1998).

    View all citing articles on Scopus
    View full text