Spatiotemporal dynamics of human object recognition processing: An integrated high-density electrical mapping and functional imaging study of “closure” processes
Introduction
One critical and highly adaptive aspect of human object recognition processes is our seemingly effortless ability to identify objects even when only partial, and often very sparse visual information is presented to the observer. The neural processes responsible for filling-in of missing information that enables eventual object recognition under partial viewing conditions (e.g., fog, partial occlusion, camouflage, poor lighting) have come to be referred to as “perceptual closure” processes (e.g., Bartlett, 1916, Snodgrass and Feenan, 1990, Foley et al., 1997). The present study uses the excellent temporal resolution of high-density electrical recordings in combination with the precise spatial localization abilities of functional imaging to investigate the spatiotemporal dynamics of these perceptual closure processes.
In the past decade, both functional imaging and electrophysiological studies have been used separately to investigate the neural processes responsible for basic object recognition. Functional imaging studies have revealed a cluster of regions within the so-called ventral visual stream that play an important role in cue-invariant object recognition (e.g., Malach et al., 1995, Kanwisher et al., 1997, Grill-Spector et al., 1998, Haxby et al., 1999). This cluster of regions is known as the lateral–occipital complex (LOC). The LOC is situated at the lateral and ventral aspects of the occipital lobe, which includes dorsal–lateral–occipital lobe close to area MT/V5 and ventral fusiform (Malach et al., 1995, Lerner et al., 2001). To our knowledge, only a single functional imaging study, which used positron emission tomography (PET), has directly investigated perceptual closure processes, and this study suggested that closure processes were also carried out within regions of the LOC (Gerlach et al., 2002). Two other related studies (Op de et al., 2000, Denys et al., 2004) have also shown activation in the regions of LOC while presenting degraded and scrambled shapes, presumably resulting from the activity of closure processes, although neither study was designed to expressly investigate these processes.
Closure processes have been mostly studied using electrophysiological techniques. In a series of studies from this laboratory (Doniger et al., 2000, Foxe et al., 2001, Doniger et al., 2001a, Doniger et al., 2002), line drawings of common objects, which were systematically fragmented to varying degrees, were used to study the timing of the neural processes responsible for perceptual closure. Our paradigm involved the presentation of sequences of fragmented pictures, whereby we began at very high levels of fragmentation and then incrementally increased the amount of information present in a given image until just enough of the object was present for subjects to “close” the picture (Snodgrass and Corwin, 1988, Doniger et al., 2000). Our studies revealed a robust event-related potential component termed the NCL (for negativity associated with closure) that tracked the neural processes related to perceptual closure. This component was manifest as a relative negativity over bilateral occipito-temporal scalp and occurred in the 230–400 ms timeframe (Doniger et al., 2000), typically peaking at 290–300 ms. The scalp topography of this component suggested that it largely reflected neural activity from the LOC or nearby structures. In addition, based on our previous studies (Doniger et al., 2001a), we also hypothesized the existence of other generators outside the LOC (e.g., frontal regions) involved in closure. While topographic mapping has suggested the involvement of LOC regions in the generation of the NCL, this has not been directly assessed. In the present study, we employed an integrative neuroimaging approach, combining the spatial accuracy of fMRI with the temporal resolution that ERP offers, to study the spatiotemporal dynamics of closure processes. We conducted a modified version of the perceptual closure experiment used in our previous work, while collecting both imaging and electrophysiological measures on the same subjects. This allowed us to coregister hemodynamic responses with electrical dynamics through the application of source analysis methods (Scherg and Picton, 1991, Grave de Peralta et al., 2001). Our data show that the bulk of NCL activity is indeed generated within regions of the LOC. Furthermore, the timing of these processes, which can be derived from the electrical recordings, challenges models of perceptual closure that posit simple feed-forward mechanisms. Rather, our data suggest that closure processes occur relatively late in processing and are likely due to recursive processing within these structures, subsequent to an initial relatively automatic phase of object processing. Furthermore, through the application of source analysis, we were able to elucidate the time course of activity within the various generators involved in closure, allowing us to speak to temporal inter-relationships between these nodes of the object recognition circuit.
Section snippets
Participants
Seven (4 female) neurologically normal paid volunteers, aged 21–35 (mean = 26), participated in the ERP study. Of these, six also participated in the fMRI section of the study in a counterbalanced order. Due to a mild bout of claustrophobia in the scanner, one of the seven ERP subjects had to be excluded from this portion of the study. All subjects provided written informed consent, and the procedures were approved by the Institutional Review Board of the Nathan Kline Institute. All subjects
fMRI data acquisition
A 3 T SMIS (Marconi) system equipped with a head volume coil at the Center for Advanced Brain Imaging (CABI) at the Nathan Kline Institute was used to acquire T2*-weighted echo-planar functional images (EPIs) (TR = 2000, FA = 90, matrix size = 64 × 64, FOV = 224 mm, voxel size = 3.5 mm3) emphasizing the blood oxygenation level dependent (BOLD) response. In each run, 230 volumes (20 contiguous functional slices) were localized in an oblique-coronal orientation that covered the occipital lobes,
fMRI results
The P values of the statistical maps obtained for each condition were Bonferroni corrected for multiple comparisons. Effects were accepted as significant only when P (corrected) < 0.001. Both experimental conditions (unscrambled and scrambled) activated widespread and substantially overlapping cortical networks. To characterize the cortical activation associated with closure, we contrasted the relative activation to “unscrambled” pictures versus “scrambled” pictures. This revealed significantly
Discussion
The high spatial resolution of fMRI was married with the millisecond temporal resolution of ERPs to assess the generators of perceptual closure processes, as indexed by the NCL component of the visual evoked potential. A major role for regions of bilateral LOC and frontal cortex in these processes was confirmed, and a concomitant role for dorsal visual regions was suggested. Here, we discuss the spatiotemporal dynamics of these object recognition processes in the context of the timing and
Conclusion
High-density electrical mapping, source analysis, and functional neuroimaging were used to map the spatiotemporal dynamics of perceptual closure processes. The data strongly support a model in which object recognition is achieved through interplay between the LOC and frontal cortical regions. The current data, together with findings from previous studies, indicate that there are dorsal and ventral contributions to object recognition and that frontal regions are activated for explicit object
Acknowledgments
This work was supported by grants from the National Institute of Mental Health (MH65350 to JJF and MH49334 to DCJ) and a grant from the National Institute on Aging (AG22696 to JJF). SM is supported by a Kirschstein National Research Service Award (NRSA) post-doctoral fellowship (MH068174). The authors would like to express their sincere appreciation to Dr. Vance Zemon for his insights. Deirdre Foxe and Beth Higgins provided invaluable technical help with data collection. We would also like to
References (90)
Anatomic basis of cognitive–emotional interactions in the primate prefrontal cortex
Neurosci. Biobehav. Rev.
(1995)- et al.
Meaningful processing of meaningless stimuli: the influence of perceptual experience on early visual processing of faces
Cognition
(2002) Integrated model of visual processing
Brain Res. Brain Res. Rev.
(2001)- et al.
Parallel versus serial processing: new vistas on the distributed organization of the visual system
Curr. Opin. Neurobiol.
(1995) - et al.
The role of feedback connections in shaping the responses of visual cortical neurons
Prog. Brain Res.
(2001) - et al.
Picture naming by young children: norms for name agreement, familiarity, and visual complexity
J. Exp. Child Psychol.
(1997) - et al.
Visual perceptual learning in human object recognition areas: a repetition priming study using high-density electrical mapping
NeuroImage
(2001) - et al.
Brain activity related to integrative processes in visual object recognition: bottom–up integration and the modulatory influence of stored knowledge
Neuropsychologia
(2002) - et al.
Dual pathways connecting the dorsolateral prefrontal cortex with the hippocampal formation and parahippocampal cortex in the rhesus monkey
Neuroscience
(1984) - et al.
Cue-invariant activation in object-related areas of the human occipital lobe
Neuron
(1998)
The effect of face inversion on activity in human neural systems for face and object perception
Neuron
The cognitive neuroscience of mental imagery
Neuropsychologia
Electric source imaging of human brain functions
Brain Res. Brain Res. Rev.
Neural representation of visual objects: encoding and top–down activation
Curr. Opin. Neurobiol.
Visuo-spatial neural response interactions in early cortical processing during a simple reaction time task: a high-density electrical mapping study
Neuropsychologia
Mapping of scalp potentials by surface spline interpolation
Electroencephalogr. Clin. Neurophysiol.
Processing of one, two or four natural scenes in humans: the limits of parallelism
Vision Res.
Neurophysiological evidence for two processing times for visual object identification
Neuropsychologia
Two bilateral sources of the late AEP as identified by a spatio-temporal dipole model
Electroencephalogr. Clin. Neurophysiol.
Effects of repetition and competition on activity in left prefrontal cortex during word generation
Neuron
Recovering meaning: left prefrontal cortex guides controlled semantic retrieval
Neuron
fMRI retinotopic mapping—Step by step
NeuroImage
A cortical mechanism for triggering top–down facilitation in visual object recognition
J. Cogn. Neurosci.
An experimental study of some problems of perceiving and imagining
Br. J. Psychol.
Recognition-by-components: a theory of human image understanding
Psychol. Rev.
Linear systems analysis of functional magnetic resonance imaging in human V1
J. Neurosci.
Posterior parietal cortex in rhesus monkey: II. Evidence for segregated corticocortical networks linking sensory and limbic areas with the frontal lobe
J. Comp. Neurol.
The anatomical connections of the macaque monkey orbitofrontal cortex. A review
Cereb. Cortex
The processing of visual shape in the cerebral cortex of human and nonhuman primates: a functional magnetic resonance imaging study
J. Neurosci.
Cortical sources of the early components of the visual evoked potential
Hum. Brain Mapp.
Changing plans: a high density electrical mapping study of cortical control
Cereb. Cortex
Activation timecourse of ventral visual stream object-recognition areas: high density electrical mapping of perceptual closure processes
J. Cogn. Neurosci.
Impaired sensory processing as a basis for object-recognition deficits in schizophrenia
Am. J. Psychiatry
Impaired visual object recognition and dorsal/ventral stream interaction in schizophrenia
Arch. Gen. Psychiatry
The face-specific N170 component reflects late stages in the structural encoding of faces
NeuroReport
Dynamic predictions: oscillations and synchrony in top–down processing
Nat. Rev., Neurosci.
Distributed hierarchical processing in the primate cerebral cortex
Cereb. Cortex
Investigations of closure processes: what source-monitoring judgments suggest about what is “closing”
Mem. Cogn.
Flow of activation from V1 to frontal cortex in humans. A framework for defining “early” visual processing
Exp. Brain Res.
Early visual processing deficits in schizophrenia: impaired P1 generation revealed by high-density electrical mapping
Neuroreport
Filling-in in schizophrenia: a high-density electrical mapping and source-analysis investigation of illusory contour processing
Cereb. Cortex
The relationship between global and local changes in PET scans
J. Cereb. Blood Flow Metab.
Statistical parametric maps in functional imaging: a general linear approach
Hum. Brain Mapp.
Feedforward and feedback connections between areas V1 and V2 of the monkey have similar rapid conduction velocities
J. Neurophysiol.
Functional imaging of mirror and inverse reading reveals separate coactivated networks for oculomotion and spatial transformations
NeuroReport
Cited by (113)
ERP and MEG correlates of visual consciousness: The second decade
2020, Consciousness and CognitionMemory influences visual cognition across multiple functional states of interactive cortical dynamics
2019, Psychology of Learning and Motivation - Advances in Research and TheoryCitation Excerpt :The N3 complex is the first ERP to differ between perceptually-matched pictures of objects that have been successfully categorized at the entry level relative to those that were unidentified (Doniger et al., 2000; Schendan & Kutas, 2002), decreasing in amplitude with better categorization and naming success. In one approach to timing of visual knowledge, the N3 complex was shown to be smaller for categorized real objects compared to “pseudo” (unknown, unreal) object versions (Kroll & Potter, 1984) with matched visual features (Schendan et al., 1998), or other novel, visual structures (Daffner, Mesulam, Scinto, Acar, et al., 2000; Folstein & Van Petten, 2008; Folstein, Van Petten, & Rose, 2008; Gruber & Muller, 2006; Gruber, Trujillo-Barreto, Giabbiconi, Valdes-Sosa, & Muller, 2006; Holcomb & McPherson, 1994; McPherson & Holcomb, 1999; Sehatpour et al., 2006). Amplitude of the N3 increases as the categorization task becomes more specific (Schendan et al., 1998).
Brain mechanisms for perceiving illusory lines in humans
2018, NeuroImage