Abstract
We examined the responses of neurons in posterior parietal area 7a to salient stimuli appearing alone or within multiple-stimulus displays in monkeys trained only to maintain fixation. Discharges in a population of parietal neurons encoded the location of the salient stimulus, although the latter had no task significance for the monkey. Neuronal selectivity for the location of the salient stimulus depended solely on its intrinsic difference from the background elements in the array and not on the color of the stimulus per se. These results were similar to those reported in monkeys trained to actively locate a salient stimulus in a multiple-stimulus display. A lower percentage of neurons with significant selectivity for the salient stimulus was observed in the fixation-only animals. These neurons took longer for the selective responses to emerge and showed a lower power of discrimination. The findings suggest that the posterior parietal cortex automatically detects and encodes the location of salient stimuli even when they are unrelated to the behavioral task.
Introduction
Vertebrate animals are unable to process simultaneously all information available to their sensory receptors and must focus on one or a few stimuli at a time, a process known as selective attention. The posterior parietal cortex, which is the end stage of the dorsal visual pathway, is thought to play an important role in attention. Parietal lesions in humans produce severe deficits in the ability to detect and orient visual attention (Critchley, 1966; Heilman, 1979; Posner and Cohen, 1984), and imaging studies demonstrate parietal activation in tasks that require focusing and shifting attention (Corbetta and Shulman, 2002). Orienting attention is not a unitary process. The willful allocation of attention (e.g., when one is actively scanning a visual scene for a known stimulus) is thought to be a process different from the purely stimulus-driven capture of attention (e.g., when a stimulus “pops out” in a display by virtue of its difference from the background). The two modes are referred to as “top-down” and “bottom-up” orienting of attention, respectively (Itti and Koch, 2001). Human imaging studies suggest that the posterior parietal cortex is engaged in both of these processes (Corbetta et al., 2000).
Neurophysiological studies in monkeys have discovered powerful modulation during the voluntary allocation of attention in posterior parietal area 7a and the lateral intraparietal area (LIP) but have provided little evidence for a role in bottom-up orienting other than the detection of a single stimulus (Yin and Mountcastle, 1977; Robinson et al., 1978; Bushnell et al., 1981; Steinmetz et al., 1994; Gottlieb et al., 1998; Bisley and Goldberg, 2003). This has reinforced an alternative proposal, which is that bottom-up attention may be allocated purely by the competition of visual stimuli within the ventral visual stream (Desimone and Duncan, 1995).
Previous studies assessed responses in area 7a of the posterior parietal cortex to displays containing stimuli that pop out from their background (Constantinidis and Steinmetz, 2001a,b). The task required monkeys to identify a salient stimulus in a multiple-stimulus display and to release a lever when a subsequent stimulus appeared at the same location. Area 7a neurons responded preferentially to a salient stimulus and encoded its spatial location. Although the location or identity of the salient stimulus was not known to the animal before the appearance of the visual display, the animals were trained to actively search for salient stimuli. Neuronal responses have been shown to be modulated after training in tasks that require color discrimination in area LIP and the frontal eye fields (Bichot et al., 1996; Toth and Assad, 2002). It is possible that the preferential responses to a salient stimulus observed in the previous studies was the effect of willful attention allocation dictated and reinforced by the task contingencies rather than a bottom-up effect of the salient stimulus. The current experiments sought to distinguish between the two possibilities by recording in monkeys trained only to fixate. We tested whether area 7a would encode the location of the salient stimulus when the stimulus itself was irrelevant to the animal's behavior and its salience was defined purely by bottom-up factors.
Materials and Methods
Two male rhesus monkeys (Macaca mulatta) weighing ∼5 kg were used in the experiments. All animal experimentation was performed in compliance with the guidelines set forth by the National Institutes of Health as approved by the Johns Hopkins University Animal Care and Use Committee.
Behavioral task. The animals sat in a primate chair with their heads fixed and viewing a computer monitor, placed 45 cm away, under dim ambient illumination. The animals were trained to pull back a lever at the onset of a 0.5° target light and to maintain eye position within a ±1° window around this target. Eye position was monitored by means of an infrared eye position scanning system (model RK-716; ISCAN, Burlington, MA) that was capable of a 0.3° resolution in eye position for central fixation. Eye monitoring of an initial set of recordings in one monkey was performed with a sclera eye coil system, as described previously (Constantinidis and Steinmetz, 2001a). Single- or multiple-stimulus displays were presented on the screen for 500 msec while the animal fixated. These stimuli were of no behavioral significance for the monkeys. In ∼10% of the trials, selected randomly during the experiment, no stimulus other than the fixation target was presented. The animals were required to release the lever within 500 msec of the extinction of the fixation target to receive a liquid reward. The animals were trained only in the fixation task.
The stimuli were 4° green or red squares of equal luminance, presented at any of the nine locations of a 3 × 3, 25° grid, centered over the fixation point. These stimuli were identical to those used in our previous study (Constantinidis and Steinmetz, 2001a). Either single stimuli or arrays of nine stimuli, one of which (salient stimulus) differed in color from the rest, were presented. We referred to the background stimuli as distractors in our previous study, because the monkey was trained to ignore them to perform the match-to-sample task. We use the same term in this paper because the stimulus displays are identical, but it should be clear that neither salient stimuli nor distractors had any behavioral significance in the fixation task. In addition to arrays with one salient stimulus among eight distractors, displays of nine identical stimuli were presented as a control condition.
Neurophysiology. A 1-cm-diameter craniotomy was placed over the posterior parietal gyrus as described previously (Constantinidis and Steinmetz, 2001a), after the animals were proficient in the fixation task. Single-unit recordings with an array of seven, independently moving electrodes (Mountcastle et al., 1991) were performed subsequently. Electrodes consisted of quartz glass filaments with metal cores of tungsten-platinum alloy (impedance, 1-4 MΩ). The electrical signal from each electrode was bandpass filtered between 400 and 10 kHz and displayed on an oscilloscope. A differential-amplitude window discriminator was used to isolate action potentials of single neurons, the times of which were then digitally stored at a 0.1 msec resolution.
Our system allowed us to precisely position electrodes in coordinates relative to the center of the recording chamber. On the last day of recordings, the electrodes were submerged in fluorescent dyes before being advanced into the cortex to create a set of reference tracks to assist with the penetration reconstruction (DiCarlo et al., 1996). Animals were given an overdose of pentobarbital and transcardially perfused. Marker pins were also inserted at the center of each craniotomy. The brain was photographed and sectioned. Electrode penetrations were localized relative to the marker pin and fluorescent-marked tracks based on our log of electrode coordinates. All recordings analyzed were confined between the intraparietal and superior temporal sulcus, and electrodes were not advanced >2-2.5 mm from the top of the cortex.
Data analysis. Neuronal responses were analyzed during the 500 msec interval of stimulus presentation. A neuron was included in the analysis if its discharge rate during presentation of any stimulus was significantly elevated above the baseline period, which was defined as the 500 msec before the fixation point onset (paired t test; p < 0.05). The spatial tuning of neurons was assessed by comparing their discharge rate during the presentation of single stimuli at the nine grid locations. Neurons with significantly different responses to the nine conditions (ANOVA test; p < 0.05), were included in further analysis. The best (most responsive) and worst (least responsive) locations were identified for each neuron based on its responses to single stimuli.
We tested whether a neuron could discriminate between salient stimuli and distractors by comparing its responses to arrays with the salient stimulus appearing at each of the nine grid locations. Neurons with significantly different responses to the nine array conditions (ANOVA test; p < 0.05) were deemed to be selective for the salient stimulus. Neuronal responses were also analyzed to compare the spatial tuning for the location of the salient stimulus in the multiple-stimulus array with that for single-stimulus presentations. These were compared by computing the correlation coefficient between the neuronal responses to the stimulus presented at each of the nine locations by itself and those observed for the salient stimuli at the same locations but among distractors. We also quantified the ability of neurons to discriminate between a salient stimulus and a distractor in its receptive field using a receiver operating characteristic (ROC) analysis, as described previously (Constantinidis and Steinmetz, 2001a). The ROC curve indicates the probability that an ideal observer can discriminate between the two stimuli based on the discharge rate of the neuron.
The time courses of neuronal responses to single stimuli and multiple-stimulus arrays were examined by constructing population peristimulus time histograms (PSTHs). We compared responses to different types of stimuli at the spatial locations that elicited the best and worst single-stimulus responses. The spike trains of individual trials were converted to spike density functions by convolving each spike with a Gaussian pulse of 10 msec SD (Richmond et al., 1987; Schall and Hanes, 1993). Responses from multiple neurons were averaged together to produce a population PSTH. We estimated the time when responses to two different stimuli diverged from each other as the first of at least two consecutive 10 msec bins with a significant difference in the mean discharge rate (paired t test; p < 0.05).
Results
Database
Neuronal activity was recorded using an array of microelectrodes from area 7a of the posterior parietal cortex of two monkeys trained to hold back a lever and maintain fixation on a central spot while visual stimuli were flashed on a screen. The activity of 262 visually responsive neurons was recorded. Of those, 170 neurons displayed significant selectivity for the spatial location of a single stimulus presented at nine different spatial locations (one-way ANOVA; p < 0.05). Our analysis was based on this sample of 170 neurons. We identified for each neuron the best and worst stimulus location based on its responses to single stimuli. A contralateral bias was evident in our sample: 46.5% responded maximally to a contralateral stimulus, 41.2% responded to a stimulus along the central meridian, and 12.4% responded to an ipsilateral stimulus. The receptive fields of these neurons were generally large; on average, visual responses significantly elevated over the baseline were elicited by 3.1 of our single stimuli that were spaced 25° apart.
Behavioral performance was consistent during the presentation of all types of stimuli. Three categories of errors were observed: early releases of the lever, while the fixation point was still on; late releases of the lever, longer than the 500 msec reaction window that we imposed; and breaks in fixation. The latter were most often caused by blinking. Average performance during trials involving presentation of single stimuli and arrays was 79.2 and 79.7%, respectively (97.8 and 97.9%, excluding fixation breaks). Only correct responses were analyzed for all discharge rates reported here.
Responses to arrays of stimuli
The selectivity of the response to the salient stimuli in multiple-stimulus arrays was tested by comparing responses to arrays of stimuli consisting of nine squares arranged in a 3 × 3 grid, one of which (salient stimulus) differed from the others (distractors) in color. A total of 53 neurons (31%) responded differentially, depending on the location of the salient stimulus (one-way ANOVA; p < 0.05). Figure 1 illustrates a neuron responding vigorously to a salient stimulus but only minimally to a distractor in its receptive field. The responses of the neuron to the array varied systematically, depending on the location of the salient stimulus within the receptive field of the neuron (Fig. 2). The neuron exhibited spatial tuning for the location of the salient stimulus that closely mirrored its tuning for the single stimuli presented at the same locations.
The responses to both red and green distractor arrays were tested to ensure that response selectivity could not be accounted for by simple preference for the color of the salient stimulus. Only 26 of 170 neurons in our entire sample (15%) and 6 of 53 (11%) neurons with significant selectivity for the location of the salient stimulus in the array displayed a significant preference for one of the two colors. The result indicates that a population of area 7a neurons selectively represents the location of a salient stimulus in a multiple-stimulus scene, regardless of stimulus color.
Eye position during the presentation of visual stimuli was analyzed trial by trial to verify that any differential responses were not caused by systematic deviations in the monkeys' eye position within the eye movement window. No systematic deviation of averaged eye position as a function of the location of the salient stimulus location was found for either monkey (one-way ANOVA test; p > 0.9). Given that the size of area 7a receptive fields (encompassing 3.1 stimuli on average) was nearly two orders of magnitude larger than our fixation window, it is unlikely that systematic eye displacements within the fixation window would have any significant effect on the discharge rate. Eye position in the 500 msec interval after the offset of the fixation point (when animals were free to move their eyes) was also examined to test the possibility that monkeys planned eye movements toward the salient stimuli during the stimulus presentation but postponed these movements until the termination of fixation control. No significant difference in the monkeys' eye position was found after the fixation point was turned off that depended on the location of the salient stimulus (one-way ANOVA test; p > 0.4).
Neuronal responses to multiple-stimulus arrays were generally lower than to a single stimulus in the receptive field (Fig. 3, left). The mean response rate of the entire population of neurons was 22.1 ± 1.1 spikes per second for a single stimulus compared with 13.6 ± 1.1 spikes per second for an array with the salient stimulus appearing in the receptive field. Responses were significantly higher (paired t test; p < 0.001) to arrays with the salient stimulus appearing in the receptive field than responses to arrays with only distractors in the receptive field which evoked 9.9 ± 0.8 spikes per second on average. These results indicate a preferential response to salient stimuli appearing in the receptive field with an overall suppressive effect of multiple stimulus displays across the population of neurons.
The time course of responses to salient stimuli were also examined (Fig. 4). Responses to the single stimulus rose with the fastest time course and continued to rise for 200 msec before reaching a peak and starting to decline. The responses to arrays with the salient stimulus in and out of the receptive field followed similar time courses. An initial burst of activity from 50 to 150 msec after stimulus onset was followed by a slight decline. The two response curves began to deviate after 150 msec, and their difference reached statistical significance 180 msec after stimulus onset (paired t test; p < 0.05). Responses to arrays with the salient stimulus inside the receptive field started to rise again from that point, whereas responses to arrays with the salient stimulus outside of the receptive field continued their decline. Response to arrays with identical elements and no salient stimulus (Fig. 4B) followed an intermediate path, declining with a slower time course.
Comparison of responses during passive fixation and during active attention
In many ways, the results of the present experiments were similar to the results reported for monkeys trained to localize the salient stimulus and later recall its spatial location (Constantinidis and Steinmetz, 2001a). A population of area 7a neurons in the previous experiments was found to respond differentially to arrays of stimuli, depending on the location of the salient stimulus. Responses to an array of stimuli with the salient stimulus appearing inside the receptive field in the previous study were also intermediate between the responses to a single stimulus and to an array of stimuli with the salient stimulus appearing outside of the receptive field (Fig. 3, right).
The differences in responses between naive and trained animals, however, were also substantial. Naive animals exhibited a lower proportion of neurons selective for the array (31 vs 68%). Even among neurons with significant selectivity, the spatial tuning of responses to the salient stimulus was more degraded under conditions of passive fixation (Fig. 5A), as judged by a lower correlation coefficient between single-stimulus and array responses as the location of the salient stimulus varied (mean r = 0.51 vs 0.67). The solid and dotted lines in Figure 5 represent the means for the current and previous study, respectively. Similarly, the responses of neurons exhibiting significant selectivity for the location of the salient stimulus under passive fixation displayed lower power of discrimination between a salient stimulus and a distractor appearing in their receptive fields (Fig. 5B), as judged by the area under the ROC curve (mean area under ROC curve, 0.77 vs 0.86). Finally, selectivity for the salient stimulus emerged later in the responses of the untrained animal; the first significant bin in the population PSTH averaging responses from all neurons appeared at 180 msec after the onset of the display compared with 120 msec in animals trained to detect the location of the salient stimulus.
Discussion
The responses of neurons in posterior parietal area 7a were shown to represent the location of a salient stimulus in a multiple-stimulus array in animals trained only to maintain fixation. Neuronal responses discriminated between salient stimuli and distractors, although the stimuli were positioned on a 25° grid that is much less dense than what has been used in previous psychophysical and neurophysiological experiments. A similar selectivity for the location of a salient stimulus has been observed previously in the frontal eye fields of monkeys instructed to with-hold any responses to the stimuli (Thompson et al., 1997; Schall, 2004). In the present experiments, the salient stimuli were irrelevant for the execution of the task; the animals were not trained in behaviors guided by the color stimuli shown to them. These results demonstrate that area 7a automatically detects and encodes the location of salient stimuli defined entirely by bottom-up mechanisms.
Selectivity for the stimulus that attracts attention
Differential responses to arrays of multiple stimuli depended on the saliency of the stimulus appearing in or out of the receptive field rather than its color. A color preference among area 7a neurons appeared unlikely from the outset considering that posterior parietal neurons show only weak color selectivity, even after animals have been trained to perform judgments based on stimulus color (Constantinidis and Steinmetz, 2001a; Toth and Assad, 2002). Indeed, only a small fraction of all area 7a neurons in the current study exhibited significant color selectivity.
Similarly, the selectivity observed could not be accounted for by deviations in eye position during the stimulus presentation or by eye movements at the end of the trial. These animals were never trained to perform visually guided saccades. Analysis of eye position during stimulus presentation and after the end of the trial revealed no systematic eye movements that could be related to the location of the salient stimulus in the array.
The responses of area 7a neurons were weaker for multiple-stimulus displays than those observed for a single stimulus, suggesting an overall suppressive effect of multiple stimuli in and around their receptive fields. Neuronal responses were highest when salient stimuli appeared in the receptive field, lowest when the salient stimulus appeared out of the receptive field, and intermediate for displays with nine identical elements and no salient stimulus. The pattern of responses that we observed is consistent with neuronal activity reflecting the locus of attention. In the case of the identical-stimulus display, attention may either continue to be focused in the fixation point or randomly wander to one of the nine stimuli, which may fall in or out of the receptive field in different trials, or attention could be spread over the whole set of stimuli.
Comparison of passive fixation with active attention
Neuronal responses to multiple-stimulus arrays that we report here were similar to the results of previous experiments in animals trained to identify and localize a salient stimulus, in area 7a (Constantinidis and Steinmetz, 2001a), the frontal eye fields (Schall and Hanes, 1993; Schall et al., 1995), and the superior colliculus (Basso and Wurtz, 1997). In the previous area 7a experiments, a population of neurons responded differentially to arrays of multiple stimuli and signaled the spatial location of a salient stimulus in them. Responses to multiple-stimulus arrays were suppressed compared with single stimuli; however, responses to the arrays were greater when the salient stimulus appeared inside the receptive field (Fig. 3). These results suggest that the selectivity that we observed in the previous experiments was not purely the result of training the animals to actively search and identify salient stimuli in an array. The current results demonstrate that area 7a automatically detects and signals the spatial location of salient stimuli. Our findings are consistent with psychophysical results showing bottom-up influences both in simple displays and in complex, natural scenes (Treisman and Gelade, 1980; Parkhurst et al., 2002).
There were also some important differences between the current and previous studies. In the active-attention experiment, a higher percentage of neurons displayed significant selectivity for the location of the salient stimulus in the array, and the responses of these selective neurons displayed higher correlation between single and multiple stimulus arrays as well as higher discrimination power (Fig. 5). Perhaps the most dramatic effect of training involved the magnitude of neuronal responses to salient stimuli in arrays, which approximated the responses to single stimuli (Constantinidis and Steinmetz, 2001a). Finally, the selectivity for the salient stimulus emerged later in time (Fig. 4). Training monkeys to discriminate between stimuli based on color has been shown to accelerate the emergence of selective neural signals in the frontal eye fields by up to 80 msec (Bichot et al., 1996); this is a similar effect to that reported here.
Role of posterior parietal cortex in bottom-up attention processes
Selective attention serves to bring stimuli of interest in the fore-ground of neural processing so that they can be identified and scrutinized. These latter functions are likely performed from neurons in the ventral visual pathway, including areas V4, TEO, and TE, which are known to be selective for stimulus features important for object recognition (Riesenhuber and Poggio, 2002). Indeed, the responses of cortical neurons in the ventral visual pathway are greatly modulated, depending on whether a stimulus appearing in their receptive field is currently attended or not (Moran and Desimone, 1985; Chelazzi et al., 1993; Motter, 1994; Connor et al., 1997; Luck et al., 1997; Reynolds et al., 1999). Less is known, however, about how neural mechanisms direct attention to stimuli in the first place.
One hypothesis is that the brain maintains a saliency map that directs attention to regions of interest (Koch and Ullman, 1985; Niebur and Koch, 1996). The current results suggest that the posterior parietal cortex as well as other brain structures with which it is interconnected, such as the dorsolateral prefrontal cortex, could provide such a saliency map. These brain structures in turn could affect the processing of information by neurons in the ventral visual stream. Although the neuronal pathways through which they might exert such action are still unclear, the gating of visual information by the connections through the pulvinar is one possibility. Prefrontal and parietal cortices do appear capable of controlling neuronal processing, as evidenced from their role in top-down attention. Artificial microstimulation of area LIP and the frontal eye fields can enhance the allocation of attention and improve behavioral performance in detection tasks (Moore and Fallah, 2001; Cutrell and Marrocco, 2002) and directly enhance processing of neurons in area V4 (Moore and Armstrong, 2003). Conversely, parietal inactivation impedes the detection of a visual stimulus (Wardak et al., 2004), although the latter is presumably still represented in the activity of ventral stream neurons.
An alternative view of bottom-up attention mechanisms posits that the focus of attention can be directed as the result of a competition process entirely within the ventral visual stream (Desimone and Duncan, 1995). A stimulus standing out from its background is preferentially represented in the activity of neurons at all stages of the visual system, starting as early as V1 (Knierim and van Essen, 1992; Lamme, 1995; Hegde and Felleman, 2003). The competition between neural representations therefore may lead to the survival of only the most salient stimulus in the scene, which is routed to higher brain centers such as inferior temporal cortex for object recognition (Niebur and Koch, 1994). If that were true, then the selectivity for the location of the salient stimulus in the array that we observed in area 7a might be an epiphenomenon of the selection process, with no further significance for the allocation of attention. Although such a conclusion is possible, it could be argued as easily that these differential neuronal responses to particular stimuli in fact constitute attention (Schall, 2004).
A third alternative is that selectivity for the salient stimulus could emerge independently in the ventral and dorsal visual streams and serve different purposes, namely to define perception or guide action (Goodale and Milner, 1992). Knowing whether parietal selectivity precedes or follows completion of the selection process in the ventral pathway would be important in resolving this issue. Although responses to target stimuli that an animal is actively searching for have been studied in the ventral stream (Chelazzi et al., 1993; Motter, 1994; Mazer and Gallant, 2003), data regarding saliency defined entirely by bottom-up processes are not yet available.
Lesion studies argue most forcefully for a direct role of the posterior parietal cortex in guiding bottom-up attention to influence perception. Damage of the posterior parietal cortex results in profound deficits in the ability to direct attention, correctly perceive feature conjunctions, and filter distractors in the field of view, even when patients are asked to perform discriminations that otherwise could be mediated entirely by the ventral visual stream (Friedman-Hill et al., 1995, 2003). These results suggest that the posterior parietal cortex automatically provides a map of visual salience, in the absence of which attention cannot be properly allocated.
Footnotes
This work was supported by National Institutes of Health Grant MH55029 and through a subcontract of Johns Hopkins University with C.C. We thank Jeff Schall and Ernst Niebur for their valuable comments on this manuscript.
M.A.S. contributed to this article in his private capacity; no official support or endorsement by the National Institutes of Health, the Department of Health and Human Services, or the federal government is intended or should be inferred.
Correspondence should be addressed to Dr. Christos Constantinidis, Department of Neurobiology and Anatomy, Wake Forest University School of Medicine, Medical Center Boulevard, Winston-Salem, NC 27157-1010. E-mail: cconstan{at}wfubmc.edu.
M. A. Steinmetz's present address: Center for Scientific Review, National Institutes of Health, 6701 Rockledge Drive, Bethesda, MD 20892.
Copyright © 2005 Society for Neuroscience 0270-6474/05/250233-06$15.00/0