Abstract
It is well established that attention modulates visual processing in extrastriate cortex. However, the underlying neural mechanisms are unknown. A consistent observation is that attention has its greatest impact on neuronal responses when multiple stimuli appear together within a cell’s receptive field. One way to explain this is to assume that multiple stimuli activate competing populations of neurons and that attention biases this competition in favor of the attended stimulus. In the absence of competing stimuli, there is no competition to be resolved. Accordingly, attention has a more limited effect on the neuronal response to a single stimulus. To test this interpretation, we measured the responses of neurons in macaque areas V2 and V4 using a behavioral paradigm that allowed us to isolate automatic sensory processing mechanisms from attentional effects. First, we measured each cell’s response to a single stimulus presented alone inside the receptive field or paired with a second receptive field stimulus, while the monkey attended to a location outside the receptive field. Adding the second stimulus typically caused the neuron’s response to move toward the response that was elicited by the second stimulus alone. Then, we directed the monkey’s attention to one element of the pair. This drove the neuron’s response toward the response elicited when the attended stimulus appeared alone. These findings are consistent with the idea that attention biases competitive interactions among neurons, causing them to respond primarily to the attended stimulus. A quantitative neural model of attention is proposed to account for these results.
Experiments on attention in extrastriate visual cortex can be divided into two types. Those that have used a single receptive field stimulus have found that attention can increase the magnitude of neuronal responses (Bushnell et al., 1981; Mountcastle et al., 1987; Spitzer et al., 1988; Treue and Maunsell, 1996; Gottlieb et al., 1998). In contrast, studies using multiple receptive field stimuli have found that the effect of attention depends on the neuron’s stimulus selectivity. If two stimuli appear together within a neuron’s receptive field, the response is smaller when attention is directed to the poorer stimulus relative to when attention is directed to the preferred stimulus (Moran and Desimone, 1985; Treue and Maunsell, 1996; Luck et al., 1997).
The purpose of the present experiments was to test a model that can unify these two streams of research by explaining both types of results as arising from a common neural mechanism. This “biased-competition model” (Desimone and Duncan, 1995) depends on two assumptions. (1) When multiple stimuli appear together, they activate populations of neurons that automatically compete with one another. (2) Attending to a stimulus biases this competition in favor of neurons that respond to the attended stimulus. We tested these hypotheses by recording neuronal responses in areas V2 and V4, where attention has been studied previously using both single and multiple receptive field stimuli.
We tested the first hypothesis of the model in Experiment 1. We measured neuronal responses to two stimuli, both preferred and nonpreferred, within the receptive field when the monkey was not required to attend to either stimulus. The stimuli were presented one at a time and also together as a pair. If the first hypothesis of the model is true, then the response to a preferred stimulus should be suppressed by the nonpreferred stimulus, because of the action of the competing neuronal population activated by that stimulus.
We tested the second hypothesis of the model in Experiment 2. As in Experiment 1, we measured neuronal responses to two receptive field stimuli, presented individually and as a pair. We then measured the response to the pair while the monkey attended to each individual stimulus. If the second hypothesis of the model is true, then this should cause the pair response to move toward the response that was elicited when the attended stimulus appeared alone.
A simple three-parameter implementation of the biased-competition model demonstrates that it can satisfy the two linear constraints that are imposed by the results of these experiments. Using the parameters derived to fit these data, the model also fits previously published data on response modulation when attention is directed to a single receptive field stimulus. We conclude by describing two easily tested predictions of the model.
MATERIALS AND METHODS
Surgery
Many of the details of the recording techniques have been described previously (Miller et al., 1993a). Briefly, three adult male rhesus monkeys (Macaca mulatta) were surgically implanted with a head post, a scleral eye coil, and recording chambers. Surgery was conducted under aseptic conditions with isofluorane anesthesia, and antibiotics and analgesics were administered postoperatively. Preoperative magnetic resonance imaging (MRI) was used to identify the stereotaxic coordinates of V2 and V4. V4 recording chambers were placed over the prelunate gyrus. Additional plastic recording chambers were used for V2 recordings, centered 15 mm lateral and 15 mm dorsal to the occipital pole. The skull remained intact during the initial surgery, and small holes (∼3 mm in diameter) were later drilled within the recording chambers under ketamine anesthesia and xylazine analgesic to expose the dura for electrode penetrations.
Confirmation of recording sites
At the beginning of the study, several penetrations were made in each chamber to ensure that the electrode was in the appropriate visual area. This was determined by assessing receptive field sizes, topographic organization, and feature preferences at each site. All implants were nonferromagnetic (plastic recording chambers, titanium screws, and brass head posts), so it was possible to verify the locations of our recording sites using additional MRI scans. After our experimental data were collected, we rescanned two monkeys with a marker electrode (sharpened tungsten microelectrode; Frederick Haer & Co., Brunswick, ME) inserted in each recording chamber at the coordinates used during recording. We used a plastic cylinder that fit snugly inside the recording well to hold the marker electrode in place during the scan. At each end of the cylinder was a grid that was perforated with small holes, spaced 1 mm apart (Christ Instruments, Damascus, MD). Each marker electrode was lowered through the grids and into the brain to a depth of ∼2 cm beneath the dura using the same micropositioner and x–y stage that had been used during recording. Before the micropositioner andx–y stage were removed, a drop of glue was applied to hold the marker electrode in the grid. After the micropositioner and x–y stage were removed, the end of the electrode that was protruding from the recording well was then cut, and a plastic cap was placed over the recording chamber during the scan.
These marker electrodes were clearly visible in each scan. The positions of these markers, the positions of electrode tracks made during recording, and the positions of the holes in the skull beneath each recording chamber all verified that our recording sites were appropriately located in areas V2 and V4. The third monkey, from which eight neurons were recorded, has not been rescanned.
Recording technique
Recordings were obtained from a tungsten microelectrode that was controlled by a hydraulic microdrive. We made no effort to select neurons from a particular layer of cortex. We recorded from the first neurons encountered that could be clearly isolated and had sufficiently large receptive fields (see Receptive field mapping). The portion of area V4 where we recorded was directly beneath the recording chamber, so the first cells encountered were those in the superficial cortical layers. Neurons in area V2 were recorded by passing the electrode through V1 on the opercular surface, through the underlying white matter, and into the portion of V2 that lies on the posterior bank of the lunate sulcus. Therefore, the first cells encountered in V2 recordings were typically in the deep layers. Thus, there may be a bias toward deeper recordings in V2 and more superficial recordings in V4.
In most cases, two neurons could be recorded simultaneously and differentiated on the basis of the size and shape of the spike waveform, and an on-line spike-sorting computer was used to classify these spikes by means of a template-matching procedure. Although this system allowed the concurrent recording of two neurons, spikes arising from both neurons simultaneously (within a 1 msec interval) could not be detected.
Stimuli
The stimuli used throughout all experiments in both cortical areas were selected from a set of 16 stimuli composed of all combinations of four oriented bars (0, 45, 90, and 135°) presented in four colors (red, blue, green, and yellow). The bars were 0.25° of visual arc wide by 1° in length. The colors were chosen to be photometrically equiluminant, with a luminance of 8.60 cd/m2, presented against a gray background of luminance 0.65 cd/m2. In Experiment 1, all stimuli were 250 msec in duration. In Experiment 2, stimulus duration ranged from 50 to 250 msec.
Receptive field mapping
A manually (computer mouse) controlled flashing bar stimulus was used to establish the outer boundaries of the multiunit receptive field. After cells were isolated, this same flashing bar stimulus was used to estimate the position in the visual field where stimuli would generate the strongest response (the “hot spot” of the cell). Each of the 16 possible stimuli described above was then presented at this position, and the stimulus that gave the greatest response was identified. This stimulus was then repeatedly presented in a random sequence at 11 positions falling at regular intervals along an arc of equal eccentricity centered on the hot spot and extending bilaterally into the surround of the receptive field. The responses of the cell at these 11 positions constituted a one-dimensional profile of the receptive field. Two of these 11 positions were used throughout the rest of the recording session. These two positions were selected to give approximately equivalent responses to the mapping stimulus and to be clearly inside the receptive field. To achieve these two goals, we found that it was necessary to place stimuli closer together for neurons with smaller receptive fields. Therefore, the stimuli used to test neurons in area V2 typically were closer together than were those used to test neurons in area V4. Because most receptive fields in both V2 and V4 were approximately symmetric around the hot spot, the two positions were typically approximately symmetric around the hot spot. For neurons with small receptive fields (including most V2 neurons), it was usually the case that the 11 positions used in this automatic mapping procedure were spaced closely together. We were careful to choose positions that were far enough apart to avoid overlap between the stimulus pairs that would appear together in the main experiment. If a receptive field was too small to fit two stimuli easily at equally potent positions inside the receptive field, the neuron was excluded, and we attempted to isolate a different neuron.
Experiment 1: characterization of V4 neurons’ responses to stimulus pairs
Stimulus configurations and experimental procedures.The stimulus configuration used in Experiment 1 is shown in Figure1A. The monkey was rewarded for passively fixating a fixation spot at the center of the computer screen while stimuli were presented within the receptive field of the neuron under study. Stimuli could appear at one of two possible locations within the receptive field, which were selected as described above. At the beginning of a recording session, the stimulus that would appear at position one, designated the reference stimulus, was chosen from the set of 16 possible stimuli (four orientations × four colors) described above. The identity of the reference stimulus was fixed throughout the recording session. On each trial, the stimulus that would appear at position two, designated a probe stimulus, was selected at random from the same set of 16 possible stimuli. We sought to test neuronal responses to stimulus pairs across the full spectrum of possible reference–probe selectivity values. Therefore, the reference stimulus was chosen sometimes to be the stimulus (among the set of 16 possible stimuli) eliciting the largest response, sometimes to be that eliciting the smallest response, and sometimes to be that eliciting a response that fell between the largest and the smallest response. There is no reason to believe that the best stimulus of the 16 was the “optimal” stimulus for the cell.
On any given trial, stimuli appeared in one of three possible configurations. (1) the reference stimulus appeared at position one, 2) a probe stimulus appeared at position two, or 3) the reference stimulus appeared at position one together with a probe stimulus at position two. Whenever a trial included a probe stimulus (i.e., configurations 2 or 3), the identity of the probe stimulus was selected for that trial at random from the set of 16 possible stimuli. A recording session consisted of 540 complete trials. These were composed of 60 trials in which the reference stimulus appeared alone, 240 trials in which each of the 16 possible probes appeared alone (15 repetitions of each probe), and 240 trials in which each of the 16 possible probes appeared with the reference stimulus (15 repetitions of each pair).
Data analysis. For each cell, we computed the average firing rate over a 250 msec window (stimulus duration) beginning 70 msec after stimulus onset (typical V4 neuron response onset). We chose this time window to cover the neuron’s full response period. To verify that our results are not an artifact of this particular time window, we have repeated this analysis using time windows that included only the first 100 msec of response, the second 100 msec of response, and the time window that was used in Experiment 2. All of these analyses yielded qualitatively similar results.
Averages were computed in three stimulus configurations: (1) the reference stimulus appearing alone, (2) each of the 16 probe stimuli appearing alone, and (3) each of the 16 resulting reference–probe pairs. We normalized all responses by dividing by the highest firing rate observed within that time window in any stimulus condition. We then computed the difference between the normalized response of the cell to the reference stimulus (REF) and each probe (PROBEi). This yielded 16 selectivity values, denoted SEi, for each cell: SEi = PROBEi − REF. This selectivity index can range from −1 to +1, with negative values indicating that the reference stimulus elicited the stronger response, a value of 0 indicating identical responses to reference and probe, and positive values indicating that the probe stimulus elicited the stronger response.
We then computed an index that quantified the change in firing rate that resulted from adding the probe stimulus to the reference stimulus. This sensory interaction index (SIi) is the difference between the normalized response to the reference stimulus (REF) and the normalized response to the pair composed of the reference stimulus and the ith probe stimulus (PAIRi): SIi = PAIRi − REF. Like the selectivity index, the sensory interaction index can take on values from −1 to +1. Negative values indicate that the response to the pair was smaller than the response to the reference stimulus (i.e., adding the probe stimulus suppressed the neuronal response). A value of 0 indicates that adding the probe stimulus had no effect on the neuron’s response. Positive values indicate that adding the probe increased the neuron’s response. For each neuron, we quantified the relationship between selectivity and sensory interactions by performing a linear regression on the 16 selectivity and sensory interaction indices. A criterion level ofp < 0.05 was used in all statistical analyses.
These indices were computed to test the first assumption of the biased-competition model. According to the model, when two stimuli appear within a neuron’s receptive field, the pair response is predicted to fall between the responses elicited when the stimuli appear individually. Thus, the response to a preferred reference stimulus (SE < 0) is predicted to be suppressed by the addition of a poor probe stimulus within the receptive field (SI < 0). Likewise, the response to a poor reference stimulus (SE > 0) should be increased by the addition of a preferred probe stimulus within the receptive field (SI > 0). Finally, if a reference stimulus and a probe stimulus generate equivalent responses (SE = 0), then the pair response is predicted to be equal to either individual stimulus response (SI = 0). Thus, according to the biased-competition hypothesis, the relationship between sensory interactions and selectivity should be positive and should pass through the origin (SE = SI = 0).
The possibility that stimulus onset may have captured attention. One possible concern is that the appearance of a stimulus can capture attention, even if behaviorally irrelevant. If this occurred for the stimuli used in the present experiment, it might have caused a change in neuronal response. Although we cannot eliminate this possibility, we do not believe that this presents a serious problem. The empirical relationship found between selectivity and sensory interactions under passive fixation in Experiment 1 was replicated in Experiment 2, in which the monkey actively attended to stimuli presented simultaneously outside the receptive field. Thus, any exogenous attention effects in Experiment 1 were evidently small in magnitude, possibly because the monkey learned to ignore the peripheral stimuli after thousands of stimulus presentations. Second, our conclusions do not depend on the absolute magnitude of neuronal responses. Instead, they depend on a comparison of responses to the different stimuli. Provided that attention was not directed preferentially to a particular stimulus, any effect of attention would not be expected to influence the observed dependence of sensory interactions on selectivity.
Experiment 2: characterization of the effect of attention on sensory interactions in areas V2 and V4
Stimulus configurations and experimental procedures.The attention task, which is similar to a task described previously (Luck et al., 1997), is illustrated in Figure 1, B andC. At the beginning of a recording session, a reference stimulus and a probe were selected from the same set of 16 possible stimuli used in Experiment 1. These two stimuli were used throughout a recording session.
Stimuli could appear at four locations: two locations within the receptive field and two other locations outside the receptive field. To minimize the possibility that the extrareceptive field stimuli appeared within the surround of the receptive field, we placed these stimuli across the vertical meridian. As an added precaution, we avoided recording from cells whose receptive fields were near the vertical meridian. For the majority of recordings, the across-meridian stimuli appeared at positions that were mirror images of the receptive field locations, as depicted in Figure 1B. For a few recordings, the across-meridian stimuli appeared at positions that were above the horizontal meridian, 180° from the receptive field. The results of the experiment did not seem to depend on which of these two configurations was used. Nevertheless, it remains possible that for some cells, the extrareceptive field stimuli may have fallen within the surround of the receptive field. However, these stimuli appeared in all attention conditions and all configurations of receptive field stimuli (probe, reference, and pair). Therefore, any effect that they potentially may have had on neuronal responses would not be expected to bias our results. Also, similar patterns of attention effects were observed in areas V2 and V4, despite the fact that V4 receptive fields and surrounds are significantly larger.
During a block of trials, the monkey had to attend to one of the positions and ignore the others to detect a target stimulus at the attended location. In the “attend-away” condition, the monkey attended to stimuli appearing at one of the two positions across the vertical meridian from the receptive field while the reference stimulus, the probe stimulus, or the pair appeared within the receptive field. In the “attend-receptive-field-stimulus” condition, the reference and probe stimuli both appeared together within the receptive field, and attention was directed to either the reference stimulus or the probe stimulus within the receptive field. Simultaneously, unattended stimuli appeared at the two positions across the vertical meridian from the receptive field.
We directed the monkey’s attention to a given location as follows. At the beginning of each block of trials, the monkey received a few (typically three to five) “instruction trials” that indicated where it was to attend for that block of trials. On these instruction trials, a bright cue box appeared at the location to be attended (see Fig.1C.) The monkey’s task was to detect the presence of a diamond-shaped target stimulus appearing at the cued location. This target appeared at the end of a variable-length sequence of zero to six nontarget (reference or probe) stimuli. The length of a given sequence varied from trial to trial. Therefore, the monkey could not know in advance when the target diamond would appear and had to attend to the cued location throughout the trial to distinguish the target from the nontargets, release the bar, and earn reward. Stimulus sequences appeared synchronously at the other positions, and distractor targets occasionally appeared embedded within them. If the monkey released the response lever after the appearance of a distractor target, the trial was aborted, and another trial began after a brief delay. After the monkey was reliably releasing the response lever when the target appeared at the cued location, the cue was removed, and the monkey had to continue performing the task without the cue.
It was rarely the case that the monkey would work long enough for us to complete an experiment, find a new set of cells, and complete a second experiment within a single recording session. However, it was often possible to continue recording from the same neuron using an additional stimulus pair. Therefore, to maximize the amount of data collected, we recorded from a neuron with one or more additional reference–probe pairs, whenever possible.
Data analysis. Neuronal responses were analyzed for trials occurring after the spatial cue was removed (i.e., the instruction trials were excluded from the analysis). We measured neuronal responses during a 150 msec time window beginning 120 msec after stimulus onset (the period over which we typically observed attentional modulation). To verify that our results are not an artifact of this particular time window, we have repeated this analysis using time windows that depend on stimulus duration and on response onset time and windows that varied from cell to cell to cover the period of attentional modulation. All of these analyses yielded qualitatively similar results. However, it is worth noting that, as a result of variability in the timing of sensory interactions and attention effects, we did miss some effects that fell outside the time window. See, for example, the figure that shows the response of a neuron for which sensory interactions and attention effects began before the beginning of this time window (see Fig.7).
Because nontarget stimuli greatly outnumbered target stimuli and distractor targets, the nontarget responses could be measured more reliably than the target responses. Therefore, our results are based on analysis of responses to nontarget stimuli. However, under similar experimental conditions, Luck et al. (1997) have compared the effect of attention on neuronal responses to target versus nontarget stimuli. They found that spatial attention has comparable effects on responses to target and nontarget stimuli.
Responses were measured in five conditions. In the attend-away condition, we measured responses to (1) the reference stimulus, (2) the probe stimulus, or (3) the pair, while the monkey attended away from the receptive field. In the attend-receptive-field-stimulus condition, we measured the pair response, while attention was directed to (4) the reference stimulus or (5) the probe.
As in Experiment 1, we normalized each cell’s responses by dividing all firing rates by the highest firing rate observed, for that cell, in any of the five conditions. Using these normalized responses, we then computed a selectivity index, SE, for each reference and probe: SE = PROBE − REF. We then computed a sensory interaction index for each of the three attentional conditions (attend away, attend to reference, and attend to probe). This was the difference between the response to the reference stimulus (REF) and the response to the pair of stimuli, with attention directed either away from the receptive field (PAIRa), toward the reference stimulus (PAIRr), or toward the probe (PAIRp): SIa = PAIRa − REF, SIr = PAIRr − REF, and SIp = PAIRp − REF. These indices are comparable with the indices derived in Experiment 1, except that SIr and SIp correspond to sensory interactions when attention was directed to the reference and probe stimulus, respectively.
Experiment 2 included many more experimental conditions than did Experiment 1. Therefore, to avoid a combinatorial explosion, it was necessary to reduce the number of probe stimuli from the 16 probes used in Experiment 1. Typically the monkey worked long enough to enable us to record from at most four different reference–probe pairs. For many cells, only one pair could be tested completely. It was therefore impossible, in Experiment 2, to quantify the relationship between selectivity and sensory interactions across stimuli within single cells. Instead, these comparisons were made across neurons within each cortical area. As in Experiment 1, we quantified the relationship between selectivity and sensory interactions in each cortical area by performing linear regressions on the selectivity and sensory interaction indices derived for each cell.
Model simulations
A simple model neural circuit, illustrated in Figure2, was used to simulate the results of Experiments 1 and 2. The model, which is a simple feedforward competitive neural network, is defined by the four equations shown at the bottom of Figure 2. The model includes two classes of cells. The circle at the top of Figure 2 represents the neuron (output) being measured. The two circles at the bottom of the diagram in Figure 2represent populations of upstream neurons (inputs) that respond to the reference (left) and probe (right) stimuli. Lines connecting the inputs to the output represent feedforward synaptic connections. Inhibitory inputs are assumed to depend on inhibitory interneurons (not shown).
Equations 1 and 2 (Fig. 2) describe the total excitatory and inhibitory inputs, respectively, to the measured cell. The total excitatory input to the cell (E) is simply the sum of the activities of the two input populations multiplied by their respective excitatory weights, as shown in Equation 1. The total inhibitory input to the cell (I) is the sum of the activities of the two input populations multiplied by their respective inhibitory weights, as shown in Equation 2.
Equation 3 (Fig. 2) describes how the firing rate of the output neuron (y) changes over time. This equation was originally introduced by Grossberg (1973) to provide an explanation of how feedforward competitive neural networks can be constructed to avoid saturating their responses to strong inputs (e.g., high-contrast stimuli) while remaining sensitive to weak inputs. See Grossberg and Levine (1975) and Grossberg (1976, 1980) for further discussion.
The first term [(B − y)E] governs excitatory input. B is the maximum response of the cell. Therefore, (B − y) is always positive. If excitatory input is greater than zero, then (B − y)E is positive, resulting in an increase in response that grows smaller as the cell’s responsey approaches its maximum rate. The second term (−yI) governs inhibitory input. If inhibitory input is greater than zero, then − yI is negative, resulting in a decrease in response toward zero. The third term (− Ay) is a passive decay term.
The net effect of excitatory and inhibitory input is described by Equation 4 (Fig. 2), which is the equilibrium response of the output neuron. The passive decay parameter A and the cell’s maximum response B are constants. Therefore, the equilibrium response depends on the relative contributions of the excitatory inputE and the inhibitory input I. Large values ofE will drive the equilibrium firing rate toward the cell’s maximum firing rate B. Large values of I will cause the cell to remain silent.
Attention is assumed to increase the strength of the signal coming from the population of cells activated by the attended stimulus. The exact mechanism by which this increase could occur is unknown. It is implemented here by increasing the efficacy of synapses projecting to the measured cell from the population activated by the attended stimulus. Increasing the strength of the signal from the attended stimulus population causes it to have a greater influence on the total mix of excitation and inhibition. Consequently, the response of the cell is driven toward the response that would be elicited if the attended stimulus were presented alone.
For all simulations, the maximum neuronal firing rate B was arbitrarily set to 1, and the passive decay parameter A was set to 0.2. For each model neuron, the excitatory and inhibitory weights projecting from the populations of neurons activated by the reference and probe stimuli were selected at random from a uniform distribution ranging from 0 to 1. To simulate the stochastic nature of neural responses, ±10% random noise, selected from a uniform distribution, was added to the response of the cell in each condition. Attention was implemented by increasing by a factor of 5 the excitatory and inhibitory synaptic weights projecting from the input neuron population responding to the attended stimulus. No other parameters appear in the model.
Simulation of Experiment 1. The responses of each model neuron to the reference stimulus, the 16 probes, and the corresponding 16 pairs were computed as follows. The reference stimulus and each of the 16 probes were assumed to activate their own input populations. Each of these input populations was assigned an excitatory and an inhibitory weight at random from a uniform distribution ranging from 0 to 1. In the circuit diagram shown in Figure 2, the input on the left is intended to correspond to the reference stimulus, which is constant for a given cell in the simulation of Experiment 1. The input on the right is intended to correspond to one of the 16 probe stimuli.
For each probe, the model was activated in three conditions. In the reference condition, the reference input activity level was set to 1, and the probe input was set to 0. In the probe condition, the probe input was set to 1, and the reference input was set to 0. In the pair condition, the probe and reference inputs were both set to 1. In each of the three conditions, the equilibrium response of the model neuron was computed according to Equation 4 in Figure 2. The resulting responses were then used to compute the indices of selectivity and sensory interaction, as described for Experiment 1.
Simulation of Experiment 2. The model simulation of Experiment 2 was conducted in the same manner as was the simulation of Experiment 1, with two changes. First, only one probe and one reference were presented to each model neuron. So, in the circuit diagram shown in Figure 2, the left input is intended to correspond to the reference stimulus, and the right input is intended to correspond to the probe. Second, to incorporate attention to the reference and probe stimulus, the model was also simulated in two additional conditions. In both of these conditions, the reference and probe input activity levels were both set to 1. In the attend-reference condition, the strengths of synaptic weights from the reference stimulus input were multiplied by a factor of 5. In the attend-probe condition, the strengths of synaptic weights from the probe stimulus input were multiplied by a factor of 5.
RESULTS
Experiment 1
Experiment 1 was designed to examine how the responses elicited by a single stimulus within the receptive field (the reference stimulus) are influenced by the addition of a second receptive field stimulus (the probe), in the absence of attentional modulation. Depending on the mechanisms that govern responses to stimulus pairs, adding the probe might be expected to result in an increase, a reduction, or a more complex change in the pair response, compared with the response elicited by the reference stimulus. An increase in response could occur as a result of additional recruitment of V1 afferents by the second stimulus. A reduction in response could occur as a result of diminished bottom-up or top-down excitatory drive. Response suppression by extrareceptive field stimuli has been observed in area V1 (Knierim and Van Essen, 1992; Levitt and Lund, 1997). Response suppression has also been observed in higher-order areas that provide feedback to areas V2 and V4 (Miller et al., 1993b; Rolls and Tovee, 1995). Alternatively, the individual stimulus responses might bear no systematic relationship to the response elicited by the pair. For example, the pair response might depend on factors other than the firing rates elicited by the individual stimuli, such as the geometric relationships between the stimuli (Kapadia et al., 1995; Sillito et al., 1995) or their color contrast (Kiper et al., 1997). Or, V2 and V4 cells might simply treat the pair as a third, independent stimulus, with its own arbitrary response.
In contrast to these alternatives, the biased-competition hypothesis predicts that the pair response should fall between the responses to the reference and probe stimuli. According to the hypothesis, stimuli activate competing populations of neurons. To the extent that a probe stimulus has any influence on the neuronal response, the probe should move the pair response toward the response the probe would give if it had been presented alone. Adding a low-firing rate probe should drive down the response to a high-firing rate reference stimulus. Adding a high-firing rate probe should drive up the response to a low-firing rate reference stimulus. If probe and reference stimuli individually elicit identical responses, then this same response should be generated when they appear together as a pair.
We recorded the responses of 18 neurons from area V4 of one monkey. Our results indicate that in area V4, the responses of neurons to pairs of bar stimuli are a weighted average of the individual stimulus responses. This is illustrated in Figure3, which shows the responses of a typical cell to the reference stimulus, the 16 different probes, and the resulting 16 stimulus pairs. Figure 3, A–C, shows the effect of adding three of the probe stimuli. For a probe that elicited a lower mean response than did the reference (Fig. 3A), the addition of the probe was suppressive. For a probe that elicited an average response approximately equal to the response elicited by the reference stimulus (Fig. 3B), the pair response was similar to the responses to the probe and reference. For a probe stimulus that elicited a stronger response than did the reference stimulus (Fig.3C), the addition of the probe caused an increase in the cell’s mean response.
This relationship held across all 16 probe stimuli, as illustrated in Figure 3D. Each point corresponds to the indices of sensory interaction (y-axis) versus selectivity (x-axis) for each probe stimulus. Points labeledA–C correspond to the examples shown in Figure 3,A–C. The data were positively correlated [r2 = 0.53;r2 significantly different from 0,F(1,15) = 16.91 and p = 0.001] and fell along a line with a positive slope (+0.67), indicating that the effect of adding a probe stimulus was proportional to the selectivity of the cell’s response to the reference and the probe stimulus. Adding a probe tended to suppress the neuronal response if the probe presented alone elicited a smaller response than did the reference stimulus. A probe tended to increase the neuronal response if the probe alone elicited a larger response than did the reference stimulus. The most suppressive probes tended to be those that elicited the smallest responses, when presented alone. The probes causing the greatest increase in response tended to be those that elicited the largest responses, when presented alone. The intercept (−0.02) was not significantly different from 0 [t(14) = −0.40;p = 0.3486]. Thus, probes such as the one corresponding to point B in Figure 3D, which gave responses similar to that of the reference stimulus, had little or no effect when added to the reference stimulus.
Sensory interactions are, for this cell, approximately proportional to selectivity. Therefore, the slope of the best-fit line provides a convenient way to quantify the relative influence exerted by the stimuli.
The equation of the best-fit line relating selectivity and sensory interaction indices can be written: SIi = wSEi + offset, where w is the slope of the regression equation, and offset is the increase or decrease in response that is not accounted for by selectivity (i.e., the vertical intercept of the best-fit line). This equation can be rewritten: PAIRi − REF =w (PROBEi − REF) + offset. Rearranging terms, this can be expressed: PAIRi = wPROBEi + (1 − w)REF + offset.
Thus, the response to the pair (PAIRi) is the average of the response to the probe (PROBEi) and the response to the reference stimulus (REF), weighted by the slope w, plus the offset term. The slope of the best-fit line (w) indicates how heavily the pair response is weighted toward the response to the probe. The value (1 − w) is the weighting factor of the reference stimulus. The slope w for the cell illustrated in Figure 3 was 0.67, and the offset was not significantly different from 0. Therefore, for this cell, the pair response can be described as a weighted average of the responses to the probe and reference stimuli, with the reference stimulus exerting less influence (0.33) than the probe (0.67). Note that, for this neuron, the reference stimulus exerted less influence than the probe despite the fact that the response to the reference was larger than the responses to 13 out of 16 probes (probes with negative selectivity values). Surprisingly, the degree of influence exerted by a given stimulus does not seem to depend only on the magnitude of the response elicited by that stimulus.
Although this weighting factor varied from cell to cell, responses to the pair typically conformed to this pattern. Figure4 shows examples of six cells (including the example from Fig. 3D for comparison) that illustrate the range of correlations we observed. Across the population, sensory interactions were proportional to selectivity. However, responses to the pair showed varying degrees of reference–probe sensitivity. For some cells, such as the one illustrated in Figure 4D, the pair response depended strongly on reference–probe selectivity (intercept = 0.05; slope = 0.8162), indicating that the pair response was determined primarily by the probe stimulus and not by the reference. For others, such as the cell illustrated in Figure 4F, the responses to the pairs were approximately equal to the response to the reference stimulus, regardless of the size of the responses to the probes (intercept = −0.006; slope = 0.11). The cell was selective for the probes, but there was no corresponding change in the responses to the pairs, which were approximately equal to the response to the reference stimulus. Sixteen of 18 cells (89%) had regression slopes between 0 (pair response equal to the response to the reference stimulus, regardless of the response to the probe) and 1 (pair response equal to the response to the probe). The two cells with slopes outside this range had small slopes (−0.07 and −0.06) that were not significantly different from 0 [F(1,15) = 0.1728 and p = 0.6835; F(1,15) = 0.0669 and p = 0.80, respectively]. Across the population, the mean regression intercept was 0.01, which was not significantly different from 0 [t(17) = 0.5619;p = 0.5815], indicating that, on average, the addition of the probe stimulus did not result in a net change in neuronal response that could not be attributed to the selectivity of the cell for reference and probe.
When the probe influenced the neuronal response, it typically moved the pair response toward the response that was elicited by the probe alone. Across cells, the neuronal response was significantly changed by the addition of the probe for 83 stimulus pairs tested (two-tailedt test, p < 0.05). For 35 out of 83 (42%) of these, the probe suppressed the pair response, and for the remaining 48 out of 83 (58%), the probe increased the pair response. Of the 35 pairs for which the probe was significantly suppressive, 34 out of 35 (97%) of these probes were less preferred than was the reference stimulus (i.e., poor probes that suppressed the neuronal response). Of the 48 pairs for which the probe increased the response, 37 out of 48 (77%) of these probes were relatively more preferred than was the reference stimulus (i.e., preferred probes that increased the neuronal response). Thus, when the probe caused a significant change in the neuronal response, this change was toward the response elicited by the probe 86% (71 out of 83 probes) of the time.
These data are incompatible with some possible models of sensory processing in areas V2 and V4. In particular, we can eliminate models in which the response to a pair of stimuli is greater than the response to the preferred stimulus appearing alone or less than the response to the poor stimulus alone. We can also eliminate models in which the pair is treated as a third, independent stimulus, with its own arbitrary response.
Experiment 2
The second experiment was designed to examine the relationship between (1) selectivity, (2) the sensory interactions resulting from adding a probe stimulus within the receptive field of the cell, and (3) the effect of directing attention to either the reference or probe stimulus. In agreement with the finding of Luck et al. (1997), we often observed increases in the spontaneous firing rate of neurons when attention was directed to a location within the receptive field, during the period before the stimulus appeared. However, in the present experiment, we were interested in characterizing the effects of attention on neuronal responses evoked by stimuli within the receptive field. Therefore all of the results described below are based on stimulus-evoked responses.
We recorded in areas V2 and V4, where previous studies have found attentional modulation of neuronal responses. We measured the responses of 158 neurons in three monkeys (86 in V2; 72 in V4). For some cells, the monkey worked long enough to record responses from more than one reference–probe pair. Responses were analyzed for all reference–probe pairs for which reference and probe each gave significant responses, relative to the neuron’s spontaneous firing rate, with attention directed away from the receptive field of the cell. A total of 208 stimulus pairs in 67 V2 cells and 138 stimulus pairs in 57 V4 cells gave significant responses (two-tailed t test,p < 0.05) for both reference and probe.
For these 124 neurons (346 stimulus pairs), we analyzed the relationship between selectivity and sensory interactions. Consistent with the results of Experiment 1, the effect of adding a probe depended on the cell’s selectivity for reference and probe. This is illustrated in Figure 5, which shows the relationship between selectivity (x-axis) and sensory interactions (y-axis) for cells in V2 (Fig. 5A) and V4 (Fig. 5B). In both cortical areas, there is a strong correlation between selectivity and sensory interactions [V2,r2 = 0.58;r2 significantly different from 0,F(1,207) = 285.9 and p < 0.000001; V4, r2 = 0.61;r2 significantly different from 0,F(1,137) = 213.3 and p < 0.000001]. This relationship appears linear and passes near the origin (intercept, −0.01 and +0.08 for V2 and V4, respectively). In V2, the intercept (−0.01) is not significantly different from 0 [t(206) = −1.082; p = 0.14]. However, in V4, the intercept (0.08) is significantly >0 [t(136) = 6.1476; p < 0.000001], indicating that adding the probe stimulus within the receptive field caused an increase in response that does not depend on the cell’s selectivity for the reference and probe stimuli. However, the magnitude of this increase is very small, relative to the changes in response that depend on selectivity for reference and probe. Both populations have slopes near 0.5 (+0.53 and +0.55 for V2 and V4, respectively) that are not significantly different from 0.5 [t(206) = 1.0395; p = 0.15;t(136) = 1.2403; p = 0.11], indicating that, on average, reference and probe stimuli exerted approximately equivalent influence over pair responses. Thus, as in Experiment 1, these results are incompatible with models in which the response to a pair of stimuli falls outside the range of responses defined by the two individual stimuli presented alone. As in Experiment 1, these results are also incompatible with models in which the pair is treated as a third, independent stimulus, with its own arbitrary response.
When attention was directed to one of the stimuli, this caused a substantial reduction in the influence of the nonattended stimulus. If the neuronal response was reduced as a result of adding the probe, then this suppressive effect was diminished when attention was directed to the reference stimulus. Likewise, if adding the probe increased the neuronal response, then directing attention to the reference stimulus caused the response to move back toward the reference stimulus response. This is illustrated in several figures (see Figs. 6-9) that show responses of individual neurons in areas V2 and V4. Figure6A shows the responses of a V2 neuron for which adding the probe stimulus reduced the response. Attention was directed away from the receptive field of the cell (attend-away condition). The reference stimulus elicited a robust response (Fig. 6A, dotted line). The pair response (Fig. 6A, dashed line) was strongly suppressed by the presence of the probe stimulus. The response to the probe (Fig. 6A, solid line) is shown for comparison. Figure 6Bshows the responses of the same neuron, except that in this case, thedotted line shows the response of the cell to the pair when attention was directed to the reference stimulus. The majority of the suppression caused by the probe stimulus was eliminated when attention was directed to the reference stimulus.
For other neurons, adding the probe stimulus increased the neuronal response, and this increase was eliminated by attention. This is illustrated in Figure 7, A andB. This V2 neuron gave a moderate response to the reference stimulus that was substantially increased by the addition of the probe stimulus. Attention to the reference stimulus filtered out most of the increase that resulted from adding the preferred probe. Similar effects were observed in area V4, as illustrated in Figures8 and 9. As in V2, when the response to the probe was lower than the response to the reference, adding the probe typically suppressed the neuronal response (Fig. 8). When the response to the probe was higher than the response to the reference stimulus (see Fig. 9), adding the probe typically increased the neuronal response. In V4, as in V2, the effect of attention was to filter out the effect of the nonattended stimulus.
Across neurons, adding the probe typically caused the neuronal response to move toward the response elicited by the probe alone. This sensory interaction could be magnified by directing attention to the probe or reduced by directing attention to the reference stimulus. This is illustrated in Figures 10 and11, which show the relationship between selectivity, sensory interactions, and attention in V2 and V4, respectively. Figure 10, A and B, shows the relationship between selectivity and sensory interactions for neurons whose responses to a given reference–probe pair showed a statistically significant change in response when attention was directed to the probe stimulus, as determined by a two-tailed, unpaired t test (p < 0.05). For neurons tested with more than one reference–probe pair, each pair was tested independently. A given pair was included if the response elicited by the pair changed significantly when attention was directed to the probe stimulus. Therefore, some neurons appear more than once, if more than one pair elicited a response that was significantly changed by attention to the probe. A total of 55 out of 67 neurons (82%) tested with 96 out of 208 reference–probe pairs (46%) showed statistically significant changes in their responses to pairs when attention was directed to the probe stimulus.
The increased influence of the attended probe stimulus can be observed by comparing data in Figure 10, A and B. Figure10A shows the relationship between selectivity and sensory interaction when attention was directed away from the receptive field. The indices are correlated (r2 = 0.60), and the relationship appears linear. The slope was 0.47, which was not statistically different from 0.5 [t(94)= −0.6704; p = 0.2521], indicating that, with attention directed away from the receptive field, reference and probe stimuli exerted approximately equal influence over neuronal responses. The intercept of the best-fit line (−0.04) was slightly but significantly <0 [t(94) = −2.4731;p = 0.0076]. For this subpopulation, adding a second stimulus within the receptive field caused a small (4% of maximum response) reduction in mean response, in addition to the larger changes in firing rate that were related to selectivity.
When attention was directed to the probe stimulus, this magnified the sensory interactions caused by the probe. This is reflected in a steeper relationship between selectivity and sensory interaction indices. Figure 10B shows the indices for the same cells shown in Figure 10A, but in this case, the pair responses used to compute the sensory interaction indices were measured when attention was directed to the probe. Attention increased the slope of the regression line, from 0.47 to 0.69 (increase of 0.22). A comparison of the regression slopes (Snedecor and Cochran, 1967) indicated that this increase was significant [F(1,190) = 14.4; p = 0.0002]. This increased slope was also significantly different from 0.5 [t(94) = 3.1946; p = 0.0001], indicating that when attention was directed to the probe stimulus, the probe exerted greater influence over the pair response. In addition to this change of slope, there was a small (0.066) but significant increase in the average response to the pair [unpaired ttest, t(95) = 2.3052; p = 0.02]. This is reflected in an upward shift in the best-fit line. Thus, in addition to the magnified influence of the attended probe, attention caused an increase in response that was unrelated to selectivity. Selectivity and sensory interactions were still strongly correlated (r2 = 0.59) with attention directed to the probe stimulus.
When attention was directed to the reference stimulus, this caused the pair response to move toward the response elicited by the reference stimulus alone. This is illustrated in Figure 10, C andD, which shows the relationship between selectivity and sensory interactions for neurons whose responses to a given reference–probe pair changed significantly when attention was directed to the reference stimulus. As in the previous analysis, significance was determined by a two-tailed, unpaired t test (p < 0.05). A total of 56 out of 67 neurons (84%) using 97 out of 208 reference–probe pairs (47%) showed statistically significant changes in the pair response when attention was directed to the reference stimulus.
Figure 10C shows, for this population, that the relationship again appears to be linear. The intercept (−0.04) was slightly but significantly <0 [t(95) = −2.5228;p = 0.0067]. The slope (0.55) was not significantly different from 0.5 [t(95) = 1.2924;p = 0.0997], indicating that, across this subpopulation, reference and probe exerted approximately equivalent influence over neuronal responses with attention directed away from the receptive field.
The increased influence of the attended reference stimulus is evident in the reduced slope in Figure 10D. This graph shows indices for the same population of cells shown in Figure10C, but in this case, the pair response used to compute each sensory interaction index was recorded when attention was directed to the reference stimulus. The slope dropped from 0.55 in the attend-away condition (Fig. 10C) to 0.24 with attention directed to the reference stimulus (Fig. 10D; a reduction of 0.31). This reduction in slope was significant [F(1,192) = 22.16; p = 0.000005]. The reduced slope was also significantly different from 0.5 [t(95) = −4.4587; p = 0.00001], indicating that, with attention directed to the reference stimulus, the reference exerted greater influence over the pair response. The reduced influence of the probe stimulus is also reflected in a diminished correlation coefficient, fromr2 = 0.67 with attention directed away from the receptive field to r2 = 0.16 with attention directed to the reference stimulus. As with attention directed to the probe (Fig. 10B), there was a small (0.04) increase in the mean response to the pairs, but this was not statistically significant [unpaired t test,t(96) = 1.3993; p = 0.16].
In summary, across V2 cells that showed significant attentional modulation, attention strongly determined which stimulus drove the cell’s response to the pair. When attention was directed to the probe stimulus, the pair response was a weighted average of 69% of the response to the probe plus 31% of the response to the reference stimulus. When attention was directed to the reference stimulus, the pair response was a weighted average of 24% of the response to the probe plus 76% of the response to the reference stimulus. In addition, there was a small (4–6%) and marginally significant increase in mean firing rate that was unrelated to the individual stimulus responses.
The average effect of attention was reduced when computed over the entire population, including responses to pairs that were not significantly modulated by attention. Across this entire population, the sensory interaction/selectivity slope with attention directed away from the receptive field was 0.53. The slope increased by 0.14 to 0.67 when attention was directed to the probe and decreased by 0.19 to 0.34 when attention was directed to the reference stimulus. The total shift, from 0.67 down to 0.34 (a shift of 0.33), was 27% smaller when computed over the entire population than was the shift of 0.45 observed for responses that showed significant attention effects.
Figure 11 shows comparable results for neurons recorded in area V4. A total of 39 out of 57 neurons (68%) tested with 61 out of 138 reference–probe pairs (44%) showed a significant (two-tailedt test, p < 0.05) change in pair response when attention was directed to the probe. The relationship between selectivity and sensory interactions appears linear. With attention directed away from the receptive field (see Fig.11A), the selectivity and sensory interaction indices for these neurons are correlated (r2 = 0.55), and the slope of the best-fit line was 0.49. This slope was not significantly different from 0.5 [t(59) = −0.1663; p = 0.4343], indicating equivalent influence of reference and probe with attention directed away from the receptive field. The best-fit line was shifted slightly but significantly [t(59) = 2.8226; p = 0.0032] upward (+0.06), indicating that, in addition to sensory interactions related to selectivity, the addition of the second stimulus caused a small increase in response. With attention directed to the probe (Fig.11B), the indices were still highly correlated (r2 = 0.60). As in V2, attention to the probe significantly increased the slope, from 0.49 to 0.83 [increase of 0.34; F(1,120) = 22.618; p = 0.000006]. The increased slope was also significantly different from a slope of 0.5 [t(59) = 3.7178; p= 0.0002], indicating greater influence by the attended probe.
As in area V2, attending to the reference stimulus typically caused the pair response to move toward the response that was elicited by the reference stimulus. A total of 37 out of 57 neurons (65%) in 59 out of 138 stimulus configurations (43%) showed a significant effect of directing attention to the reference stimulus. For these cells (Fig.11C), the relationship between selectivity and sensory interaction appears linear. The best-fit line was shifted slightly upward (+0.06), indicating that the addition of the second stimulus caused an increase in response that was unrelated to selectivity [t(57) = 3.0362; p = 0.0018]. The slope of the best-fit line was 0.6, which was significantly >0.5 [t(57) = 1.8835; p = 0.0324]. This indicates that for this subpopulation, there was a small but marginally significant bias in favor of the probe stimulus with attention directed away from the receptive field. Note that probe and reference stimuli were selected daily from the same stimulus set, and it was impossible to know in advance which of two stimuli would exert greater control over the pair response. Therefore, we can only assume that for this subset of neurons, we happened to pick probes that exerted, on average, slightly greater influence over responses to the pairs than did the corresponding reference stimuli.
Attention to the reference stimulus overcame this bias, as reflected by the reduced slope in Figure 11D. Directing attention to the reference significantly [F(1,116) = 20.796; p = 0.00001] decreased the slope from 0.60 down to 0.21 (decrease of 0.39). The reduced slope was also significantly different from 0.5 [t(57) = −3.2059; p = 0.0011], indicating greater influence by the attended reference stimulus. As in V2, the reduced influence of the unattended probe stimulus was also reflected in a diminished correlation coefficient, from r2 = 0.66 with attention directed away from the receptive field tor2 = 0.09 with attention directed to the reference stimulus. There were also small but significant increases in the average response, when attention was directed to the probe stimulus [mean shift = 0.13; unpaired t test,t(60) = 3.8168; p = 0.0003] or to the reference stimulus [mean shift = 0.10; unpairedt test, t(58) = 3.7941;p = 0.0004]. These increases are reflected in an upward shift of the best-fit lines (Fig.11B,D).
In summary, across V4 cells that showed significant attentional modulation, we observed a shift of control that was comparable in magnitude with the shift observed in V2. When attention was directed to the probe stimulus, the pair response was a weighted average of 83% of the response to the probe plus 17% of the response to the reference stimulus. When attention was directed to the reference stimulus, the pair response was a weighted average of 21% of the response to the probe plus 79% of the response to the reference stimulus. In addition, there was a small (10–13%) but significant increase in mean firing rate that was unrelated to the individual stimulus responses.
As in V2, the average effect of attention was reduced when computed over the entire population, including responses to pairs that were not significantly modulated by attention. Across the entire V4 population, the sensory interaction/selectivity slope with attention directed away from the receptive field was 0.55. The slope increased by 0.19 to 0.74 when attention was directed to the probe and decreased by 0.21 to 0.34 when attention was directed to the reference stimulus. The total shift, from 0.74 down to 0.34 (a shift of 0.40), was 35% smaller when computed over the entire population than was the shift of 0.62 observed for responses that showed significant attention effects.
Although these data indicate that when attention effects were observed, their direction and magnitude depended on the direction and magnitude of underlying sensory interactions, it was not the case that sensory interactions alone guaranteed the presence of attention effects. In both areas, we regularly found stimulus pairs for which the addition of the probe caused significant sensory interactions, with no corresponding effect of directing attention to the reference stimulus. A total of 48 out of 208 stimulus pairs tested in area V2 (23%) and 34 out of 138 stimulus pairs tested in area V4 (25%) caused sensory interactions that were not accompanied by attention effects. Thus, when attention is directed to a stimulus, the responses of some neurons continue to be influenced by the presence of the unattended stimulus.
Model simulation results
A simple neural circuit that satisfies the constraints imposed by the data from Experiments 1 and 2 is illustrated in Figure 2 and is described in detail in Materials and Methods. To test whether the model is consistent with the results of Experiment 1, we simulated a total of 100 model neurons, differing only in their randomly assigned weights. For each model neuron, we computed the response to the reference stimulus alone, the responses to each of the 16 probes, and the responses to each of the resulting stimulus pairs. Selectivity and sensory interaction indices were then computed for each probe. These indices are shown for six representative model neurons in Figure12 (compare with data in Fig. 4). Across the population, the median slope was +0.506, indicating that, on average, reference and probe exerted approximately equivalent influence over the model neuron’s responses to the pair. However, as we observed for the cells recorded in Experiment 1, some model neurons (such as those shown in Fig. 12B–D) had steeper slopes, whereas others (such as those in Fig.12E,F) had shallower slopes, corresponding to a greater influence of the probe and reference, respectively.
According to the model, this range of slopes is the result of differences in the strength of the (randomly chosen) projections from the population of input neurons that respond to the reference stimulus. Cells for which the reference stimulus had weak projections (such as those illustrated in Fig. 12B–D) had steeper slopes because the probe stimuli made up the majority of the input to the cell. Cells for which the reference stimulus projections were stronger (such as those illustrated in Fig.12E,F) had shallower slopes because responses to pairs were dominated by the inputs from the reference stimulus.
Figure 13 shows the results of simulating Experiment 2 (i.e., the effects of attention). Again, 100 model neurons were simulated with the same parameters used to simulate Experiment 1. As in the previous simulation, excitatory and inhibitory weights were chosen at random for both the reference and the probe stimuli. Figure 13A shows a scatter plot of these indices. As in the recording data (compare with data in Figs.10A,C, 11A,C), when attention was directed away from the receptive field, the slope of the best-fit line relating selectivity and sensory interactions for the model neurons was ∼0.5. The reference and probe had approximately equal influence over the pair response. The best-fit line passes near the origin (+0.07), indicating that there was a small increase in response beyond that depending on selectivity.
To simulate Experiment 2, we modeled the effect of attention by increasing the strength of inputs driven by the attended stimulus, as described in Materials and Methods. Figure 13B shows the indices of selectivity and sensory interaction that were computed when attention was directed to the probe stimulus. As in the experimental data (compare with data in Figs. 10B,11B), attention caused a moderate increase in the mean firing rate across model neurons. This is reflected in an upward shift (+0.10) in the line relating sensory interactions to selectivity, similar to the upward shifts of +0.06 and +0.11 that were observed in V2 and V4, respectively. In contrast to the small change in intercept, there was a large effect of attention on the slope, which increased from 0.52 to 0.78 (increase +0.26), reflecting enhanced influence of the attended probe stimulus. This increase in slope is comparable with the increases measured in V2 (+0.22) and V4 (+0.34).
Figure 13C shows the effect of directing attention to the reference stimulus. Attending to the reference resulted in a small (+0.04) upward shift in the line relating sensory interaction to selectivity, which is comparable with upward shifts of +0.06 and +0.11 observed in V2 and V4, respectively. Again, the change in intercept was small, compared with the reduction in slope that occurred when attention was directed to the reference stimulus. Attention drove the slope from 0.52 down to 0.18, reflecting the reduced influence of the probe stimulus when attention was directed to the reference stimulus (compare with data in Figs. 10D,11D). This change in slope (−0.34) is comparable with the changes observed in V2 (−0.31) and V4 (−0.39).
These model predictions are robust in that they do not depend on many parameters that are fine-tuned to achieve an adequate fit. Rather, the magnitudes of attention effects relate directly to individual parameters of the model. The predicted magnitude of the upward shift is a function of the passive decay parameter A. If the model neuron has a larger rate of passive decay, its equilibrium response rate is further below its saturation response. Therefore a given increase in input strength resulting from attention causes a greater increase in response and a larger upward shift in the line relating sensory interactions to selectivity. The size of the predicted change in slope depends on the magnitude of the magnification factor that is applied to the strength of the inputs from the attended stimulus. A larger magnification factor causes the pair response to move further toward the response elicited by the attended stimulus, resulting in a larger change in slope.
DISCUSSION
In Experiment 1, we found that, during passive fixation, the neuronal response to a pair of oriented bars depends linearly on the responses to the individual bars. If reference and probe are selected to give identical responses, then the pair response is typically indistinguishable from the responses to the individual stimuli. However, if the orientation and color of the probe are adjusted to cause it to elicit a larger response than that of the reference stimulus, the pair response typically increases. The magnitude of this increase grows in proportion to the response elicited by the probe. Changing the probe to a nonpreferred orientation or color typically reduces the pair response. As the probe becomes more nonpreferred, it typically becomes proportionally more suppressive. Thus, the degree of influence exerted by a stimulus over the neuronal response to the pair is not simply proportional to the magnitude of the response evoked by that stimulus. Instead, the influence exerted by a stimulus and the response it elicits when presented alone must be considered to be separate variables. These findings were replicated in Experiment 2 with attention directed away from the receptive field.
In Experiment 2, we found that when attention is directed to one of two receptive field stimuli, its effect depends on these underlying sensory interactions. In the absence of sensory interactions, attention to either individual stimulus typically is limited to a moderate increase in mean response. When sensory interactions do occur, the magnitude and direction of the observed attention effects depend on the magnitude and direction of the underlying sensory interactions. If the addition of the probe suppresses the neuronal response, then attention to the reference stimulus typically filters out some of this suppression. If adding the probe facilitates the response, attention to the reference typically filters out some of this facilitation. Attending to the probe magnifies the change that was induced by the addition of the probe.
These linear relationships between selectivity, sensory interactions, and attention effects provide several constraints on the set of possible models of ventral stream visual processing. Because of the many stages of complex processing that occur between the retina and cortical areas such as V2 and V4, these constraints narrowly circumscribe the set of possible models. However, as our simulations show, these results can be understood within the context of the proposed model.
Model predictions
In addition to providing a way to satisfy these constraints, the model also makes predictions about neuronal responses under conditions that were not tested in the present experiments. First, it predicts the conditions under which attention to a single receptive field stimulus should result in an increase in neuronal response. According to the model, attention increases the bottom-up drive reaching the measured neuron, which forces the neuron’s response upward, toward its maximum firing rate for that particular stimulus. If the bottom-up inputs driven by a particular stimulus are strong enough that the cell’s response has saturated, then attention is predicted to have no influence on the response. However, if the response is not saturated, then attention is predicted to increase it. Thus, a prediction of the model is that attention should increase neuronal responses to stimuli that elicit responses within the dynamic range of the cell. These would include stimuli that activate populations of afferents that project weakly to the measured cell or stimuli of low brightness or color contrast.
In addition, the model makes a novel prediction about how neuronal responses should depend on the relative salience of two receptive field stimuli, when attention is directed away from the receptive field. Specifically, if the salience of one receptive field stimulus is increased relative to the salience of another receptive field stimulus, this should cause the pair response to move toward the response elicited by the first stimulus. For example, suppose the response to a preferred stimulus is suppressed by the addition of a less-preferred stimulus. Then, according to the model, increasing the luminance contrast of the less-preferred stimulus should increase the strength of the inputs from that stimulus, resulting in greater suppression of the response to the pair. This is predicted to occur even when the less-preferred stimulus elicits a significant excitatory response on its own. Finally, the model predicts that the increased influence of the more salient stimulus can be offset if attention is directed to the lower salience stimulus.
Baseline shift
The model can also account for a number of previously reported results, such as the observation (Luck et al., 1997) that the spontaneous firing rate of V2 and V4 neurons increases when attention is directed to a location within the receptive field. According to the model, attention increases the efficacy of synapses projecting from afferent neurons whose receptive fields are at the attended location. As a result of this increase, spontaneous activity among these afferents is predicted to be better able to activate the measured neuron, resulting in higher spontaneous activity in the measured neuron. If the synapses of inputs projecting from the afferent neurons are weak or sparse, this shift in baseline firing rate is predicted to be small. It is predicted to be larger for afferents with stronger projections to the measured cell. In agreement with this prediction,Luck et al. (1997) found that the increase in spontaneous activity is larger when attention is directed to the center of the receptive field (stronger afferent projections) versus a position near the edge of the receptive field (weaker afferent projections).
Attention to a single receptive field stimulus
The model is also consistent with previously reported spatial attention effects in the ventral stream using single stimuli within the receptive field (Haenny et al., 1988; Spitzer et al., 1988; Maunsell et al., 1991). These studies have reported no change or small increases in responsiveness with attention directed to the receptive field stimulus. These findings are compatible with the model’s prediction that increases in response will be observed for a single stimulus, provided the stimulus has not already saturated the neuronal response. With the same parameters used to simulate the results of the present experiments, the model predicts a mean increase of 17.5% in neuronal response to a single stimulus with attention, which falls within the range of effects reported in these studies.
According to the model, these increases in response should depend on the magnitude of the attentional signal. Stronger top-down attentional feedback is assumed to result in larger increases in input strength for the attended stimulus. Therefore, the magnitude of the response increase caused by attention to a single stimulus in a difficult task is predicted to be equal to or greater than any increase observed in an easy task, using identical stimuli. In agreement with this, Spitzer et al. (1988) reported moderate (18%) increases in neuronal responsiveness in V4 when attention was directed to a single stimulus in a difficult discrimination task but not in a less-demanding task.
Three additional spatial attention studies conducted with a single receptive field stimulus should be considered within the context of the present results. Motter (1993) has reported that in the ventral stream, attention to a stimulus inside the receptive field can cause increases or decreases in response when stimuli appear outside the receptive field. Connor et al. (1996, 1997) have reported that the response to a single receptive field stimulus can increase or decrease, depending on which of several extrareceptive field stimuli is attended. One possible explanation for both of these findings is that attention modulated sensory interactions resulting from the addition of the extrareceptive field stimuli.
None of these studies compared the response of the receptive field stimulus with and without the extrareceptive field stimuli, with attention directed away from the receptive field. Therefore, it is unknown whether the extrareceptive field stimuli induced sensory interactions. However, extrareceptive field stimuli are known to modulate the responses of cells in the areas examined in these studies. Cells in V4, for example, have large, stimulus-selective, silent surrounds that can be either inhibitory or excitatory (Schein and Desimone, 1990). Because of the relationship between attention effects and sensory interactions demonstrated in the present experiment, it would be useful to know whether the attention effects observed in these three studies were accompanied by sensory interactions resulting from the presence of the extrareceptive field stimuli.
Comparing these results with those of previous experiments with multiple receptive field stimuli
The attention effects observed in the present experiment are compatible with those of previous studies that have examined the effect of attention when multiple stimuli appeared within the receptive fields of neurons in the ventral stream (Moran and Desimone, 1985; Luck et al., 1997). These studies have found neuronal responses to be larger when attention was directed to the preferred stimulus relative to when the poor stimulus was attended. Among these studies, the experiment that is more closely related to the present experiment is the study ofLuck et al. (1997), which used the same stimuli and the same behavioral task. Luck et al. (1997) reported that, among V4 neurons that showed attention effects, responses were, on average, 63% higher when attention was directed to the preferred stimulus than when attention was directed to the poor stimulus. Using the same selection criteria, we find that, on average, responses were 69% higher. We find attention effects of comparable magnitude in area V2, where, on average, responses to stimulus pairs were 79% higher when attention was directed to the preferred stimulus relative to when attention was directed to the poor stimulus.
The remaining studies of attention in the ventral stream did not explicitly manipulate spatial attention. Instead, they manipulated nonspatial variables such as whether the stimuli matched the form of a cue (Haenny et al., 1988; Chelazzi and Desimone, 1994; Ferrera et al., 1994; Motter, 1994) or whether the monkey was engaged in a particular task (Fischer and Boch, 1985). In these studies, the stimulus-evoked responses and/or baseline firing rates of neurons were found to vary depending on behavioral condition, but the relationship between such nonspatial attention effects and the findings of the present study is not yet clear.
The purpose and limitations of the model
The biased-competition model provides a unified, quantitative framework within which to place a number of observed and predicted attention effects. However, our implementation of this model is not intended to be an account of the actual neural circuitry underlying visual attention. Many of the details of this circuitry are simply unknown. For instance, the source of the biasing feedback is unknown, as are the neural elements that are the targets of feedback in the cortex. In the absence of detailed knowledge of the circuitry underlying attention, it is not yet possible to distinguish between a number of alternative models. These include models that implement competitive interactions using lateral inhibitory connections and that assume that the attentional bias is mediated either by a direct excitatory signal or by invoking synchronous discharge among cells whose receptive fields overlap with the focus of attention (for example, see Koch and Ullman, 1985; Anderson and Van Essen, 1987;Niebur et al., 1993; Olshausen et al., 1993; Ferrera and Lisberger, 1995; Grossberg, 1995, 1999a,b; Stemmler et al., 1995; Pouget and Sejnowski, 1997; Borisyuk et al., 1998). Our purpose in providing a simple but mathematically complete implementation of the biased-competition model is to provide a demonstration proof that biased competition can satisfy the constraints imposed by the present experiments while remaining compatible with results that have been reported using single receptive field stimuli. Because it is simple, has a closed-form solution, and depends on only three parameters, it is possible to use the model to make quantitative predictions that can be tested experimentally, to refute the model, or to determine better how it is implemented in the brain.
Biased competition in the dorsal stream
Recent experiments suggest that similar mechanisms may be at work in the dorsal stream. Ferrera and Lisberger (1995) have found that the onset time of a smooth pursuit eye movement to a target moving in one direction can be increased by the presence of a distractor moving in the opposite direction or reduced by the presence of a distractor moving with the target. They have modeled this result using a winner-take-all competitive network that receives top-down feedback that biases competition between the target and distractor. They have also found that the responses of some neurons in areas MT and MST to a moving stimulus depend on whether the stimulus is a target of a smooth pursuit eye movement (Ferrera and Lisberger, 1997) and have suggested that this might reflect a top-down biasing signal. In related experiments, Treue and Maunsell (1996) have found that attention modulates the responses of directionally selective neurons in areas MT and MST. They found that the response to a single stimulus is increased in magnitude when the stimulus is attended. However, larger attention effects were observed when attention was directed to one of two receptive field stimuli. When attention was directed to a dot moving in the neuron’s preferred direction of motion, the response was greater than when attention was directed to a dot moving in the opposite direction. Recanzone et al. (1997) have found that neurons in areas MT and MST respond to pairs of stimuli in a manner that is highly consistent with what we have found in the ventral stream; namely, the response to a stimulus moving in a nonpreferred direction was increased by the addition of a second stimulus moving in the preferred direction. Likewise, the response to a stimulus moving in a non-null direction for the cell was suppressed by the addition of a stimulus moving in the null direction. Taken together, these results seem to suggest that biased competition may be a basic computational strategy that has been adopted throughout the visual system and possibly in other modalities as well.
Footnotes
This work was supported by the National Institute of Mental Health Intramural Research Program. J.H.R. was supported by a fellowship from the McDonnell-Pew Foundation. L.C. was supported in part by a grant from the Human Frontier Science Program Organization.
Correspondence should be addressed to Dr. Robert Desimone, Chief, Laboratory of Neuropsychology, 49 Convent Drive MSC 4415, Building 49, Room 1B80, Bethesda, MD 20892-4415.