Abstract
The purpose of saccadic eye movements is to facilitate vision, by placing the fovea on interesting objects in the environment. Eye movements are not made for reward, and they are rarely restricted. Despite this, most of our knowledge about the neural genesis of eye movements comes from experiments in which specific eye movements are rewarded or restricted. Such experiments have demonstrated that activity in the lateral intraparietal (LIP) area of the monkey correlates with the monkey's planning of a memory-guided saccade or deciding where, on the basis of motion information, to make a saccade. However, other experiments have shown that neural activity in LIP can easily be dissociated from the generation of saccadic eye movements, especially when sophisticated behavioral paradigms dissociate the monkey's locus of attention from the goal of an intended saccade. In this study, we trained monkeys to report the results of a visual search task by making a nontargeting hand movement. Once the task began, the monkeys were entirely free to move their eyes, and rewards were not contingent on the monkeys making specific eye movements. We found that neural activity in LIP predicted not only the goal of the monkey's saccades but also their saccadic latencies.
Introduction
The role of the lateral intraparietal area (LIP) in the generation of saccades is not clear. Much evidence has implicated LIP in the generation of saccades: (1) it is monosynaptically connected to the superior colliculus and frontal eye field (Ferraina et al., 2002), both of which are known to play a role in the generation of saccades; (2) neurons in LIP discharge in response to the target appearance, during the delay period, and often give a slight presaccadic burst during a memory-guided saccade (Gnadt and Andersen, 1988); (3) when monkeys have to choose a saccade target on the basis of motion information presented at a central window, LIP predicts the saccade goal and weakly correlates with reaction time (Roitman and Shadlen, 2002); (4) high current stimulation of LIP induces saccadic eye movements (Thier and Andersen, 1998); and (5) transient inactivation of LIP slightly affects the latency and accuracy of memory-guided saccades and the latency of visually guided saccades (Li et al., 1999).
Other evidence argues against a reliable role for LIP in the generation of saccades. Wardak et al. (2002) failed to find any effect of LIP inactivation on visually guided or memory-guided saccades, although given the choice of two targets, one in the ipsilateral field and one in the contralateral field, monkeys with inactivated LIP were more likely to make a saccade to the ipsilateral field. LIP inactivation did, however, interfere with contralateral visual search in a fixation search task (Wardak et al., 2004). Furthermore, unlike microstimulation of the superior colliculus or the frontal eye field, saccadic eye movements are only induced with high current stimulation of LIP (Shibutani et al., 1984), especially when the monkey is fixating. Neural activity in LIP can also be dissociated from saccades. Thus, although LIP neurons do not respond well to stable objects in the environment (Gottlieb et al., 1998), they do respond in an enhanced manner to saccade-irrelevant distractors flashed away from the saccade goal during the delay period of a memory-guided saccade. This response can, at the time of the saccade, be the strongest activity in LIP, and yet the metrics of the preplanned saccade are unchanged (Powell and Goldberg, 2000). However, this stimulus does grab the monkey's attention, and the activity in LIP correlates with this, to the point of predicting when the monkey's attention returns from the distractor to the saccade goal (Bisley and Goldberg, 2003). Thus, under conditions in which saccades are strictly controlled, the responses of neurons in LIP correlate with attention but not with saccades.
In all of these studies, rewards were contingent on whether monkeys made or did not make specific eye movements, and, for a water-deprived monkey, the lack of reward caused by a wrong eye movement is aversive (Bowman et al., 1996). However, under normal circumstances, eye movements facilitate vision and generally do not themselves cause rewards or aversive results. To understand the relationship of neural activity in LIP to naturally occurring saccades, we trained monkeys on a visual search task in which they had to signal the results of the task with a hand movement and were free to move their eyes as they pleased. We hypothesized that, if LIP neurons are involved in the process of saccade target selection, then their activity should correlate with saccadic latency. However, if they are involved only in attentional processes and not specifically in saccade target selection, then their activity may still indicate where the saccade will go without correlating with saccadic latency. We found that, under free-viewing visual search, activity in LIP predicted both saccade goal and latency.
Parts of this paper have been published previously in abstract form (Ipata et al., 2004).
Materials and Methods
All of the protocols were approved by the Animal Care and Use Committees at Columbia University and the New York State Psychiatric Institute as complying with the guidelines established in the United States Public Health Service Guide for the Care and Use of Laboratory Animals. Two male rhesus monkeys (Macaca mulatta) were used in these experiments.
We controlled all experiments using the REX system (Hays et al., 1982) and recorded single-unit activity with glass-insulated tungsten electrodes introduced through a guide tube positioned in a grid (Crist et al., 1988).
Behavioral task.
We trained experimentally naive monkeys to perform a free-viewing visual search task in which they were required to report the orientation of a target among seven distractors (Fig. 1A). The target consisted of an upright or inverted black capital T. The monkeys had to release the left bar if the upright T appeared in the array or the right bar if the inverted T appeared. The distractors were black cross-like stimuli, of the same width, length, color, and luminance of the target, but whose vertical and horizontal components crossed each other at different levels. One of the distractors was green and brighter than the target and the other distractors; in this study, we did not examine the responses to this stimulus. The target and distractors were positioned along a circle. The monkeys learned the task with an eccentricity of 10°. Subsequently, we altered the eccentricity chosen so that at least one of the objects was in the center of the receptive field of the neuron under study when the monkey fixated the center of the array. The position of the target among the distractors varied randomly from trial to trial. At 10° eccentricity, the target and distractors were 1.0° wide and 2.9° tall, and their size was adjusted for cortical magnification as the eccentricity of receptive field center changed.
The free-viewing visual search task and the response of the LIP population. A, The free-viewing visual search task. After an initial fixation period (1–1.75 s), the search array appeared. One of the stimuli, either the target (an upright or inverted T) or a distractor, appeared in the center of the receptive field of the neuron. Monkeys had 3 s to report the orientation of the target by releasing one of two bars. During the presentation of the search array, no constraints were imposed on the monkey's eye movements, and they were not required to fixate the target before giving the response. B, C, The average normalized activity from one- and two-saccade trials is plotted against time for monkeys R and Z. These represent 76 and 73% of all trials for monkeys R and Z, respectively. Activity from trials in which the first saccade was made away from the receptive field is shown in blue. Activity from trials in which the first saccade was made toward the receptive field is shown in red. Trials in which the target was in the receptive field have been pooled with trials in which the distractor was in the receptive field. Histograms show the distribution of first saccade latencies relative to array onset.
After the monkeys had reached an asymptotic performance level (98% correct), we implanted them with scleral eye coils (Judge et al., 1980), head restraint devices, and recording chambers during aseptic surgery under ketamine and isoflurane anesthesia. We positioned the chambers using magnetic resonance images taken from the anesthetized animals. We then imposed a fixation requirement, which the monkeys quickly learned. Each trial started when the monkey grabbed two bars. A central point appeared, and, if the animal maintained fixation for 1–1.75 s within a 3° window, the fixation point was extinguished and the stimulus array appeared. After this, the monkeys were free to move their eyes. There was no constraint on the direction and number of eye movements, and the monkeys were neither rewarded for looking at the target nor punished, by withholding of reward, for not looking at it. In fact, one monkey never looked at the target on a minority of trials. They had 3 s to report the orientation of the target. The final step in the monkey's training was to teach them the standard memory-guided delayed saccade task (Hikosaka and Wurtz, 1983).
Data analysis.
All data analysis programs were written in Matlab (MathWorks, Natick, MA). To examine the pattern of activity, we calculated spike-density functions by convolving the spike train, sampled at 1 kHz, with a Gaussian of σ 10 ms (Richmond et al., 1987). We calculated neuronal responses over an interval of interest as the average of the number of spikes in that epoch. To create the population data, we took the square root of every data point from all of the spike-density functions for that cell. We then divided each value by the total mean of the square rooted values from all of the functions. We used this square root normalization method to decrease the effect of outliers (Wilson and Martinez, 1997).
We mapped receptive fields using a memory-guided saccade task, adjusting the target position by hand. Neurons were characterized as having visual or delay activity based on their responses during this task. We identified the receptive field of a neuron as the locations in which the visual response was significantly and at least 75% greater than the background. In some cells, the visual response at adjacent locations (flanks) was significantly greater than background but was <75% greater than the background. We did not include in the analysis trials in which the target appeared in these flanking locations or trials in which the monkey made a saccade to these flanking locations. All of the remaining locations have been classified as outside the receptive field, and we will refer to these positions when we want to specify that one stimulus or the direction of the saccade is outside the receptive field. We also excluded from the analysis trials in which the first saccade had a latency of <80 ms from all analyses to remove saccades that had been planned before the array appeared. These exclusions were all post hoc so that, during the session, the probability that the search target appeared in each location was equal and the monkeys' performance was similar for targets at each location.
We compared trials in which the monkey made a saccade to the receptive field with trials in which the monkey made a saccade away from the receptive field and the flanks. To calculate the time at which the activity from these two types of trials started to separate, we used a sliding window test. For each millisecond, we calculated the activity in a 50 ms bin centered at that time for each class of response. We then compared the activity in each pair of bins using a two-tailed t test. We defined the time of separation as the first bin of 20 consecutive bins that all had p values <0.05. Cells that showed separation after the onset of the first saccade were excluded from additional analysis.
To investigate the relationship between saccade latency and the time at which the neuron first showed a differential response to the saccade goal, we divided the trials into groups according to the latency of the first saccades for each neuron. For monkey R, this involved dividing the trials into three equally sized groups. Monkey Z had a skewed distribution with more short-latency saccades, so we divided his trials into two groups, with the fastest two-thirds of latencies in the first group and the slowest one-third of latencies in the second group.
Results
Behavior
We trained two monkeys to perform a free-viewing visual search task (Fig. 1A). Both monkeys performed the task correctly on 95–99% of the trials, with mean ± SD manual reaction times of 545 ± 101 and 560 ± 78.3 ms for monkeys R and Z, respectively. Although it was not a requirement of the task, both monkeys almost always looked at the target before responding (>99 and 88–99% of trials per session for monkeys R and Z, respectively). The monkeys made almost every eye movement to either a distractor or the target and almost never made a saccade to a blank space between array objects. They required, on average, 1.7 saccades per trial to do the task. Mean ± SD first saccadic latencies were 151 ± 23 and 146 ± 22 ms for monkeys R and Z, respectively (Fig. 1B,C).
LIP activity and saccade goal selection
We recorded a total of 73 LIP neurons from two monkeys (42 from monkey R and 31 from monkey Z) whose activity was enhanced when the monkey made a saccade to the stimulus in its receptive field. Each neuron had significant visual activity, and 71% of them responded significantly during the delay period of the memory-guided saccade task (p < 0.05, t tests compared with background activity). Because there was no difference in the activity of cells with and without delay period activity during our task, we pooled the cells for the remainder of the analysis. Activity in LIP began to distinguish the direction of the upcoming saccade ∼90 ms after array onset. This can be seen in the spike-density functions in Figure 1, B and C. The responses diverged so that the population was more active when the monkey was going to make a saccade to the receptive field (red trace) than when the monkey was going to make a saccade away from the receptive field (blue trace). To calculate the time at which the monkey's choice of saccade direction was clearly represented in LIP, we used the sliding window analysis described above. This occurred in the population 86 ms after the array appeared in monkey R and 91 ms after array onset in monkey Z. In the interval from 70 to 120 ms after the appearance of the array, the average response of the majority of the neurons and that of the population as a whole (p < 0.001 by Wilcoxon's signed rank) was greater when the monkey made a saccade to the receptive field than when the monkey made a saccade away from it (Fig. 2A,B). The difference was even more striking in the 50 ms epoch starting 25 ms before the saccade (Fig. 2C,D). All but six of the cells had more activity when the monkey made a saccade to the receptive field, producing a strong significant difference at the population level (p ≪ 0.0001 by Wilcoxon's signed rank). Thus, under conditions of free visual search, activity in LIP correlates with the monkey's selection of the saccade goal.
LIP neurons predict the direction of the upcoming saccade. The mean activity of each cell when the first saccade was made to the receptive field is plotted against the mean activity when the first saccade was made away from the receptive field for two epochs: A, B, 70–120 ms after array onset; C, D, 50 ms starting 25 ms before first saccade.
LIP activity and saccadic latency
The time at which LIP activity distinguished the direction of the impending saccade correlated with the monkey's saccadic latency. To show this relationship, we plotted the time to saccade goal selection against the mean saccadic latencies for groups of trials from single neurons, sorted by saccadic latency (Fig. 3). Data are only shown for neurons (27 for monkey R, 13 for monkey Z) that gave statistically significant selection times for all groups, and each solid black line connects the points from a single cell. The selection time of the group means changed as saccadic latency changed when calculated from array onset (ANOVA; monkey R, p < 0.001, F = 25, DF = 2; monkey Z, p < 0.004, F = 10, DF = 1) (Fig. 3A,B), such that shorter selection times were associated with shorter saccadic latencies and longer selection times were associated with longer saccadic latencies. When the selection times were calculated from saccade onset, no consistent changes were seen (p > 0.5 in both monkeys) (Fig. 3C,D). The red lines in each plot connect the mean values calculated from all of the data presented. These lines show the relationship at the population level between the selection time and saccadic latency. When the selection times were calculated relative to array onset, the slopes of the red lines were close to 1 (compare with the dotted unity lines in Fig. 3A,B), whereas the red lines were relatively flat when the selection times were calculated backward from the onset of the saccade (Fig. 3C,D). This suggests that, for each millisecond in the selection time calculated from array onset, an extra millisecond is added to the saccadic latency and that there is an approximately equal time between selection time and the time that the saccade is made.
Saccade latency correlates with saccade goal selection time in LIP. A, B, The time from array onset to saccade goal selection is plotted against the mean first saccadic latency for each group for each cell. C, D, The time from saccade goal selection to saccade onset is plotted against the mean first saccadic latency for each group for each cell. Lines connect points from the same cell. The solid red lines connects the population means. The dotted lines show example slopes of 1.
We saw a similar result at the level of single neurons, although many cells had a mixture of saccade-onset and stimulus-onset components. To calculate the slopes for individual neurons, we used linear regression on the three (monkey R) or two (monkey Z) data points that were connected by the solid black lines in Figure 3. So for each cell, we obtained two slopes: one taken from the data aligned by array onset and one taken from the data aligned by saccade onset. The histograms in Figure 4 show the distributions of these slopes in degrees [computed as arctan(slope)] under the two alignment conditions. In both monkeys, most of the neurons have slopes between 25 and 55° when aligned by array onset and slopes between −20 and +20° when aligned by saccade onset. This is similar to the result in the population, which had a slope near 45° when aligned by array onset and near 0° when aligned by saccade onset. The scatter plots show the slope calculated from saccade onset plotted against the slope calculated from stimulus onset. In both monkeys, there is a significant correlation between the two values (p < 0.05 from linear regression). The lines of best fit (Fig. 4, solid black lines) are both close to −1 (monkey R, slope of −1.278, R2 = 0.4995; monkey Z, slope of −0.745, R2 = 0.32), so theoretically, if the slope aligned by saccade onset is 0°, then the slope aligned by array onset should be near 45° and vice versa. This works well with the fit for monkey R, with intercepts of 52 and 41°, but the fit for monkey Z is not as clean, giving intercepts of 17.5 and 23.5°. To test the prediction at the single-neuron level, we added the two slopes for each cell and found a mean of 42.7, which was not significantly different from 45 (p = 0.433, t test), for monkey R and a mean of 26.1, which was significantly different from 45 (p = 0.02), for monkey Z. Thus, in the population, there is a strong, almost unitary relationship between when the saccade goal direction is represented in LIP and when the saccade is made. However, this average arises from a continuum composed of neurons with different mixtures of relationships of latency from array onset and latency to saccade onset, although it is important to point out that most of the neurons lie in the section of the continuum in which the time that the saccade goal is represented in LIP correlates far better with saccadic latency than with array onset.
The relationship between saccade goal selection and saccadic latency for single LIP neurons from monkeys R and Z. The scatter plots show the slopes (in degrees) from the data aligned by saccade onset plotted against the slopes from the data aligned by array onset. The solid lines show the best fit linear regression to the data. The histograms show the distributions of the slopes under the two conditions. Only cells that gave slopes for both conditions are plotted.
Discussion
In this experiment, we have shown that, when a monkey makes saccades in the service of visual search rather than to earn reward, activity in LIP correlates with both the choice of saccade goal and the latency of the saccade. We will discuss these findings in the context of previous studies on the role of the lateral intraparietal area and the neurophysiology underlying the generation of eye movements in visual search.
In all previous studies of the neurophysiology of eye movements in primates, rewards have been contingent on whether the animals make or do not make particular eye movements. However, in real life, eye movements are made to facilitate vision and rarely by themselves yield a positive or aversive result, except for the occasional social taboo. In the task used here, monkeys reported their decision by a nontargeting hand movement. As expected, the average reaction time for the hand movement was long, >500 ms. However, there was no reward contingency on the eye movements, and, as a result, the mean first saccadic latencies were even shorter than the 200 ms reported for simple saccades made from one point to another (Fischer and Boch, 1983). These short latencies, also found by Motter and Belky (1998), are in striking contradistinction to the saccadic latencies found even in easy oddball visual search (Thompson et al., 1996), in which the risk of not receiving a reward may act as a brake on the saccadic system. It could be that, in these studies, the monkeys took advantage of a speed–accuracy tradeoff to prolong their saccadic reaction time and avoid the unpleasantness of failing to receive a reward. In our experiment, there was no such thing as a wrong eye movement that led to an unrewarded trial. Because the monkey failed to fixate the target on the first saccade approximately half the time, it is clear that the monkey did not treat a saccade to a distractor as a particularly aversive event.
When the monkeys made a saccade to the receptive field, the activity of each visually responsive neuron in LIP became greater than when the monkeys made a saccade away from the receptive field. We defined the time at which the two responses differed significantly as the saccade goal selection time. By definition, this time is an average measure, depending on the means of activity from saccade to and saccade away trials. We found that, on average, the interval from array onset to saccade goal selection time [the perceptual duration of Thompson et al., 1996] correlated with saccadic reaction time, with the slope of the population data approaching an angle of 45°, but the interval from saccade goal selection time to saccade onset (the motor duration of Thompson et al., 1996) did not, with a slope approaching 0°. At the level of the single cell, there was a continuum, although it is important to point out that most of the neurons lie in the section of the continuum in which the time that the saccade goal is represented correlates more strongly with saccadic latency than with array onset.
Studying a pop-out search task in which the monkey had to make a saccade to the oddball in an array, Thompson et al. (1996) found that, in the frontal eye field, saccadic reaction time correlated better with motor duration than with perceptual duration. McPeek and Keller (2002) found a similar result for visual on- responses in the superior colliculus, although they did find visuomovement cells whose movement bursts correlated with saccadic reaction time. It is unlikely that this represents a fundamental difference between these areas and LIP. Instead, it is more likely related to the monkey's strategy in performing the task, as attested to by the longer latencies in these pop-out tasks. Using more difficult search tasks, Schall and colleagues (Bichot et al., 2001; Sato et al., 2001) found better correlations with perceptual duration, but, in these examples, the discrimination latencies were themselves longer than our saccadic latencies. It could be that the early specification of saccade latency arises from the coalescence of a visual and a motor burst as occurs in the superior colliculus for express saccades (Sparks et al., 2000). However, this is unlikely because, in our data, some of the neurons with a clear relationship to saccadic reaction time had no delay or presaccadic activity in the delayed saccade task, so the discrimination must then occur as a modulation of the visual response alone.
Although on the average the neural separation times correlated with perceptual and not motor duration, the individual cells lay on a continuum, with some cells having correlations closer to motor duration than perceptual duration. This could be attributable to intrinsic differences among the cells. Alternatively, it could be attributable to the monkey's choosing to emphasize motor duration or perceptual duration at different times, and the continuum therefore arising from the weighting of different strategies on different days. We cannot distinguish between these alternatives. Similar relationships between slopes at the single-neuron level may also be seen in the frontal eye field, and the results seen by Schall and colleagues under the different difficulty levels could reflect a shift in the population along the continuum as required by the demands of the task.
The tight relationship between LIP activity and saccadic reaction time demonstrated in the current experiment differs from the relationship to visual attention demonstrated by similar neurons in an experiment designed to force the monkey to inhibit saccades but to make a perceptual decision (Bisley and Goldberg, 2003). It is unlikely that the function of LIP changes from experiment to experiment or monkey to monkey. Instead, we propose that LIP represents a salience map, in oculomotor coordinates, of the visual field (Koch and Ullman, 1985). This salience map can be used by areas in the visual system to which LIP projects, such as the inferotemporal cortex (IT) and visual cortical area 4 (V4) (Baizer et al., 1991), to determine the focus of attention. Simultaneously, it can be used by the oculomotor areas to which LIP projects, such as the superior colliculus and the frontal eye field (Ferraina et al., 2002), to determine the goal of the next saccade, if a saccade is appropriate.
Under most circumstances, saccades and attention are closely linked, so close that attention from which saccades are divorced has been labeled “covert attention” and attention that is linked to saccades has been labeled “overt attention” (Posner, 1980). Although we did not independently measure attention in this experiment, it is not unreasonable to assert that the monkeys were attending to the goals of their saccades. Thus, when a monkey makes a saccade to an attended object, activity in LIP is tightly linked to attention and to the motor programming of saccades. When the monkey is forbidden to make a saccade to an attended object, activity in LIP is correlated with attention despite the absence of an eye movement. However, because this artificial separation is rare in real life, under normal circumstances, like natural visual search, eye movements, attention, and activity in LIP are ineluctably bound together.
Footnotes
-
This work was supported by National Eye Institute Grants R01 EY014978-01 and R24 EY015634-01 (M.E.G.), the Whitehall, James S. MacDonnell, and W. M. Keck Foundations, and the David Mahoney Chair at Columbia University. Preliminary development of the task was done at the Laboratory of Sensorimotor Research, National Eye Institute (Bethesda, MD). We thank Dr. Mohammed Osman and Dr. Girma Asfaw for veterinary care, Yana Pavlova for wonderful and expert assistance with animal care, and Latoya Palmer for facilitating everything.
-
↵*M.E.G. and J.W.B. contributed equally to this work.
- Correspondence should be addressed to Anna E. Ipata, 1051 Riverside Drive, Unit 87, Room 561, New York State Psychiatric Institute Kolb Annex, New York, NY 10032. Email: ai2019{at}columbia.edu