 |
||||
|
||||
|
||||
|
||||
Previous Article | Next Article
The Journal of Neuroscience, November 1, 2002, 22(21):9475-9489
Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task
Jamie D. Roitman1 and Michael N. Shadlen2 1 Program in Neurobiology and Behavior, and 2 Howard Hughes Medical Institute, Department of Physiology and Biophysics, and Regional Primate Research Center, University of Washington, Seattle, Washington 98195-7290
ABSTRACT
TOP
ABSTRACT
INTRODUCTION
Decisions about the visual world can take time to form, especially when information is unreliable. We studied the neural correlate of gradual decision formation by recording activity from the lateral intraparietal cortex (area LIP) of rhesus monkeys during a combined motion-discrimination reaction-time task. Monkeys reported the direction of random-dot motion by making an eye movement to one of two peripheral choice targets, one of which was within the response field of the neuron. We varied the difficulty of the task and measured both the accuracy of direction discrimination and the time required to reach a decision. Both the accuracy and speed of decisions increased as a function of motion strength. During the period of decision formation, the epoch between onset of visual motion and the initiation of the eye movement response, LIP neurons underwent ramp-like changes in their discharge rate that predicted the monkey's decision. A steeper rise in spike rate was associated with stronger stimulus motion and shorter reaction times. The observations suggest that neurons in LIP integrate time-varying signals that originate in the extrastriate visual cortex, accumulating evidence for or against a specific behavioral response. A threshold level of LIP activity appears to mark the completion of the decision process and to govern the tradeoff between accuracy and speed of perception.
Key words: lateral intraparietal area (LIP); decision; reaction time; random-dot motion; electrophysiology; vision; psychophysics
INTRODUCTION
A hallmark of higher brain function is the ability to form decisions from sensory data to guide appropriate behavioral responses. Decisions tend to be more accurate if subjects are given longer exposure to a stimulus (Green and Luce, 1973
; Wickelgren, 1977
A simple form of decision making occurs when an observer discriminates between two possible interpretations of a visual stimulus. Newsome and colleagues (1989)
Many neurons in the lateral intraparietal area (LIP) respond to visual stimuli that are the target of a planned saccadic eye movement (Gnadt and Andersen, 1988
To characterize neural activity as decision related, the period of decision formation must be clearly defined. In previous experiments, monkeys were given a fixed amount of time to view the motion stimulus and decide its direction. The actual moment when the monkey committed to a behavioral response was not known, so one could not distinguish neural activity associated with decision formation from activity that represented the planned eye movement. We therefore trained monkeys to perform the same task, but to respond as soon as a decision was made. The combination of threshold psychophysics with a reaction time measurement allowed us to examine both the accuracy of the monkeys' perception and the time required to reach a decision. Near psychophysical threshold, we were able to record neural activity during an extended epoch of decision formation, which was determined on each experimental trial. We found that the activity of LIP neurons modulated as the monkey viewed the motion stimulus, predicting both the perceptual decision and the time required to reach it.
Parts of this work have been published previously in abstract form (Roitman and Shadlen, 1998
MATERIALS AND METHODS
We recorded from 54 neurons in the LIP of two rhesus monkeys (both female, 4.5-5 kg) trained to perform a reaction-time direction-discrimination task. Each monkey was implanted with a head-holding device, a recording cylinder for transdural introduction of electrodes (Crist Instruments, Damascus, MD), and a scleral eye coil for monitoring eye position (Fuchs and Robinson, 1966
Neurons were selected according to anatomical and physiological criteria. Postoperative MRI was used to identify LIP and to direct the placement of recording electrodes within the recording chamber. The three coronal images in Figure 1 show the recording locations from one of our monkeys. The arrowheads border the sites where neurons were recorded. A minority of neurons (5-10%) studied in this monkey may have been located within the histological boundaries of the lateral occipitoparietal zone (in "A"), whereas most of the neurons were located in LIPv ("B" and "C") (Lewis and Van Essen, 2000
View larger version (101K):[in this window]
[in a new window]
Figure 1. Representative MRI of recording sites from one monkey. The coronal images show the area of the intraparietal sulcus (ips) studied in these experiments. The recording grid can be seen above the ips. The arrowheads next to the ips represent the boundaries of the locations from which neurons were recorded. Images were obtained using STIR acquisition in 1.5T using carotid RF coils adjacent to the head. Slices are centered 1.5 mm apart (3 mm thickness). The diagram is a lateral view of the brain showing the location of the corresponding coronal images.
We used a memory-guided saccade task to select LIP neurons that were active during saccade planning (Hikosaka and Wurtz, 1983
Direction discrimination task. Monkeys performed a single-interval, two-alternative forced-choice direction-discrimination task (Fig. 2). They were tested in two conditions: reaction time (RT) and fixed duration (FD). The conditions shared the following features. A trial began when the monkey fixated a central fixation point. Two choice targets then appeared, with one located in the RF of the neuron under study (T1) and the other located in the opposite hemifield (T2). The random-dot motion stimulus appeared in a 5°-diameter aperture centered at the fixation point. The stimulus was presented on a computer monitor with a frame rate of 75 Hz. A set of dots was shown for one video frame and then replotted three video frames later. When replotted, a subset of dots was offset from their original location to create apparent motion while the remaining dots were relocated randomly. Therefore, the first pairing of coherent dots occurred after three video frames (40 msec), although random motion was present by the second video frame (13 msec). The net direction of motion was toward one or the other choice target. Both the direction and strength of motion (the percentage of coherently moving dots) were chosen randomly on each trial. The monkey's task was to decide the direction of motion and to indicate its decision with a saccadic eye movement to the appropriate choice target. The monkey received a liquid reward for correct choices and was rewarded on half the trials that used 0% coherent motion. Errors were followed by a short time out (extra 750 msec added to the intertrial interval).
In the RT task (see Fig. 2A), the choice targets were displayed for a variable interval before the onset of the motion stimulus. The duration of this prestimulus interval was randomly selected from an exponential distribution (mean = 700 msec). This randomization served to discourage anticipation of the onset of the motion stimulus. We found that randomization of the prestimulus interval in this fashion was essential for training on the RT task. When the onset of the stimulus was predictable, RT was faster than in these experiments and varied less across the range of coherences [see also Green et al. (1983)
In the FD task (Fig. 2B), the choice targets were displayed for 700 msec followed by the appearance of the random-dot motion stimulus. The monkey then viewed the motion stimulus for 1 sec. This was followed by a random delay of 500-1500 msec, followed by the disappearance of the fixation point, which cued the monkey to report its choice of direction. Until then, the monkey was required to maintain fixation within a window of ±0.5° (monkey N) or ±1.5° (monkey B).
The RT and FD tasks were conducted in alternating blocks consisting of 10-40 trials at each of the six levels of motion strength. The monkeys were informed of the task condition by the color of the fixation point, which was red for FD and blue for RT. Note that in the RT trials, the monkey could take as little or as much time as it needed to make its decision. However, the reward was withheld for a minimum of 800 msec (monkey B) or 1200 msec (monkey N) after onset of random-dot motion, no matter how quickly the monkey indicated its choice. This strategy created an incentive to respond within ~1 sec of motion onset, but no incentive to go any faster. On each trial we obtained a measurement of the monkey's direction judgment and, on the RT version of the task, the amount of time taken to achieve it. We refer to the period from onset of random-dot motion to saccade initiation as the reaction time.
Analysis of behavioral data. For each experiment, the monkey's sensitivity to motion was estimated by plotting the probability (p) of a correct choice as a function of motion coherence (C). The accuracy data were fit by a cumulative Weibull function (Quick, 1974
using a maximum likelihood fitting procedure. The discrimination threshold,
(1) is the coherence level at which the monkey would make 82% correct choices. The second parameter,
, describes the slope of the psychometric function.
Analysis of neural data. All physiological data reported in this paper were acquired from trials in which the monkeys completed the direction-discrimination task by choosing one of the two choice targets. Spike times were recorded to 1 msec precision and aligned to events in the trial. In the RT task, there are two relevant time scales: one that relates the time of spikes to the onset of random-dot motion, the other to initiation of the saccadic eye movement response. Unless indicated otherwise, when reporting data aligned to the onset of motion, we exclude all spikes that occurred from 100 msec before the saccadic eye movement. Therefore, the averages shown on the left portion of Figures 7A, 11, and 12 exclude any perisaccadic enhancement that is commonly observed in LIP neurons (Barash et al., 1991a
We use various regression tests to evaluate the role of motion strength and other factors on the neural response. In general, these regression models provide a convenient test of the null hypothesis that the factor in question does not affect the measured response. They also offer a sensible way to combine data from several neurons: unless indicated otherwise, the regression models make use of an identifier variable, Iu (also known as a dummy variable), to indicate neuron identity. This maneuver adjusts for differences in overall response level between neurons and allows the measurement from each neuron to affect the regression fit with leverage commensurate to its reliability. Unless indicated otherwise, all fits were obtained using weighted least squares (Draper and Smith, 1966
) is distributed as multivariate Normal. Here we list the specific regression models with minimal explanation so as to complement the development in Results.
Effect of motion strength, RT group, and task during specific epochs. To determine whether the neural response was affected by the strength of motion at a particular time (see Figs. 6, 7, and 9), we fit:
where Yt is the spike rate in the epoch (as specified in the text), C is the coherence level from that trial (0-0.512), Iu is the dummy variable that identifies the neuron (Iu = 1 when u = n and 0 otherwise), and
(2A) We used the same strategy to determine whether the response was affected by RT during an epoch before saccade initiation (see Fig. 8). Instead of motion strength, we tested the effect of RT on spike rate, measured at designated times:
where Tgroup is the median RT for the group of trials used to generate the response functions in Figure 8A.
(2B) We obtained data from both the FD and RT versions of the task in a subset of neurons (see Fig. 10). For these neurons we were interested in whether the responses differed according to the behavioral paradigm. For specific epochs during the trials, we expanded Equation 2A to incorporate the effect of task type:
using the same conventions as above. Itask equals 1 or 0 for RT and FD trials, respectively. Any difference in activity between the two paradigms is estimated by
(2C) To examine whether the buildup and decline of activity during motion viewing differs during correct and error trials (see Fig. 11), we modified Equation 2C to compare correct and error trials:
where Icorrect equals 1 or 0 for correct and error trials, respectively. The analysis excludes all 0% coherent motion trials and is performed separately for T1 and T2 choices. The null hypothesis (H0:
(2D) We were also interested in whether the response recorded in particular epochs was related to the amount of time it took the monkey to make its choice (see Figs. 12, 13). For each neuron, all correct trials for each coherence level in which the monkeys had chosen T1 were sorted into a "short" or "long" group on the basis of the RT on that trial. To test the effect of RT group, Equation 2A was expanded to:
using the same conventions as above. In this equation, IRT equals 1 for trials in the short RT group and 0 otherwise.
(2E)
where Ip identifies each unique pattern of random dots and
(2F) Time course of response. To examine the effect of motion strength on the time course of the neural response, we modeled spike rate as a linear function of time and examined the effect of motion strength on the slope of these lines. For these analyses, spike rate (Y) was measured in 20 msec bins, and time (T) denotes the center of the bin. The effect of random-dot coherence (C) on time course of the response can be estimated by fitting:
where T × C is the term describing the interaction between time and motion strength. Responses for trials in which the monkeys selected T1 or T2 correctly were analyzed separately. The interaction term instantiates the possibility that the time-dependent change in spike rate is affected by motion strength. Therefore, the null hypothesis is that
(3A) We also were interested in whether the neural response changed as a function of time early in the trials even when RT was long. Trials were selected according to choice (T1 or T2 correct) and RT (for example, 700-799 msec; see Fig. 8), permitting a reduction of Equation 3A to:
(3B) For correct T1 choices, we tested whether the time course of the neural response was related to the amount of time it takes the monkey to make its choice (see Figs. 12 and 13 and Eq. 2E). For each coherence level, the response was estimated for each RT group with a modification of Equation 3A:
using the same conventions as above. The null hypothesis is that RT group does not affect the slope of the response (H0:
(3C) Saccade metrics. We were interested in whether the variability in the neural response during motion viewing could be accounted for by variability in the eye movements themselves. To test this, we first determined whether saccade metrics were affected by the strength of the motion stimulus. For each saccade, we measured its maximum velocity (Vmax), average velocity (
), duration (Tdur), amplitude (A), reaction time (RT), and accuracy (ACC, 1/distance of saccade endpoint from target, divided by target eccentricity). We tested each parameter to determine whether it varied with coherence for each direction of motion, using the model:
where S is the value for Vmax,
(4A) We then tested whether these saccade parameters could explain the apparent relationship between spike rate and motion stimulus strength. Equation 2A was expanded to incorporate the potential confounding variables:
where C is motion strength and Yt is the response on each trial during the 40 msec epoch ending at the median RT for 51.2% coherence trials. Only saccade metrics that varied as a function of coherence level are included. The null hypothesis is that motion strength does not affect the level of LIP activity once saccade metrics are known (H0:
(4B) Response on single trials. We used a maximum likelihood procedure to estimate the rate of increase in the spike rate from single trials, using spikes from 200 msec after dots onset until 100 msec before saccade initiation. Only correct T1 choices were included in this analysis. The spike rate,
(t), was approximated as a line:
We solved for values of
(5) To evaluate the relationship between the ramp-slope and RT, we used weighted multiple regression to factor out the influence of cell identity and coherence on ramp-slope:
where k is the ramp-slope for each trial (from Eq. 5), C is motion coherence, and RT is the reaction time. As above,
(6) Because behavioral and physiological results were similar in the two monkeys (see Fig. 6, Table 1), all population analyses were performed on combined data from both monkeys.
RESULTS
We recorded from 54 LIP neurons in two rhesus monkeys while they performed a direction discrimination task (Fig. 2). We will first show how the speed and accuracy of the monkeys' direction decisions depended on motion strength. Then we will examine the activity of LIP neurons during the period of decision formation. Finally, we will expose a weak relationship between RT and the activity of LIP neurons on single trials.
View larger version (14K):[in this window]
[in a new window]
Figure 2. Direction discrimination tasks. Monkeys discriminated the direction of motion in a dynamic random-dot display. The color of the fixation point signified whether the experiment follows a reaction time or fixed duration protocol, which were conducted in separate blocks. A, Reaction time version. After fixation, two choice targets appeared in the periphery. One of the targets was within the response field (RF) of the neuron, indicated by the gray shading. After a variable delay period, dynamic random dots appeared in a 5° diameter aperture. The fraction of coherently moving dots and the direction of motion, toward one of the choice targets, were selected at random from a predetermined list of values. The monkey was allowed to make a saccadic eye movement to a choice target at any time after onset of random-dot motion to indicate the direction of perceived motion. A liquid reward was administered for choosing the correct target in the direction of motion and on half the trials in which there was no coherent motion. See Materials and Methods for additional details. Reaction time (RT) is defined as the interval from motion onset to saccade initiation. B, Fixed duration version. After the monkey fixated the central point, two choice targets appeared for 700 msec. The monkey maintained fixation through a 1 sec motion-viewing period followed by a variable duration memory delay. When the fixation point was extinguished, the monkey reported its judgment by making an eye movement to a choice target.
Speed and accuracy of direction judgments
Performance accuracy on the RT and FD tasks depended on the motion strength of the stimulus. Accuracy data from a single recording session are shown in Figure 3A. The monkey's performance varied from chance (50% correct, data not shown) to perfect discrimination as the motion strength increased from 0 to 51.2% coherence. The fitted psychometric function revealed a threshold (
View larger version (13K):[in this window]
[in a new window]
Figure 3. Behavioral data from one experiment. A, Psychometric functions from RT and FD versions of the direction discrimination task. RT and FD tasks were performed in alternating blocks of ~60 trials. The probability of a correct direction judgment is plotted as a function of motion strength and fit by sigmoid functions (see Materials and Methods, Eq. 1). Vertical lines indicate psychophysical thresholds (
Overall, the accuracy of the direction discrimination was not compromised when tested in the RT condition (Table 1). For 38 experiments in which we obtained data on both the FD and RT tasks, the ratio of thresholds (
View this table: [in this window]
[in a new window]
Table 1. Psychophysical performance for RT and FD experiments The amount of viewing time required to achieve such performance varied inversely as a function of motion strength. Figure 3B shows the mean RT for correct choices in the same experiment depicted in Figure 3A. RT varied from 350 ± 9 msec (mean ± SEM) for the strongest motion (51.2% coherence) to 876 ± 35 msec for the weakest motion strength. Summary statistics for all 54 experiments are provided in Table 2. RT tended to be slightly faster for T1 choices than T2 choices, presumably because of the extensive practice given the monkey during RF mapping. The distributions of RT associated with any one condition tend to exhibit positive skew. Various statistics have been advanced to describe the moments and shape of the RT distribution (Luce, 1986
View this table: [in this window]
[in a new window]
Table 2. Reaction time (msec) on correct and error trials The rapid reaction times accompanying strong motion indicate that the monkeys did not procrastinate in reporting their decisions once attained. This is remarkable considering that there was no incentive to respond any faster than 800 or 1200 msec (depending on monkey). In contrast, on the weaker motion strengths, the monkeys often took more than 1 sec to reach a decision (e.g., in 20% of the trials at 3.2% coherence) (Table 2). Presumably, the ability to view a difficult stimulus for a longer time accounts for the small difference in performance on the RT and FD tasks. Overall, the pattern of results indicates that the monkeys took about as long as required to achieve the level of performance to which they had grown accustomed on the 1 sec FD task. Although we did not attempt to manipulate the tradeoff between speed and accuracy in these experiments, one of the monkeys tended to respond with short latencies when first exposed to the RT task. We observed that this monkey's performance improved over weeks as it took more time to respond to weaker stimuli. This observation suggests that the additional viewing time on difficult trials was devoted to solving the task, consistent with previous studies (Gold and Shadlen, 2000
Neural response during direction discrimination task
The direction of motion and the location of targets in the discrimination task were arranged so that the activity of the neuron under study would indicate the monkeys' decisions. One of the choice targets (T1) was in the response field of the neuron; the other choice target (T2) was placed in the opposite hemifield. The random-dot motion was in a 5° aperture centered at the fovea and was directed toward either T1 or T2. The monkey was trained to interpret such motion as an instruction to make an eye movement to the corresponding target. On the basis of our selection criterion (Materials and Methods), we expected neurons to respond more when the monkey planned an eye movement to the target in its RF (Shadlen and Newsome, 1996
Neural activity increased during the period of decision formation when the monkey's judgment resulted in an eye movement to the choice target in the RF (T1) but not before an eye movement to the other target (T2). Figure 4 illustrates the responses obtained in the RT condition of the same experiment as Figure 3. When motion was strong, the monkey made its decision rapidly, and the response modulation was apparent for only ~150 msec before saccade initiation (Fig. 4, top, 51.2% coherence).
View larger version (30K):[in this window]
[in a new window]
Figure 4. Response of an LIP neuron during the RT-direction-discrimination task. Data were obtained from the block of RT trials depicted in Figure 3. Only correct choices at two motion strengths are shown. The diagram at the top indicates whether the monkey's behavioral response was an eye movement into or out of the response field (gray shading). Spike rasters and response histograms are aligned to the beginning of the monkey's eye movement response (sac). Carets denote the onset of random-dot motion. Trial rasters are sorted by RT. The monkey took longer to decide the direction of the weaker (6.4% coherent) motion. Notice the buildup and attenuation of activity that occurred during motion viewing (spike histogram binwidth = 20 msec). Spikes/s, Spikes per second.
When the motion was weaker, the time required to reach a decision was prolonged. As shown in Figure 4 (bottom, 6.4% coherence), the response of the neuron began to increase several hundred milliseconds before the execution of the saccade to the target in the RF. The example exposes a dividend of the combined RT threshold-discrimination task. When the task is easy, it is difficult to interpret the neural data because the sequence of events from stimulus onset to saccade execution occurs within a short time frame. However, when the task is more difficult, the decision is formed over a prolonged period, which can be differentiated from the immediate preparation of an eye movement.
A similar pattern of activation was evident during blocks of trials in which the monkey viewed motion for a fixed duration. Figure 5 illustrates the responses obtained in the FD condition of the same experiment as Figure 3. The activity increased during the period of motion viewing on trials in which the monkey judged the direction as toward the RF, and this change in activity persisted through the delay period until the go signal. When the monkey judged the direction of motion as away from the RF, the activity decreased and remained suppressed through the delay period. This pattern of persistent activity during the delay period helps to distinguish LIP neurons from visual sensory neurons like those in area MT (Seidemann et al., 1998
View larger version (28K):[in this window]
[in a new window]
Figure 5. Response of an LIP neuron during the FD-direction-discrimination task. Data were obtained from the block of FD trials depicted in Figure 3 (same neuron as Fig. 4). The diagram at the top indicates whether the monkey's behavioral response was an eye movement into or out of the response field (gray shading). Spike rasters and histograms are aligned to two events in each trial. In the left portion of each axis, the responses are aligned to the onset of motion, which is then followed by a 1 sec motion-viewing period. In the right portion of the axes, the delay period (del) response is shown aligned to saccade initiation (sac). The break in the panel is attributable to the variable length of the delay period. The elevated spike rate accompanying T1 choices was more pronounced for the easier (51.2% coherent) motion (spike histogram binwidth = 20 msec).
The chief advantage of the RT task is that we can examine the neural activity during the period of decision formation, before the monkey is committed to an eye movement response. In particular, we may ask whether the stimulus has an effect on the neural activity that cannot be accounted for by the initiation of the saccade. We examined the spike rate as a function of motion strength during a short epoch after the onset of random-dot motion (Fig. 6). We chose for this analysis a 100 msec epoch ending at the median RT for the strongest motion strength, thus permitting us to estimate spike rate from at least half of the trials at every coherence level. Figure 6A shows the spike rates associated with T1 and T2 choices for the neuron depicted in Figure 4, plotted as a function of motion strength and fit by a line. For this neuron, the average response associated with T1 choices increased by 42.2 spikes per second per 100% coherence (CI: 14.6-69.9; p < 0.001) (Eq. 2A, H0:
16.5 spikes per second per 100% coherence (CI:
View larger version (19K):[in this window]
[in a new window]
Figure 6. Effect of motion strength on LIP response during decision formation. Graphs depict the effect of stimulus strength on single neuron activity for trials ending in the same eye movement. All data are from correct choices in the RT version of the discrimination task. A, Effect of motion strength on firing rate for the neuron shown in Figures 3 and 4. Spike rate (mean ± SEM) was measured in the same 100 msec epoch for each motion strength. The epoch was chosen to end at the median RT for trials using the strongest (easiest) motion strength. Lines are weighted least squares fits (Eq. 2A) performed separately for T1 and T2 choices (T1, filled symbols, solid line; T2, open symbols, dashed line). The activity of this neuron increased 42.2 spikes per second per 100% coherence for T1 choices (CI: 14.6-69.9) and decreased 16.5 spikes per second per 100% coherence for T2 choices (CI:
We performed this analysis on each of the neurons in our sample. For each neuron, we used this regression procedure to estimate the change in spike rate per 100% coherence on trials that would ultimately culminate in T1 or T2 choices. These values are displayed for each monkey in the histograms in Figure 6B. For the population of neurons studied, the modulation of the response during motion viewing was related to motion strength. Arrows indicate the average change in firing rate for each monkey. The dependence of the response on coherence during motion viewing was found across the population of neurons studied and was similar in the two monkeys.
The evolution of activity accompanying motion viewing furnishes insight into the neural basis of decision formation. Figure 7A shows the averaged responses of 54 neurons recorded while monkeys performed the RT task. On the left half of the graph, the responses are aligned to the onset of random-dot motion. They show activity accompanying motion viewing and exclude any perisaccadic response. On the right half of the graph, the responses are aligned to the onset of the eye movement response. They show activity leading to the monkeys' behavioral response and exclude any response modulation that accompanies onset of random-dot motion. The next three paragraphs focus primarily on the left portion of the graph, during the period in which the monkey is viewing random-dot motion but has not committed to a choice.
View larger version (27K):[in this window]
[in a new window]
Figure 7. Time course of LIP activity in the RT-direction-discrimination task. A, Average response from 54 LIP neurons. Responses are grouped by motion strength and choice as indicated by color and line type. The responses are aligned to two events in the trial. On the left, responses are aligned to the onset of stimulus motion. Response averages in this portion of the graph are drawn to the median RT for each motion strength and exclude any activity within 100 msec of eye movement initiation. On the right, responses are aligned to initiation of the eye movement response. Response averages in this portion of the graph show the buildup and decline in activity at the end of the decision process. They exclude any activity within 200 msec of motion onset. The average firing rate was smoothed using a 60 msec running mean. Arrows indicate the epochs used to compare spike rate as a function of motion strength in the next panels. Arrows a and b mark the 40 msec epoch ending at the median RT for 51.2% motion trials (370-410 msec after stimulus onset); arrows c and d mark the 40 msec epoch ending 30 msec before saccade initiation. Only correct choices are included in these graphs for motion coherences >0%. B, Effect of motion strength on firing rate during decision formation. Response averages were obtained from 54 neurons in the 40 msec epochs corresponding to arrows a and b above. When motion was toward the RF (solid line; epoch a), the spike rate increased linearly as a function of motion strength. When motion was away from the RF (dashed line; epoch b), the spike rate decreased as a function of motion strength. Note that, for the 0% coherence stimulus, there was no net direction of motion, but the activity was greater when the monkey chose the T1 direction. Symbols represent weighted means ± SEM. Lines are weighted least squares fits to Equation 2A (*p < 0.05; H0:
Before the onset of motion, there was a modest level of activity attributable to the presence of one of the choice targets in the RF of the neuron. Approximately 90 msec after the onset of random-dot motion, there was a transient dip and recovery in the activity lasting ~100 msec that did not depend on the strength of motion (indicated by color) or the monkey's choice (solid or dashed lines). The activity then increased or decreased in a manner that reflected the strength of motion and the monkey's ultimate choice. On trials that culminated in T1 choices (direction judged as toward the RF), the activity increased in a ramp-like fashion. The rate of growth in the response was largest for the strongest motion and smaller as the coherence decreased. The slopes of the functions ranged from 21.5 spikes per second squared to 88.8 spikes per second squared as motion strength was varied from 0 to 51.2% coherence (CI: 16.5-26.6 and 49.2-128.3, respectively; p < 0.0001) (Eq. 3A, H0:
Importantly, the ramp-like modulation in discharge accompanied motion viewing and was not an immediate antecedent to the saccadic eye movement. The response averages shown in the left half of Figure 7A are drawn to the median reaction time and do not include any activity in the 100 msec preceding saccade initiation. These curves therefore exclude any enhancement (or attenuation) that occurred just before the eye movement. They reveal clear differences in the neural processing that accompanied the formation of difficult versus easy decisions. It will prove useful for comparison to tabulate the mean response in the 40 msec epoch denoted by arrows a and b in Figure 7A. The mean responses (±SEM) for each coherence level for both directions are shown in Figure 7B. These means, drawn from at least half of the trials at all motion strengths, demonstrate a systematic dependency on motion strength: an increase in activity of 38.4 spikes per second per 100% coherence when motion was toward T1 (a, CI: 23.9-52.8 spikes per second; p < 0.001) (Eq. 2A, H0:
Because many LIP neurons modulate their activity in relation to eye movements (Barash et al., 1991a
In fact, by the time the monkey was committed to a particular eye movement response, differences in neural activity attributable to motion strength were substantially attenuated or absent. This is shown by the solid curves on the right half of Figure 7A, in which the activity was aligned to the monkeys' eye movement responses. On the trials in which the monkeys chose the T1 target (in the RF), the response achieved a common value of 68.6 ± 0.9 spikes per second in the epoch 30 to 70 msec before the saccades, indicated by arrow c. The mean responses for each coherence level during this epoch are shown in Figure 7C. In this interval (c), the neural response was not affected by motion strength (1.1 spikes per second per 100% coherence; CI:
The responses preceding T2 choices (outside the RF) also exhibited a precipitous change before saccade initiation, but unlike the T1 choices, the responses remained associated with motion strength. For example, in the 40 msec epoch indicated by d in Figure 7, A and C, there was a decrease of
The gradual change in LIP activity leading up to the monkey's choice is better appreciated by examining trials sharing the same duration. In Figure 8A, we have grouped the trials by the monkey's RT, using T1 choices from all motion strengths. Each curve shows the average spike rate from sets of trials that end within 25 msec of each other, for example 400-425 msec, plotted as function of time from the saccade. Unlike Figure 7A, the averages shown do not exclude any spikes between stimulus onset and saccade. Trials ending in short RT follow a steeper trajectory than those ending in long RT. For example, in the epoch from 200 msec after motion onset to 100 msec before saccade, the firing rate increased 64.1 spikes per second squared for the 500 msec RT group and 25.7 spikes per second squared for the 900 msec RT group (CI: 34.4-93.9 and 14.1-37.2; both p < 0.01) (Eq. 3A, H0:
View larger version (33K):[in this window]
[in a new window]
Figure 8. Time course of activity on trials with similar reaction time. A, Population average responses for T1-choice trials. The responses are aligned to saccade initiation. Color designates the RT of the trials included in the average, which fall within 25 msec of the time indicated (e.g., 400-425 msec). All spikes are included in these averages (n = 54 neurons). Average firing rate was smoothed using a 60 msec running mean. B, The gradual change in spike rate is evident hundreds of milliseconds before the monkey discriminates motion. Trials with long RT were selected for a closer examination of an early portion of motion viewing period, from 200 to 500 msec after motion onset. This epoch corresponds to the beginning of the coherence-dependent response after the stereotyped dip after motion onset and ends at least 200 msec before the saccadic response. Trials are grouped by RT spanning a 100 msec range: 700-799, 800-899, and 900-999 msec, indicated by red, green, and blue, respectively. Points show the average firing rate, calculated in non-overlapping 40 msec bins. A representative error bar (± 1 SEM) is shown for one data point. Lines are weighted least squares fits (Eq. 3B) performed separately for each RT group. Solid lines and filled symbols correspond to T1 choices; dashed lines and open symbols correspond to T2 choices. The slope (
Importantly, grouping the data by RT allows us to appreciate a gradual change in spike rate for hundreds of milliseconds before the monkeys initiated their eye movement responses. Figure 8B shows the responses in the epoch from 200 to 500 msec after motion onset (that is, subsequent to the transient dip) at the very beginning of the coherence-dependent portion of the response. The responses are shown for the three longest RT groups (700-799, 800-899, 900-999 msec). Points represent the mean spike rate from 40 msec epochs and were fit by lines using weighted regression. The slopes of these regressions and their confidence intervals are shown in Figure 8C (p < 0.05 for all six fits) (Eq. 3B, H0:
The buildup and decline in spike rate also occurred in the FD version of the task (Fig. 9A), where a decision does not immediately lead to an eye movement. Early in the motion-viewing period (left panel), the response modulated in a ramp-like fashion. The mean responses at the time point marked by the arrow a (corresponding to arrow a in Fig. 7A) are shown in Figure 9B (a). The response associated with T1 choices increased 25.8 spikes per second per 100% coherence (CI: 20.1 to 31.6 spikes per second; p < 0.0001) (Eq. 2A, H0:
View larger version (27K):[in this window]
[in a new window]
Figure 9. Time course of LIP activity in the FD-direction-discrimination task. A, Average response from 38 LIP neurons. Responses are grouped by motion strength and choice as indicated by color and line type. On the left, responses are aligned to the onset of stimulus motion. On the right, responses are aligned to initiation of the saccadic response. Response averages in this portion of the graph show activity during the delay period. Otherwise, we use the same labeling conventions as in Figure 7A. Arrows indicate the epochs used to compare spike rate as a function of motion strength in the next panels. Arrows a and b mark the 40 msec epoch from 370 to 410 msec after stimulus onset, corresponding to time points from Figure 7; arrows c and d mark the epoch ending 30 msec before saccade initiation. B, Effect of motion strength on firing rate during motion viewing. Same conventions as in Figure 7B. C, Effect of motion strength on firing rate at the end of the delay period and shortly before the eye movement response. Same conventions as Figure 7C. Motion strength did not affect the level of spike discharge late in the delay period.
The effect of motion strength is apparent early in the delay period, but by the end of the delay, the activity reflected only the monkey's choice of saccade target. Responses preceding saccades to T1 bear striking similarity to the enhancement seen in the RT task. Like the RT data, there was a stereotyped increase in activity that did not vary systematically with motion strength in the epoch 30-70 msec before the saccade (c in Fig. 9A, C) (change of 1.9 spikes per second per 100% coherence; CI:
These qualitative similarities belie important quantitative differences in the responses on the FD and RT tasks. Figure 10 compares the activity from 38 neurons from which we were able to obtain data using both FD and RT tasks. In the epoch before onset of random dot motion (Fig. 10A), activity was on average 4.4 spikes per second higher in the RT task (CI: 3.2 to 5.5; p < 0.0001) (Eq. 2C, H0:
View larger version (13K):[in this window]
[in a new window]
Figure 10. Comparison of neural activity on RT and FD tasks. Scatter plots depict average spike rates from three comparable epochs for 38 neurons studied in both tasks. A, Response before onset of motion. Response averages are from the 40 msec epoch before onset of random-dot motion, when choice targets and fixation point are visible. All correct choices are included in the response averages. B, Response during motion viewing. Response averages are from the 100 msec epoch ending at the median RT for the easiest motion strength (310-410 msec). Averages are computed separately for T1 and T2 choices, as indicated. Only correct choices accompanying 12.8% coherent motion are shown. C, Response preceding eye movements. Response averages are from the period 30-70 msec before the eye movement response. Averages are computed separately for T1 and T2 choices, as indicated. All correct choices are included.
The observations in Figures 7-9 are consistent with the idea that LIP reflects the mounting evidence for or against a T1 choice. The evidence is represented in the ramp-like activity during the period of decision-making
before the monkey is committed to a choice. This period ends ~50 msec before the saccade in the RT task and some time during motion viewing (or just after) in the FD task. The pattern of activity in the RT task in particular suggests that the neurons are accumulating some quantity
