 |
||||
|
||||
|
||||
|
||||
The Journal of Neuroscience, July 25, 2007, 27(30):8122-8137; doi:10.1523/JNEUROSCI.1940-07.2007
Previous Article | Next Article
Behavioral/Systems/Cognitive
Linking Neuronal and Behavioral Performance in a Reaction-Time Visual Detection Task
Chris Palmer, Shao-Ying Cheng, andEyal Seidemann Department of Psychology and Center for Perceptual Systems, University of Texas at Austin, Austin, Texas 78712
Abstract
Top
Abstract
Introduction
Perceptual decisions are likely to be based on signals that are provided by populations of neurons in early sensory cortical areas. How these neural responses are combined across neurons and over time to mediate behavior is unknown. To study the link between neural responses and perceptual decisions, we recorded the activity of single units (SU) and multiple units (MU) in the primary visual cortex (V1) of monkeys while they performed a reaction-time visual detection task. We then determined how well the target could be detected from these neural signals. We found that, on average, the detection sensitivities supported by SU and MU in V1 are comparable with the detection sensitivity of the monkey even when considering neural responses during brief temporal intervals (median duration, 137 ms) that ended shortly before the monkey's reaction time. However, we observed systematic differences between the overall shape of the neurometric functions and the monkey's psychometric functions. We also examined the quantitative relationship between SU and MU activity and found that MU responses are consistent with the sum of the responses of multiple SU, most of which have low stimulus selectivity. Finally, we found weak but significant trial-to-trial covariations between V1 activity and behavioral choices, demonstrating for the first time that choice probability can be observed at the earliest stages of cortical sensory processing. Together, these results suggest that the activity of a large population of V1 neurons is combined suboptimally by subsequent processing stages to mediate behavioral performance in visual detection tasks.
Key words: visual cortex; visual detection; neurometric function; choice probability; population coding; electrophysiology
Introduction
Stimuli in the environment elicit neural responses that are distributed over large populations of neurons in early sensory cortical areas. These distributed signals must then be read out, or decoded, by subsequent processing stages, to mediate behavior. The nature of this decoding process and, in particular, the identity and number of neurons that contribute to a given perception and the way signals from these neurons are combined to mediate behavior are unknown.Hypotheses regarding the number of neurons that contribute to perception range from a handful of neurons to many thousands of neurons (for review, see Parker and Newsome, 1998
). In several previous studies, the sensitivity of the subject was found to be comparable to the average sensitivity of single neurons (Britten et al., 1992
The current study examines the link between activity of primary visual cortex (V1) neurons and behavioral performance in a visual detection task. Determining the nature and quality of the signals carried by individual V1 neurons is a prerequisite for understanding how these signals might be combined to guide behavior. A central goal of the current study was therefore to characterize the detection performance of single V1 neurons. In contrast to most previous studies in which neuronal and behavioral performances were compared over a fixed and typically long stimulus presentation interval, in the current study we examined neural responses over a brief interval that ended shortly before the monkey's reaction time (RT), ensuring a more realistic comparison between neural and behavioral performances.
In addition, because perceptual decisions are likely to be based on signals from multiple neurons, multiple-unit (MU) activity may provide useful information regarding the quality of the signals in small pools of V1 neurons. It is commonly assumed that MU signals represent the summed spiking activity from multiple neighboring SU. This assumption, however, has not been examined quantitatively. A secondary goal of our study was therefore to determine the quantitative relationship between SU and MU activity in V1.
Finally, to determine the potential contribution of V1 neurons to detection, we examined choice probability (CP), which is the trial-to-trial covariation between V1 activity and behavioral choices.
Specifically, we addressed four primary questions: (1) How much task-related information is carried by V1 neurons? (2) How similar is the performance of simple models that base their decisions on the activity of V1 neurons to the monkeys' performance? (3) Are MU responses consistent with the sum of SU responses? and (4) Is variability in V1 responses correlated with variability in behavioral performance? Importantly, significant CP has not been observed in V1 in previous studies (Nienborg and Cumming, 2006
Materials and Methods
Subjects and surgery
Three monkeys (Macaca mulatta) were used in this study. Monkeys underwent two surgical procedures. In the first procedure, a head-restraining device was implanted, and two custom-designed recording chambers were positioned over the skull above V1 in both hemispheres. After a recovery period, the animal went through an extensive period of training on the visual detection task. Animals were trained using standard operant conditioning techniques in which water and juice were used as positive rewards. During training and recording sessions (2–5 h long), each animal was seated comfortably in a primate chair with its head restrained. Once the animal reached a stable level of performance on the task, a second surgery was performed to prepare the monkey for optical and electrophysiological recordings. In this surgery, a cranial window was opened, and the dura was resected and replaced by a transparent artificial dura (Arieli et al., 2002The animals used in this study were also used in voltage-sensitive dye (VSD) imaging experiments (Chen et al., 2006
All surgical procedures were performed under deep anesthesia, using strictly sterile techniques, in a dedicated surgical suite. All procedures were approved by the University of Texas Institutional Animal Care and Use Committee and conformed to National Institutes of Health standards.
Task and visual stimulus
Each trial began when the animal achieved fixation in a small window (<2° full width) around a 0.1 x 0.1° central fixation point displayed against a uniform gray background (Fig. 1A, first panel). After the initial fixation period, the fixation point dimmed (Fig. 1A, third panel). In the target-absent trials (50% of trials), the monkey had to maintain gaze on the fixation point for an additional 1200 ms to receive the reward. In the target-present trials (50% of trials), the monkey was rewarded for making a saccade to the location of the Gabor target. The Gabor target appeared at a fixed location 300 ms after the dimming of the fixation point (Fig. 1A, fourth panel) and was removed 300 ms later or after the monkey detected the target. To get the reward, the monkey had to make a saccadic eye movement to the target within 600 ms of its onset (but at least 70 ms after target onset) and maintain gaze on the target location for 300 ms.The parameters of the Gabor patch were adjusted to match the preferred parameters of the recorded SU or MU [mean eccentricity, 4.23°; range across experiments, 2.0–9.2°; mean
of Gabor patch, 0.37°; range, 0.167–0.667°; mean spatial frequency, 1.90 cycles per degree (c.p.d.); range, 0.62–4.37 c.p.d.; no orientation bias seen]. These parameters were not changed within a block of trials. Four to seven contrast levels (minimum of 2% up to a maximum of 50%) were used for the target, encompassing the animal's detection threshold. Within a block of trials, target-present trials at different contrasts were randomly interleaved with an equal number of target-absent trials.
Visual stimuli were presented on a gamma-corrected high-end 21 inch color display at a fixed mean luminance of 30 cd/m2. The display subtended 20.5 x 15.4° at a viewing distance of 108 cm, had a pixel resolution of 1024 x 768, had 30-bit color depth, and had a refresh rate of 100 Hz.
Electrophysiology
A tungsten microelectrode (0.5–1.5 M; Frederick Haer Company, Bowdoinham, ME) ensheathed in a protective metal guide tube was lowered to just above V1 through a rubber gasket positioned within a transparent plastic cover sealing the recording chamber. Once the guide tube penetrated the rubber gasket, we advanced the electrode until it extended out from the guide tube by 3–5 mm and then locked it in place. We then advanced the electrode through the artificial dura with a hydraulic microdrive (Narishige, Tokyo, Japan) until a single neuron and/or a cluster of MU was isolated. A dual slope/height window discriminator (Bak Electronics, Germantown, MD) was used to isolate spikes from single neurons. A second independent window was used to accept spikes from several neurons, so MU activity could be recorded concurrently with SU activity.
When recording SU and MU simultaneously, spikes from the SU channel were excluded from the events of the MU channel. Across experiments, the slope and the delay of the slope/height window discriminator were manually adjusted to maximize the chance of detecting spikes. We used the lower threshold of the height window to control the rate of acceptance of a voltage deflection as MU events. The position of this criterion is arbitrary. A high criterion is likely to lead to acceptance of spikes from a small number of single neurons that are near the tip of the electrode. A lower criterion is likely to lead to acceptance of spikes from a larger number of neurons but potentially also to acceptance of some voltage deflections that are not attributable to action potentials. One of the goals of the current study was to examine the effect of pool size on neural performance in the detection task. We therefore varied the level of the lower criterion of the height window across experiments, aiming to sample MU sites with a wide range of pool sizes. Because the number of neurons that contribute to the MU activity is likely to be related to the level of baseline (spontaneous) MU activity, we adjusted the criterion to obtain different levels of baseline MU activity at different recording sites. Across our data set, the range of MU baseline firing rates was
1.2 events/s to
After isolation, we qualitatively analyzed the receptive field properties of the recorded neuron(s). If recording from both an SU and an MU cluster simultaneously, the stimulus was modified to maximize the single neuron responses. This was done using a custom software package (courtesy of G. DeAngelis, Washington University, St. Louis, MO) that allowed interactive variation of the parameters (location, size, orientation, and spatial frequency) of a sinusoidal grating while monitoring neural responses. Based on the initial analysis of the receptive field properties, we performed an additional quantitative assessment using a sine-wave grating presented for 300 ms for a minimum of five trials per condition to identify the preferred orientation of the neuron(s). At some of the sites, we also quantitatively assessed the preferred size (at a fixed spatial frequency of 3 c.p.d.) and spatial frequency (with size fixed at the preferred value) of the recorded neuron(s). To assess the preferred size and spatial frequency, a block of trials with Gabor patches of various sizes or spatial frequencies was run. The smallest size or lowest spatial frequency that gave the maximal response was used for the target detection block. In general, the size increased, and the spatial frequency decreased with increased receptive field eccentricity (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). Bandwidth did not change systematically across eccentricity.
Analysis of behavioral data
For a target-present trial, the trial was scored a hit if the monkey made an eye movement to the target within the specified time frame (70–600 ms after target onset), and the trial was scored a miss if the monkey failed to shift its gaze out of the fixation window. For target-absent trials, the trial was scored a correct rejection if the monkey maintained fixation for the entire length of the trial. The trial was scored a false alarm if the monkey moved its gaze toward the target location. If the animal made a saccade outside of the specified time frame, or made a saccade to a non-target location, the trial was immediately aborted and not used for behavioral or neurophysiological analysis. RT for hits and false alarms was denoted as the duration of the interval between target onset and the time at which the monkey's gaze first left the fixation window (en route to the target location).Behavioral performance was fitted using maximum likelihood estimation by a modified cumulative Weibull function (Green and Swets, 1966
where P(C) is the probability that the subject would report that the target was present as a function of target contrast C;
and ß are the offset and slope terms, respectively; and
is the false-alarm rate. The detection threshold (T) was defined as the target contrast at which the monkey was 75% accurate taking into account both the hit and the false-alarm rates. The overall accuracy was defined as the rate at which the monkey correctly performed the task across all of the trials: accuracy = (#hits + #correct rejects)/#total trials.
Behavior monitoring and data acquisition were performed by a personal computer (PC) running software for real-time neurophysiological recordings from alert animals (Tempo; Reflective Computing, St. Louis, MO). This computer interfaced with an infrared eye-tracker (Dr. Bouis Devices, Karlsruhe, Germany) for high-quality analog eye position monitoring. Eye position signals were sampled with 16-bit resolution at 250 Hz. Electrophysiological signals were sampled at 1 kHz. The data acquisition computer also interfaced with the system used to acquire electrophysiological data (Bak Electronics, Mount Airy, MD). In addition, this computer controlled a dedicated PC with a high-end graphics card that was used for stimulus presentation.
Analysis of physiological data
Integration period. Neural responses were integrated during a short period that started 36 ms after stimulus onset and ended at variable times depending on the monkey's RT (Fig. 1B). We selected 36 ms for the beginning of the integration period because it was approximately the shortest latency of the response to high-contrast targets. The default maximal time for the integration interval was 200 ms after stimulus onset. However, if the RT for a trial was <220 ms, the integration period ended 20 ms before the monkey initiated the saccade to the target. We ended the integration period 20 ms before the saccade to allow for sufficient time between the formation of a decision and the monkey's actual eye movement. We refer to this 20 ms period as the minimal motor response time (MMRT). We also considered other possible values of MMRT and maximal integration time. In addition, we compared results obtained using variable integration times with results obtained using a fixed integration time.Jackknife procedure. We used a jackknife procedure to ensure that we did not overestimate the performance of the ideal observer (see Results). In this procedure, a separate analysis was performed for each of the n trials in an experiment. For each trial, an optimal criterion was established based on the remaining n – 1 trials. The criterion was then applied to the spike count/rate value from the single excluded trial. The trial was classified as correct if the spike count/rate exceeded the criterion for a target-present trial, or if it was below the criterion for a target-absent trial. This procedure was repeated for every trial in an experiment to obtain the neurometric function.
Bootstrap analysis. We performed two types of bootstrap tests. In the first test, we determined whether a single parameter (overall accuracy, T, ß, or
To generate new bootstrap samples for the behavioral and the neural performances, each point on the neurometric and psychometric functions was replaced by drawing a random number from a binomial distribution with parameters n and p, where n is the number of trials and p is the proportion of trials in which the subject reported that the target was present for that contrast level. The new data points were then fitted with the modified Weibull function to give new estimates for overall accuracy, T, ß, and
To determine whether the overall shapes of the two functions were significantly different, we combined the three parameters used to describe the neurometric and psychometric functions (T, ß, and
To find the criterion that minimizes the difference between the neurometric and psychometric functions, we used the procedure described above to compute d' between the neurometric and psychometric functions for each possible criterion and selected the criterion that minimized d'.
Monte Carlo analysis of independent SU. We simulated the performance of a pool of independent, similarly tuned SU using a simple Monte Carlo analysis. To simulate a pool of N units, N SU were randomly chosen (with replacement) from the set of recorded single neurons. To simulate a single trial, the response of each neuron was randomly selected from the responses of the neuron at that contrast, and the pooled response on that trial was computed as the sum of the responses from the N neurons. Each simulated experiment included 20 trials for each of five target contrasts and 100 blank trials. We then computed the neurometric function from these simulated pooled neural responses and extracted the fitted parameters of the neurometric function. These steps were repeated 100 times with different random pools of the same size, and the mean and SD of the parameters for the neurometric functions were noted. The above steps were repeated for pool sizes of two neurons up to 33 neurons.
Analysis of CP
We performed a separate CP analysis for target-present trials (hits vs misses) and for target-absent trials (correct rejects vs false alarms). For each SU and MU site, only target contrasts for which at least three trials were obtained in each behavioral category were analyzed. For a given contrast, firing rates in all trials (regardless of behavior) were first converted into z-scores by subtracting the mean response and dividing by the response SD. z-score values were then combined across contrasts to form two distributions for the two possible behaviors (say hits vs misses). CP values were computed for each site as the area under the receiver operating characteristic (ROC) curve (Green and Swets, 1966Analysis of eye movements
We investigated the effects of four indicators of eye movements during the period in which neural responses were averaged: (1) average distance from the fixation point, (2) average distance from the target, (3) average velocity, and (4) maximal instantaneous velocity. The average distances are the mean distance between the center of gaze and the fixation point or the target location during the integration period. The average velocity is the overall distance the eye moved during the integration period divided by its duration. Instantaneous velocity is the maximum instantaneous velocity of the eye during the integration period.To determine whether eye movements affect the relative neural and behavioral sensitivities, we split each experiment in half and computed overall accuracy for the half containing the trials with the lower-than-median eye movement values and the half with the higher-than-median eye movement values for each of the four indicators (Table 1). Ten experiments were excluded from this analysis because splitting the data to two halves left too few trials to evaluate the performances of the model and the monkey.
View this table:
[in this window]
[in a new window]
Table 1. Effect of eye movements on neural and behavioral accuracies
To examine the potential contribution of eye movements to CPs, we determined whether neural responses were significantly correlated with the four eye movement indicators.Neuron database
The results here are based on 62 MU recordings and 33 SU recordings. SU/MU recordings had to meet three criteria to be included in our analysis: (1) minimum of eight repetitions per condition; (2) overall behavioral accuracy within 70–95% (so that we could reliably estimate the parameters of the psychometric function); and (3) stable baseline neural activity across the entire block (discussed below). Of the 55 single neurons that were initially screened, 22 were excluded because isolation was not maintained for long enough to collect a minimum of eight repetitions per condition. All of the original 78 MU experiments resulted in eight repetitions per condition. Four experiments were excluded based on the behavioral accuracy falling outside of the accepted range. In addition, the recordings of 12 MU experiments were found to be unstable over time.To assess the stability of the MU responses across a block of trials, baseline activity for each trial was computed as the spike rate in the 300 ms interval before the fixation point dimming. Baseline activity was averaged over bins of 10 consecutive trials. Then, the trial order was randomly shuffled, and again, baseline activity was averaged over bins of 10 consecutive trials. This was done 1000 times to obtain 95% confidence intervals on the baseline measurement. If a significant number of the original data points (bins of 10 trials) fell outside of the 95% confidence interval, then the block was considered unstable and removed from the database. We used the binomial distribution to determine how many times the data had to fall outside of the 95% confidence interval to be considered significantly unstable.
Results
Monkeys were trained to detect a small oriented visual target and indicate target presence by shifting gaze to the target location as soon as it was detected (Fig. 1A). The target appeared in half of the trials. In the remaining target-absent trials, the monkey was rewarded for maintaining fixation. Target contrast was randomly selected from several contrast levels spanning the monkey's detection threshold.
View larger version (17K):
[in this window]
[in a new window]
Figure 1. Visual detection task. A, The monkey initially shifted gaze to a fixation point at the center of the screen. The fixation point then dimmed. Three hundred milliseconds later, a target could appear in the periphery. The monkey was allowed to make a saccade to the target location as soon as it had detected the target. Three hundred milliseconds later, the target was extinguished, but the monkey could still make a saccade to the target for another 300 ms. B, Time line for computing mean firing rates and spike counts. Mean firing rates and spike counts were computed over a period starting 36 ms after target onset and ending 20 ms before the monkey's saccade to the target [allowing for motor preparation (Motor Prep.) time] or 200 ms [maximal (max) integration time] after target onset, whichever came first. In this example, the rate/count was computed over just the light gray bar. The dark gray bar shows the overlap between the maximal integration interval and the motor preparation interval (black bar). C, Psychometric function describing how often the monkey reported the presence of the Gabor target as a function of the contrast of the target. The solid line is the best-fit Weibull function (see Materials and Methods). The dashed vertical line indicates the monkey's threshold. Note that threshold is defined as the contrast at which performance is 75% correct combined across target-present and target-absent trials. Because the false-alarm rate in this experiment was close to zero, threshold is the contrast at which the hit rate was
While the monkey performed the detection task, we used extracellular recording techniques to measure the activity of SU and MU in V1. The results reported here are based on recordings from 33 SU and 62 MU sites in five hemispheres of three monkeys. To maximize the probability that the recorded neurons would contribute to performance in the task, at each recording site, the stimulus was tailored to match the preferred stimulus parameters of the recorded neuron(s) (see Materials and Methods).We started our analysis by assessing the behavioral sensitivity of the monkey and comparing it with the detection sensitivity that can be supported by the activity of SU and MU in V1. Next, we compared the shapes of the neurometric and psychometric functions. We then proceeded to analyze the quantitative relationship between the SU and the MU responses. Finally, we examined the trial-to-trial covariation between V1 activity and behavioral choices.
Assessing behavioral detection sensitivity
Figure 1C shows the behavioral performance of the monkey in one experiment. Each point represents the proportion of trials in which the monkey indicated that the target was present as a function of target contrast. Target contrast of zero represents the half of the trials in which the target was absent. Performance was fitted with a psychometric function (solid black line) derived from a modified Weibull function (Quick, 1974The time it took the monkey to initiate a saccade to the location of the target depended on the target contrast. Figure 2A shows the mean RT of the three monkeys at different target contrasts collapsed across all experiments. As target contrast was lowered, average RTs became significantly longer (significance test for regression slope; p = 0.008). These results suggest that longer durations are required to reach perceptual decisions regarding the presence of the target at lower target contrasts.
View larger version (9K):
[in this window]
[in a new window]
Figure 2. Mean RTs. A, Mean RTs as a function of binned target contrast collapsed across all experiments (n = 74). Data points lie in the middle of the bin. B, Mean RTs for experiments in which the false-alarm rate was above 15% (n = 21). RTs at target contrast equal to zero correspond to false-alarm trials. Target contrast bins are (0, 2–3, 4–6, 7–10, 11–15, 16–20, 21+). Solid line represents best linear fit of mean RT as a function of log contrast in target-present trials. Error bars are SEs across experiments.
Interestingly, the average RT in target-absent trials (false alarms) was shorter than the average RT at low target contrasts. This difference, however, was attributable to the fact that in many experiments the monkey made almost no false alarms, and in those experiments RTs tended to be longer. In fact, in the subset of experiments with false-alarm rates >15%, RTs were very short even at the lowest target contrasts (Fig. 2B). In this subset of the experiments, average RTs were indistinguishable in false alarms and in target-present conditions with hit rates comparable to the false-alarm rates (paired t test, p = 0.74; n = 18). These results suggest that, in some experiments, the monkey tended to make fast saccades to the target location even in the absence of sufficient sensory evidence. Such "fast guesses" could explain the drop in the average RT in false-alarm trials.Fast guesses could also lead to a drop in the monkey's detection sensitivity. This effect, however, appears to be small. In the two animals with significant false-alarm rates, behavioral thresholds in experiments with above the median false-alarm rates were not significantly different from behavioral thresholds in experiments with below the median false-alarm rates (paired t test, p > 0.1 for both monkeys).
Target eccentricities varied across sites from
View larger version (11K):
[in this window]
[in a new window]
Figure 3. The relationship between target eccentricity and behavioral and neuronal detection thresholds. A, Scatter plot of behavioral thresholds as a function of target eccentricity. Because the target parameters were matched to the preferred parameters of the recorded V1 neuron(s), the size of the target increased with eccentricity, and the spatial frequency decreased with eccentricity (supplemental Fig. 1, available at www.jneurosci.org as supplemental material). B, Scatter plot of neural detection thresholds for the rate model versus target eccentricity. C, Scatter plot of the ratio of neural over behavioral detection thresholds versus target eccentricity. Black data points are MU, and gray data points are SU. Black lines show the best-fit linear regressions for the MU data, and the gray lines show the best-fit linear regressions for the SU data. The Pearson correlation coefficient r and its significance value p are indicated in each panel for SU and MU separately. contr., Contrast.
To compare the monkey's detection sensitivity with the detection sensitivity of SU and MU in V1, we next used an ideal observer analysis that allowed us to evaluate the neural responses with the same metrics used to describe the monkey's performance.Ideal observer analysis of neural detection sensitivity
We used the responses of one SU site and one MU site as example experiments (Fig. 4). Both SU and MU responses in the example experiments increased monotonically with target contrast. We used a simple model based on signal detection theory (Green and Swets, 1966
View larger version (20K):
[in this window]
[in a new window]
Figure 4. Peristimulus time histograms from two representative recording sites. A, SU. B, MU. Neural responses were averaged over repetitions (n = 10 for each target contrast; n = 50 for target-absent trials). Bin size is 25 ms, and data points lie in the middle of the bin period. Vertical blue lines denote stimulus onset (time 0) and offset (time 300 ms). For conditions with at least four hits, colored arrows indicate the median RT. Horizontal black lines denote typical times of neural integration. The key shows target contrasts; 0% contrast = target-absent trials. SU and MU recordings are from different sites. sp, Spikes.
Neural responses of SU or MU sites were first integrated over a short temporal window (median duration, 137 ms) that started 36 ms after stimulus onset and ended either 20 ms before the onset of the monkey's saccadic response, allowing for a MMRT, or 200 ms after stimulus onset, whichever occurred first (Fig. 1B) (see Materials and Methods).We initially considered two models for detecting the target from the neural responses in single trials. The first model (summation) used the raw integrated responses to detect the target. In the second model (rate), the integrated responses in each trial were divided by the duration of the integration interval (which varied from trial to trial) to obtain an average spike/event rate.
Spike counts and spike rates were compiled into separate histograms for each target contrast (Fig. 5A,B, summation, C,D, rate), with target-present trials shown as filled bars alongside distributions of an equivalent number of target-absent trials (open bars). There was almost no overlap between the spike count and spike rate distributions in target-present and target-absent trials at the highest target contrast (Fig. 5A–D, top panels). As the target contrast was lowered, significant overlap was seen.
View larger version (24K):
[in this window]
[in a new window]
Figure 5. Response distributions and neurometric/psychometric functions. A, Distributions of spike counts for target-present trials (filled bars) and target-absent trials (open bars) from the SU recording session shown in Figure 4A. Spikes were counted during the integration period described in Figure 1B. B, MU spike counts from the recording session shown in Figure 4B. Dashed vertical lines in A and B are optimal criteria for the summation model. C, D, Same as A and B, but for spike rates rather than spike counts. E, F, Comparison of the monkey's performance with that of the count and rate models that base their decisions on the neural responses in single trials. Each point represents the probability that the observer (monkey or model) would indicate the presence of the target. E, SU neurometric and psychometric functions (same experiment as Fig. 4A). F, MU neurometric and psychometric functions (same experiment as Fig. 4B). Overall accuracy is listed in each panel. Detection thresholds (T) are shown with thin vertical lines. Error bars show 95% confidence intervals for threshold (horizontal) and false-alarm rate (vertical). Prob., Probability.
Our primary goal here was to compare the neural sensitivity implied by the distributions in Figure 5A–D with the behavioral sensitivity of the monkey. A common measure of the detection sensitivity implied by two distributions is the area under the ROC curve (Green and Swets, 1966To ensure that we did not overestimate the detection sensitivity of the neuron(s), the analysis was performed for each trial separately using a jackknife procedure (Efron and Tibshirani, 1993
Similar to the behavioral thresholds, neural detection thresholds were correlated with stimulus eccentricity (Fig. 3B) for both SU and MU. As stimulus eccentricity increased, neural detection threshold also tended to increase. This is somewhat surprising because the parameters of the visual target were matched to the preferred parameters of the recorded neuron(s). The ratios of the neural to behavioral threshold were not significantly correlated with eccentricity (Fig. 3C), indicating that eccentricity had a similar effect on neural and behavioral sensitivity in our data set.
Comparing neural and behavioral detection sensitivities
In the example SU site, the performance of both models was comparable to that of the monkey (overall accuracy of summation model, 80%; rate model, 82%; monkey, 83%). In the example MU site, the summation model performed significantly worse than the monkey (overall accuracy of summation model, 67% vs monkey, 79%; bootstrap test, p = 0.008). The performance of the rate model, in contrast, was comparable to that of the monkey (overall accuracy, 78%).Across our data set, the summation model was significantly less accurate than the monkey (Table 2). This was true whether the neural performance was based on SU or MU responses. The average accuracy of the rate model, on the other hand, was not significantly different from the average accuracy of the monkey (Table 2, Fig. 6A) for both SU and MU.
View this table:
[in this window]
[in a new window]
Table 2. Quantitative comparison of the psychometric functions with the ideal-observer neurometric functions
View larger version (31K):
[in this window]
[in a new window]
Figure 6. Scatter plots of the parameters of the monkey's psychometric functions versus the parameters of the neurometric function for the rate model. A, Overall accuracy. B, Detection threshold (T). C, Slope (ß). D, False-alarm rate (FA;
Additional analysis (discussed below) indicated that the accuracy of the summation model could not be significantly improved by integrating neural responses over shorter or longer intervals. These results demonstrate that a model that bases its decisions on spike counts of SU or MU in V1 is relatively inefficient and is inconsistent with the monkey's performance in the detection task. The rate model, however, showed an average detection sensitivity that was comparable to that of the monkey. We therefore proceeded to quantitatively compare the shape of the neurometric functions of the rate model with the shape of the psychometric functions.In the example experiments, the detection thresholds of the rate model (T = 5.0% in the SU site; T = 6.62% in the MU site) were also comparable to the monkey's detection thresholds (T = 5.2% in the SU site; T = 7.25% in the MU site). Across all experiments, average detection thresholds were slightly lower for the monkey than for the rate model for both SU and MU (Table 2, Fig. 6B).
Behavioral and neural thresholds varied somewhat between the three monkeys. In particular, one monkey had significantly lower neural and behavioral thresholds than the other two animals, probably because of the lower eccentricities of the recording sites in this animal (Table 3).
View this table:
[in this window]
[in a new window]
Table 3. Threshold and stimulus parameters per monkey
In the 21 experiments in which both SU and MU were recorded simultaneously from the same microelectrode, there was no significant difference between SU and MU in overall accuracy and threshold under the rate model (Table 4). The quantitative relationship between SU and MU activity is further analyzed below.
View this table:
[in this window]
[in a new window]
Table 4. Comparison of neurometric functions for SU and MU recorded simultaneously
Our finding that the detection sensitivities of SU and MU in V1 using the rate model were comparable to the monkey's detection sensitivity demonstrates that V1 neurons are highly sensitive to stimuli near detection threshold even when using brief, behaviorally relevant integration periods. In addition, it also shows that detection sensitivity of SU is comparable to that of MU.Quantitative comparison of the shapes of the neurometric and psychometric functions
Although the overall accuracies and thresholds of the monkey and the rate model were comparable in the example experiments, there were significant differences in the shapes of the neurometric and psychometric functions. In the SU recording site (Fig. 5E), the slope (ß) of the neurometric function for the rate model was significantly shallower than the slope of the psychometric function (p = 0.012, bootstrap test) (see Materials and Methods). In the MU recording site (Fig. 5F), the neural false-alarm rate (Across our data set, we found systematic differences between the shapes of the psychometric and the neurometric functions (Table 2). Most notably, the slopes of the neurometric functions tended to be significantly lower than those of the psychometric functions (Table 2, Fig. 6C), demonstrating that task difficulty affects neural and behavioral performances differently. These results indicate that the performance of an ideal observer model that bases its decision on the spike rate of a SU or a small pool of neurons in V1, although highly sensitive, is inconsistent with the behavioral performance of the monkey.
An additional difference between the neurometric and psychometric functions was that for many of the SU sites the neurometric function had a lower false-alarm rate than the monkey (Fig. 6D). The reason for this low false-alarm rate is simple. For many SU, in the vast majority of target-absent trials the neuron did not fire at all during the short integration interval (Fig. 5A). Because the ideal observer had to use a criterion that was larger than zero, the false-alarm rate of the ideal observer was typically very low. Because the level of baseline activity was much higher for MU sites, a rate model that based its decisions on MU activity had an average false-alarm rate comparable to that of the monkey (Table 2, Fig. 6D).
Similar results were obtained in a site-by-site bootstrap analysis that compared the overall shape of the neurometric and psychometric functions (simultaneously taking into account T, ß, and
The models that we considered so far used the optimal criterion to separate target-present from target-absent trials. In reality, the monkey may not be able to use the optimal criterion to perform the detection task. We therefore determined whether suboptimal criteria could provide a significantly better match between the psychometric and neurometric functions. For each SU and MU site, we varied the detection criterion and found the criterion that produced a neurometric function that was most consistent with the psychometric function (see Materials and Methods). We then compared the average parameters of the psychometric function with the average parameters of these new neurometric functions (Table 5).
View this table:
[in this window]
[in a new window]
Table 5. Comparison of the psychometric functions with the neurometric functions that are most consistent with behavior
For the SU and MU sites, the neurometric functions were still significantly shallower, on average, than the psychometric functions even when using the criterion that minimized the difference between the neurometric and psychometric functions at each site. Furthermore, the accuracies of the model were now significantly lower, on average, than the accuracies of the monkey (Table 5).The most consistent difference between the neurometric and psychometric functions was the steeper slopes of the psychometric functions. For the vast majority of the SU (30 of 33) and the MU (45 of 62) sites, no criterion (optimal or suboptimal) could be found that would produce a neurometric function with an equal or steeper slope and an equal or lower threshold than those of the monkey.
In addition, we performed a simple Monte Carlo analysis to examine how the shape of the neurometric function changes as a function of the size of a pool of independent, similarly tuned SU (see Materials and Methods). The performances of simulated pools of different sizes were compared with the average performance of the monkey (Fig. 7, solid horizontal lines). As the size of the pool increased, the accuracy of the pooled response became higher and the threshold became lower, exceeding the monkey's detection sensitivity for pools of eight or more neurons (Fig. 7A,B). False-alarm rates initially increased but then decreased to a level lower than that of the monkey at large pool sizes. In the range of five to seven pooled SU, the average accuracy, threshold, and false-alarm rate for the pooled responses were all comparable to those of the monkey, yet the slope of the neurometric function remained lower than the slope of the monkey's psychometric function. In fact, there was no pool size for which the slopes could be matched. We consider possible ways in which the neurometric function could be steepened in the Discussion.
View larger version (29K):
[in this window]
[in a new window]
Figure 7. Monte Carlo simulations of the performance of summed independent SU as a function of the number of neurons in the pool. A, Expected accuracy (percentage correct). B, Expected threshold. contr., Contrast. C, Expected slope. D, Expected false-alarm rate. Each data point is the mean of 100 random combinations (with replacement) of SU from our data set. Error bars are SD. Solid lines are the mean values for the monkeys' behavior during the 33 SU recordings. Dashed lines are the mean values for the 62 MU recordings. The point at n = 1 is the average value for the SU.
The dashed horizontal lines in Figure 7 show the average parameters of the neurometric functions based on MU responses. The average detection sensitivity of the MU responses was significantly lower than the predicted detection sensitivity based on simulations of pools of eight or more independent and similarly tuned SU. Additional analysis of the quantitative relationship between SU and MU responses described below revealed some possible sources for this discrepancy.Overall, our results demonstrate significant inconsistencies between neural and behavioral performances when using models that base their decisions on spike counts or spike rates of SU or MU in V1. In general, the performance of models based on MU activity was more similar to the performance of the monkey (Tables 2, 4, 5). These results suggest that perceptual performance in our detection task is based on signals provided by a larger and/or a more widely distributed population of neurons than the ones recorded by our MU signals.
Effect of varying integration period parameters
To test the robustness of our results, we evaluated the performance of the rate and summation models using various values for the maximal integration time and MMRT. Although there was a slight improvement in the average accuracy of the rate and summation (count) models based on SU activity when using a maximal integration time longer than 200 ms (Fig. 8A), the improvement was not significant. The rate model based on the MU activity also did not improve significantly for maximal integration times above 200 ms (Fig. 8B). In contrast, the accuracy of the summation model based on the MU activity peaked for a maximal integration time of
View larger version (19K):
[in this window]
[in a new window]
Figure 8. Effect of varying the integration period parameters on accuracy and threshold. A, B, Mean accuracy as a function of the maximal integration time (50 ms up to 450 ms) for SU recordings (A) and for MU recordings (B). C, D, Mean threshold as a function of the maximal integration time for SU (C) and for MU (D). E, F, Mean accuracy as a function of MMRT (0 ms up to 60 ms) for SU (E) and for MU (F). G, H, Mean threshold as a function of MMRT for SU (G) and for MU (H). In A, B, E, and F, the MMRT was fixed at 20 ms. In C, D, G, and H, the maximal integration time was fixed at 200 ms. Solid horizontal lines indicate the mean accuracy or threshold for the monkey, and the dashed lines indicate the corresponding SEM. Asterisks indicates data points where the overall accuracy (threshold) was significantly lower (higher) than the monkey based on a t test. Error bars are SEM. Max, Maximum; contr., contrast; SUA, single-unit activity; MUA, multi-unit activity.
When higher MMRT values (30–60 ms) were used, overall accuracy of both models decreased significantly (Fig. 8E,F), with the summation model performing consistently worse than the rate model, and threshold increased significantly (Fig. 8G,H). Most of this decrease in performance for longer MMRT came from the high-contrast target conditions (usually 25% target contrast), where the model in many experiments performed well below 100% and sometimes worse than at lower target contrasts. Because RTs were very short on high-contrast trials (Fig. 2), assuming a long MMRT left extremely short intervals for the evaluation of neural signals in such trials, which led to decreases in performance.Because many of the monkey's RTs were shorter than 200 ms, it was often the case that target-absent trials had longer integration periods than target-present trials. To exclude any potential bias because of this variable integration time, we repeated our analyses using a fixed integration period of 200 ms (regardless of the monkey's saccade time). The neurometric functions remain almost identical under this analysis (supplemental Table 1, available at www.jneurosci.org as supplemental material), demonstrating that our results are not systematically biased by the variable integration duration.
The differences in the performance between the summation and the rate models were more pronounced for the MU data than for the SU data (Table 2), particularly for longer integration times (Fig. 8A,B). Because the integration durations tended to be short in target-present trials and longer in target-absent trials, the additional integration time reduced the separability between spike counts from target-present and target-absent trials but had little effect on the rate model. The MU responses were more sensitive to this reduced separability because the relative differences between the evoked response and the baseline response were much smaller for the MU responses than for the SU responses (see analysis of the quantitative relationship between SU and MU responses below).
Possible effect of eye movements
Small eye movements within the fixation window (microsaccades and drifts) could have affected our detection sensitivity measurements. Eye movements could cause the stimulus to move outside of the receptive field of the recorded neuron(s). This could lead to a decrement in the neural performance and could bias the comparison of neuronal and behavioral detection sensitivities if the monkey could effectively integrate signals from different eye positions. To rule out this possibility, for each experiment we used a simple test to capture the effect (if any) of small, fixational eye movements during the integration period. We investigated four indicators of eye movements (see Materials and Methods). For each indicator, we recalculated the overall accuracies of the monkey and the rate model for half of the trials with the best fixation and half of the trials with the worst fixation during the integration period. Both the monkey and the rate model performed slightly better on the best fixation trials across all four indicators (Table 1), but these differences were not significant. These results indicate that small eye movements during the integration period are not likely to have influenced our result.Quantitative comparison of SU and MU responses
Our results indicate that MU in V1 have detection sensitivities that are not significantly different from single V1 neurons (Table 4). It is somewhat surprising that MU were not more sensitive than SU. The results of our simple Monte Carlo simulation (Fig. 7) demonstrate that a pool of eight or more independent and similarly tuned single neurons would be significantly more sensitive than the average MU site. Similarly, a simple calculation shows that it is enough to sum responses from two neurons that have comparable detection sensitivities and are weakly correlated to obtain a pooled performance that is more sensitive than each neuron on its own [Chen et al. (2006)
View larger version (10K):
[in this window]
[in a new window]
Figure 9. Neural threshold of MU sites as a function of baseline firing rate. The solid line is a best-fit regression line, with the Pearson correlation coefficient r and its significance value p noted. contr., Contrast.
There are at least three ways in which the detection sensitivity of the MU responses could be degraded relative to the detection sensitivity predicted by our Monte Carlo simulations (Fig. 7). First, V1 neurons are selective to multiple parameters of the visual stimulus, and therefore a significant fraction of the neurons that contribute to the MU activity may carry little or no signal regarding the target [e.g., neurons for which the stimulus was in the null phase relative to the receptive field or neurons that are tuned to higher spatial frequencies (Geisler and Albrecht, 1992To explore the possibility that nonselective SU contribute to the MU signals, we compared the average baseline and evoked MU responses with those expected by summation of SU from our data set. For each SU and MU site, we computed the average number of spikes during the 200 ms after the time of stimulus onset at each contrast level. These responses were then fitted with a Naka-Rashton contrast response function of the following form:
where C is the target contrast; R(C) is the average number of spikes at contrast C; Rmin and Rmax are the baseline and maximal response, respectively; C50 is the half-saturation contrast; and n is an exponent. In general, this function provided an excellent fit to the data, accounting for >95% of the variance for 28 of the 33 SU sites and 55 of the 62 MU sites.
As expected, baseline and maximal response were much higher for MU than for SU (median Rmax: SU, 6.72; MU, 30.84; median Rmin: SU, 0.17; MU, 4.54) (Fig. 10A,B). On the other hand, contrast response functions for SU and for MU were similar in terms of their exponent and half-saturation (median n: SU, 3.81; MU, 3.08; median C50: SU, 8.45; MU, 8.38) (Fig. 10C,D).
View larger version (13K):
[in this window]
[in a new window]
Figure 10. Comparison of contrast response function parameters for SU and MU. Mean response as a function of contrast was fitted for each SU and MU site using a Naka-Rashton contrast response function (see Results). A, Histogram of Rmax values. Open bars, SU; filled bars, MU. Gray arrows indicate median value for SU; black arrows indicate median value for MU. B, Histogram of Rmin values. C, Histogram of the exponent n. D, Histogram of half-saturation constant C50. B–D have the same conventions as in A.
If MU activity was the sum of the response of multiple SU that were all similarly tuned, as more SU were added to the MU pool, we would have expected both the baseline response Rmin and the selective response Rmax – Rmin to increase at a similar rate. If, on the other hand, some of the SU that contributed to the MU signal were only weakly tuned to the target, those neurons would have contributed to the baseline response but would have added little to the selective response. Under the assumption that the MU signals are the sum of the responses of multiple SU, we can estimate the expected total number of SU that contribute to each MU pool, NT, by dividing the Rmin for each MU site with the average Rmin of the SU sites. Similarly, we can estimate the equivalent number of SU that could account for the selective component of the MU response, NS, by dividing the selective response of the MU site with the average selective response of all the SU sites.To determine the contribution of selective SU to the MU pool, we examined the relationship between NS and NT for all the MU sites. Figure 11A shows a scatter plot of NS versus NT for each of the MU sites. Surprisingly, NS is much lower than NT. In fact, NS does not exceed 9 although NT can be as high as a 100. Furthermore, although NS is quite variable, the average NS as a function of NT saturates at a value of
View larger version (6K):
[in this window]
[in a new window]
Figure 11. Relationship between MU activity and SU activity. A, Scatter plot of the equivalent number of selective SU, NS, that can account for the selective MU response versus the expected total number of SU, NT, that contribute to the baseline MU response (see Results for details). Gray circles, SU; black circles, MU. The solid curve is the fit to the observed MU data with a saturating function. B, Scatter plot of NS over NT as a function of NT. The same conventions as in A are shown. The solid curve is derived from the curve in A.
To further examine the relationship between NS and NT, Figure 11B shows the ratio of NS to NT as a function of NT. As NT increased, the ratio of NS to NT dropped dramatically, reaching a mean value of 0.5 at NT ofFigure 11 demonstrates that MU activity in our task is inconsistent with the sum of the responses of similarly tuned selective SU. These results imply that as the threshold of the window discriminator was lowered, most of the events that were detected by the window discriminator were not coming from selective neurons.
We are left with two options. Either these events came from spikes of neurons that were not tuned, or were only weakly tuned, to the target, or alternatively, these events came from sources other than SU spikes. One potential way to distinguish between these two possibilities is to examine the response variability of the SU and MU responses. The variability in the spike count of single neurons in a short interval is known to be proportional to the mean, with proportionality constant (also known as Fano factor) in the range of 1.2–1.6 (Tolhurst et al., 1983
To compare the Fano factor of the SU and MU responses, for each SU and MU site we fitted a linear regression to the relationship between the variance to the mean across all contrasts and took the slope of the regression line to be the Fano factor of the SU or MU site. The histograms in the right panel of Figure 12 show the distributions of the Fano factors of the MU and the SU responses. As with SU, the variance of the MU responses was proportional to the mean over a wide range of baselines with comparable Fano factors for SU and MU sites. This result is consistent with the hypothesis that the MU responses are indeed the sum of spiking activity of many SU.
View larger version (10K):
[in this window]
[in a new window]
Figure 12. Fano factors (variance-to-mean ratios) for SU and for MU sites. Left, Scatter plot of Fano factor of SU (gray circles) and MU (black circles) as a function of NT. Dashed lines indicate the expected relationship between the MU Fano factor and NT under different assumptions regarding the correlation r between the SU that contribute to the MU signals. The solid black line is the best-fit linear regression for the relationship between the MU Fano factor and NT. The slope of this curve corresponds to a weak correlation between the SU that contribute to the MU signal (see Results for more details). Right, Histogram of Fano factor for SU (open bars) and MU (filled bars). Arrows indicates the mean Fano factor for the SU sites (gray) and the MU sites (black). The average Fano factor for the MU sites was slightly higher than that for the SU sites (mean SU, 1.34; mean MU, 1.74; t test, p = 0.055).
To explore the possibility that the SU that contribute to the MU response are correlated, and that these correlations lead to further reduction in the sensitivity of the MU responses, we examined the relationship between the Fano factor and the baseline of the MU responses. If the MU responses were the sum of N independent, identical SU responses, the MU Fano factor, FN, should have been the same as the Fano factor of the SU. If, on the other hand, the
