Abstract
Previous studies have shown that sensory neurons that are the most informative of the stimulus tend to be the best correlated with the subject's perceptual decision. We wanted to know whether this relationship might also apply to short time segments of a neuron's response. We asked whether spikes that conveyed more information about a motion stimulus were also more tightly linked to the perceptual behavior. We examined single-neuron activity in middle temporal (MT) area while monkeys performed a motion-detection task. Because of a slow stimulus update (every 27 ms), activity in many MT neurons was entrained and phase-locked to the stimulus. These stimulus-entrained neuronal oscillations allowed us to separate spikes based on phase. We observed a large amount of variability in how spikes at different phases of the oscillation encoded the stimulus, as revealed by the spike-triggered average of the motion. Spikes during certain phases of the cycle were much more informative about the presence of coherent motion than others. Importantly, we found that the phases that were the most informative about the motion stimulus were also more correlated with the behavioral performance and reaction time of the animal. Our results suggest that the relationship between a neuron's spikes, the stimulus, and behavior can vary on a time scale of tens of milliseconds.
Introduction
How is the activity of neurons in the sensory areas of cortex related to our perceptual abilities? This question is fundamental to the broader understanding of how the brain underlies behavior (for review, see Parker and Newsome, 1998; Romo and Salinas, 2001). Previous studies have shown that the activity of many cortical sensory neurons covaries with the behavior of animal subjects performing perceptual tasks (Celebrini and Newsome, 1994; Britten et al., 1996; Dodd et al., 2001; Cook and Maunsell, 2002b; Parker et al., 2002; Uka and DeAngelis, 2004; de Lafuente and Romo, 2005; Nienborg and Cumming, 2006) (but see de Lafuente and Romo, 2005). Importantly, the correlation between neuronal activity and perceptual choice tends to be strongest for neurons that are the most informative about the stimulus (Celebrini and Newsome, 1994; Britten et al., 1996; Cook and Maunsell, 2002b; Parker et al., 2002; Purushothaman and Bradley, 2005). This result suggests that a subject's perceptual choice is preferentially based on the activity of a subset of neurons that convey the most reliable task-related information.
The link between perception and neural activity has typically been studied in the context of a neuron's spike rate. However, many other possible stimulus-coding strategies have been examined. For example, different components of a neuronal response, such as the onset, sustained and offset portions, have been shown to encode different aspects of the stimulus (Lamme, 1995; Gawne et al., 1996; Friedrich and Laurent, 2001; Pack and Born, 2001; Reich et al., 2001; Bair et al., 2002; Shapley et al., 2003; Hegde and Van Essen, 2004; Roelfsema et al., 2007). Furthermore, many studies, both theoretical (Hopfield, 1995; Thorpe et al., 2001; Mehta et al., 2002; Chacron et al., 2004; Guyonneau et al., 2005; Gutig and Sompolinsky, 2006) and experimental (Celebrini et al., 1993; O'Keefe and Recce, 1993; Konig et al., 1995; deCharms and Merzenich, 1996; Gawne et al., 1996; Victor and Purpura, 1996; de Ruyter van Steveninck et al., 1997; Mechler et al., 1998; Fries et al., 2001, 2007; Lu et al., 2001; Johansson and Birznieks, 2004; Chacron et al., 2005; Womelsdorf et al., 2006; Sadeghi et al., 2007) have proposed that the timing of action potentials plays a role in stimulus coding. These results raise the possibility that some spikes carry more information about a behaviorally relevant stimulus than others. Thus, if the most informative neurons exert greater influence on perceptual choice, do the most informative spikes also have a greater influence on perceptual choice?
Addressing this hypothesis would seem challenging because estimating the sensory information contained in a single spike is not readily possible. However, we were able to approach this question using data from a motion-detection experiment where the visual motion stimulus was updated at a slow enough rate (approximately every 27 ms) to induce oscillations in the activity of middle temporal (MT) neurons. From these neuronal oscillations, we were able to group spikes based on phase and then ask if sensory and choice-related information varied as a function of phase.
Using two measures of sensory encoding (spike-triggered average and neurometric value), we found that spikes encoded a motion stimulus differently depending on the phase of the neuronal response. Additionally, spikes that occurred during the phases that were most informative about the motion stimulus were also more correlated with both the perceptual choice and reaction time of the animal. Our analysis suggests that the link between the activity of single neurons and perceptual choice can vary on a relatively fast timescale of tens of milliseconds.
Materials and Methods
Behavioral task.
The data set analyzed for this study comes from several previous studies (Cook and Maunsell, 2002a,b, 2004). Monkeys (Macaca mulatta) were trained to perform two variants of a motion-detection task. In the first set of experiments, the animals performed a spatially cued motion-detection task (see Fig. 1A). The trial began when the monkey depressed a lever and fixated on a central point. The goal of the task was to release the lever when coherent motion began in one of two random dot patches diametrically opposite of the fixation point. After the cue was presented, 0% coherent motion began in the two patches followed by coherent motion occurring in one of the two patches at a random time (flat hazard function) 500–8000 ms afterward. The location of the coherent motion was cued to the monkey at the start of the trial with static dots and this cue was valid on 80% of the trials. Trials with invalid cues were discarded for this analysis. The strength of coherent motion was varied between three levels (low, medium, and high), with the monkey correctly detecting the coherent motion 50, 90, and 99% of the time for the three levels, respectively. The coherence levels for each experiment were adjusted depending on the eccentricity, speed and radius of the random dot patch to produce the target performance. Ninety-three MT neurons were recorded in two monkeys.
The second set of experiments (see Fig. 1B) was similar to the first except that only a single random dot patch was presented and only two coherence levels (low and high) were used. Twenty-five MT neurons were recorded for this stimulus configuration in one monkey. The correlations between the neuronal response and the animal's behavioral performance was not appreciably different between the single and double patch experiments (Cook and Maunsell, 2002b).
In both sets of experiments, the direction and the speed of the coherent motion were matched to the preferred direction and speed of the neuron under study. The coherent motion lasted 750 ms and the monkey had to release the lever from 200 to 750 ms after the onset of coherent motion to obtain a juice reward. Trials where the monkey failed to release the lever or released the lever too late were deemed missed trials. Only correct and missed trials were included in the analysis. Trials where the monkey released the lever too early or was unable to maintain fixation were discarded from the analysis.
Visual stimulus.
The animal sat 62 cm from a computer monitor (±17° × ±13° of visual angle; 1600 × 1200 pixels; 75 Hz refresh). The stimuli consisted of two patches of white dots (each dot 0.25° diameter; 78 cd/m2) on a dark gray background (12 cd/m2) with a dot density of 2.1 dots/degree2. Each patch of dots was updated on every other video frame (approximately every 27 ms) using the following procedure. The dots in each patch were evenly divided into two groups. On each update, one group was replaced with new, randomly positioned dots, whereas dots in the other group were displaced by a fixed distance. The dots in this latter group determined the motion coherence. For 0% coherence, all of the dots in this group moved a fixed distance in a random direction. For coherent motion greater than zero, a proportion of the dots moved with a fixed distance in the same direction. This proportion determined the strength of the coherent motion. On the next update (27 ms later), the groups were switched. This arrangement insured that all of the dots had a lifetime of four video frames (i.e., two stimulus updates) before they were replaced. Because half the dots are always randomly replotted regardless of the proportion of dots moving coherently, our motion had a maximum strength of 50% coherence. For example, at 25% coherent motion, half the dots are randomly replotted, one-quarter are moving with the same fixed distance and direction, and one-quarter are moving with the same fixed distance in a random direction.
Data collection.
Using standard extracellular recording techniques (Gibson and Maunsell, 1997), recordings were made from well isolated single neurons in area MT in both animals. When a neuron was isolated, the receptive field (RF) was mapped using a manually controlled bar while the animal fixated on a central spot. The diameter of the RFs ranged from 3.9 to 10.7° (median, 7.4°). RF center eccentricities ranged from 3.9 to 11.1° (median, 7.9°). The preferred speed was also judged using a bar moved by hand. The animals were trained to perform the task at slow or moderate motion speeds, so neurons with a preferred speed between 4 and 12°/s were usually selected. Once the RF location, size, preferred direction, and speed were determined, the motion-detection task was then run, and the neuron was recorded from for as long as possible. For some neurons, a memory saccade task was also run, but these data were not used in this analysis. The number of completed trials per coherence level for the motion-detection task ranged from 15 to 175 (median, 35). The monkey's performance varied with patch location, size, and motion speed, which were determined by the response properties of the neuron under study. Consequently, different neurons were tested with different coherence levels. The animal's eye position was measured every 5 ms using a scleral search coil (Robinson, 1963; Judge et al., 1980) and the times of action potentials were recorded to the nearest millisecond.
Detect probability and neurometric value.
Detect probability (DP) expresses the ability to predict the behavioral outcome of a trial in a detection task given the neuronal response (Cook and Maunsell, 2002b). DP is analogous to the choice probability calculation used in discrimination tasks (Celebrini and Newsome, 1994; Britten et al., 1996). The neurometric value expresses the ability to predict which one of two stimuli was presented given the neuronal response.
DP and neurometric value are formulated in the same way using traditional receiver-operating characteristic (ROC) analysis (Green and Swets, 1975). Briefly, given two random samples from different distributions, the DP and the neurometric value are the probabilities that one can correctly determine from which distribution the samples were drawn. In other words, it is a measure of how separate two distributions are from each other. Specifically, given two distributions of neuronal responses, p(x) and q(x), the maximum probability that we could correctly determine from which distribution a sample was drawn from is P(correct)= ∫0∞ ∫x∞ p(x)q(y)dydx, assuming that, on average, a sample drawn from q(y) is greater than a sample drawn from p(x).
For detect probability, q(y) corresponds to the distribution of spike rates for correct trials and p(x) for failed trials. For the neurometric value, p(x) and q(y) are the distribution of spike rates corresponding to the 0% and coherent motion, respectively. A DP of 1.0 indicates the neuronal distributions, p(x) and q(y), do not overlap and the behavioral outcome is completely predictable from the neuronal responses. Likewise, a neurometric value of 1.0 indicates the motion stimulus encoded (either 0% coherent or coherent) is completely predictable from the spike rate. A DP or neurometric value of 0.5 indicates a chance prediction of the behavioral outcome or the state of the coherent motion, respectively.
For the DP calculation, only neuronal responses occurring during low coherent motion trials from 50 to 150 ms after coherent motion onset were included. We only used the low coherent motion trials to compute DP because this was the only condition that produced similar numbers of correct and failed trials. For the neurometric value, we compared the neuronal response from 50 to 150 ms after coherent motion onset to the neuronal response from 100 to 0 ms before the coherent motion onset for all three coherence levels. The neurometric value was computed individually for each of the three motion coherences and then averaged.
Reaction time correlation coefficient.
Reaction time (RT) correlation coefficients provide the correlation between the neuronal response and the reaction times. Given the neuronal responses and the respective RTs, the Pearson product-moment correlation coefficient is calculated between these two sets. Because neither the spike count distributions nor the reaction time distributions were normally distributed, the calculations were also performed using the Spearman's rank correlation coefficient. The correlation coefficient (γ) for each of the three coherence levels was calculated individually and then a weighted (by the number of trials) average was taken. The RT correlation coefficient was computed using the neuronal response from 50 to 150 ms after coherent motion onset.
Extracting global motion from the stimulus.
We reduced our random dot stimulus to a one dimensional time series of global motion strength along the preferred/null axis of the neuron (see Fig. 4A). To estimate the net motion between two successive frames of random dots, we applied a previously described method (Barlow and Tripathy, 1997) that computed the correspondence between dots in two sequential frames. We calculated the motion vector between each pair of dots in successive frames. Thus, if our patch contained N dots, we computed N2 motion vectors for each pair of successive frames, where the magnitude of the vectors represented the speed. We then scaled each vector by passing its magnitude through a speed filter. The speed filter was a Gaussian distribution in the log speed domain (Nover et al., 2005) centered at the preferred speed of the neuron (determined by sweeping a bar through the receptive field) and with an SD of 0.5°/s. We filtered speed because speed-tuning profiles were not collected from the neurons, however, our results were insensitive to the parameters of the speed filter. Each weighted motion vector was then projected onto the neuron's preferred/null direction of motion axis. The sum of all of the projected vectors was the motion strength at that time point. This calculation was repeated for every pair of successive random dot patches to create a time series of the motion strength in the preferred/null axis of the neuron. The resultant time series was normalized to have an SD of one and then smoothed with a Gaussian filter with an SD of 3 ms.
Results
The goal of this study was to examine how the phase of stimulus-driven oscillatory activity in MT neurons was related to both the encoding of the motion stimulus and behavior. The results are based on the activity of single neurons recorded from two monkeys performing a motion-detection task (Fig. 1) (see Materials and Methods).
motion-detection task. A, The first set of experiments used two random dot patches and included 93 neurons. Once the animals fixated, a static cue was presented to indicate the most likely position of where the coherent motion would occur. Next, 0% coherent motion was presented in both the RF path of the neuron and in a patch diametrically opposite. Coherent motion randomly began in one of the patches from 500 to 8000 ms with a flat hazard function. The coherent motion lasted for 750 ms and the animal had to release the lever from 200 to 750 ms after the coherent motion onset in order to obtain a reward. B, The second set of experiments used only a single random dot patch and included 25 neurons. This paradigm was identical to the one above, except that only a single random dot patch that overlapped the RF of the neuron was used.
In this task, the monkeys released a lever when a patch of random dot motion began moving in a coherent manner. The location and size of the stimulus was matched to the classical RF of the MT neuron under observation, and the direction and speed of the coherent motion step was matched to the neuron's preferred tuning. Because we were interested in accurately correlating neuronal responses with behavioral outcome, we selected neurons for analysis based on a criterion of at least three correct and three missed trials for the low coherent motion condition. This resulted in 115 of the original 118 MT neurons in this data set selected for analysis.
Stimulus-driven oscillations of MT neurons
Although the stimulus was presented on a monitor with a 75 Hz refresh rate, because of hardware limitations, the dot locations in each patch were updated on every other refresh period. Because of this, many of our MT neurons demonstrated oscillatory firing with a frequency that matched the updates of the motion stimulus, which occurred once every 27 ms (every other frame). An example of this stimulus-driven oscillatory behavior is shown for an MT neuron in Figure 2A. The activity of the neuron was entrained with a fixed phase to the stimulus updates (vertical lines). The oscillatory behavior occurred during both the 0% coherent motion and the coherent motion step. Although a 27 ms separation between motion updates may not be optimal for MT neurons (Churchland and Lisberger, 2001), it was still sufficient to robustly drive our population of cells.
Analysis of an example neuron. A, Raster plot and binned response of an example MT. Motion stimulus updates are thin vertical lines. The monitor's vertical refresh rate was 75 Hz or twice the stimulus update rate. The neural activity is locked to the motion updates and is greatest just before each motion update. Calibration: 40 spikes per second. B, The average spike rate during the 0% coherent motion as a function of phase for the same neuron. The data were smoothed with a 1 ms Gaussian filter and normalized so that the area under the curve was equal to one. Top, The phase is relative to the stimulus updates. Bottom, The curve is shifted so that the peak is aligned at the center, and the phase is given relative to this peak. The oscillatory amplitude (given in arbitrary units) is the difference between the maximum and minimum values of the curve. The black section of the curve (corresponding to 1–13 ms the left of the peak) is defined as the rising phase, and the gray section of the curve (corresponding to 1–13 ms to the right of the peak) is defined as the falling phase. C, The STA for the rising (black) and falling (gray) phases of the example neuron. Positive amplitudes indicate motion in the neuron's preferred direction (see Fig. 4A). The STA was computed from the 0% coherent motion and was much larger for the rising phase than for the falling phase. D, The neurometric value for the rising phase (black), falling phase (gray) and the entire cycle (white) of the example neuron. The neurometric value was calculated using ROC analysis applied to the responses 100–0 ms before the coherent motion step and 50–150 ms after the coherent step. Activity on the rising phase is more informative of the coherent motion step than activity on the falling phase. E, The average spike rate for correct (solid) and failed (dashed) trials for the low-coherence motion steps only. The spike rate for correct trials is greater on the rising phase, but less than the spike rate for failed trials on the falling phase. F, The DP using spikes occurring on the rising phase (black), falling phase (gray), and the entire cycle (white) of the low coherence motion. DP was calculated using the response 50–150 ms after the onset of the coherent motion step. Activity on the rising phase is positively correlated with the animal correctly detecting the motion signal and is greater than the DP using all spikes. Activity on the falling phase is negatively correlated with the animal correctly detecting the motion stimulus.
To characterize the strength of the stimulus-induced oscillatory response, we collapsed the average spike rate during the 0% coherent motion to a single update, smoothed the curve using a Gaussian with an SD of 1 ms, normalized the area under the curve, and then centered the peak of activity at zero. For our example neuron, the neuronal response as a function of phase is illustrated in Figure 2B. The top of Figure 2B shows the neural activity as a function of phase aligned to the stimulus updates. As with Figure 2A, it also shows that neural activity tends to peak a few milliseconds before the next stimulus update. In the bottom panel of Figure 2B, the curve is shifted so that the peak in neural activity is aligned to the center and the phase is measured relative to this peak. All subsequent plots of neural activity versus phase are shown in this manner. This response versus phase relation was computed for all neurons, and the oscillatory amplitude (i.e., the strength of the oscillations) was expressed as the maximum minus the minimum value.
The distribution of oscillatory amplitudes for our population of 115 neurons is shown in Figure 3A. Our example neuron in Figure 2 had prominent oscillatory firing and a corresponding high oscillatory amplitude of 0.11. Two other neurons with lower oscillatory scores are illustrated in Figure 3, B and C. The average normalized spike rate as a function of phase for our population of neurons is shown in Figure 3D.
Oscillatory neural activity for the population of neurons. A, Histogram of the oscillatory amplitudes of the neural oscillations. For each neuron, we calculated the spike rate at each phase, the result was smooth with a Gaussian filter and then the amplitude was taken to be the maximum minus the minimum value. B, C, Average spike rate as a function of phase; for example, neurons display medium and low-level oscillatory behavior, respectively. D, Average spike rate as a function of phase for all 115 MT neurons. Dashed lines are SEM.
The dependence of stimulus encoding on neuronal phase
We first wanted to know whether action potentials at different phases of a neuron's oscillatory response differed in how they encoded the motion contained in our random-dot stimulus. To make this question more explicit, we asked whether the spike-triggered average (STA) of the motion stimulus occurring during the rising phase of the oscillatory cycle was different from the STA occurring during the falling phase of the oscillatory cycle.
An STA with a large positive lobe indicates that, on average, highly coherent motion in the preferred direction preceded each spike, which implies that the neuron was very selective for coherent motion in the preferred direction. An STA that is relatively flat indicates that the neuron was not very selective for coherent motion. We wanted to know whether the STA was dependent on the phase of the neuronal oscillatory response. If we assume a common model of a cortical neuron as a linear filter (which is estimated by the STA) followed by a static nonlinearity, known as the linear-nonlinear Poisson model (Paninski et al., 2004), then the average input to the nonlinearity is equal to the area under the STA times the average motion strength. Thus, we used the area under the STA to quantify how informative spikes are about the motion strength.
To compute the STA, we projected the motion in our random-dot stimulus onto the preferred/null axis as determined by the neuron's directional selectivity (see Materials and Methods). Although a previous study had described the random-dot motion using the dimensions of direction, speed, and time (Cook and Maunsell, 2004), we chose to use a reduced description of the motion time series because it improved the signal-to-noise ratio of our STAs. Given the motion time series and the corresponding spike train, the STA was calculated by averaging the motion stimulus (in the preferred/null axis) that preceded each spike (Fig. 4A). Only the responses to the 0% coherent motion were used to compute the STA.
Phase dependence of the spike-triggered average. A, Diagram of the stages used to compute the STA of a neuron. From the successive random dot patches (top), we computed the time series of motion strength in the preferred/null axis of the neuron, smoothed by a Gaussian filter (middle). Given this time series, and the corresponding spike train (bottom), we computed the STA. B, Average aligned STAs for all 115 neurons for the rising (black) and falling (gray) phases. For each neuron, the time of the peak of the overall STA (the STA calculated using all spikes) was determined and was defined as time 0. The STAs from spikes from the rising and falling phases were then aligned to this reference time. On average, there was little difference between the two STAs. Dashed lines are SEM. C, Histogram of the percentage difference in areas between the STA from the rising phase and falling phase for the group of oscillatory neurons (see Materials and Methods). Many neurons possessed large differences between the STAs of the rising and falling phases.
The STAs for our example cell are shown in Figure 2C. We computed two STAs using either spikes occurring on the rising phase (Fig. 2B, black curve, 1–13 ms to the left of the peak) or spikes occurring on the falling phase (Fig. 2B, gray curve, 1–13 ms to the right of the peak). The rising and falling phases were always defined relative to the peak generated during 0% coherent motion. For this cell, the two STAs varied in size between the rising (black) and falling (gray) phases of the neuronal response. Spikes occurring on the rising phase were much more selective for coherent motion in the preferred direction than were spikes occurring during the falling phase.
Across our population, the mean area of the STA from spikes from the rising phase was 11.6% greater than that of the falling phase, although the difference was marginally significant (two-sided t test, p = 0.058). This is illustrated by the average STAs for our population of neurons (Fig. 4B). Because neural latencies differ across neurons, we chose to align the STAs when calculating the average STAs of the population. Thus, for each neuron, we calculated the time of the peak of the overall STA (the STA calculated using all spikes), and this time was defined as time 0. The STAs from spikes from the rising and falling phases were then aligned to this reference time.
Although the mean difference in area was not appreciably different from zero, we found a surprisingly large amount of variability in the size of the STA between the two phases, as shown in Figure 4C. This histogram illustrates the distribution of the percentage difference in the areas between the STAs from the rising and falling phases for our 115 neurons. There are several possible explanations for the variability in stimulus encoding (as measured by STA area) illustrated in Figure 4C. For example, the finite number of spikes from each cell may have introduced experimental noise that produced naturally variable STAs between the two phases of the neuronal oscillations.
To test whether the variance in the difference between the two STAs was greater than expected by chance, we randomly assigned spikes with 50% probability to the rising or falling phases, recalculated the STAs for the rising and falling phases, and found the difference between the two. The population was simulated 1000 times to obtain an estimate of the amount of variance one would expect by chance. The variance in the real population was 3.1 times greater than the variance one would expect by chance (two-sided z test, p < 10−7).
Another possibility is that some phases of the neuronal oscillation encode the coherent motion more reliably than others. This second possibility is motivated by past studies that suggest the sensory information conveyed by a neuron can evolve over time (Lamme, 1995; Gawne et al., 1996; Friedrich and Laurent, 2001; Pack and Born, 2001; Reich et al., 2001; Hegde and Van Essen, 2004; Roelfsema et al., 2007). We addressed this second hypothesis more closely by using the behavioral data that was simultaneously collected with the neuronal recordings.
Past studies have demonstrated that neurons conveying the most informative task-related information tend to show the strongest correlations with behavior (Celebrini and Newsome, 1994; Britten et al., 1996; Cook and Maunsell, 2002b; Parker et al., 2002; Purushothaman and Bradley, 2005). We wanted to know whether this hypothesis extended to the phases of our oscillatory activity. In other words, was the phase of the neuronal response that most reliably encoded the coherent motion also the phase that was most strongly correlated with behavior? This is an important question because it addresses the time-scale on which the relationship between neuronal activity and sensory perception can change.
To answer this question, we first describe how we correlated the phase of the neuronal activity with behavior using standard ROC-based choice analysis. In addition to the STA described above, we introduce a second method for quantifying the phase-dependent encoding of the coherent motion using an ROC-based neurometric analysis. Finally, we combine all these measurements to show that the phase of the neuronal oscillation that most reliably encoded the coherent motion also tended to have the strongest correlation with behavior.
Neurometric value and detect probability as a function of phase
The STA analysis above was based only on neuronal responses to the 0% coherent motion. The monkey's task, however, was to detect the onset of a coherent motion step (Fig. 1). Therefore, we wanted to know whether the information in the neuronal response to the coherent motion step was phase dependent. For this, we computed a neurometric value using ROC analysis (see Materials and Methods). The neurometric value was calculated using the neuronal response occurring during the 100 ms just before the onset of the coherent motion pulse compared with the neuronal response that occurred 50–150 ms after the coherent motion pulse. A neurometric value of 1.0 indicates an ideal observer could discriminate the response to the coherent motion from the response to the 0% motion with perfect reliability, whereas a value of 0.5 indicates chance performance.
Figure 2D shows the neurometric analysis applied to our example cell. We computed the neurometric value using spikes from either the rising (black bar) or falling phases (gray bar). For this cell, the neurometric value is higher for the rising phase compared with the falling phase. As a result, spikes on the rising phase were better able to signal the presence of coherent motion than spikes on the falling phase in agreement with the phase-dependent STA for this cell shown in Figure 2C.
It is important to emphasize, however, that the neurometric value can be dependent on the number of spikes used in the analysis. Thus, Figure 2D also shows that the neurometric value using all the spikes combined from both phases provides the most informative signal for detecting the onset of the coherent motion (white bar). This suggests that if the monkey were behaving as an ideal observer, combining the spikes from both phases would be a better strategy for detecting the onset of the coherent motion. The neurometric value, however, does not provide any insight into how the monkey actually used the activity from this cell to detect the coherent motion. To address this question, we next turn to the ROC-based detect probability calculation.
For our example cell, Figure 2E compares the average response (as a function of phase) for correct and failed trials from 50 to 150 ms after the onset of low-level coherent motion only. This particular cell demonstrated a phase dependent difference in response between correct (solid line) and failed trials (dashed line). Spiking activity associated with correct trials was greater than failed trials on the rising phase and became less than failed trials on the falling phase.
To quantify how the neuronal response was correlated with the animal's behavior as a function of phase, we computed the DP (see Materials and Methods). DP measures the ability to predict the behavioral outcome of a trial given the neuronal response. We calculated DP using the neuronal responses from 50 to 150 ms after coherent motion onset and only used responses from the low-coherence trials (which produced ∼50% correct detections). DP is an ROC calculation that is analogous to the choice probability measure commonly used to express the correlations between neuronal activity and behavioral choice in discrimination tasks (Celebrini and Newsome, 1994; Britten et al., 1996; Parker and Newsome, 1998; Uka and DeAngelis, 2004). A DP value of 1.0 indicates that an ideal observer could predict the behavioral outcome of a trial (either correct or failed) using the neuronal response. A DP of 0.5 indicates chance predictability.
For our example cell, we found DP differed between the rising and falling phase of the neuronal response (Fig. 2F). The DP analysis shows that activity of the rising phase is well correlated with behavior (Fig. 2F, black bar) (DP, 0.64) and is even stronger than the DP from all spikes (white bar) (DP, 0.57). On the falling phase (gray bar), activity was negatively correlated with behavior (greater activity was associated with failed instead of correct trials) producing a DP of 0.36. Thus, for this particular cell, the correlation between the neuronal response and behavior was asymmetric across neuronal phase.
Population phase dependence of motion encoding, signal detection, and behavior
For our example cell, the STA, neurometric, and DP calculations suggest that there was a phase asymmetry in both how the coherent motion was encoded and the correlation with the animal's behavior. We next wanted to know whether those relationships were observable in our population of neurons.
As discussed above, a large positive STA implies that the neuron is selective in its firing, and will more likely respond only to coherent motion in the preferred direction and speed. One would suspect that spikes that are more likely generated by preferred coherent motion would naturally be more predictive of whether the neuron is encoding 0% coherent or coherent motion in the preferred direction. Thus, in Figure 5A, we examined whether neurons with different STAs between the two phases would also have a corresponding difference between the neurometric values of the two phases. For our population of 115 neurons, the asymmetry between the STAs of the two phases is well correlated with asymmetry in the neurometric value (r = 0.55, p < 10−9). The filled symbols are neurons whose STA satisfied a minimum signal-to-noise criterion of 2.75 (see below). Thus, the phase with the larger STA (computed from 0% coherent motion) also tended to be the phase that most reliably signaled the onset of the coherent motion.
Population phase dependence of motion encoding, signal detection, and behavior. A, The relation between the percentage difference in the area of the STAs of the rising and falling phases (x-axis) and the difference between neurometric values for the rising and falling phases (y-axis) for our 115 neurons. The areas of the STA were computed in a 27 ms window centered at the peak of the STA from all spikes. The black dots represent neurons with STAs that satisfied the criterion of an SNR of at least 2.75 (for details, see Results. The strong and weak encoding phases). B, The difference in neurometric value between the two phases (x-axis) versus the difference in the detect probability of the two phases (y-axis). C, The relationship of the percentage difference in area between the STAs and the two phases (x-axis) with the difference in the DP (y-axis).
It has generally been observed that the most informative sensory neurons also tend to show the best correlations with behavior. Thus, we wanted to know whether the neuronal phase with the best neurometric value was also more correlated with behavior. In Figure 5B, we show the relation between the asymmetry in the neurometric value between the two phases and the asymmetry in the DP. Consistent with this hypothesis, the phase of the cycle that is more capable of signaling the presence of coherent motion, whether it be the rising or falling phase, also tends to be more correlated with the behavior (r = 0.32, p = 0.0005). Thus, not only does the correlation between how a stimulus is encoded (neurometric) and its relation to perceptual choice (DP) exist at the single cell level, but this relationship also seems to hold for certain spike times within a cell's response. We also examined the correlation between the phase-asymmetry in the STA with the phase-asymmetry in DP (Fig. 5C). Consistent with the above results, these two asymmetries were also weakly correlated (r = 0.25, p = 0.007). Similar results were also observed for other time windows used to compute neurometric value and DP as a function of phase.
Together, this analysis suggests that the task-related information (as measured by STA and neurometric value) conveyed by the rising and falling phases of the neuronal response had a systematic relationship to the correlation between neuronal response and behavior (as measured by DP). A likely explanation of this result is that neuronal latencies varied depending on the strength of the pattern of moving dots in our stimulus. We examine this and other possibilities, such as possible biases induced by spike rate, below.
One limitation of the analysis in Figure 5 was that it was based on all neurons in our data set regardless of their tendency to oscillate. In addition, the STA estimates were inherently noisy for many neurons. The goal of our next analysis was to overcome these limitations by focusing on a subset of neurons with the most robust STAs and oscillations.
The strong and weak encoding phases
If one phase of the neuronal oscillation was more informative about the stimulus than the other, how much more correlated is that phase with behavior? To answer this, we grouped activity based on the size of the STA. We refer to the phase with the larger STA as the strong encoding phase and the phase with the smaller STA the weak encoding phase. We then examined how the strong and weak encoding phases encoded the motion stimulus and were related to the formation of a perceptual decision.
Because this calculation relies on an accurate measurement of the STA, we only included neurons whose STA satisfied a minimum signal-to-noise ratio (SNR). We based our SNR on a similar definition as used in a past analysis of this data (Cook and Maunsell, 2004). Our SNR was defined as the variance among the values of the STA in a 27 ms window centered at the peak (signal) divided by the variance among the values of the STA in the two 27 ms windows lying adjacent to this central window (noise). The idea was that the two adjacent windows represent the noise in our STA by capturing the chance correlations between the stimulus and neuronal response. We arbitrarily set the minimum SNR to 2.75, resulting in 30 neurons that satisfied the criterion. The STA of our example cell shown in Figure 2C had a robust SNR of 189.9. Figure 5 (filled symbols) illustrates that our 30 best cells with the most reliable STA spanned the range of our population with respect to asymmetries in the STA, neurometric and DP between the rising and falling phases.
In Figure 6A, we show the average STA from our 30 best neurons for the strong encoding and weak encoding phases. Because the strong encoding phase was defined as the phase with the larger STA, the strong encoding phase was naturally more selective for motion in the preferred direction compared with the weak encoding phase. As there was a correlation between the SNR of the STA and oscillatory amplitude (r = 0.55, p < 10−9), our best neurons also demonstrated stronger oscillations in response to our 0% coherent stimulus compared with the overall population (Fig. 6B).
The strong and weak encoding phases. A, The average STA for the strong (black) and weak (gray) encoding phase from the group of 30 neurons whose STA had an SNR >2.75 (Fig. 5, filled symbols). The phase with the larger STA by area was designated the encoding phase. B, The average spike rate as a function of phase for the 30 neurons. The oscillatory amplitude of this group was larger than the oscillatory amplitude of the entire population (Fig. 3D). C, The average neurometric of these 30 neurons using spikes from the strong encoding phase (black), weak encoding phase (gray) and the entire cycle (white). The associated histogram to the right shows the pairwise differences between the strong and weak encoding values for each neuron. The vertical arrow indicates the median value. D, Same as in C, but showing the average DP. E, Same as in C, but showing the average RT Pearson's correlation coefficient.
The prediction from Figure 6A is that the strong encoding phase (i.e., phase with the largest STA) should also be the most informative phase about the onset of the coherent motion. Thus, we compared the neurometric values for the strong and weak encoding phases of our subset of neurons with high SNR. It is important to note that the strong/weak classification was based on the response to the 0% coherent motion, whereas the neurometric value was based on both the 0% coherent and coherent motion. The neurometric value for the strong encoding phase (mean, 0.65 ± 0.02 SEM) (black bar) was significantly greater than the average value for the weak encoding phase (mean, 0.59 ± 0.01) (gray bar) as shown in Figure 6C (pairwise difference two-sided t test, p = 0.009). The histogram on the right shows a shift in the distribution of the pairwise difference in the neurometric value between the strong and weak encoding phases. Thus, the strong encoding phase tended to be more informative about the onset of the coherent motion than the weak encoding phase. Because activity on both phases (usually) increased during coherent motion, both provided information about the onset of the coherent motion. Thus, using all the spikes provided a slight increase in the neurometric value (Fig. 6C, white bar).
We next examined how neural activity was correlated with perceptual choice for the strong and weak encoding phases. Figure 6D shows that the DP for the strong encoding phase (0.56 ± 0.02) (black bar) was marginally greater than the DP from all spikes (white bar) (0.53 ± 0.02; pairwise difference, two-sided t test, p = 0.082). However, the DP for the weak encoding phase was significantly below chance at 0.46 ± 0.02. The associated histogram reveals that neural activity on the strong encoding phase was more positively correlated with perceptual choice than the weak encoding phase in 22 of the 30 neurons, and the mean difference between the two DPs was significantly greater than zero (pairwise difference two-sided t test, p = 0.0018). Thus, spikes occurring during the weak encoding phase for our subset of 30 neurons were significantly and positively correlated with the animal failing to detect the motion stimulus. Spikes in the strong encoding phase, by comparison, were positively correlated with the animal correctly detecting the motion. This raises the possibility that the brain could combine activity from both phases by weighting them with positive and negative weights, respectively. However, because activity is correlated between subsequent phases (see below), this might not be advantageous.
To further confirm that the strong encoding and weak encoding phases had different relationships with behavior, we also examined the (Pearson's) correlation between the neuronal response and reaction time (γ) (see Materials and Methods). In this analysis, a negative value indicates that greater activity is correlated with faster reaction times. In Figure 6E, we show that activity on the strong encoding phase had an average correlation coefficient (−0.130 ± 0.023) that nearly matched the correlation coefficient using all spikes (−0.147 ± 0.023). However, activity on the weak encoding phase was less correlated with reaction time (−0.064 ± 0.021). The associated histogram illustrates that the RT coefficient for the strong encoding phase is greater for a majority of the neurons and the mean difference of −0.066 was significantly less than zero (pairwise difference, two-sided t test, p = 0.033). Because neither the spike count distributions nor the reaction time distributions were not normally distributed, we repeated our calculations using the Spearman's rank correlation coefficient. The pairwise difference between the correlation coefficient of the strong encoding and weak encoding phases was −0.078, also significantly less than zero (pairwise difference, two-sided t test, p = 0.024).
Because it has been reported previously that gamma band synchronization between spiking and activity and the LFP before a stimulus change in a detection task is correlated with reaction times (Womelsdorf et al., 2006), we also calculated the Pearson's correlation coefficient between neural activity and reaction time from 100 to 0 ms before coherent motion onset. Although there was a small difference between the correlation on the strong encoding and weak encoding phases (strong, −0.022 ± 0.015; weak, 0.020 ± 0.022; overall, −0.008 ± 0.021), the differences were not significant.
By analyzing our 30 best neurons based on the quality of their STA, we found a phase-dependent relationship between the encoding of the motion stimulus and how the neuronal activity was correlated with behavior. We next examine several potential mechanisms that may account for the observed phase-dependent encoding of motion information.
Potential mechanisms
One possibility is that the mean or variance in the number of spikes varied between phases, biasing our results. The correlation between the asymmetry of the neurometric and the asymmetry in the STA (Fig. 5A) is because there was a greater increase in firing rate on the strong encoding phase. For low-coherent motion (used for the DP analysis), the average spike count was 13% greater (pairwise mean) and the SD was 10% greater for the strong encoding phase than the weak encoding phase. To control for the difference in spike rate between phases, we calculated the difference in spike rate between the strong encoding and weak encoding phase during low coherent motion for each neuron. Based on this difference, we randomly removed spikes from the phase with greater activity to equalize the mean activity between the two phases. Each trial was simulated 100 times, and then we recalculated the correlations between the changes DP and the STA and neurometric value. These new values are consistent with our previous results (change neurometric value vs change DP, r = 0.30, p = 0.001; change STA vs change DP, r = 0.24, p = 0.010). We also performed the same calculation equalizing total spike count between the two phases, instead of spike rate. Again, the results were left almost unchanged (change neurometric value vs change DP, r = 0.32, p = 0.0006; change STA vs change DP, r = 0.24, p = 0.009). Because equalizing the spike rate and the spike count between the two phases when calculating the DP does not alter its significant correlation with the neurometric value and the STA, we do not believe that the number of spikes was a major factor in the asymmetric relationship between the phase of stimulus encoding and behavior.
Another possible reason for the observed asymmetry in the DP between phases could be stimulus variability. More precisely, if there was more coherent motion for correct trials than for failed trials, this could have produced more spikes during the strong encoding phase as well increased the chance the animal detects the coherent motion. If this was the case, then the percentage difference between the amount of coherent motion in the preferred direction between correct and failed trials should be correlated with the difference in the DP between the strong encoding and weak encoding phases. However, we found no such correlation for the entire population of 115 neurons (r = −0.10, p = 0.30) or for the population of 30 neurons with high SNR STAs (r = 0.05, p = 0.80). Thus, we do not believe that stimulus variability was the reason for the difference in the DP between the strong encoding and weak encoding phases.
A third possible explanation for our finding was that it was the result of correlations between the neural activity in each rising phase and the subsequent falling phase. For example, suppose activity on the rising and falling phases were negatively correlated, so that greater activity on the rising phase tended to reduce the amount of activity on the subsequent falling phase. If the onset of coherent motion first increased neural activity on the rising phase, then activity on the falling phase would be suppressed, shifting the average phase of activity to the left. This possibility is somewhat similar to previous studies on how short-term plasticity can induce phase shifts in neural activity (Fortune and Rose, 2002; Fortune, 2006).
To explore this possibility, we computed the correlation coefficient between the numbers of spikes in each rising phase with the subsequent falling phase (similar results were obtained using the spike count in each falling phase and the subsequent rising phase) during 0% coherent motion. Although the mean correlation coefficient for all 115 neurons was quite weak (r = 0.039), most neurons possessed a significant (p < 0.05) correlation coefficient, with 39 having a negative correlation coefficient (32 significant) and 76 having a positive correlation coefficient (66 significant).
This correlation between the activity in each rising and subsequent falling phases was associated with a phase shift in the neural activity toward the encoding phase when coherent motion occurred. This was revealed by the significant correlation (r = −0.20, p = 0.030) between the correlation coefficient between activity on the rising and falling phases and the change in the neurometric value between the strong encoding and weak encoding phases. This is fairly intuitive because the encoding phase was generally the phase with the shorter latency (see Discussion), and likely to be first affected by the onset of coherent motion. For a neuron with negative correlation between activity on the rising and falling phases, coherent motion would increase activity disproportionably more on the encoding phase, resulting in a greater neurometric value on the strong encoding phase compared with the weak encoding phase. Although the correlation coefficient between rising and falling phases was correlated to the change in neurometric values between the strong encoding and weak encoding phases, no significant correlation existed with the change in DP between the two phases (r = −0.02, p = 0.82). Thus, although the correlation between the activity on the rising and falling phases might shift the neural activity, it does not appear to be the source of the asymmetry in the correlation with behavior. However, we cannot say what role it might play, if any, in encoding the motion stimulus.
Although we cannot rule out the existence of other possible explanations for our observations (see Discussion), our result suggests that the spikes that convey more information about the presence of coherent motion are weighted more heavily in forming a perceptual decision.
Discussion
In this study, the activity of many MT neurons in response to a random moving dot stimulus oscillated with the same frequency as the stimulus updates. This allowed us to investigate how neural activity at different phases was correlated with both the stimulus and the behavior. We found that neuronal phases that were more informative about the stimulus, both in terms of the STA and the neurometric value, were also more correlated with the animal's detection performance and reaction time. Importantly, the asymmetry between stimulus encoding and behavior occurred on a relatively short time scale of a few tens of milliseconds.
Encoding of the motion stimulus
Oscillations seem to serve two broad, and possibly nonexclusive roles, in stimulus encoding. In the first, oscillations can synchronize ensembles of neurons to fire together (Gray et al., 1989; Singer, 1999; Usrey and Reid, 1999; Perez-Orive et al., 2002; Friedrich et al., 2004). Active conductances in dendrites can then sum these synchronously arriving spikes in a superlinear a manner, a term dubbed “coincidence detection” (Softky, 1994; Wang et al., 2000; Stuart and Hausser, 2001; Schaefer et al., 2003). These oscillations in the local field potential thus act as a timing device and, because these oscillations can remain coherent across large distances (Engel et al., 1990; Donoghue et al., 1998; Destexhe et al., 1999), they are capable of synchronizing neural activity across different areas in the brain (Engel et al., 1991; Destexhe et al., 1999). This mechanism is widely observed throughout the brain, and thought to play a role in a wide range of systems (for review, see Engel et al., 2001; Averbeck and Lee, 2004; Sejnowski and Paulsen, 2006).
In the second role, the stimulus is encoded by the phase at which an action potential is generated. This is the encoding scheme used by place cells in the hippocampus, where the phase of the spike encodes the relative location of the animal inside the neuron's receptive field (O'Keefe and Recce, 1993). There are two important properties for such a encoding scheme. First, in order for the phase of the spike to be meaningful downstream, oscillations must be coherent between the upstream and downstream areas. Second, incoming EPSPs must have different effects on an oscillating target cell depending on the phase of its arrival (Lampl and Yarom, 1993; Schaefer et al., 2006).
Unlike the encoding by synchronization scheme mentioned above, a phase encoding scheme as only been observed in the hippocampus. However, it has been proposed that a phase encoding scheme might be more prevalent, and serve as a more general encoding scheme through out the cortex (Hopfield, 1995; Fries et al., 2007). Although the oscillations we observed in our data were specific to the stimulus used, the results presented here suggest that a cortical phase encoding scheme may be possible.
What are the mechanisms behind our observed phase encoding? One possibility is that spikes at phases that encoded stronger coherent motion were the same spikes generated with a short latency. It has been shown that latencies are shortest in response to preferred stimuli (Bolz et al., 1982; Celebrini et al., 1993; Lagae et al., 1994; Gawne et al., 1996; Lisberger and Movshon, 1999; Maunsell et al., 1999; Raiguel et al., 1999; Johansson and Birznieks, 2004). If this is the case, then there should be a temporal offset between the STAs of the two different phases, with the peak of the larger STA occurring before the peak of the smaller STA. To verify this, we took the difference between the time of the peak of the STA from the rising phase minus the time of the peak from the falling phase. We then correlated the sign of this temporal difference with the sign of the difference in area between the rising and falling STAs. There was significant negative correlation between the two sets (r = −0.254, p = 0.0061), indicating that the STA with a peak that occurred first was, on average, the STA with the larger area. This suggests that the variability in encoding between the phases was caused, in part, by spikes produced with different latencies.
It has been proposed that the visual cortex may encode aspects of the visual stimulus through its response latency. However, these past studies have usually considered latency as the first spikes produced by a step-change in a visual stimulus. Thus, a latency code usually only refers to the initial response of the neuron to a stimulus. Oscillations, by comparison, provide a reference point by which latency of a spike can be determined from the phase in the presence of ongoing activity. The idea that oscillations could provide a reference allowing for a continuous latency encoding scheme has also been proposed previously by Fries et al. (2007), and it bears many similarities to other studies proposing that oscillations may serve to convert signal strength into a phase code (Hopfield, 1995; Thorpe et al., 2001; Mehta et al., 2002). Although the phase encoding observed in our study is the result of the long delay between motion updates, it is of interest whether oscillations of the local field potential in visual cortex would allow for this encoding scheme to work under more general conditions, as observed in the hippocampus (O'Keefe and Recce, 1993). Alternatively, the relative phase difference in firing between neurons with different stimulus preferences might allow for this phase encoding scheme to function without the need for coherent oscillations between MT and downstream areas (Konig et al., 1995; Thorpe et al., 2001). The idea is that during coherent motion, MT neurons tuned to the direction of the coherent motion will fire in advance of neurons tuned to other directions, with the biggest phase difference for neurons tuned to the opposite direction. If downstream neurons could detect the temporal differences between incoming EPSPs arriving from differently tuned neurons (as in the auditory cortex) (Carr and Konishi, 1990), then this might eliminate the need for coherent oscillations to exist between MT and downstream.
Relation to the behavioral response
It is well documented that the activity of a single sensory neuron can covary with perceptual report (Celebrini and Newsome, 1994; Britten et al., 1996; Dodd et al., 2001; Cook and Maunsell, 2002b; Parker et al., 2002; Uka and DeAngelis, 2004; de Lafuente and Romo, 2005; Nienborg and Cumming, 2006). Additionally, it has also been shown in several visual detection/discrimination tasks that the neurons that most reliably encode the stimulus are also the neurons most correlated with the behavior (Celebrini and Newsome, 1994; Britten et al., 1996; Cook and Maunsell, 2002b; Parker et al., 2002; Purushothaman and Bradley, 2005). This has led to the hypothesis that perceptual decisions depend more on a small subset of highly influential neurons that carry the most reliable sensory information.
In our study, certain spike times encoded the coherent motion more reliably than others. By comparing the difference in encoding between phases with the difference in the correlation with behavior between phases, we determined that the phase most informative about the motion stimulus was also more (positively) correlated with the perceptual behavior of the animal. This relation was present as early as 50–150 ms after the coherent motion onset and is unlikely that top-down attentional modulation of MT contributed to this result.
There does exist evidence linking synchronous neural activity, as described in the first part of the Discussion, to behavioral performance in a discrimination task (Stopfer et al., 1997; Womelsdorf et al., 2006). However, to our knowledge, no previous study has shown phase-dependent neural activity to be correlated with behavior.
Implications for decision models
When we restricted our analysis to the 30 neurons with the best-defined STAs, we found that the difference between how well the two phases are correlated with behavior is large. In fact, our DP analysis showed that the activity on the phase with the smaller STA is actually correlated with the animal's failure to detect the coherent motion. This result is interesting because current models on how simple decisions are formed in the brain propose that evidence is accumulated toward one decision or another by integrating the spiking activity from sensory areas (Kim and Shadlen, 1999; Mazurek et al., 2003; Huk and Shadlen, 2005). In these models, spike times are irrelevant and the spike rate is the determining factor in decision-making. This implies that activity at any phase of the cycle should be positively correlated with the animal's detection performance. It is unclear how our observation that activity during half the cycle of the oscillation can be correlated with the animal failing the trial is consistent with these integrator models. Further studies will be required to determine how oscillating sensory activity is temporally processed to form decisions.
Footnotes
-
We thank Drs. J. Maunsell, C. Pack, and M. Vollrath for their critical comments on this manuscript. We also thank J. Smith, N. Ghandehari, and A. Wilhelm for their support during the analysis and D. Murray and T. Williford for their expert technical assistance.
- Correspondence should be addressed to Nicolas Y. Masse, Department of Physiology, McGill University, 3655 Sir William Osler, Montreal, Quebec, Canada QC H3G 1Y6. nicolas.masse{at}mail.mcgill.ca
