Abstract
Previous work has revealed a remarkably direct neural correlate of decisions in the lateral intraparietal area (LIP). Specifically, firing rate has been observed to ramp up or down in a manner resembling the accumulation of evidence for a perceptual decision reported by making a saccade into (or away from) the neuron's response field (RF). However, this link between LIP response and decision formation emerged from studies where a saccadic target was always stimulating the RF during decisions, and where the neural correlate was the averaged activity of a restricted sample of neurons. Because LIP cells are (1) highly responsive to the presence of a visual stimulus in the RF, (2) heterogeneous, and (3) not clearly anatomically segregated from large numbers of neurons that fail selection criteria, the underlying neuronal computations are potentially obscured. To address this, we recorded single neuron spiking activity in LIP during a well-studied moving-dot direction–discrimination task and manipulated whether a saccade target was present in the RF during decision-making. We also recorded from a broad sample of LIP neurons, including ones conventionally excluded in prior studies. Our results show that cells multiplex decision signals with decision-irrelevant visual signals. We also observed disparate, repeating response “motifs” across neurons that, when averaged together, resemble traditional ramping decision signals. In sum, neural responses in LIP simultaneously carry decision signals and decision-irrelevant sensory signals while exhibiting diverse dynamics that reveal a broader range of neural computations than previously entertained.
Introduction
Neural responses in the lateral intraparietal area (LIP) of rhesus monkeys appear to be a neural correlate of decision formation for where to look next (Mountcastle et al., 1975). During performance of a well-studied motion direction–discrimination task (Newsome and Paré, 1988), LIP firing rates ramp up before a saccade into the neuron's response field (RF). Firing rates ramp more steeply for easier, faster decisions based on stronger sensory evidence (Shadlen and Newsome, 1996, 2001; Roitman and Shadlen, 2002) and thus have been framed quantitatively as instantiating the accumulation of evidence in favor of one saccadic response over another (Gold and Shadlen, 2001; Mazurek et al., 2003; Palmer et al., 2005; Wong et al., 2007; Yang and Shadlen, 2007; Beck et al., 2008; Kiani et al., 2008). In short, LIP spike rates appear to be a direct neural correlate of the formation of a decision to move the eyes.
Given such a striking neural correlate of a cognitive process, a key question is how LIP neurons perform this computation of converting fleeting sensory inputs into cognitive signals reflecting the relevant sensory history. Presumably, a neural time-integrator would need to exhibit long responses to sensory input to support the accumulation of evidence over time. Indeed, LIP neurons are highly sensitive to the appearance of a visual stimulus in the RF, and some respond for as long as several seconds to this stimulus after it has vanished (“persistent activity,” e.g., Gnadt and Andersen, 1988). However, in the motion discrimination task used to observe decision signals in LIP, the RF has always contained a visual stimulus (a saccadic “choice target”). Furthermore, only neurons exhibiting persistent activity were targeted for recording. Both practices (visual stimulation of the RF during decision-making, and exclusive sampling) potentially obscure how LIP neurons transform inputs into outputs. Specifically, it is not known whether visual stimulation of the RF (by a choice target) interacts with the decision-related signals that have received primary focus. Second, the relationship between persistent activity (the conventional screening criterion for LIP cells) and ramping decision signals is unknown. Third, it is unclear the degree to which individual LIP neurons reflect the ramping decision signals primarily evident in averaged population activity.
We therefore recorded from LIP neurons while we systematically manipulated the presence/absence of the choice targets during the decision-making task. Our results show the following: (1) decision signals are multiplexed with decision-irrelevant visual signals; (2) the presence of decision-related activity is not strongly predicted by a cell's persistent activity; and (3) substantial single-unit heterogeneity reveals diverse response motifs that do not always resemble a decision variable until averaged together. These findings enrich our understanding of the neural computations in LIP that underlie the “neural correlates” observed there and motivate further study of how LIP itself is “read out” (and perhaps demultiplexed) by downstream brain structures (Hanes et al., 1995; Hanes and Schall, 1996; Basso and Wurtz, 1998; Gold and Shadlen, 2000, 2003; Horwitz et al., 2004; Curtis and Lee, 2010).
Materials and Methods
Procedures were standard and intentionally as similar as possible to prior work to allow our results to be directly relevant to similar studies (Roitman and Shadlen, 2002; Huk and Shadlen, 2005). Here, we briefly review the procedures and emphasize the few methodological changes.
Monkeys.
Two male, adult rhesus monkeys (Macaca mulatta) were kept and handled in accordance with National Institutes of Health guidelines and the Institutional Animal Care and Use Committee at the University of Texas at Austin. Standard surgery procedures were performed to place a recording chamber and head-holder (Crist Instrument). The recording chamber was placed over a craniotomy in the posterior parietal lobe (lateral 12, posterior 5). The recording chamber and head-holder were embedded in dental acrylic that covered the top of the head.
Apparatus.
Monkeys sat in a primate chair (Crist Instrument) in a sound-insulated, RF-shielded acoustic booth (Acoustic Systems). Eye movements were monitored by a video eye-tracker (ASL 6000, Applied Science Laboratories). Fluid reward for correct responses was dispensed by a solenoid-gated system (custom-built). Visual stimuli were displayed on a CRT display (resolution 800 × 600, refresh rate 100 Hz, Iiyama). Experiments were controlled by a PC (Dell Dimension 8250) running REX software (National Institutes of Health LSR). This computer also fed video commands to a Macintosh G4 running MATLAB v 5.2.1 (The MathWorks) under OS 9 and the Psychophysics Toolbox (Brainard, 1997), which were then displayed to the monkey.
Neurophysiology.
A positioning grid (Crist Instrument) was inserted into the recording chamber. A guide tube was inserted through a grid hole so that the tip of the tube just punctured the dura. An electrode (tungsten, glass-coated, 1.0–3.0 MΩ, Alpha Omega) was then lowered through the guide tube into the brain by a remotely operated microdrive (Frederick Haer).
Action potentials were identified online, using a dual-window time-voltage discriminator (Bak Electronics). Data were analyzed using custom MATLAB code. Post hoc analyses of interspike intervals and response levels over time were performed to confirm isolation of single units.
Instructed saccade tasks.
Instructed saccade tasks were used to characterize and select cells for recording in the main experiments. After isolating a cell (and hand-mapping the RF using the tasks described immediately below), data were collected while the monkey performed ∼100 trials of randomly interleaved trials of the visually guided and memory-guided saccade tasks (Fig. 1A). In both tasks, the monkey fixated a central fixation point and a saccadic target appeared either in, or 180 degrees opposite, the RF of the neuron. In a memory-guided saccade trial, the target disappeared after 100 ms. The monkey was required to maintain fixation for a variable delay until the fixation point disappeared (1150–1700 ms, uniform distribution), at which point he could saccade to the target location for a liquid reward.
Dot motion direction–discrimination task.
In the main experiment, trials from the “Targets-ON” (Fig. 1B, top row) and “Targets-FLASH” (Fig. 1B, bottom row) conditions were randomly interleaved. In both tasks, the monkey fixated on a central fixation point and two saccadic choice targets then appeared. One choice target was located in the RF of the neuron and the other choice target was diametrically opposite. The Targets-ON condition was the standard dot motion task, with an experimenter controlled viewing duration (500–1000 ms, uniform distribution). Targets-FLASH trials were identical to the Targets-ON trials, except that the choice targets were extinguished 100 ms after appearing (and 100 ms before the start of the motion stimulus).
In a follow-up experiment, Targets-ON trials were randomly interleaved with “Targets-NONE” trials. Targets-NONE trials were identical to the Targets-ON trials, except that the choice targets were never presented. By making the timing of events leading up to the onset of the motion stimulus short and regular across trials, monkeys could expect the onset of the motion stimulus, even when the choice targets were not presented.
The motion stimulus appeared 500 ms after fixation was achieved, within a circular aperture (5 degrees in diameter) centered 5 degrees away from the fixation point and at a 90-degree angle from the axis defined by the two choice targets. The algorithm for generating the motion display was identical to that used in prior LIP studies. Dots within the motion aperture were 0.1 degrees in diameter at a density of 16.7 dots/degree2/s and “moved” by being replotted in later frames at either in a random location or, at a probability equivalent to the coherence value of the trial, in a location in the direction of a choice target (the “correct” target for that trial). The standard motion coherence values of 0, 3.2, 6.4, 12.8, 25.6, and 51.2% were used. At 0% coherence, the monkeys were rewarded with 50% probability. After the motion had been displayed for a variable time period (500–1000 ms), it was extinguished and the monkey was required to maintain fixation for another 500-ms delay (“wait” period) until the fixation point disappeared, at which point he was allowed to saccade to the correct choice target location for a reward. If the monkey took >450 ms after the go-signal to make a saccade into one of the two potential response windows, the trial was considered incomplete. The monkey received a liquid reward 200 ms after a correct response was registered. Eye movements were monitored on-line and inspected off-line to ensure they met standard criteria. We chose to use a relatively liberal online window for acceptable saccades (as wide as 10 degrees) because we did not want the monkeys to experience different success rates for Targets-ON versus Targets-FLASH (and Targets-NONE) trials as a result of the likely differential accuracy of saccades to visible versus remembered target locations. The differences in neural response observed between conditions are unlikely to be explained by saccade metrics (discussed in Results), and the monkeys were well trained and did not appear to be acquiring undesirable strategies associated with the saccade window size.
We opted to use an experimenter-controlled, variable duration form of the decision task because we were originally interested in comparing the temporal dynamics during memory-guided instructed saccades and decision trials with the same durations (and hence needed to control these durations, instead of letting the monkey decide when to respond, as in “response time” versions of the task). After noting the unexpectedly large and complex interactions revealed by choice target manipulations during the decision tasks, we decided to continue using the variable-duration task for several reasons. First, the effects we focused on were so temporally persistent that knowing the exact epoch of decision-making was not critical to interpretation. Second, we were able to observe what happened during a wait/delay period after motion viewing but before the saccade. Third, neural activity measured during the response-time version of the decision task may reflect additional contingencies (such as a minimum time to reward) used during training to generate long reaction times on easier trials. However, because knowing the duration of the likely “decision epoch” is an important part of interpreting the relation between the physiology and decisions, we performed additional psychophysical experiments using very short, variable, experimenter-controlled durations to assess the period of psychophysical temporal integration (the results of which confirmed the results of several prior studies that have also shown integration over the first 400–500 ms of dot viewing) (Gold and Shadlen, 2003; Kiani et al., 2008).
Cell selection.
Neurons on the lateral bank of the intraparietal sulcus were only considered for data collection if they were encountered at least 2000 microns below the dura and were spatially selective enough to have a clear RF during instructed saccade tasks (Fig. 1A). Cells were considered for further analysis if they exhibited spatially selective responses for at least 40% of the time in any of the three response epochs in either instructed saccade task. We defined three response epochs: (1) the epoch of visual response after target appearance; (2) the delay epoch as the monkey waited for go-signal; and (3) and the motor epoch around the saccade. The visual response epoch was 30–200 ms after target appearance in visually guided saccade task; the delay epoch was 300 ms after target onset until the go-signal in both visually and memory-guided saccade tasks; the motor response epoch was 100 ms before to 75 ms after the saccade in both visually and memory-guided saccade tasks. The criterion value of 40% and epoch windows were chosen to match estimates of selectivity by eye on the standard spike count histogram.
Selectivity during each epoch was determined as the proportion of time bins within that epoch where firing rate was different between trials when the monkey made a saccade into the RF (“In-RF” trials) and when he made a saccade to the null location (“Out-RF” trials). Firing rate was considered different between conditions within a given time bin if the upper and lower bounds of the middle 68% of bootstrapped firing rate values did not overlap (i.e., the bootstrapped equivalents of ±SEM). For inclusion in the dataset of cells considered to have persistent activity, this criterion had to be satisfied only for the memory delay epoch in the memory-guided saccade task.
Although we collected data from cells that were spatially selective in any epoch of either instructed saccade task, we were biased to record from the cells with persistent activity so as to ensure enough data were collected from these neurons. Substantial data from these neurons were required for comparison with other laboratories where only these neurons are studied.
Fewer neurons were collected from Monkey J because, despite a large number of recording attempts, neurons with spatially selective responses in the instructed saccade task were encountered infrequently, and ultimately concerns about the implant required removal of the recording chamber. Given the moderate number of total cells from this monkey (n = 16) and the compelling similarity of single-cell responses between these cells and ones seen in the other monkey, we did not think that additional data collection and/or monkey use was warranted.
Data analysis.
In all plots and analyses of neural response, only correct trials were used (except for 0% coherence trials, for which there is no correct response, and hence were grouped according to the monkey's target choice). An additional exception was the calculation of d′ during the decision epoch, where all complete trials were used.
The time bins used to compute firing rate were 50 ms wide, unless a running mean was used, in which case a sliding 100-ms window was used. If firing rate was normalized, then a cell's spiking frequency was normalized so its dynamic range spanned the range (0–1).
To characterize the variability of firing rate, firing rate was bootstrapped 100 or 1000 times (indicated in plot captions) by resampling with replacement the same number of original trials in the original condition to produce synthetic firing rates for each condition. Then the high and low values in each time bin that bound the middle 68% (±SEM) of bootstrapped firing rate values were gathered into two vectors. These two vectors, high and low, were then drawn around the actual mean firing rate to indicate variability.
To summarize conventional, directional dependence of firing rate on motion strength (steeper ramping of neural response up and down for higher motion strengths directed, respectively, toward and away from the RF), the response vector of Out-RF choices was subtracted from that of In-RF choices for each motion strength and plotted as a function of time during motion viewing. Directional dependence of neural activity on motion strength was also shown by subtracting the average response of Out-RF choices from that of In-RF choices during the same epoch (0–700 ms) and plotting those values as a function of motion strength.
To illustrate nondirectional dependence of firing rate on motion strength, the firing rate vectors of In-RF and Out-RF choices for each absolute value of motion strength were averaged together for a single cell, thus yielding one response vector for each motion strength. Averaging these response vectors across cells produced the population response for population-level plots of nondirectional coherence dependence.
Behavioral data of proportion correct as a function of motion viewing duration were fit with a line resulting from plateaued accuracy in the tasks used during neural recording (trials had at least 500 ms of motion viewing), whereas the supplemental behavioral data of shorter motion viewing durations (100–900 ms of motion viewing) were fit with a saturating exponential. Variability was measured by bootstrapping (1000 reps). Variability was graphically indicated in plots by fitting the same functions to the 16th and 84th percentiles of the bootstrapped data points.
To quantify how predictive the firing rate of a cell was for the monkey's eventual saccade, we calculated d′ values for individual neurons during trial epochs of interest. d′ was calculated for each neuron during the memory period of the memory-guided saccade task and again during the decision-making epoch. In the memory-guided saccade task, spikes occurring 300 ms after target appearance and before the go-signal were used to calculate d′. In the decision task, spikes occurring 200–700 ms after dot motion onset were used to calculate d′. This time period was chosen as the decision-making epoch to be in accord with behavioral data (which suggests that decisions are well made by 500 ms of dot motion viewing) and because of the ∼200-ms latency of decision-related LIP responses (Churchland et al., 2008), which could be processed while the monkey waited after motion ended for the go-signal.
To calculate how well choice could be predicted from spike rate by an ideal observer during decision formation, conventional receiver operating curve (ROC) analyses were performed (using methods described by Shadlen and Newsome, 2001) for each motion strength for the 47 cells with persistent activity in Targets-ON and Targets-FLASH trials. Predictive index values from a ROC analysis for each neuron were calculated for a given trial type and a given motion strength, in a motion onset-aligned time window of 100 ms (advancing in 10-ms increments).
To illustrate the different response motifs of cells across the population, individual cell responses were subjected to a k-means clustering algorithm. The single-neuron response vectors first were computed by calculating the firing rate for each of the 80 spatially selective cells during the initial 700 ms of motion viewing for In-RF and Out-RF choices (collapsed over coherence) with a 100-ms running mean. Each cell's firing rate was then normalized to range between 0 and 1, and the response vectors were concatenated. (Although this is an admittedly coarse manner of summarizing each cell's response dynamics, visual inspection of the resulting clusters suggests that it captured important aspects that allowed for distinguishing between different response types.) These 80 response vectors were then grouped by a k-means clustering algorithm, which iteratively minimizes the Euclidian distance between the response vectors of each group. The number of groups for output was chosen as 6 after qualitatively comparing output for a range of group numbers. The classification of heterogeneity did not change substantially when slightly larger numbers of groups were considered.
Results
We recorded extracellularly from 80 spatially selective neurons in LIP of two rhesus macaques (M. mulatta) while they performed instructed saccade and perceptual decision-making tasks. The conventional instructed saccade tasks were either visually guided overlap saccades (in which a single target was presented, and the monkey was cued to make a saccade to it; Fig. 1A, top row) or memory-guided saccades (in which the target was flashed early in the trial, and the monkey was later cued to make a saccade to the remembered location; Fig. 1A, bottom row). The decision-making tasks were standard two-alternative forced choice motion direction–discrimination tasks in which the monkey made a saccade to one of two locations to indicate his decision about the direction of a random-dot kinetogram. In the Targets-ON condition (Fig. 1B, top row), the two saccade choice targets were always present, whereas they were merely flashed on and off at the beginning of the trial in the Targets-FLASH condition (Fig. 1B, bottom row), or were not presented at all in a follow-up experiment (Targets-NONE condition).
Visual stimulation of the response field (RF) changes neural response during decision-making for the LIP neurons in our sample with persistent activity (n = 47). A, Visually guided (top) and memory-guided (bottom) instructed saccade tasks. In both tasks, the monkey began by fixating a central spot. A saccadic target then appeared either in the RF of the neuron or in the location diametrically opposite. In the memory-guided saccade task, the target disappeared 100 ms after its onset, whereas it remained on in the visually guided saccade trial for the whole trial. After a variable delay period, the fixation point disappeared, cuing the monkey to make a saccade to the target location for reward. B, Targets-ON (top) and Targets-FLASH (bottom) decision tasks. Monkeys performed a 2AFC dot motion direction–discrimination task. On half the trials (interleaved), the choice targets remained on throughout the trial (Targets-ON); in the other half of trials, the targets were only flashed at the start of the trial (Targets-FLASH). One choice target was placed in the RF of the neuron, and the other was placed in the location diametrically opposite. In the Targets-FLASH task, the targets disappeared before dot motion onset, whereas the targets remained on in the Targets-ON condition for the whole trial. Strength of motion (% coherence) varied from trial to trial. Duration of the dot motion was controlled by the experimenter (500–1000 ms during physiology, uniform distribution). C, Population response is similar during visually guided and memory-guided saccade tasks. Firing rate of 47 LIP neurons with persistent activity (normalized to the dynamic range of each neuron) is shown during both visually and memory-guided saccade tasks (solid and dashed line, respectively). Left, Responses aligned to target appearance (“Target on”). Right, Aligned to saccade (“Saccade”). Dashed vertical line marked “Target off” indicates time at which the target was extinguished during memory-guided trials. Green indicates trials when the target appeared in the RF of the neuron (“In-RF”); and red, when it appeared in a location opposite (“Out-RF”). Shaded regions indicate variability produced by the middle 68% of 100 bootstrapped values of firing rate. D, Population response during decision-making depends on visual stimulation of the RF. Firing rate (normalized same as Fig. 1) of 47 cells with persistent activity is shown during both Targets-ON and Targets-FLASH decision trials (solid line and dashed line, respectively). Firing rate is aligned to trial events marked by vertical lines. Dashed vertical line indicates when the targets disappeared in the Targets-FLASH condition (“Targets off”). Green indicates trials in which the choice saccade was made to the In-RF location; and red, to the Out-RF location (both curves are collapsed over all coherences). Only correct trials are included in this analysis, except for 0% coherence trials, which are segregated by choice. Shaded regions indicate variability middle 68% of 1000 bootstrapped firing rate values. See Figures 7 and 2, respectively, for plots of behavioral performance and neural response for each coherence.
In the initial sections of the Results, we focus on the 57 neurons that exhibited strong persistent activity during memory-guided saccades (47 in the Targets-FLASH experiments, 10 in the Targets-NONE experiments), so as to best relate to prior work that has applied similar cell selection criteria (e.g., Shadlen and Newsome, 1996; Huk and Shadlen, 2005; Churchland et al., 2008; Bennur and Gold, 2011). We initially describe effects of visual stimulation by the choice targets. First, a simple analysis shows that a large fraction of LIP response during decision formation is driven by the simple presence of the choice target in the RF. Second, a finer-grained analysis reveals the surprising effect that decision-irrelevant visual target responses interact with the onset of the moving dots (which are the decision-relevant sensory stimulus). This interaction results in a coherence-dependent modulation of the response that is likely also to be decision-irrelevant. When we examined single-neuron responses to see how these signals might be multiplexed at the single-neuron level, we noticed significant variability from neuron to neuron in the form of these effects. This motivated us to consider heterogeneity more thoroughly.
The later sections of the Results focus on characterizing the neural heterogeneity in LIP in the context of this putative multiplexing of decision-related and decision-irrelevant signals. We therefore consider a broader sample of all 80 spatially selective LIP neurons, selected solely on the basis of having a spatially selective response during visually guided and/or memory-guided saccade tasks. Using the neural heterogeneity to gain leverage on several standard assumptions, we found little evidence that cells with persistent activity form an identifiable “decision making” subpopulation. Instead, the data raise the possibility that population-temporal integration may be implemented primarily by the aggregate activity of several types of temporal response motifs, many of which differ qualitatively from a direct neural correlate of accumulating evidence in favor (or against) a particular saccadic response.
Together, these two LIP neuron response characteristics (multiplexing and heterogeneity) demonstrate that the neural computations in LIP may be distinct, or at least far more nuanced, than the population-level neural correlate of evidence accumulation that has been observed during perceptual decision-making tasks.
Baseline assessment of neural responses to instructed saccades
We initially restricted our analysis to the neurons with strong persistent activity during memory-guided saccades (as described in Materials and Methods, Cell selection). Experimenters (including ourselves) have long assumed that such neurons are more prone to carry decision signals (Shadlen and Newsome, 1996), the logic being that persistent activity indicates the capability of such neurons to integrate information over time, which is a key element in forming decisions in the motion discrimination task (Huk and Shadlen, 2005; Palmer et al., 2005; Kiani et al., 2008).
As expected, these cells exhibited higher firing rates when the saccade target was presented in the RF compared with outside it, throughout the entire trial, from target onset, through the delay period, until the saccade itself (Fig. 1C). Response levels were very similar regardless of whether the target stayed on (visually guided saccades) or was only flashed early in the trial (memory-guided saccades), confirming the presence of strong “memory” activity. Because this persistent activity between target onset and the later saccade was not affected by whether the saccade target stayed illuminated, one might expect that these neurons should be ideal candidates for carrying decision-related activity that is not affected by the presence or absence of a visual saccade target during the motion direction–discrimination task. However, this is an empirical question, and we sought to test it.
Decision signals are combined with a strong response due to visual stimulation of the RF
After assaying each neuron with the instructed saccade tasks described above, single-unit spiking activity was measured while the monkey performed two versions of the decision task. In the Targets-ON task (Fig. 1B, top), the monkey discriminated the direction of dot motion (two-alternative forced-choice) and communicated his decision by making an eye movement to one of two choice targets. In the Targets-FLASH task (Fig. 1B, bottom), the monkey performed the same dot motion direction–discrimination task, but this time the choice targets were only flashed briefly (100 ms) at the start of each trial, before dot motion began. Importantly, this meant that there was no visual stimulation of the cell's RF during decision-making. After the go-signal, the monkey communicated his choice by making an eye movement to one of the two previously illuminated choice target locations. These Targets-FLASH trials can be thought of as a “memory-guided” version of the Targets-ON condition, similar to the two types of instructed saccade tasks.
Targets-ON and Targets-FLASH trials were randomly interleaved and were identical, except for whether or not the targets remained on during the whole trial or were only briefly illuminated right before the onset of the dot motion. This manipulation of the choice targets would not, by design, be expected to exert a substantial effect on decision-making; and indeed, psychophysical performance was very similar in the two conditions (Monkey J, 79.5% correct Targets-ON, 78.2% Targets-FLASH; Monkey P, 78.2% Targets-ON, 78.7% Targets-FLASH).
Despite similar psychophysical performance, LIP responses were very different in Targets-ON and Targets-FLASH trials (Fig. 1D). The population neural response in both conditions followed the same general pattern through the various phases of the trial, but responses were substantially lower throughout Targets-FLASH trials. Specifically, the average firing rate during Targets-FLASH decision trials was approximately 25% lower during motion viewing and the subsequent wait period. This smaller response during Targets-FLASH trials was evident soon after the targets were extinguished and persisted throughout the trial with similar magnitude for all motion coherences (Fig. 2). The emergence of this effect so soon after offset of choice targets in Targets-FLASH trials suggests a primarily visual origin, although the significant difference in spike rates then persisted all the way through the decision period and out to the saccade itself. The effect was evident regardless of whether individual cell responses were normalized to their respective dynamic range (Fig. 1D), examined in raw spike averages (data not shown), computed for each coherence separately (Fig. 2), or plotted with other normalization schemes (data not shown).
The main effect of Targets-FLASH on LIP response is similar across all conditions (each motion coherence) that were summed together in Figure 1D. Population firing rate during Targets-ON and Targets-FLASH trials for each coherence. Each panel shows the firing rate of the n = 47 neurons during Targets-ON and Targets-FLASH trials for a different coherence value. Green indicates In-RF choice; red, an Out-RF choice. Solid line indicates Targets-ON trials; dashed line, indicates Targets-FLASH trials. Darker color indicates higher motion coherence. Firing rate was computed as a 100 ms running mean.
This large, and apparently decision-irrelevant, effect of target presence/absence is particularly striking because a similar effect was not present in these neurons when comparing the response during visually and memory-guided instructed saccades (Fig. 1C). Thus, the spike rate in this particular subset of LIP neurons (i.e., ones that exhibit strong memory activity) does not directly map on to the amount of accumulated evidence; instead, a large component of the response during decision-making simply reflects whether or not there is a small red spot (the choice target) in the RF.
The mixture of decision-related ramping signals with a decision-irrelevant response to the choice targets shows that LIP neurons multiplex this information; during a trial, spike rate simultaneously codes for target presence as well as the eventual saccade. Because this multiplexing persisted throughout the trial, substantial constraint is placed on the relation between LIP activity and decisions. However, a finer-grained analysis of decision-irrelevant interactions poses even greater challenge to the mapping between LIP spike rates and decision formation.
Coherence-dependent responses unrelated to decisions are produced by an interaction between decision-related and decision-irrelevant components
The interaction between the choice targets and decision signals was found to be more complex when we analyzed the coherence dependence of the ramping responses during motion viewing. To our chagrin, initial inspection of standard spike count histograms did not reveal coherence-dependent ramps that have been reported in a number of prior studies (Shadlen and Newsome, 1996, 2001; Roitman and Shadlen, 2002; Huk and Shadlen, 2005; Law and Gold, 2008) (Fig. 3, top row). Further scrutiny revealed that, for higher coherences, the ramping slope in firing rate was indeed steeper, but the ramping emerged from a lower starting point. For lower coherences, the inverse was true: shallower ramps but emerging from a higher starting point.
LIP population response during decisions depends on motion strength in two ways: conventional (directional coherence dependence) and unexpected (nondirectional coherence dependence). Normalized firing rate of n = 47 cells is aligned to the start of dot motion. Only correct trials are plotted, as well as all trials of 0% motion coherence, where there is no correct choice. Firing rate computed as a 100-ms running mean. Stronger motion strengths are indicated by darker shades. Top, Firing rate during decision epoch for each direction (green and red indicate motion toward and away from RF, respectively) and motion strength (darker colors indicate higher motion strength) shows two simultaneous forms of motion strength dependence: steeper ramping for higher motion strengths simultaneously occurs with overall lower firing rate for higher motion strengths. Middle, The conventional form of firing rate dependence on motion strength is isolated. The firing rate of In-RF target choices minus Out-RF choices is plotted across the decision epoch. Higher motion strength trials show bigger response differences between In-RF and Out-RF choices. Bottom, The unexpected inverse dependence of firing rate on motion strength is isolated. Firing rate is plotted across the decision epoch for each motion strength. The firing rate vector for each motion strength was calculated by first computing the firing rate vectors for the two separate directions of a motion strength (shown in the standard PSTH in the top row), and then averaging those two vectors together. This analysis reveals that higher motion strength trials have lower firing rates.
To better visualize this mixture of slope and starting point dependencies on motion coherence, we decomposed responses during the dot motion epoch into 2 coherence-dependent components. We took the data shown in the conventional spike count histogram (Fig. 3, top tow) and simply calculated: (1) the difference between In-RF and Out-RF choice responses for each coherence (which we refer to as the directional coherence dependence signal) (Fig. 3, middle row), and (2) the response averaged over both In-RF and Out-RF trials for each coherence (thus creating a nondirectional coherence dependence signal) (Fig. 3, bottom row).
In the Targets-ON task, the expected coherence dependence observed in prior studies using similar stimuli is evident in the directional coherence-dependent signal (Fig. 3, left column, middle row, and Fig. 4A, top row). Coherence-dependence was muted in the Targets-FLASH task (Fig. 3, right column, middle row, and Fig. 4B, top row). To further compare the decision-related ramping activity of our LIP sample to that observed in prior work, a classic ROC analysis was performed (Fig. 4, bottom row) (Shadlen and Newsome, 2001, their Fig. 10). Behavioral choices were predicted based on single-neuron activity, calculating the probability of correctly guessing the choice as a function of time and motion coherence (knowing only the preferred saccadic target of the neuron under study). This resulted in an evolving ROC pattern where predictive power grew more steeply across time for higher coherences. This confirms that our LIP sample exhibits decision-related responses similar to those in prior work.
The putative neural correlate of evidence accumulation can be derived from LIP population response. Top, The response difference between In-RF and Out-RF choices for n = 47 cells during motion viewing (first 700 ms) is plotted for each coherence for A (Targets-ON) and B (Targets-FLASH) trials, illustrating larger response differences for higher coherences. Bottom, ROC analysis of the same data again illustrates putative evidence accumulation signals because during trials of stronger motion strengths, the distributions of spike counts between In-RF and Out-RF choices were less overlapping, so an ideal observer could more correctly predict the monkey's behavior across time for both A (Targets-ON) and B (Targets-FLASH) trials.
The unexpected, nondirectional coherence dependence emerged earlier than the directional coherence dependence and was of a similar overall magnitude (Fig. 3, left column, bottom row). This nondirectional, inverse coherence dependence was also clearly visible in single neurons (Fig. 5). In the Targets-FLASH task, we also observed both directional and nondirectional coherence dependencies, although the nondirectional component was somewhat muted in magnitude and exhibited different temporal dynamics (Fig. 3, right column).
Single-neuron responses show inverse, nondirectional dependence on motion strength. Each column shows one neuron's response (letter and numbers above the plots indicate unique recording session codes). Top: The firing rate of individual neurons can be strongly, inversely related to the strength of motion in both Targets-ON (left) and Targets-FLASH (right) decision tasks. Firing rate vectors for In-RF and Out-RF choices (correct trials only except for 0% coherence trials) are computed separately and then averaged together, producing the neural response to each motion strength plotted here. Darker lines represent firing rate for higher motion strengths. Bottom: Each neuron is a “conventional” LIP cell in that it exhibits robust persistent activity in the memory-guided saccade task. Plotting format same as in Figure 1C.
The smaller nondirectional coherence dependence in Targets-FLASH trials was a first clue that it resulted from an interaction between the motion stimulus and the choice targets. To better understand this interaction, we performed an additional set of recordings to observe LIP responses when no choice targets were on screen at any point during a trial. We therefore recorded from 10 more LIP neurons (with strong persistent activity in memory-guided saccades) as one of the monkeys performed the decision task where Targets-ON trials (as described above) were interleaved with Targets-NONE trials (which were identical, except that the targets were never illuminated) (Fig. 6A).
Targets-NONE decision task reveals that a total lack of visual stimulation of the RF reduces the inverse coherence dependence. A, Targets-NONE decision task schematic. This task is the same as Targets-ON (Fig. 1B, top), except that choice targets never appear. Targets-NONE trials were interleaved with Targets-ON trials in a recording session. B, Neural population response during decision tasks. Firing rate of n = 10 cells with persistent activity is shown during both Targets-ON and Targets-NONE decision tasks (solid and dashed line, respectively). Same format as Figure 1D. C, Population response of n = 10 cells during decision, separated by motion strength is aligned to the start of dot motion. Same format as Figure 3.
In these Targets-NONE trials, the response pattern now contained a more conventional pattern of coherence-dependent ramping and was not dominated by a substantial nondirectional component (Fig. 6C, right). The large nondirectional component was still clearly present in these neurons during Targets-ON trials, replicating the surprising result of the main experiment (Fig. 6C, left).
Although it is standard to focus attention to the period of time when the animal was likely forming a decision, the pattern of our results (i.e., effects of target presence/absence that persist throughout the entire trial, effects of motion coherence early in trials that are inverted and nondirectional) makes such an exercise less critical. However, it of course remains important for the sake of generalization to demonstrate that the animals were performing the task similarly to prior studies. Indeed, psychophysical performance followed expected patterns. Accuracy (proportion correct) was generally flat across the range of viewing durations we used during the electrophysiological recordings (500–1000 ms; Fig. 7). This demonstrates that the majority of decisions were made by the monkeys before even our shortest period of dot motion viewing (500 ms), a result expected given prior results (Gold and Shadlen, 2003; Kiani et al., 2008). To more precisely define the epoch of decision formation, we collected additional psychophysical data from one monkey in (purely behavioral) sessions of intermixed Targets-ON and Targets-FLASH trials that included shorter motion viewing durations (100–900 ms, uniform distribution). In these trials, accuracy plateaued for all motion strengths by ∼400 ms of motion viewing for both conditions (Fig. 7D), following patterns very similar to prior studies (Gold and Shadlen, 2003). Although these results do not rule out the potential for subtle behavioral differences between conditions or relative to other work, conventional analyses did not reveal anything out of the ordinary that might explain the time course or magnitude of the effects we observed in the physiology.
The effect in Figures 1D and 3B is likely not the result of a difference in behavioral performance between conditions because behavioral performance was similar between Targets-ON, Targets-FLASH, and Targets-NONE decision tasks. Proportion correct is plotted as a function of motion viewing duration, separated by task and motion strength. Transparent shading indicates ±1 SEM by bootstrapping trials (see Data analysis). Data in this figure are taken from the electrophysiological recording sessions and from additional sessions (some coherences not shown for visual clarity; similar effects were observed). Behavioral performance for Monkey P in A, and Monkey J in B during the Targets-ON and Targets-FLASH decision tasks. Monkey P, 68 sessions (57,222 trials); Monkey J, 16 (11,172 trials). C, Performance during Targets-ON and Targets-NONE decision trials. Only Monkey P; 7978 trials. D, Performance during Targets-ON and Targets-FLASH decision task in which motion viewing on a trial was 100–900 ms. Only Monkey P; 11,251 trials.
These interactions between choice targets, motion, and decision activity are not likely the result of differences in saccade metrics (between Targets-ON and Targets-FLASH or Targets-NONE trials). One might be concerned that saccades to extinguished target locations are less accurate and precise than saccades to targets that are still illuminated and that this might explain the lower LIP responses during Targets-FLASH and Targets-NONE trials. However, the constellation of results argue against this proposition playing a major role here. First, these effects emerged early in the trial and maintained an approximately constant magnitude until the saccade, inconsistent with a primarily perisaccadic locus. Second, the neurons we analyzed showed no such drop during memory-guided saccades compared with visually guided saccades, which would be expected to contain a similar saccade-metric confound (Fig. 1C). Third, some individual neurons (found in both monkeys) exhibited the opposite pattern: increased firing rate in target-absent trials (Fig. 8). Fourth, in our Targets-NONE experiment, the population response was actually higher than the Targets-ON condition at and around the time of the saccade (Fig. 6B). Fifth, and most compellingly, consider the trials when the monkey chose the Target outside of the neuron's RF: both Targets-FLASH and Targets-NONE trials have an even lower firing rate than Targets-ON trials at the time of saccade, which is opposite expectation if saccade accuracy variable were to blame for firing rate (i.e., less accurate saccades to the Out-RF choice target would not be expected to further decrease Out-RF choice trial responses).
Single neurons demonstrate the main effects and also reveal idiosyncrasies. Effects of RF stimulation in 2 different neurons during decision tasks. Each plot shows a single neuron. Top, Higher firing rate in Targets-ON trials than in Targets-FLASH trials (i.e., confirming the main effect shown in the population responses in Fig. 1D). Bottom, Higher firing rate for Targets-FLASH trials (i.e., an inverted effect relative to the population average). Same color key as Figure 1D.
The form of nondirectional and inverted coherence dependence observed in Targets-ON trials is an unexpected effect of RF stimulation and is not obviously interpretable as a direct neural correlate of the accumulation of evidence. Instead, the pattern of results is consistent with a sensory mechanism, such as divisive normalization, which operates across the entire visual field, without respect to the relevance or irrelevance of a signal to decisions. The stronger the motion coherence in a portion of the visual field, the larger the reduction in response of neurons with an RF in a distant portion of the visual field. This interaction with the more conventional response components likely occurred because we used a shorter time interval between appearance of choice targets and the onset of dot motion (200 ms) than was used in other experiments recording in LIP during a motion discrimination task (at least 500 ms in Shadlen and Newsome, 2001; Roitman and Shadlen, 2002; Huk and Shadlen, 2005; Churchland et al., 2008; Rorie et al., 2010).
We note that the two primary effects we have discussed (higher responses for Targets-ON than Targets-FLASH trials, and an interaction between the target onset and a response to motion coherence) did not occur in isolation: although described separately for the sake of exposition, both were evident in the same dataset. Even during simple versions of this perceptual decision-making task, LIP responses are a function of the decision process, the timing and presence of simple RF stimulation, and the interaction between these factors.
Single neuron examples
Although the effects of simple target manipulations on averaged responses illustrate that decision and decision-irrelevant visual signals can exist simultaneously in LIP, the underlying single-neuron computations remain occluded. We gained further insight by examining the decision signals displayed by single neurons. Many cells showed the effects described above in the “population” averages, demonstrating similar responses at the level of single neurons (Figs. 5, 8, top rows). However, although some single neurons exhibited responses to Targets-ON versus Targets-FLASH trials that were similar to those observed in the subpopulation average (Fig. 8, top), we also saw compelling counter examples that showed the opposite pattern (i.e., higher responses to Targets-FLASH than Targets-ON trials; Fig. 8, bottom). These observations motivated us to examine single-neuron responses in more detail as an additional clue into the underlying mechanistic computations within LIP. This investigation yielded two additional insights described in the following two sections.
Dissociation of decision signals from persistent activity
Up to this point, we restricted our analysis to only 47 of the 80 spatially selective neurons in our dataset because they exhibited strong persistent activity. The reason for this conventional sampling criterion derives from the logical similarity between evidence accumulation and persistent activity: a cell that can integrate evidence over time in decision-making would theoretically need persistent “memory” activity to accrue units of evidence across time. However, just as scrutiny of single neurons revealed a dramatic departure from the population response in the results of cellular diversity described above, we decided to test this assumption by expanding our analysis to the individual responses of all spatially selective cells, regardless of whether they had strong persistent activity (n = 80) (see Materials and Methods, Cell selection, for quantitative criterion for spatial selectivity).
Across this less-restricted LIP sample, persistent activity was not a strong predictor of whether a cell exhibited choice-predictive signals during decision-making (Fig. 9). An example cell with no persistent activity but strong choice-predictive response is displayed in Figure 9A. To capture the relationship between persistent activity and choice-predictive activity across all cells, selectivity (d′) of each cell during the decision epochs as a function of its selectivity during the memory delay period (Fig. 9B). The resulting correlation is weak albeit statistically significant (r2 = 0.12, p = 0.001): the scatterplot does not compellingly justify using the presence/absence of persistent activity for accepting/rejecting individual LIP neurons (similar results were obtained using other assessments of persistent activity, including qualitative judgments, e.g., those typically done “on-the-fly” during experimental sessions).
Weak relation between persistent activity and decision-related signals. A, Single neuron example where persistent activity during the memory delay of instructed saccade task is clearly dissociated from decision signals during decision-making. This neuron shows weak persistent activity during the memory-guided saccade task (left), yet strong decision signals during the decision task (right). B, Scatterplot shows weak relationship between persistent activity and selectivity in the decision epoch for the entire population of 80 cells. Spatial selectivity of each cell (n = 80) is plotted in units of d′. d′ during motion discrimination of the decision task is plotted as a function of d′ during the memory delay period of the memory-guided saccade task. r2 = 0.12, p = 0.0014 (type II regression, gray line). In the decision task, decision epoch was 200–700 ms after dot motion onset.
This pattern of results shows that it is unlikely that there is an easily identifiable and special subset of LIP neurons that have a privileged relation to decision processes. Instead, the surprisingly weak dependence of decision-related activity on persistent activity motivated an even more general reassessment of single-neuron responses and their relation to the accumulation of evidence during the discrimination task. Although there is indeed a statistically significant relation and it is possible that other quantification schemes might reveal a stronger relation, the point here is that the relation is subtle enough that online judgments by experimenters do not receive strong support. Other selection issues, such as the overall responsivity of LIP neurons, may also deserve more principled consideration (one can note glaring differences in overall response levels in “population average” plots across studies).
Temporal dynamics suggest a diversity of computations
Many single-cell responses did not show conventional ramping decision signals but instead exhibited temporal dynamics during motion viewing (“response motifs”) which could not be as easily interpreted as a neural correlate of evidence accumulation. Figure 10A (left column) shows the choice-separated peristimulus time histogram (PSTH, averaged over coherences) for six example neurons. One exhibits canonical ramping responses (fourth row, cell ID p033011); the others show varying degrees of departure from this pattern. Figure 10A (right column) shows the difference in spike rate between In-RF and Out-RF (correct) choices as a function of coherence, confirming that many of the cells exhibit the dependence on motion coherence often asserted as a signature of decision relevance, despite exhibiting a noncanonical response motif.
Diverse responses of individual LIP neurons combine to show conventional ramping activity of the population response. A, The responses of 6 different example neurons are shown. Left, The firing rate of each example neuron is shown for the first 700 ms of dot motion. Right, Directional coherence dependence of each neuron is shown by plotting the difference between average response for In-RF and Out-RF choices during motion discrimination (first 700 ms of dot motion) as a function of coherence. The slopes of the fit lines are reported on the plots in units of spikes per 100% coherence ±SEM. B, Combination of the diverse responses of the 6 cells in A yields an average response with conventional decision signals. C, The population response of the entire population of cells (n = 80). Slopes of the fit lines are reported in the same units as A, except that normalized firing rate units are used instead of spikes.
The average response of these six diverse example cells (Fig. 10B) results in both a PSTH (left) and a pattern of directional coherence dependence (right) that approximates the expected response dynamics reported in prior work. Although just a simple mathematical exercise, this plot reinforces the logical point that the population response can appear to mimic the accumulation of evidence, despite the fact that many of the individual constituent neurons appear to be reflecting other neural computations that are less easily interpreted in terms of decision formation.
The selection of these six example cells was not a wholly arbitrary exercise, moreover. The six example cells we chose actually exhibited temporal dynamics that are representative of the six groups of cells revealed by a k-means clustering (Materials and Methods, Data analysis), which provided a coarse but automated way to group cells with similar temporal dynamics. These six groups (of 9–15 cells each) exhibited highly diverse temporal dynamics during the first 700 ms of motion viewing (Fig. 11A), from cells with only moderate or late choice sensitivity and mostly decreasing firing rate (top 2 groups) to cells with greater, earlier choice sensitivity and more severe ramping for Out-RF choices (middle 2 groups). Also, cells with no gradual growth in choice selectivity were common (fifth group down), as well as cells with constant growth in selectivity (bottom group). It was also notable that all of these groups showed averaged saccade-task responses that suggested strong persistent activity (Fig. 11B). Furthermore, some of the groups with nonclassic motifs still exhibited significant coherence dependence (right column, e.g., Group 1), whereas others with more “expected” motifs showed weaker (nonsignificant) coherence dependence (e.g., Groups 5 and 6). As shown in the 6-cell toy problem above, the average of all 80 cells produces the more conventional ramping responses that mimic the accumulation of evidence, despite the heterogeneity of the individual elements (Fig. 10C).
Eighty cell responses are segregated into 6 diverse groups based on their response dynamic during dot motion. Each row displays the responses of one group. A, The response of each neuron in a group is plotted during decision formation (thin lines), along with the group average response (thicker lines). Firing rate is normalized and plotted as a running mean (100 ms bins) for In-RF and Out-RF choices. These response vectors were used originally to assign the cells to groups with an automated k-means algorithm (see Data analysis). B, The average response of each cell group exhibits persistent activity in the memory-guided saccade task. Average response is plotted during the instructed saccade tasks for each cell group. C, The directional coherence dependence of each cell group is shown by plotting the average response difference between In-RF and Out-RF choices (during first 700 ms of dot motion) as a function of coherence. Slopes are reported in units of normalized firing rate per 100% coherence ± SEM.
Although the results of this clustering reveal a diversity of temporal dynamics that in turn imply a wide range of neural computations beyond “accumulation to threshold,” even this analysis obscures more compelling deviations from our expectations. Specifically, we encountered several (n = 15) cells that exhibited preference for one choice target during the decision epoch, and opposite preference after the dot motion and leading up to the saccade (Fig. 12B shows two examples). Interestingly, both example cells exhibited reasonable persistent activity during instructed saccades (Fig. 12A) and coherence dependence (Fig. 12C) and thus would have qualified for inclusion in standard population-level analyses. But their idiosyncrasy is even more striking when one appreciates that one cell's saccadic direction preference in the instructed saccade task matched the cell's perisaccadic preference during the decision task (top cell), whereas the other cell's preference in the instructed saccade task did not switch during motion viewing (bottom cell). Rather, this latter cell showed “flipping” behavior after motion viewing, and only during the Targets-ON trials. It did not flip at all during the Targets-FLASH trials. Notably, this cell came from the most canonical-looking cell group (Fig. 11, fourth group down).
Dramatic heterogeneity of LIP can be observed outside of the conventionally focused upon epochs of the memory delay period in the instructed saccade task and the decision formation period of the decision task. The individual responses of two example neurons illustrate that, although a neuron may exhibit persistent activity and even directional coherence dependence during decision formation, neural responses are categorically distinct from “conventional” cells because these neurons switch spatial selectivity after dot motion viewing. Each row shows the response of one cell. A, PSTH of each cell during the memory-guided saccade task (dashed line) shows that both cells exhibited conventional persistent activity. Solid lines indicate activity during visually guided saccades. B, PSTH of each cell during the dot motion decision task shows that both cells exhibit choice-predictive activity during decision formation. However, both cells switch their spatial selectivity after the dot motion epoch in Targets-ON trials (the bottom cell does not switch selectivity during Targets-FLASH trials). Solid and dashed lines indicate, respectively, Targets-ON and Targets-FLASH trials. C, Each cell appears to have some directional coherence dependence, as illustrated by plotting the average response difference between In-RF and Out-RF choices (during first 700 ms of dot motion) as a function of coherence. Slopes are reported in units of spikes/s per 100% coherence ±SEM.
Together, the examination of single-neuron responses in LIP reveals that an even wider range of LIP neurons appears to carry decision-related signals than was previously assumed; however, the temporal form of their responses, which is often a departure from simple ramping, implies that many LIP neurons do not explicitly reflect the accumulation of evidence because they might be awash in other signals. These “other” signals are either from the unknown sources that underlie unique response motifs or from the RF visual stimulation we directly manipulated. Although averaged responses do better mimic the accumulation of evidence, the other signals are still obvious, as conveyed by the large sensory signals from target visibility seen in the population response.
Discussion
Our results show that decision signals in LIP are multiplexed with decision-irrelevant signals of substantial magnitude and of various forms. At the single-neuron level, explicit neural correlates of decision formation are not particularly common and are not strongly related to the degree of persistent activity exhibited by those neurons. Instead, single neurons exhibit diverse but categorizable temporal motifs that are difficult to interpret in psychological (decision-making) terms. However, at the population level, the aggregate response of these varied neural dynamics yields a signal that looks more like a neural correlate of the accumulation of evidence. But the neural characteristics of multiplexing and heterogeneity suggest that an explicit neural correlate of decision formation may not exist in the brain, or, if one is extracted by reading out LIP signals, this would require significant computations (which themselves would deserve considerable future study) (Mirpour and Bisley, 2012). Alternatively, decisions could be supported by coarser downstream mechanisms that do not fully or precisely demultiplex LIP signals but approximate a decision variable in certain contexts. More generally, these results invite reconsideration of the central role in decision-making that has been ascribed to this area: On one hand, multiplexing raises the possibility that LIP could be a less critical stage; on the other hand, the neural heterogeneity we observed suggests that LIP may actually support a broader range of sensorimotor transformations involved in decision-making.
In some ways, these results point toward a step backwards in our understanding of LIP; there is not a fixed one-to-one correspondence between absolute LIP spike rate and accumulated evidence, and the responses of many individual neurons are qualitatively difficult to reconcile as direct correlates of any posited decision-making mental process. Our results may also seem to dilute the attractive correlations between LIP activity and perceptual decisions. However, we prefer to highlight that our observations serve as a critical window into the nuanced computations performed by these neurons and motivate another wave of hypothesizing and experimentation to unpack these signals further.
These results should also help to integrate studies using the motion direction–discrimination task with a wide array of other work in LIP. Several lines of work have investigated signals in LIP that are distinct from pure oculomotor decisions, such as shape selectivity (Sereno and Maunsell, 1998; Sereno and Amador, 2006; Janssen et al., 2008), category specificity (Bennur and Gold, 2011; Freedman and Assad, 2011), salience (Colby and Goldberg, 1999; Ipata et al., 2009; Gottlieb and Snyder, 2010), color (Bennur and Gold, 2011), direction selectivity (Fanini and Assad, 2009), and passive visual responses (Ben Hamed and Duhamel, 2002). Our results suggest that significant findings from such work do not reflect distinct “modes” of LIP function specific to particular experimental contexts but instead reveal signals that can be mixed with decision- or saccade-related activity. LIP responses have been interpreted as reflecting the sum of visual, cognitive, and motor components, modulated by a wide-field response normalization (Dorris and Glimcher, 2004; Sugrue et al., 2004; Ipata et al., 2009; Falkner et al., 2010; Louie et al., 2011). Our results are not only consistent with this viewpoint but make it clear that decision-related (“cognitive”) and decision-irrelevant signals can both sum together over time and also interact in more complex ways that preclude simple mappings back to decision variables.
Even within the context of the motion direction–discrimination task, some previously reported findings might fruitfully be considered in light of the potential for mixtures of decision-related and decision-irrelevant signals. For example, one recent study (Churchland et al., 2008) compared LIP activity when 2 versus 4 alternatives were discriminated (with corresponding numbers of choice targets). The lower responses during 4 alternative trials than 2 alternative trials can be interpreted as a direct reflection of a lower starting point for evidence accumulation. Our results suggest a complementary mechanism for lowering responses, driven simply by the larger number of visual targets on the screen (i.e., full-field divisive normalization). Other experiments using a third (postdecision wagering) target (Kiani and Shadlen, 2009) also invite consideration of wide-reaching interactions between visual stimuli. Of course, these cutting-edge studies involve richer characterizations that link these neural effects to psychophysical effects, but the potential for changes in the visual displays to affect LIP responses (and perhaps decisions themselves) should be considered in future work.
Although heterogeneity is no surprise when recording from cortical neurons in association cortex, the dynamics of individual neurons provide important clues into the actual neural computations performed in LIP (Premereur et al., 2011). The disparate response motifs observed suggest that an array of distinct signal types are received and/or computed within LIP, that these signals have temporal properties very distinct from early visual areas, and that these single neuron dynamics are unlikely to be direct and explicit neural correlates of psychological processes. Although it is tempting to contemplate that these response types map directly onto distinct cell types in LIP, we can only propose that this is a matter worthy of future investigation. Perhaps this idiosyncrasy and diversity provide LIP with computational flexibility, providing a bank of transformations of sensory and motor signals that can be used as appropriate to the task (Bernacchia et al., 2011). Coupled with our conclusions regarding the multiplexing of visual and decision signals, LIP looks less like a decision formation mechanism that performs the key step of sensory “read out,” and more like an aggregator of sensorimotor signals with potential behavioral relevance that itself must be selectively read out in a context-appropriate manner.
These results also shed light (or perhaps, cast shadow) upon the relation between persistent activity and decision-related signals. Although several decades of research have assumed that persistent activity during a memory-guided saccade task implied that a cell was capable of performing temporal integration (Shadlen and Newsome, 1996), our results call this assumption into question and more generally suggest that temporal integration is likely not the result of a fixed intrinsic property of each cell that generalizes across behavioral contexts. Rather, the responsibility for maintaining a signal across time may be distributed across a heterogeneous network (Miri et al., 2011), and task-relevant networks might be composed of cells with specific local or long range outputs.
Together, these results motivate a shift of emphasis to how LIP signals could be read out by later mechanisms that demultiplex the interacting visual and decision-related signals (Mirpour and Bisley, 2012), as well as how neurons with different temporal motifs contribute to a single aggregate signal or can be more selectively accessed to support a wide range of sensorimotor functions (Machens, 2010). The relation between spikes in LIP and perceptual decision-making may thus be considerably more enigmatic than previously postulated (Gold and Shadlen, 2002).
Footnotes
This work was supported by National Institutes of Health Grant R01-EY017366 to A.C.H. We thank Kyler Eastman for help with physiological data collection, Jacob Yates and Benjamin Naecker for supplemental behavioral data collection, and Jonathan Pillow and Lawrence Cormack for helpful comments on the manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Dr. Miriam L. R. Meister at her present address: Yerkes National Primate Research Center, 954 Gatewood Road NE, Atlanta, GA 30329. miriam.meister{at}emory.edu
