Switching behavior based on multiple rules is a fundamental ability of flexible behavior. Although interactions among the frontal, parietal, and sensory cortices are necessary for such flexibility, little is known about the neural computations concerning context-dependent information readouts. Here, we provide evidence that neurons in the lateral intraparietal area (LIP) accumulate relevant information preferentially depending on context. We trained monkeys to switch between direction and depth discrimination tasks and analyzed the buildup activity in the LIP depending on task context. In accordance with behavior, the rate of buildup to identical visual stimuli differed between tasks and buildup was prominent only for the stimulus dimension relevant to the task. These results indicate that LIP neurons accumulate relevant information depending on context to decide flexibly where to move the eye, suggesting that flexibility is, at least partly, implemented in the form of temporal integration gain control.
SIGNIFICANCE STATEMENT Flexible behavior depending on context is a hallmark of human cognition. During flexible behavior, the frontal and parietal cortices have complex representations that hinder efforts to conceptualize their underlying computations. We now provide evidence that neurons in the lateral intraparietal area accumulate relevant information preferentially depending on context. We suggest that behavioral flexibility is implemented in the form of temporal integration gain control in the parietal cortex.
A hallmark of human cognition is the flexibility to select an appropriate action in response to identical sensory events depending on the environment or context. For decades, mounting evidence has suggested that the prefrontal cortex (PFC) is involved in cognitive control enabling flexible task switching (Miller and Cohen, 2001; Sakai, 2008; Stoet and Snyder, 2009). For example, neurons in the PFC encode abstract task rules (White and Wise, 1999; Asaad et al., 2000; Wallis et al., 2001; Mansouri et al., 2006; Buschman et al., 2012) and PFC lesions abolish the ability to switch as the rule dictates (Buckley et al., 2009).
Recent physiological studies have begun to reveal how prefrontal and parietal cortical neurons alter dynamically their representations to guide context-based decision making. Prefrontal and parietal neurons typically have mixed representations and encode all aspects of a task, including sensory, motor, and context-related components (Mante et al., 2013; Siegel et al., 2015). Although these representations evolve into a single decision over time, the mechanism of this computation over time remains controversial.
We investigated whether and how a neural correlate of perceptual decision formation changes depending on task context. Neurons in the lateral intraparietal area (LIP) are involved in computing a decision variable by temporally accumulating sensory evidence (Shadlen and Newsome, 2001; Roitman and Shadlen, 2002; Huk and Shadlen, 2005; Gold and Shadlen, 2007). Therefore, we investigated whether and how the process of evidence accumulation changes depending on context by studying LIP neuron responses during reaction time task switching. Two monkeys were trained to switch from trial to trial between direction and depth discrimination tasks. Both tasks have been studied previously and the underlying sensory representations are well established (Britten et al., 1992, 1996; Salzman et al., 1992; DeAngelis et al., 1998; Uka and DeAngelis, 2003, 2004). After stimulus onset, the firing rates of LIP neurons gradually increased depending on both motion strength and depth strength. The rate of buildup differed between the two tasks, suggesting that LIP neurons accumulate relevant information depending on context to decide flexibly where to move the eye. Therefore, the LIP presumably computes a more explicit decision variable compared with mixed representations of the PFC.
Materials and Methods
Subjects and surgery.
Two male Japanese macaques (Macaca fuscata) weighing 7 kg (monkey M) and 6 kg (monkey K) were used. All animal care, training, and experimental procedures were in accordance with the National Institutes of Health guidelines and were approved by the Juntendo University Animal Care and Use Committee. Animals were prepared for experiments using standard aseptic procedures. A post for the head restraint and a recording chamber were chronically implanted in each monkey. To monitor eye movement, scleral search coils were implanted into both eyes of monkey K and into one eye of monkey M (Judge et al., 1980). A cylindrical recording chamber was mounted over the occipital cortex ∼12 mm lateral and 21 mm anterior to the occipital ridge at an angle of 25° above the horizontal plane (Rao et al., 2012). For each monkey, structural magnetic resonance imaging was used to guide the placement of electrodes within the chamber to access the LIP.
The monkeys were seated with head restraint in a primate chair. A visual stimulus was presented on a 22-inch cathode ray tube monitor (HM204DA; Iiyama) positioned 57 cm in front of the monkeys' eyes. The monitor subtended a visual angle of 40° × 30°. Stimuli were presented dichoptically using a pair of ferroelectric liquid crystal shutters (DisplayTech) mounted just in front of the monkeys' eyes. Stereo half-images for the left and right eyes were presented alternately at a frame rate of 100 Hz (i.e., a 50 Hz refresh for each eye) and were synchronized with the vertical refresh of the video input. The monkeys viewed random-dot stimuli while maintaining fixation on a small dot (0.15°) on the screen.
The random-dot stimuli were presented using an OpenGL accelerator board with quad-buffer stereo support (Quadro FX 1400; NVIDIA). Each random-dot stimulus was presented within a circular aperture. The dot density was 64 dots per square degree per second, with each dot subtending ∼0.1°. The starting position of each dot was newly randomized for each trial. The stimulus consisted of red dots to minimize ghosting effects (stereo cross-talk was <3%). Precise binocular disparities and smooth motions were achieved by plotting dots with subpixel resolutions using anti-aliasing provided by the OpenGL board.
Behavioral tasks and data acquisition were controlled with a commercial software package (TEMPO; Reflective Computing) and all data analyses, including the online data visualization, were performed using MATLAB (The MathWorks).
Memory-guided saccade task.
The memory-guided saccade task was used to locate the response field (RF) of individual LIP neurons. While the monkey fixated a central fixation point, a saccadic target was presented for 300 ms at one of eight equidistant (12° eccentricity) peripheral locations spaced evenly around the fixation point. After a 1 s delay, the fixation point disappeared, at which time the monkey was required to make a saccade to the remembered location of the target. Correct responses were rewarded with a drop of water or juice.
Reaction time task switching.
The monkeys were initially trained to perform the direction and depth discrimination tasks separately. After the monkeys were trained in each discrimination task, they were trained to switch between tasks. In each trial, the color of the fixation point (magenta or cyan) indicated which task the monkeys should perform. Magenta indicated the direction discrimination task and cyan indicated the depth discrimination task. The two tasks were randomly interleaved from trial to trial so that the monkeys were unable to predict the context of the upcoming trial. After the monkey fixated for 300 ms, two peripheral choice targets appeared. The two targets were 180° apart and equally eccentric (12°). The stimulus appeared within a circular aperture (8° in diameter) 8° from the fixation point (left for monkey M and right for monkey K). The temporal delay between target onset and stimulus onset was drawn from a truncated exponential distribution (0.3–2.8 s) to minimize anticipation of the stimulus onset (Roitman and Shadlen, 2002). In the direction discrimination task, the monkeys were required to make a saccade to the lower target when the dots moved down and to the upper target when the dots moved up. In the depth discrimination task, the monkeys were required to make a saccade to the lower target when the dots were nearer than the plane of fixation and to the upper target when the dots were farther. Once the stimulus was presented, the monkeys were free to indicate their choice at any time. When their gaze left the fixation window, the stimulus was extinguished and this time was designated as the saccade onset. Correct responses were rewarded with a drop of water or juice. To discourage fast guesses, the reward was withheld for 1 s after the onset of the stimulus (Roitman and Shadlen, 2002). Task difficulty was titrated by manipulating the percentage of coherently moving dots (motion coherence) or binocularly correlated dots (binocular correlation) in the stimulus. Motion coherence and binocular correlation varied independently from trial to trial.
A tungsten microelectrode (FHC) with impedance values between 0.5 and 2.0 MΩ (at 1 kHz) was used to record the extracellular activity of single neurons. The electrode was advanced through the cortex through a transdural guide tube using a pulse motor micromanipulator (MO-951; Narishige) mounted on the recording chamber. Raw signals from the electrode were amplified and band-pass filtered (200–10,000 Hz) with conventional electronic equipment (Bak Electronics). Single neurons were isolated using a voltage–time window discriminator (Bak Electronics). The action potential times and trial event occurrences were stored to a disk with a 1 ms resolution. Eye position was monitored using a magnetic search coil system (Sankei Kizai) and stored on a disk at a rate of 200 Hz.
The LIP was identified functionally based on the physiological response properties during the execution of the memory-guided saccade task (Roitman and Shadlen, 2002).
After isolating a single LIP neuron, its activity was recorded while the monkey performed the memory-guided saccade task. During the task, a peristimulus time histogram and target-direction tuning curve for the sustained activity during the delay period were constructed online. These displays were inspected and the RF of the LIP neuron was defined as the target location at which the response was the strongest at any time during the trial. Because of the configuration of the task, neurons with RFs near the horizontal meridian (<30° from horizontal) were not examined further.
Neuronal activity was then recorded during the reaction time task switching. One target (Tin) was located in the RF of the neuron and the other target (Tout) was diametrically opposite. The coherently moving dots in the stimulus moved toward one of the choice targets at a speed of 8°/s. The binocular disparity of the stimulus was either −0.5° or 0.5°. Motion coherence varied among 6%, 12%, 24%, and 48%, and binocular correlation varied among 12%, 24%, 48%, and 96%. Therefore, a total of two motion directions, two binocular disparities, four motion coherences, and four binocular correlations were used for each of the two tasks, yielding 128 stimulus conditions. The task, motion direction, binocular disparity, motion coherence, and binocular correlation were pseudorandomly interleaved within a block. Whenever possible, data were collected for 20 repetitions of each unique stimulus condition and were discarded if the neuron was lost before three repetitions. Across the range of acceptable datasets, the average number of repetitions was 11 (SD = 5), and the average number of total trials was 1403 (SD = 584).
Behavioral data analysis.
For each task, behavioral performance was quantified with a psychometric function describing the relationship of signal strength (motion coherence or binocular correlation) with choice and a chronometric function describing the relationship of signal strength with reaction time. Reaction time was measured as the time from stimulus onset to saccade onset. Psychometric function was fitted using three models. All fits were performed using maximum-likelihood methods.
First, logistic regression was used, in which the probability of a preferred choice (ppref) was calculated as follows: where Coh is motion coherence and Corr is binocular correlation. Positive values of Coh and Corr indicate coherence/correlation with motion or depth corresponding to the Tin choice, whereas negative values indicate coherence/correlation with motion or depth corresponding to the Tout choice. According to the drift diffusion model of decision making (Palmer et al., 2005), the psychometric function should have a form of a logistic function of stimulus strength. Our logistic regression model incorporates coefficients for both relevant and irrelevant stimulus strength. By incorporating β for the irrelevant stimulus, the model can control sensitivity for irrelevant stimulus strength. We thus call this model the “gain control” model.
Second, we assumed that the monkey perfectly ignored irrelevant stimulus dimension during performance of each discrimination task. We further assumed that the monkey performed the wrong task occasionally, with a probability of perr (Mitani et al., 2013). We call this the “task error” model. The fitted function was as follows: We fitted psychometric data for both direction and depth discrimination tasks simultaneously with these equations.
Third, we combined the above two models. We call this model the “full” model. The fitted function was as follows:
We fitted psychometric data for both direction and depth discrimination tasks simultaneously with these equations. We used the Akaike Information Criterion (AIC; Akaike, 1974) to determine which model best described the psychometric functions. The AIC was calculated as AIC = −2lnL + 2k, where L is the maximized likelihood and k is the number of parameters.
To quantify how well the monkeys switched between direction and depth discrimination, we calculated the switch ratio (SR) of each task (Sasaki and Uka, 2009). SRs were determined from the sensitivities for direction and depth during performance of the relevant and irrelevant tasks using the gain control model as follows: where βdirection represents sensitivity in the direction discrimination task and βdepth represents sensitivity in the depth discrimination task.
Neuronal data analysis.
We assessed LIP neuron response temporally by calculating the firing rates within a 50 ms window in 5 ms steps. Responses were truncated to the time of saccade onset for each trial before averaging across trials. The rate of buildup was estimated using linear regression of firing rates from 200 ms after stimulus onset until 100 ms before the median saccade time.
To determine whether responses just before saccade depended on stimulus condition, we first calculated the firing rate during a 50–100 ms window before saccade onset for each stimulus condition using trials leading to correct Tin choices. We then used a Kruskal–Wallis test to determine whether there was a significant difference among the measured responses across all stimulus conditions.
For each neuron, we estimated the sensitivity to motion coherence and binocular correlation from the 2D maps of the buildup rates. We fit a plane using multiple linear regression for these 2D maps. The equations of the fitted planes for each task were calculated as follows: where α0–2 are fitted parameters. We used a slope parameter of α1 as the sensitivity for motion coherence and α2 for binocular correlation.
Two monkeys, M and K, were trained to switch between responding to motion direction and stereoscopic depth of a random-dot stimulus in successive trials (Fig. 1). In the direction discrimination task, the monkeys reported on motion direction. Upward motion was indicated with a saccade to the upper target and downward motion with a saccade to the lower target. In the depth discrimination task, the monkeys reported on binocular depth. Far disparity was indicated with a saccade to the upper target and near disparity with a saccade to the lower target. In each trial, the color of the fixation point indicated which task the monkeys should perform. We measured the accuracy and speed of the monkeys' decisions using a reaction time paradigm while simultaneously recording single LIP neurons (50 from monkey M and 49 from monkey K). This enabled us to examine LIP neuron activity during decision formation.
First, we analyzed how relevant and irrelevant stimulus features affected the accuracy and speed of the monkeys' choices. Figure 2 shows psychometric and chronometric data for both monkeys. Positive coherences and correlations indicated choices toward the target located in the RF of the neuron (Tin), whereas negative coherences and correlations indicated choices toward the target located diametrically opposite to the RF (Tout). For both the direction and depth discrimination tasks, accuracy was nearly perfect for strong relevant stimulus features and weak irrelevant stimulus features. During the direction discrimination task, the monkeys correctly discriminated motion direction for weak binocular correlations (Fig. 2, left top panel for monkeys M and K). However, the psychometric functions shifted horizontally depending on the strength of the depth signal, which was irrelevant to the direction discrimination task. The same was true for the depth discrimination task (Fig. 2, right top panel for monkeys M and K). The monkeys correctly discriminated depth for weak motion coherences, but the psychometric functions shifted horizontally depending on the strength of the motion signal. In addition, at strong irrelevant stimulus conditions, the monkey's performance dropped down to ∼70% correct even with the strongest relevant stimulus. These results indicated that, although the choices were mainly dependent on the relevant stimulus feature, they were affected by the irrelevant stimulus feature.
We analyzed psychometric data for each monkey with three models (Fig. 2A,B,E,F). The first, the gain control model, assumed that the monkey was able to control the relative sensitivity for the irrelevant stimulus dimension. The second, the task error model, assumed that the monkey was able to ignore the irrelevant stimulus dimension perfectly, but performed the wrong task with a given probability. The third, the full model, assumed that the monkey controlled the sensitivity for the irrelevant stimulus dimension and also performed the wrong task with a given probability. We found that the full model best described the psychometric data. For monkey M, AIC for the gain control model was 50,132, the task error model 48,966, and the full model 48,023. For monkey K, AIC for the gain control model was 53,340, the task error model 52,024, and the full model 51,117. These results suggest that the monkey was able to control the sensitivity for the irrelevant stimulus dimension, although not perfectly, and also performed the wrong task on some trials.
We next quantified how well the monkeys switched between the two tasks based on the SR. If the monkeys switched perfectly, then the SR would be 1, whereas if the monkeys were ignoring the trial type cue and solving the task the same way on each trial, then the SR would be 0. For comparison with our previous study (Sasaki and Uka, 2009), the SRs were calculated using the sensitivities from the gain control model. For monkey M, the SR for the direction discrimination task was 0.79 ± 0.14 (mean ± SD) and the SR for the depth discrimination task was 0.72 ± 0.14 (n = 50). For monkey K, the SR for the direction discrimination task was 0.67 ± 0.13 and the SR for the depth discrimination task was 0.90 ± 0.07 (n = 49). The SRs were in agreement with our previous study (Sasaki and Uka, 2009) and indicated that the monkeys switched correctly between tasks, although not perfectly. The SR values calculated from the full model were also consistent with these results. The SR values were 0.96 ± 0.23 for the direction discrimination and 0.86 ± 0.08 for the depth discrimination for monkey M and 0.72 ± 0.15 for the direction discrimination and 0.94 ± 0.06 for the depth discrimination for monkey K. The perr values from the full model were 0.11 ± 0.12 for monkey M and 0.06 ± 0.03 for monkey K. The SR values also show task asymmetries between monkeys. Specifically, monkey M performed better on the direction discrimination task than the depth discrimination task, whereas the opposite was true for monkey K. The behavioral sensitivities for the two tasks were set to be nearly comparable and therefore we did not expect more interference in one of the tasks compared with the other. Such asymmetries between tasks have been described previously (Buschman et al., 2012), although it is not known why these asymmetries occur.
Finally, we measured the speed of the monkey's decision. For both tasks, irrelevant stimulus features affected the monkey's decision in two ways. First, similar to the psychometric functions, the chronometric functions shifted horizontally depending on the degree of irrelevant stimulus features. Second, particularly for direction discrimination, the monkeys tended to decide quickly for strong irrelevant binocular correlations (Fig. 2, left bottom panel for monkeys M and K). Overall, the behavioral data revealed small interferences from stimulus features irrelevant to the task.
Task-relevant sensory evidence is accumulated preferentially in the LIP
We analyzed 99 neurons (50 from monkey M and 49 from monkey K) that exhibited spatially selective activity during a memory-guided saccade task. One target (Tin) was located in the RF of the neuron and the other target (Tout) was located diametrically opposite to Tin. We recorded all neurons that exhibited spatial selectivity at any time during the memory-guided saccade task.
Figure 3 shows the responses of a single LIP neuron after the onset of the random-dot stimulus. For clarity, only response traces for weak irrelevant stimulus features are shown separately for each task. The firing rates of the LIP neuron decreased ∼200 ms after the stimulus onset, after which the response gradually increased depending on the stimulus strength of the relevant feature for both tasks (Fig. 3A,D, left). Stronger stimuli resulted in steeper increases in firing rates for both tasks (p = 9.3 × 10−12 for direction discrimination, p = 9.4 × 10−15 for depth discrimination, F test across all irrelevant stimulus conditions). Responses associated with the Tin choice reached a common level just before the saccade regardless of stimulus strength (Fig. 3A,D, right). For the direction discrimination task (Fig. 3A, right), there was no difference in the responses during a 50–100 ms window before saccade among all the stimulus conditions (p = 0.44, Kruskal–Wallis test). The same was true for the depth discrimination task (p = 0.48, Kruskal–Wallis test; Fig. 3D, right).
The common presaccadic response was also observed when the responses were grouped by reaction time (Fig. 3C,F). Short reaction time trials were associated with steep buildups, whereas long reaction time trials were associated with shallow buildups. To quantify this relationship, trials were grouped by reaction times from 450 to 1000 ms in steps of 50 ms (±25 ms bin width) and average firing rates for the groups were calculated. We found a statistically significant negative correlation between reaction times and buildup rates across the groups for the direction discrimination task (Spearman's rank correlation, rs = −0.85, p = 9.7 × 10−4) and for the depth discrimination task (rs = −0.63, p = 0.03). Therefore, all trials reached a common level just before the saccade. These neural response patterns were consistent with previous studies of perceptual decision making using the direction discrimination task (Roitman and Shadlen, 2002) and are consistent with the drift diffusion model.
Visual inspection of the responses during the irrelevant task to the same visual stimulus also showed a buildup of activity, but the modulation was not as strong as that during the relevant task (Fig. 3B,E). Importantly, responses associated with the Tin choice reached a common level just before the saccade, even when sorted by strength of the irrelevant stimulus. Therefore, this sample LIP neuron accumulated task-relevant information preferentially regardless of the stimulus feature.
These characteristics were representative of the whole LIP neuron population (Fig. 4). For the majority of neurons (70 of 99 for direction discrimination and 72 of 99 for depth discrimination), stronger relevant stimuli resulted in larger buildup rates (p < 0.05, F test). Mean responses 50–100 ms before saccades across the population were independent of the stimulus strength: there was no difference in the presaccadic responses among all stimulus conditions for both the direction discrimination task (p = 0.996, Kruskal–Wallis test; Fig. 4A, right) and the depth discrimination task (p = 0.45, Kruskal–Wallis test; Fig. 4D, right). There was a significant negative correlation between reaction times and buildup rates for both direction discrimination (rs = −0.87, p = 3.1 × 10−4) and depth discrimination (rs = −0.69, p = 0.02).
Figures 3 and 4 only show response traces for weak irrelevant stimulus features and Figure 5 shows the results of all motion coherence and binocular correlation combinations for each monkey separately. To examine the effect of context on LIP response across the population, we quantified the rate of buildup activity by fitting a line to the response from 200 ms after stimulus onset until 100 ms before the median saccade time for each stimulus condition for each neuron and calculated the average across all neurons. The rate of buildup for motion coherence was higher for large, positive motion coherences across all binocular correlations during the direction discrimination task (p ≪ 0.0001, F test; Fig. 5A,G, relevant task), indicating that the buildup slope was steeper for large positive motion coherence regardless of binocular correlation. We quantified the relationship between the buildup rates and motion coherence (Fig. 5A,G) as sensitivity to motion coherence and the buildup rates and binocular correlation (Fig. 5C,I) as sensitivity to binocular correlation. The sensitivity to motion coherence and to binocular correlation during the direction discrimination task was computed from a plane fitted to a 2D map of the buildup rates during the direction discrimination task as a function of signed motion coherence and signed binocular correlation (Fig. 5E,K). The sensitivity for motion coherence during the direction discrimination task was 0.98 spikes/(s2 · coh) (95% confidence interval (CI) [0.77 1.2]) for monkey M and 0.46 spikes/(s2 · coh) (95% CI [0.41 0.51]) for monkey K. During the depth discrimination task, the buildup rate was less reliant on motion coherence (Fig. 5B,H, irrelevant task). The sensitivity to motion coherence during the depth discrimination task was 0.05 spikes/(s2 · coh) (95% CI [−0.19 0.29]) for monkey M and 0.05 spikes/(s2 · coh) (95% CI [0.00 0.09]) for monkey K, indicating that the buildup slope did not change substantially depending on motion coherence. The same was true for the binocular correlation buildup rate (Fig. 5C,D,I,J). The sensitivity to binocular correlation during the depth discrimination task (0.68 spikes/(s2 · corr), 95% CI [0.56 0.80] for monkey M and 0.32 spikes/(s2 · corr), 95% CI [0.29 0.34] for monkey K) was significantly higher than the sensitivity to binocular correlation during the direction discrimination task (0.36 spikes/(s2 · corr), 95% CI [0.26 0.47] for monkey M and 0.01 spikes/(s2 · corr), 95% CI [−0.01 0.04] for monkey K).
If LIP neurons only accumulate information for the relevant stimulus feature, then contours of the 2D map of buildup rates should be horizontal for the direction discrimination task and vertical for the depth discrimination task within the 2D map; however, this was not completely true. For both the direction and depth discrimination tasks, the buildup rate contours were obliquely elongated, although slightly, indicating that the rate of buildup scaled with both motion coherence and binocular correlation. These results suggest that, although interference from the irrelevant feature was observed, buildup mainly depended on the strength of the relevant stimulus feature.
Individual LIP neurons accumulate both direction and depth information
In the previous section, we showed that the population of LIP neurons preferentially accumulated task-relevant sensory evidence. Next, we tested whether this was true for individual neurons or if each individual neuron accumulated sensory evidence for only one of the two tasks when it was relevant. To quantify neural sensitivity to motion coherence and binocular correlation for each task in individual neurons, we fitted a plane to the 2D map of the buildup rates for each neuron. The sensitivity to motion coherence (or binocular correlation) denoted the change in buildup rates caused by an increase in motion coherence (or binocular correlation). Figure 6A shows the sensitivities of each neuron to both stimulus dimensions when they were relevant to the task; that is, sensitivities to motion coherence were calculated from the direction discrimination trials and sensitivities to binocular correlation were calculated from the depth discrimination trials. Most individual neurons (44/50 for monkey M and 41/49 for monkey K) were sensitive to both stimulus dimensions when they were relevant to the task (p < 0.05, F test; Fig. 6A), with the majority showing an increased buildup rate for positive coherence or correlation. Figure 6A reveals no population of LIP neurons possessing significant sensitivity to either stimulus dimension, which would have exhibited a large sensitivity for one stimulus and no sensitivity for the other. Instead, individual LIP neurons seemed to accumulate sensory evidence for both the direction and depth discrimination tasks.
Next, we plotted the sensitivities of each neuron to both stimulus dimensions for both the direction (magenta) and depth (cyan) discrimination tasks. Comparison of the sensitivity for the relevant task with that for the irrelevant task showed that sensitivity for the relevant task was stronger than that for the irrelevant task (sensitivity to motion coherence, p = 1.1 × 10−6 for monkey M and p = 8.9 × 10−6 for monkey K, Wilcoxon signed-rank test; sensitivity to binocular correlation, p = 6.2 × 10−4 for monkey M and p = 1.7 × 10−5 for monkey K, Wilcoxon signed-rank test; Fig. 6B), as illustrated by the greater prevalence of cyan dots in the upper left and magenta dots in the lower right portions of the plot. This suggests that evidence accumulation is more prominent for relevant stimulus features than for irrelevant stimulus features, and demonstrates a significant effect of context on buildup activity at the single-neuron level.
For comparison, we calculated behavioral sensitivity for motion coherence and binocular correlation. Sensitivities were taken from the β values of the gain control model fitted to the psychometric function. As expected, behavioral sensitivity for the relevant task was stronger than that for the irrelevant task (sensitivity to motion coherence, p = 7.6 × 10−10 for monkey M and p = 1.1 × 10−9 for monkey K, Wilcoxon signed-rank test; sensitivity to binocular correlation, p = 7.6 × 10−10 for monkey M and p = 1.1 × 10−9 for monkey K, Wilcoxon signed-rank test; Fig. 6C).
Task representation before stimulus viewing
In our task-switching paradigm, information on task context was provided by the color of the fixation point. Therefore, the effect of context on LIP response may have been evident early in the trial time when the fixation point was first turned on. To investigate this, we analyzed the responses during the interval from the onset of the fixation point to the onset of the choice target (Fig. 7A). The responses of individual neurons were averaged across all stimulus conditions for each task. The average firing rate across the population for the direction discrimination task (13.8 spikes/s) did not differ significantly from that for the depth discrimination task (13.5 spikes/s; p = 0.06, Wilcoxon signed-rank test). Individually, only 15 of 99 neurons showed significantly different responses between direction discrimination and depth discrimination (p < 0.05, Mann–Whitney U test; Fig. 7C). Of these, 10 neurons exhibited stronger responses for the direction discrimination task than for the depth discrimination task, while the opposite was true for five neurons. Therefore, task information was only weakly represented at the time of fixation onset.
Furthermore, we analyzed the responses to choice targets by examining the firing rate from target onset to the onset of the random-dot stimulus (Fig. 7B). Across the population, the direction discrimination response (12.3 spikes/s) was slightly but significantly higher than that for depth discrimination (11.8 spikes/s, p = 7.3 × 10−4, Wilcoxon signed-rank test). However, the difference between the two responses (0.5 spikes/s) represented only 4% of the responses for either task. Across the population, >20% of recorded neurons (23/99) exhibited differential responses between direction and depth discrimination (p < 0.05, Mann–Whitney U test), with 16 of these neurons preferring direction discrimination (Fig. 7D). These results suggest minimal to no effect of context on LIP response before viewing the visual stimulus and imply that effects of context may be evident only while viewing the visual stimulus, at least when the RFs of neurons cover the choice alternatives and not the task cue.
Our results demonstrate significant effects of context on a neural correlate of evidence accumulation in the LIP; buildup activities were prominent for the stimulus dimension relevant to the task. Importantly, we did not observe any neurons that integrated motion or depth signals exclusively. Therefore, LIP neurons represent a decision variable concerned with where the eyes move and do not represent intermediate stages such as sensory salience or context per se, which are presumably represented elsewhere in the brain.
These results were in marked contrast to a recent study examining the effects of context on neural response in the PFC (Mante et al., 2013). The response patterns of individual PFC neurons during task switching between direction and color discrimination were complex and the investigators observed no effect of context at the single-neuron level, whereas we found strong context dependency at the single-neuron level in the LIP. Our study differs from that of Mante et al. (2013) in several ways. Most notably, context was interleaved among trials in our task, but not in those used by Mante et al. (2013). This presumably does not explain the large difference in neuronal response between the two studies because monkeys should be able to focus better in blocked trials, resulting in stronger context-dependent modulation. Second, the monkeys were free to respond when comfortable in our task (a reaction time task), but not in the Mante et al. (2013) study. Stimulus-strength-dependent buildup is prominent in both reaction time and fixed duration direction discrimination tasks (Shadlen and Newsome, 2001; Roitman and Shadlen, 2002), making it an unlikely explanation for the discrepancy. Rather, this appears to be a genuine difference between the LIP and PFC. It is interesting to note that the form of evidence accumulation does not differ among the LIP, frontal eye fields, PFC, and superior colliculus when a direction discrimination task is performed in isolation (Kim and Shadlen, 1999; Horwitz and Newsome, 2001; Shadlen and Newsome, 2001). Conceivably, functional specialization among areas emerges only when confronted with context-dependent decision making.
Moreover, our results are in contrast to a recent study examining the effects of context on neural response in the LIP (Siegel et al., 2015). Although they found neurons that encoded choice information, Siegel et al. (2015) also observed neurons that encoded context or task. Similarly, Stoet and Snyder (2004) examined the effects of context on neural response in the parietal cortex. They found that 29% of neurons exhibited context-selective responses during the delay period before stimulus presentation. The incidence of context-selective neurons is similar to our LIP neurons that were selective for context during target presentation (23%, 23/99). However, the difference between the two tasks in response was minimal (∼0.5 spikes/s) in our LIP neurons, whereas Stoet and Snyder (2004) observed larger response differences (∼7 spikes/s). The differences among the Siegel et al. (2015) and Stoet and Snyder (2004) studies and our study can be accounted for by the fact that the task cue was not presented in the RF of the neurons in our study. Therefore, task information can reside in the LIP, but presumably in neurons that have RFs corresponding to the task cue. Overall, various information is distributed within the LIP, although our study suggests that these functions can be compartmentalized within a specific area.
Our behavioral data impose constraints on models of context-dependent decision making. The psychometric data suggest that at least two mechanisms are in play: one relies on gain control of evidence accumulation and the other on task selection. According to the simplest form of the drift diffusion model, the relationship between logistic regression coefficient β and drift rate coefficient k is straightforward. β is equivalent to 2A'k (Palmer et al., 2005), where A' corresponds to the (normalized) bound. Therefore, a context-dependent change in β can be thought of as a context-dependent change in either k or A'. Assuming that the bound does not change depending on context (which we cannot rule out, but is unlikely because of the similar reaction times between tasks), a context-dependent change in β likely corresponds to a context-dependent change in k. The context-dependent change in buildup rate of LIP neurons observed in our study may correspond to a context-dependent change in the drift rate coefficient k. The implementation of task error probability perr assumes that the monkey selected the wrong task on some occasions. We could not identify any LIP neurons that were related to task error. Therefore, task selection is presumably represented elsewhere in the brain.
The horizontal shift of the chronometric data (Fig. 2) suggests that decisions were based on a variable that combined relevant and irrelevant information rather than one that considered both direction and depth and then chose between them. If the latter were true and direction decisions and depth decisions were independent for all irrelevant stimulus strengths, then the reaction time should be longest for the most ambiguous relevant stimulus (i.e., 0% coherence or correlation). This would result in all chronometric data having a peak at the same position regardless of the strength of the irrelevant stimulus, in contrast to the horizontal shift in the chronometric data observed in the behavioral data. However, the amplitude of the chronometric data varied systematically depending on the strength of the irrelevant stimulus. This could not be explained by a simple task error model (data not shown). Because amplitude directly corresponds to the height of the bound in the simplest form of the drift diffusion model (Palmer et al., 2005), a stimulus-strength-dependent change in bound height (or urgency) (Cisek et al., 2009; Hanks et al., 2011) must be implemented to explain our behavioral results.
We showed previously that the firing rate of neurons in the middle temporal visual area does not change depending on context (Sasaki and Uka, 2009). Rather, the brain appears to prepare separate sensory–motor pathways for each context and then select between them at subsequent stages. This study suggests that this selection is implemented, at least in part, in the parietal cortex in the form of a temporal integration gain control. Computationally, it can be implemented as a gated or leaky integrator (Mitani et al., 2013). Biologically, it can be implemented as a reverberating attractor dynamic neural circuit (Ardid and Wang, 2013) or a recurrent network (Mante et al., 2013). According to the model described by Wang (2002) and extended to task switching by Ardid and Wang (2013), there are several ways to control the integrator gain. These include controlling the input gain, background noise, or mutual inhibition between integrators, although we cannot know which underlies gain control from this study alone. Future studies should examine the mechanism of this gain control. Moreover, we hypothesize that controlling the gain of neural integrators is vital for behavioral flexibility on a short time scale and dysfunction of the gain control of neural integrators may underlie cognitive impairments in neurological and neuropsychiatric diseases. For example, blockade of the NMDA receptor, which is known to be related to schizophrenia (Krystal et al., 1994), impairs task-switching performance (Stoet and Snyder, 2006) in a way that can be explained in terms of dysfunctions of the gain control of neural integrators. This study provide us with a useful tool for examining impairments in task switching by studying the dysfunctions of molecules and receptors that control the gain of temporal integration.
This work was partly supported by Japan Society for the Promotion of Science (KAKENHI Grants JP26290008 and JP15H01447). H.K. was supported by a High Technology Research Center Grant from the Ministry of Education, Culture, Sports, Science and Technology. T.U. was supported by a Japan Science and Technology Agency PRESTO grant. We thank Haruyo Kimizuka for technical and surgical assistance.
The authors declare no competing financial interests.
- Correspondence should be addressed to Takanori Uka, Department of Integrative Physiology, Graduate School of Medicine, University of Yamanashi, 1110 Shimokato, Chuo, Yamanashi 409-3898, Japan.