Abstract
Recent studies have shown that cognitive factors such as spatial and feature-based attention, learning, and task-switching can change the extent to which the trial-to-trial variability in the responses of neurons in sensory cortex is shared between pairs of neurons (for review, see Cohen and Kohn, 2011). Global cognitive factors related to concentration, motivation, effort, arousal, or alertness also affect performance on perceptual tasks and the responses of individual neurons in many cortical areas (Spitzer et al., 1988; Spitzer and Richmond, 1991; Motter, 1993; Bichot et al., 2001; Hasegawa et al., 2004; Boudreau et al., 2006; Niwa et al., 2012). The question of how global cognitive factors affect correlated response variability is important because these factors likely vary both across and within all psychophysical and physiological studies. Furthermore, global cognitive factors might provide a convenient platform for studying the neuronal mechanisms underlying how cognitive factors affect correlated variability because they can be manipulated easily without training complex perceptual tasks. We recorded simultaneously from groups of neurons in visual area V4 while rhesus monkeys performed a contrast discrimination task whose difficulty changed in blocks of trials. We found that correlated variability decreased when the task was more difficult, even when the visual stimuli were far outside the receptive fields of the recorded neurons. Our results suggest that studying global cognitive factors might provide a general framework for studying how cognitive factors affect the responses of neurons throughout sensory cortex.
Introduction
A large body of work has examined the relationship between psychophysical performance and neuronal responses, particularly in primate visual cortex. Despite experimenters' best efforts to control their subjects' behavior, uncontrolled fluctuations in cognitive factors have been hypothesized to affect many aspects of the responses of visual cortical neurons, including the firing rates and variability in the responses of individual neurons, the extent to which variability is shared among groups of neurons, and to explain substantial differences between measurements of shared variability in different studies (Ecker et al., 2010; Cohen and Kohn, 2011). Cognitive factors such as motivation, effort, arousal, alertness, or concentration can be considered global because their effects are not specific to neurons that encode particular stimulus features or locations. These factors are known to affect the firing rates of individual neurons in sensory (Spitzer et al., 1988; Spitzer and Richmond, 1991; Motter, 1993; Boudreau et al., 2006; Niwa et al., 2012) and prefrontal (Bichot et al., 2001; Hasegawa et al., 2004) cortex. However, the way global cognitive factors affect both private and shared variability in the responses of cortical neurons remains unknown.
Understanding how global cognitive factors affect response variability is important for two reasons. First, uncontrolled fluctuations in these factors likely affect the psychophysical and physiological results of any study, so characterizing these effects will affect the interpretation of all studies. Second, global factors are potentially easier to manipulate than other cognitive factors such as spatial and feature-based attention, which are known to affect the extent to which response variability is shared among a group of neurons. If global factors have similar effects on response variability (and potentially share underlying neuronal mechanisms), studying whichever cognitive factor is most experimentally accessible would yield general insights. Studying the mechanisms underlying global cognitive factors will be tractable in species such as rodents or marmosets for which selective attention tasks might be prohibitively difficult to train. This approach would be an attractive way to use genetic tools to address questions about how cognitive factors affect sensory cortex and how shared response variability is modulated.
Our goal was to measure how global cognitive factors affect the response variability of individual neurons and covariability in the responses of groups of neurons. We simultaneously recorded from groups of neurons in both hemispheres of visual area V4 while two rhesus monkeys performed a contrast discrimination task. We manipulated global cognitive factors by changing the average difficulty of the task in blocks of trials, thereby changing the effort the animal had to exert to maintain a satisfactory reward rate.
We found that increased task difficulty is associated with decreases in the extent to which the trial-to-trial fluctuations in the responses of pairs of neurons covary. These changes were also present (and typically larger) in neurons whose receptive fields did not overlap the visual stimulus, implying that factors such as effort or concentration likely affect all of sensory cortex. Our results suggest that studying the neuronal mechanisms underlying global factors might be generally informative about how cognitive factors affect sensory cortex.
Materials and Methods
Subjects and electrophysiological recordings.
We recorded simultaneously from groups of neurons in both hemispheres of V4 in two rhesus monkeys (Macaca mulatta; both male, 7.5 and 9.0 kg) while they performed a contrast discrimination task. All animal procedures were approved by the Institutional Animal Care and Use Committees of the University of Pittsburgh and Carnegie Mellon University. Before behavioral training, we implanted each animal with a titanium head post. After the animal learned the task (5–7 months), we implanted a pair of 6 × 8 microelectrode arrays (Blackrock Microsystems), in area V4 in each cerebral hemisphere. The two arrays were connected to a percutaneous connector that allowed simultaneous recordings from all 96 electrodes. The distance between adjacent electrodes was 400 μm, and each electrode was 1 mm long. We identified area V4 using stereotactic coordinates and by visually inspecting the sulci. We placed the arrays between the lunate and the superior temporal sulci.
We recorded neuronal activity from these arrays during daily experimental sessions for several weeks in each animal. Using our recording methods, it is nearly impossible to tell whether we recorded from the same single or multiunit clusters on subsequent days. To be conservative, our primary statistical analyses (see Fig. 3) were performed on recordings from a single experimental session from each animal. These example days were selected because the animal performed a large number of trials with good psychophysical performance and because recording quality was good.
We confirmed that the results from these example recording sessions were typical of our dataset and analyzed the neuronal and behavioral data from all recording sessions in which the animal completed at least 250 correct contrast discrimination trials in each hemifield (task described below; median, 1725 total correct trials; range, 616–2912 correct trials), achieved at least 60% correct performance during contrast discrimination trials (median overall percent correct was 73%; range, 62–86%), made choices toward one location in a hemifield no more than 2.5 times as often as the other location (thereby excluding sessions with large behavioral biases), and had good recording quality (the recordings were essentially free from electrical noise and the stimuli were appropriately placed over the receptive fields of the units). Thirty-four recording sessions fulfilled all of these criteria (27 from Monkey F and seven from Monkey J). We recorded a total of 59 single units and 710 multiunit clusters across these sessions. The example session from Monkey F contained one single unit and 24 multiunit clusters, and the example session from Monkey J contained no single units and 20 multiunit clusters. Because the majority of the units we recorded were multiunits (in particular, our combined dataset included only 20 pairs of well-isolated single neurons simultaneously recorded from the same hemisphere), we were unable to perform the analyses separately on single units. A previous study showed that the effects of spatial attention on the responses of individual units or covariability between pairs of units were indistinguishable for single and multiunits (Cohen and Maunsell, 2009). However, in the current study, we can only draw conclusions for multiunit clusters.
All spike sorting was done manually after the experiment using Offline Sorter (version 3.3.2; Plexon). We included a single unit or multiunit cluster for analysis if its response from 50 to 100 ms after stimulus onset (averaged over all trials in which stimuli were presented in the contralateral hemifield) was significantly greater than its baseline firing rate sampled 50 ms before stimulus onset (one-tailed t test, p < 0.05 with a Bonferroni correction for multiple comparisons). We based our analyses on both single units and multiunit clusters and use the term unit to refer to either. The centers of the visual receptive fields from the units recorded during an example recording session from each animal are plotted in Figure 1A.
Task and stimuli. A, Center of visual receptive fields for the single and multiunit clusters from example recording sessions for each animal. B, Contrast discrimination task. The animals were required to maintain fixation in a 1–1.5° diameter window for the duration of the trial. After the animal fixated a blank screen for 200–400 ms, one pair of grating stimuli was placed on one side of a fixation spot within the envelope of the receptive fields (RF) of the units recorded in that hemisphere. The animal's task was to determine which of the pair of stimuli had higher contrast. After the stimulus viewing time (333, 500, 667, or 800 ms), two targets appeared at the locations of the stimuli. When the targets were presented, the animal was free to move its eyes and was rewarded for saccades directed to the target corresponding to the higher contrast stimulus in the pair. C, Discrimination task contrast pairings. There were three possible stimulus pairings: easy, medium, and hard. This distinction was determined by the ratio of the contrasts of the two stimuli. Hard blocks consisted of interleaved medium and hard stimulus pairings, and easy blocks consisted of medium and easy stimulus pairings. D, Task block structure. The animals performed this task in blocks that alternated between easy and hard blocks that appeared on either the left or right side of fixation.
We presented visual stimuli using custom software (written in MATLAB using the Psychophysics Toolbox; Brainard, 1997; Pelli, 1997) on a CRT monitor (calibrated to linearize intensity; 1024 × 768 pixels; 120 Hz refresh rate) placed 54 cm from the animal. We monitored eye position using an infrared eye tracker (Eyelink 1000; SR Research) and recorded eye position and pupil diameter (1000 samples/s), neuronal responses (30,000 samples/s), and the signal from a photodiode to align neuronal responses to stimulus presentation times (30,000 samples/s) using hardware from Ripple.
Contrast discrimination task with difficulty manipulated in blocks.
Our monkeys performed the two-alternative, forced-choice contrast discrimination task illustrated in Figure 1B. A trial began when the animal fixated a central spot. After a random period of time (200–400 ms, picked from a uniform distribution on each trial), a pair of grating stimuli appeared within a single hemifield. After a randomly selected stimulus-viewing period (333, 500, 667, or 800 ms), the gratings were replaced by two saccade targets. The animals were given a juice reward for successfully making an eye movement to the target corresponding to the stimulus that had higher contrast.
The contrasts of the two gratings differed by one of three amounts, and the ratio of the contrasts of the two stimuli (or contrast ratio) is related to the difficulty of the discrimination. Easy trials contained gratings of high and very different contrasts (90 vs 40% contrast for the example session in Monkey F and 99 vs 25% contrast for the example session in Monkey J), hard trials contained gratings with low and very similar contrasts (20 vs 15% contrast for Monkey F and 12 vs 8% contrast for Monkey J), and medium trials were between the two (40 vs 20% contrast for Monkey F and 25 vs 12% contrast for Monkey J; see Fig. 1C). The trials were grouped into easy blocks that contained both easy and medium discriminations and hard blocks that contained both hard and medium discriminations (Fig. 1D). Within a block, the two difficulty levels (easy and medium or medium and hard) and the location of the higher-contrast stimulus were randomly interleaved. The hemifield in which the two stimuli were presented alternated in superblocks, each of which contained an easy and a hard block (Fig. 1D). This alternation allowed us to investigate the neuronal effects of changing task difficulty even when the stimuli were in the opposite hemifield of the receptive field of a neuron.
We based all of our analyses on the medium-difficulty trials, which occurred in every block. Therefore, differences between easy and hard blocks in psychophysical performance or neuronal responses on medium-difficulty trials cannot be attributed to differences in the visual stimuli or the difficulty of an individual trial. Instead, such differences must reflect blockwise differences in the animal's overall effort, concentration, or motivation.
Data analysis.
We examined five measures of neuronal activity calculated using spike counts from 60 to 393 ms after stimulus onset to allow for the latency of V4 responses. We selected this time period for analysis because the visual stimuli were on for the duration of this period (adjusted for the latency of V4 responses) for all trials. Performance on the shortest medium-difficulty trials was slightly worse than on longer trials (70% correct on the shortest 25% of trials vs 74% correct on the longest 75% of trials; t test, p < 0.05), but performance in the longest three quartiles of trial lengths was statistically indistinguishable (p > 0.05). These results suggest that the animals typically made their decisions early in the trial. Consistent with this idea, we found that the physiological signatures of task difficulty were much stronger early than late in the trial.
We calculated the mean response (mean number of spikes over this 333 ms period, so that the mean can be directly compared with variance), variance, and Fano factor (ratio of variance/spike count response) for each neuron in each condition as well as the covariance and spike count correlation (rSC; defined as the Pearson's correlation coefficient: the ratio of covariance/square root of the mean of the variances between spike count responses to repeated presentations of the same stimulus) for each pair of simultaneously recorded units in the same hemisphere. These measures are sensitive to outliers, so we did not analyze trials for which the response of either unit was >3 SDs away from its mean (following the convention in the study by Kohn and Smith, 2005). For each pair of units, we computed covariance or rSC separately for each stimulus condition and averaged the results. Taking the z-scored responses for each condition and computing a single value of covariance or rSC for each pair (as in the study by Ecker et al., 2010) gave qualitatively similar results.
To determine whether differences in any of the response metrics we calculated were artifacts of changes in firing rate, we subsampled our data to create matched distributions of firing rates for all four blocks of difficulty and hemifield. We first binned the firing rates to create histograms of the firing rates in each of the four block types. We then picked, without replacement, from each bin of each distribution to create subdistributions in which each subdistribution has an identical number of points in a given bin. For example, for each bin, we would determine which of the four original firing rate distributions has the fewest data points in that bin and choose a random subset of the data from the other distributions such that each of the four new distributions has the same number of data points in that bin. We repeated this resampling procedure 10,000 times, and the numbers in Figures 3⇓–5 represent the average of these resampled distributions. The error bars represent the SEM for a representative resampled distribution.
Results
Psychophysical performance on identical trials is typically better during hard than easy blocks
We looked for evidence that our task difficulty manipulation caused changes in the animal's overall effort or motivation by comparing performance on medium-difficulty trials that occurred in easy or hard blocks. Figure 2 depicts the proportion of correct trials as a function of the ratio of the contrasts of the two stimuli during easy blocks (blue) or hard blocks (magenta), for example experimental sessions from each monkey. In these two example days, performance on medium-difficulty trials was better during hard than easy blocks. These results suggest that, for these two example sessions, some aspect of the animals' global cognitive state differed in the two types of blocks, leading to better performance during periods when the animals had to work harder, on average, to obtain rewards.
Psychophysical performance. A and B depict each monkey's (F and J, respectively) psychophysical performance during the contrast discrimination task from an example day. In both cases, performance on the identical discrimination (the medium pairing) was improved during hard blocks. Error bars represent 95% confidence intervals on performance (binomial statistics). C, Histogram of the differences in performance on medium trials between the hard and easy blocks across all 34 recording sessions. Performance was better on hard than easy blocks in 71% of sessions.
Across our dataset, performance on medium trials was typically higher in difficult than in easy blocks, but this was not always the case. For the 34 experimental sessions in this study, the median performance on medium trials was 4% better on hard blocks (range of 5% better on easy blocks to 11% better on hard blocks). Performance on medium trials was better during hard blocks in 24 of 34 sessions (71%; Fig. 2C), and average performance was significantly better on hard than easy blocks (t test on mean performance on medium trials across the 34 experimental sessions, p < 10−4).
Increased task difficulty is associated with decreased covariability in neuronal responses
The results in Figure 2 show that our task difficulty manipulation affected psychophysical performance. We wondered whether these changes would be accompanied by changes in the responses of groups of units in area V4. We determined the effects of task difficulty on five common measures of neuronal activity: the mean response (spike count), variance, and Fano factor of individual units and the response covariance and spike count correlation (rSC) of pairs of units recorded simultaneously from the same hemisphere.
Several previous studies have measured the effects of task difficulty on the firing rates of individual neurons with stimuli in their receptive fields (Spitzer et al., 1988; Spitzer and Richmond, 1991; Motter, 1993; Bichot et al., 2001; Hasegawa et al., 2004; Boudreau et al., 2006; Niwa et al., 2012). Our goals were to determine (1) whether this manipulation affects private and shared variability in neuronal responses and (2) whether task difficulty modulates the responses of neurons whose receptive fields do not overlap the stimuli, which would suggest that the global cognitive factors affect neurons throughout the sensory cortex. Therefore, we calculated mean response rate and measures of response variability separately on trials when the visual stimuli were in the same or opposite hemifields as the receptive fields (contralateral or ipsilateral hemifield) of the neurons. We found strong evidence that global cognitive factors affect V4 neurons regardless of whether the stimuli overlapped their receptive fields and that global cognitive factors are most robustly associated with changes in the shared variability of V4 responses.
Figure 3 depicts physiological results for the same example datasets as in Figure 2. Although mean spike count responses (Fig. 3, first row) were predictably higher when the stimuli were in the contralateral hemifield (filled bars) than when they were in the ipsilateral hemifield (open bars), there was no significant difference in mean rate between easy blocks (blue) and hard blocks (magenta). These results were typical of our dataset: of 34 recording sessions, 15 showed significant difficulty-related changes in firing rate when the stimuli were in the contralateral hemifield (44% of sessions, t tests, p < 0.05), and 17 showed significant changes when the stimuli were in the ipsilateral hemifield (50% of sessions). However, these significant sessions were approximately evenly divided between higher firing rates for hard than easy blocks (eight sessions during contralateral blocks, eight sessions during ipsilateral blocks) and lower rates (seven sessions during contralateral blocks, nine sessions during ipsilateral blocks).
Measures of neuronal activity from example sessions. A and B depict five measures of neuronal activity (including mean spike count, variance, Fano factor, covariance, and spike count correlation) during the medium-difficulty discrimination trials from an example day from each monkey (F and J, respectively). Diagonally hatched bars designate rate-matched analysis. The error bars are SEM, and asterisks represent statistically significant difficulty-related modulation (t tests, p < 0.05). Contra, Contralateral; Ipsi, ipsilateral.
The variance in neuronal responses to repeated presentations of the same stimulus was also typically similar in hard and easy blocks (Fig. 3, second row), although there was a trend toward slightly lower variance during hard compared with easy blocks. Across all 34 recording sessions, variance was significantly lower in hard than easy blocks in three sessions of contralateral blocks (8.8% of sessions) and nine sessions of ipsilateral blocks (26%) and was significantly higher in hard than easy blocks for no sessions of contralateral blocks and one session of ipsilateral blocks (3%). This modest drop in variance combined with unchanged mean response resulted in a trend toward lower Fano factors in hard than easy blocks (Fig. 3, third row), although this decrease was only statistically significant in four sessions of contralateral blocks (12%) and eight sessions of ipsilateral blocks (24%, including the example session from Monkey J).
By far, the largest difficulty-related changes we observed were in measures of the extent to which variability in spike count responses was shared between pairs of neurons. The covariance between the responses of pairs of simultaneously recorded neurons in the same hemisphere was lower in the hard than easy blocks (Fig. 3, fourth row). This drop in covariance was statistically significant in 21 sessions of contralateral blocks (62%) and 26 sessions of ipsilateral blocks (76%). There were no sessions for which covariance was significantly higher in hard than easy blocks.
The drop in covariance was proportionally larger than the drop in the variance of individual neurons, so the spike count correlation (rSC, which is related to the ratio of covariance/variance) was lower in hard than easy blocks (Fig. 3, fifth row). The difficulty-related drop in rSC was statistically significant in 23 sessions of contralateral blocks (68%) and 22 sessions of ipsilateral blocks (65%), and there were no significant difficulty-related increases in rSC.
The example recording sessions in Figure 3 were typical of our dataset. Figure 4 depicts peristimulus time histograms (PSTHs) of the average responses of units recorded during all 34 recording sessions to stimuli presented in the contralateral (Fig. 4A, left) and ipsilateral (Fig. 4A, right) hemifields, as well as the average values of the five response measures for all sessions (Fig. 4B). This analysis implicitly treats units recorded on subsequent days as independent, which is unlikely to be true. Therefore, we rely on these averages as a way to visually assess the similarity of our example recording sessions to our dataset as a whole rather than to assess statistical significance. As in the example recording sessions, increased task difficulty did not change the mean response but was associated with slightly lower variance and Fano factor and substantially lower covariance and rSC.
A, PSTHs of the activity of all units recorded across the entire dataset aligned to the onset of stimuli in the contralateral (left) or ipsilateral (right) hemifield, on high contrast (easy; green) trials, medium trials in easy blocks (blue) or hard blocks (magenta), or low contrast (hard; blue) trials. The thin lines represent SEM. B, Mean values for each measure of neuronal activity across our entire dataset. The average of each neuronal measure from the 34 recording sessions from both monkeys plotted according to the same conventions as in Figure 3. This analysis treats each unit as an independent observation, which is likely inaccurate (see Results), so this figure should be viewed as a summary of the dataset rather than a basis for statistical comparisons. Contra, Contralateral; Ipsi, ipsilateral.
Although the average difficulty-related changes in covariance and rSC and also performance were consistent across our dataset, we did not find a consistent relationship between the day-to-day fluctuations in the difficulty-related changes in performance and covariability. This lack of a strong relationship is likely caused by differences in the recordings across days that have nothing to do with the animals' performance. For example, day-to-day variability in the locations of the stimuli likely had a strong effect on measures such as mean firing rate and therefore also covariability. Differences in recording quality across days and especially across animals also likely interfered with our ability to measure a strong relationship between fluctuations in performance and covariability.
Measures of neuronal variability can depend trivially on firing rate (de la Rocha et al., 2007; Churchland et al., 2010). Although we did not observe systematic difficulty-related changes in mean rate, some neurons did respond differently in the easy and hard blocks, and all neurons responded differently when the stimuli were in the same or opposite hemifield as their receptive fields. To control for the possibility that changes in mean rate accounted for some of our results, we analyzed subsets of our neurons so that all four distributions of mean rate (corresponding to easy and hard blocks when the stimulus was in the contralateral or ipsilateral hemifield) were identical (see Materials and Methods). This control did not qualitatively affect our results (Figs. 3, 4B, hatched bars in the right two columns). The variance and Fano factor of individual neurons remained slightly lower during hard than easy blocks, and covariance and rSC remained robustly lower during hard than easy blocks. Together, the results presented in Figures 3 and 4 show that increased task difficulty was associated with lower covariability in the responses of V4 neurons regardless of whether the stimuli were in the contralateral or ipsilateral hemisphere.
Even neurons that do not respond to the visual stimuli show a difficulty-related drop in covariance
One signature of global cognitive factors as opposed to selective factors such as spatial or feature attention is that global factors should affect all neurons, regardless of their receptive field location or tuning properties. Neurons in V4 have primarily contralateral receptive fields, but we observed a difficulty-related drop in covariance (and therefore rSC) even when the visual stimuli were in the hemifield opposite the receptive fields of the units. This covariance drop was as robust as, and typically larger than, the drop in covariance when the stimuli were in the contralateral hemifield (and therefore overlapped the receptive fields of the neurons; Figs. 3, 4). Because many of the units we recorded had receptive fields near the vertical meridian (Fig. 1A), they often responded to stimuli in both the contralateral and ipsilateral hemifields. We wondered whether the decrease in covariance that we observed was limited to units that responded to the visual stimuli, regardless of which hemifield they were presented in.
Therefore, we divided the units we recorded into two groups based on whether they responded significantly to the stimuli in the ipsilateral hemifield (t tests comparing spike count responses 50 ms before ipsilateral stimuli were presented to responses 50–100 ms after stimulus onset, p < 0.05). Across our dataset, 400 of 769 units responded significantly to ipsilateral stimuli (364 of 630 units in Monkey F and 36 of 139 units in Monkey J).
Figure 5 depicts PSTHs (Fig. 5A) and the five measures of neuronal activity for units that did (Fig. 5B, first set of bars from left) or did not (Fig. 5B, second set of bars) respond to ipsilateral stimuli. The difficulty-related changes in all five measures were extremely similar for units that did or did not respond to ipsilateral stimuli. The units that responded to ipsilateral stimuli predictably had higher spike count responses to ipsilateral stimuli (Fig. 5B, first row) and therefore higher covariance (fourth row) and rSC overall (fifth row). Consistent with previous results (Churchland et al., 2010), the units that responded to ipsilateral stimuli also had lower Fano factors than units that did not respond to ipsilateral stimuli. Importantly, both groups showed robustly lower covariance and rSC in hard than easy blocks, and this difficulty-related covariance drop survived our rate-matching controls (Fig. 5B, third and fourth columns). Our results show that increased difficulty is associated with lower covariance, regardless of whether the neurons under study respond to the visual stimuli.
A, PSTHs for neurons that did (left) or did not (right) significantly respond to a stimulus presented in the ipsilateral hemifield as determined by t tests comparing baseline and evoked activity (cutoff p < 0.05). B, Mean values for each measure of neuronal activity for neurons that did or did not respond to a stimulus presented in the ipsilateral hemifield. The same data as in Figure 4 are plotted with the same conventions but split depending on whether a neuron responded significantly to a stimulus placed in the ipsilateral hemifield (horizontal bars; t tests comparing baseline and evoked activity, p < 0.05) or not (vertical bars; t tests comparing baseline and evoked activity, p > 0.05).
The difficulty-related drop in correlated variability cannot be explained by contrast adaptation
Although the medium-difficulty trials on which our analyses are based had identical contrast in the easy and hard blocks of trials in our task, the easy and hard blocks differed by their average contrast. Easy trials had higher contrast than medium trials, and hard trials had lower contrast. Therefore, contrast adaptation likely affected the correlated variability we measured, even on medium-difficulty trials. The fact that covariability decreased even among neurons that did not respond to the stimuli (Fig. 5) suggests that the difficulty-related drop in covariability was not caused by contrast adaptation. To further control for this possibility, we split the medium-difficulty trials by whether another medium-difficulty trial or a different (easy or hard) trial immediately preceded it. The difficulty-related drops in covariance and spike count correlation were statistically significant for both contralateral and ipsilateral stimuli in both subgroups of trials (t tests, p < 0.05). However, the drops in covariability were substantially larger on trials preceded by another medium-difficulty trial than on trials preceded by trials with other contrasts (paired t tests on the difficulty-related drops in covariance and rSC for trials with contralateral or ipsilateral stimuli separately, all p values <10−4). Together, these results suggest that the difficulty-related changes we observed were caused by changes in global cognitive factors rather than contrast adaptation.
Discussion
Global cognitive factors affect correlated variability in neuronal responses
Correlated variability has been well studied in recent years because it can affect the amount of sensory information encoded in a population of neurons (Shadlen et al., 1996; Abbott and Dayan, 1999; Averbeck et al., 2006; Ecker et al., 2011) and because the pattern of shared variability and the way that it is changed by sensory, motor, and cognitive factors might provide clues about the makeup of the underlying circuit (for review, see Cohen and Kohn, 2011). Global cognitive factors, such as motivation, effort, arousal, alertness, or concentration, have been hypothesized to affect correlated variability (Ecker et al., 2010; Moreno-Bote et al., 2014), but this hypothesis has never been tested.
Our results show that increasing the difficulty of a perceptual task is associated with decreased covariance and spike count correlations between pairs of units in V4, regardless of which hemisphere the neurons are located in or whether they respond to the visual stimuli. The fact that the decrease in covariability was present even when the units did not respond to the visual stimuli suggests that global cognitive factors likely decrease covariability across the visual cortex and potentially in many other areas.
The finding that global cognitive factors affect shared variability is important for two reasons. First, changes in global factors likely happen in an uncontrolled way in every study and also differ substantially across studies and even subjects within a study. Difficult psychophysical tasks have often been used to minimize fluctuations in global cognitive factors, yet this approach limits the study of the role of these global factors in brain function. Here, by explicitly manipulating task difficulty, we have shown that such differences in subjects' cognitive states are principally associated with changes in the covariability of neuronal responses. The difficulty-related decreases in covariability could in principle reflect a reduction in the trial-to-trial variability in the animal's cognitive state, which might pave the way for improved performance on psychophysical tasks.
Second, although correlated variability has been shown to depend on many factors, including learning (Ahissar et al., 1992; Gutnisky and Dragoi, 2008; Komiyama et al., 2010; Gu et al., 2011), attention (Cohen and Maunsell, 2009, 2011; Mitchell et al., 2009; Zénon and Krauzlis, 2012; Herrero et al., 2013), or changes in the perceptual task (Vaadia et al., 1995; Cohen and Newsome, 2008; Poulet and Petersen, 2008), the neural mechanism underlying changes in correlated variability remains unknown. In model systems such as mice for which genetic tools to study cellular-level mechanisms are readily accessible, training complex attention or discrimination tasks might be prohibitively difficult (Carandini and Churchland, 2013). Our results open up possibilities for studying the mechanisms underlying correlation changes because we showed that simply changing the difficulty of any perceptual task likely induces changes to the shared variability of sensory neurons.
The decrease in covariance was by far the largest and most consistent difficulty-related change we observed, but we also found that the Fano factor (the ratio of the variance/mean spike count of individual neurons) was typically lower during hard than easy blocks. This result is consistent with other recent studies showing that the Fano factor depends on whether there is a stimulus on the screen (Churchland et al., 2010) or whether the animal attends to the receptive field of the neuron under study (Mitchell et al., 2007). In the current study, we found that increased task difficulty led to small changes in the Fano factor of V4 neurons even in the absence of a stimulus in their receptive field. Our results fit into an emerging body of observations that the increased engagement of sensory neurons, whether caused by the onset of a stimulus (Kohn and Smith, 2005; Smith and Kohn, 2008; Huang and Lisberger, 2009; Churchland et al., 2010; Snyder et al., 2014), allocating attention to the receptive field of the neuron (Mitchell et al., 2007, 2009; Cohen and Maunsell, 2009, 2011; Zénon and Krauzlis, 2012; Herrero et al., 2013), or, in the current study, increasing task difficulty, leads to decreased private and shared variability. Together, these results suggest that changes in the shared and private variance in the responses of cortical neurons can be reliable signatures of the neural computations underlying some sensory and cognitive process and may reveal general mechanisms about the sources and roles of variability in neural circuits.
Effects on mean response rate: relationship to previous studies
Several previous studies have shown that the difficulty of a perceptual task can affect the firing rates of single neurons in the prefrontal cortex (Bichot et al., 2001; Hasegawa et al., 2004), auditory cortex (Niwa et al., 2012), and visual cortex (Spitzer et al., 1988; Spitzer and Richmond, 1991; Motter, 1993; Boudreau et al., 2006). We observed difficulty-related changes in response rate in a substantial number of our recording sessions, but these were fairly evenly split between rate increases and decreases, and there was no difficulty-related difference in average responses across our dataset.
Our experimental design (analyzing neuronal responses on medium-difficulty trials embedded in blocks of different difficulty) was inspired by a study of how task difficulty interacts with spatial attention (Boudreau et al., 2006). In that study, the authors observed difficulty-related changes in the response strength of neurons in area V4. However, they found that the sign of the modulation (especially when spatial attention was directed to a stimulus in the ipsilateral hemifield) varied from subject to subject. The authors provided evidence that the neurophysiological differences between their subjects likely reflected differences in the animals' training history or strategy. Although we did not observe consistent differences between the difficulty-related firing rate modulations in our two animals, it is possible that our animals' strategies changed from day to day.
The two studies also differed in that the subjects in the study by Boudreau et al. (2006) had to alternate spatial attention between two stimuli in different hemifields, whereas we had no distracter stimuli in our task and the animals based their decisions off of a comparison of two adjacent stimuli. Boudreau et al. showed that task difficulty interacts with spatial attention to modulate neuronal responses. It is possible that difficulty-related firing rate modulations are stronger and more consistent in the presence of distracter stimuli. Our study also had a higher proportion of medium-difficulty trials, which likely meant that the difficulty of the two blocks was more similar in our study than in the study by Boudreau et al.
Despite more modest differences in difficulty between the easy and hard blocks and the day-to-day variability in the difficulty-related changes in both firing rates and psychophysical performance, we observed a consistent and large difficulty-related drop in covariability. This difficulty-related drop was present even in neurons that did not respond to the stimuli. In contrast, if the monkeys had responded to the difficult blocks by spatially attending more strongly to the stimuli and ignoring the rest of the visual field, one might have expected difficulty-related increases in covariability in undriven neurons comparable with those observed in neurons representing the unattended stimulus in spatial attention tasks (Cohen and Maunsell, 2009, 2011; Mitchell et al., 2009; Zénon and Krauzlis, 2012; Herrero et al., 2013). In future studies, it will be interesting to investigate the relationship between global cognitive factors, local factors, such as spatial attention, and different measures of the responses of single neurons and neuronal populations.
Concluding remarks
We showed that a simple task modification induces changes in the covariability of the responses of neurons in the visual cortex. These results suggest that studies in which subjects perform difficult psychophysical tasks might expect to record lower spike count correlations than studies with easy tasks or when animals simply fixate without engaging in a task. Our study also suggests a path for studying the neuronal mechanisms underlying changes in spike count correlations associated with cognitive factors by showing that simple manipulations of task difficulty or other global cognitive factors in any system are likely sufficient to induce correlation changes throughout the brain.
Footnotes
This work was supported by National Institutes of Health Grants 4R00EY020844-03 and R01 EY022930 (M.R.C.), a training grant slot on NIH Grant 5T32NS7391-14 (D.A.R.), a Whitehall Foundation grant (M.R.C.), a Klingenstein Fellowship (M.R.C.), a grant from the Simons Foundation (M.R.C.), and a Sloan Research Fellowship (M.R.C.). We thank Regina Chang for assistance with animal training and recordings, Karen McCracken for technical assistance, and Mark Histed, John Maunsell, and Alex Smolyanskaya for comments on a previous version of this manuscript.
The authors declare no competing financial interests.
- Correspondence should be addressed to Marlene R. Cohen, 4400 Fifth Avenue, MI Room 115, Pittsburgh, PA 15213. cohenm{at}pitt.edu
This article is freely available online through the J Neurosci Author Open Choice option.
