Abstract
The prefrontal cortex (PFC) is important for flexible, context-dependent behavioral control. It also plays a critical role in short-term memory maintenance. Though many studies have investigated these functions independently, it is unclear how these two very different processes are realized by a single brain area. To address this, we trained two monkeys on two variants of an object sequence memory task. These tasks had the same memory requirements but differed in how information was read out and used. For the “recognition” task, the monkeys had to remember two sequentially presented objects and then release a bar when a matching sequence was recognized. For the “recall” task, the monkeys had to remember the same sequence of objects but were instead required to recall the sequence and reproduce it with saccadic eye movements when presented with an array of objects. After training, we recorded the activity of PFC neurons during task performance. We recorded 222 neurons during the recognition task, 177 neurons during the recall task, and 248 neurons during the switching task (interleaved blocks of recognition and recall). Task context had a profound influence on neural selectivity for objects. During the recall task, the first object was encoded more strongly than the second object, while during the recognition task, the second object was encoded more strongly. In addition, most of the neurons encoded both the task and the objects, evidence for a single population responsible for these two critical prefrontal functions.
Introduction
Sustained activity is a well known correlate of short-term memory. When a short (several second or less) delay is imposed between a cue and a behavioral response based on it, neurons in several cortical areas collectively sustain activity over the memory delay and maintain task-relevant information (Fuster and Alexander, 1971; Kubota and Niki, 1971; Miyashita and Chang, 1988; Funahashi et al., 1989; Miller et al., 1996). Such “delay activity” is especially prominent and robust in the prefrontal cortex (PFC), which is thought to play a major role in the “online” temporary memory maintenance needed for goal-directed behavior (Pribram et al., 1952; Goldman-Rakic, 1990).
There is a tacit assumption that delay activity is essentially equivalent across tasks. A wide range of studies using a variety of memoranda and behavioral responses have shown more or less the same phenomenon: information related to a cue seen before a delay is maintained over that delay. For example, monkeys have been trained to perform object delayed response tasks with both eye (Wilson et al., 1993; Rao et al., 1997) and arm (Kubota et al., 1980; Fuster et al., 1982; Miller et al., 1996) movements. In both, delay activity is thought to reflect the object in short-term memory. Studies of long-term memory, however, have suggested that the neural substrates and mechanisms can differ depending on how memories are “read out” (i.e., reported). Different brain areas and neurophysiological effects seem to underlie recognition (i.e., “Have I seen this before?”) versus recall (i.e., “I remember that …”) (Delbecq-Derouesné et al., 1990; Cabeza et al., 1997; Staresina and Davachi, 2006; Tsivilis et al., 2008). Whether or not there is an analogous difference for short-term memory is not clear; this has not been directly tested at the neuronal level.
We trained two monkeys on a delayed response task in which they had to remember the identity of two objects and their order under different conditions of memory readout. In the “recognition” task, monkeys released a bar if a sequence of two objects presented after the memory delay was the same as that seen before the delay. In the “recall” task, an array of three objects was presented after the memory delay and the monkeys were required to reproduce the remembered sequence by choosing (saccading to) the two objects in the correct order. Importantly, these tasks only differ in the events at the end of the trial. Up until that point, the tasks are identical and have the same overt behavioral requirements: observe and hold in memory, first one, then two objects over short memory delays. Nonetheless, we found that differences in how memories for object sequences were reported at the end of the trial changed how they were reflected in PFC delay activity. A subset of data from the recognition task (Warden and Miller, 2007) and the recall task (Siegel et al., 2009) has been previously reported.
Materials and Methods
Behavior.
Behavior was monitored and controlled using the CORTEX data acquisition and experimental control system (www.cortex.salk.edu), and eye position was monitored and stored at 60 Hz using an infrared eye-tracking system (ISCAN). Monkeys were required to fixate within 1.5° of the fixation spot throughout the course of each trial. Trials in which breaks in fixation occurred or the monkeys failed to initiate the trial by holding the bar were discarded. Correct responses resulted in a juice reward. Monkeys performed two different behavioral tasks, the recognition (bar-release) task and the recall (saccade choice) task, either in separate recording sessions, or interleaved in blocks during the same recording session (switching task). Four novel objects (objects A–D) chosen from a database of small complex images (Corel) were used each day throughout the recording session and could appear with equal probability as the first or the second sample object. Sample sequences composed of one object repeated twice were only used in the initial recognition task recording sessions. These sequences were not used in the recall task or in the switching task.
Recognition task.
Monkeys learned the recognition task first (Fig. 1A). For this task, the monkeys initiated a trial by grasping a bar and achieving fixation. They were then presented with a sample sequence of two objects presented at the fovea. Each object was shown for 500 ms, and was followed by a delay period of 1000 ms. Monkeys were then shown a test sequence of identical temporal structure, and, if the sequence was a match, they were to release the bar during the presentation of the second test object to receive a reward. If the test sequence was not a match, the monkeys were required to continue to fixate and hold the bar until the presentation of a subsequent match sequence, at which point the bar was to be released. Nonmatch test sequences differed from the original sequence in that either the order of the objects was reversed or one of the two test objects was different. Three types of nonmatching test sequences were used to ensure that the monkeys were remembering the sequence correctly. One type of nonmatch was that in which the first object changed and the second object remained the same. This nonmatch was used to ensure that the monkey remembered the first cue—it would be impossible to correctly respond to this type of trial if the monkey only remembered the second cue. The second type of nonmatch was a sequence in which the first object stayed the same but the second object changed. This was used to test the memory of the second object. The third type of nonmatch was that in which the same objects were used, but they were presented in the reverse order. This type of nonmatch was used to ensure that the monkeys were remembering the objects in the correct order. Each type of nonmatching test sequence was used for a third of the trials. After the recognition task had been learned, neural data were recorded. The monkeys had an average correct performance of 90%, and performed well on all types of trials (first cue 91% correct; second cue 85% correct; order 95% correct; chance on all conditions was 50%), indicating that they remembered both items and the order in which they were presented. On average, 441 correct trials were performed each day.
Recall task.
The monkeys were then trained on the recall task (Fig. 1B). For this task, the trial was identical to that seen in the recognition task through the presentation of the sample sequence. However, the test phase was the presentation of a triangular array of three objects at an eccentricity of 5°. The monkeys were required to saccade, immediately and in the correct order, to the two objects that had just been seen in the sample sequence. After the recall task had been learned, we again recorded neural data. The monkeys were proficient with an average correct performance of 63% (chance level 16.7%). The monkeys' memory of the first cue and the second cue were both significantly above chance (first cue was 74% correct, chance was 33%; second cue was 69% correct, chance was 33%; given that the first cue was correct, the second cue was 83% correct, chance was 50%). The monkeys' performance on cue order was also significantly above chance (given that both items were correct, order was 85% correct, chance was 50%). On average, 469 correct trials were performed each day. There was no spatial bias in responses; each two-saccade path was represented with equal frequency among all completed trials. Performance on the eye-movement task was somewhat worse than performance on the bar-release task, perhaps due to the difficulty of quickly identifying two small images in the periphery of the visual field without the chance to visually inspect them. The monkeys performed better both as the number of items in the choice grid decreased and as they moved closer to the fovea. In both tasks the monkeys' memory for the first and second objects were similar.
Switching task.
The monkeys were then trained on the switching task, in which blocks of each task, recognition and recall, were interleaved, 100–250 trials per block. There was no explicit cue to indicate that the task had switched, as it was obvious from context. Monkeys typically performed 2–4 blocks of each task during each recording session. Performance was good on both versions of the task. During the recognition task the monkeys performed an average of 305 correct trials each day at a performance level of 93% correct, again performing each of the three trial types significantly above chance (test of first cue: 94% correct; test of second cue: 89% correct; test of order: 97% correct; chance for all was 50%). During the recall task, the monkeys performed an average of 242 correct trials each day, at a performance level of 64% correct, significantly above the chance level of 16.7%. Performance on the first cue and the second cue was good (first cue was 74% correct, chance was 33%; second cue was 72% correct, chance was 33%; given that the first cue was correct, the second cue was 85% correct, chance was 50%), and performance on order was good (given that both items were correct, order was 85% correct, chance was 50%). Performance on the recognition and recall components of the switching task was similar to performance when the tasks were performed separately.
Subjects and surgery.
The PFC in two adult rhesus monkeys (Macaca mulatta), one female and one male, was localized using magnetic resonance imaging. A recording chamber was implanted stereotaxically directly over the principal sulcus and anterior to the arcuate sulcus, and a head bolt was implanted to immobilize the head during neural recordings. All surgeries were performed under aseptic conditions while the animals were anesthetized with isoflurane. The animals received postoperative antibiotics and analgesics and were always handled in accord with National Institutes of Health guidelines and the Massachusetts Institute of Technology Committee on Animal Care.
Neurophysiology.
We used grids with 1 mm spacing (Crist Instrument) and custom-made independently moveable microdrives to lower eight dura-puncturing Epoxylite-coated tungsten microelectrodes (FHC) until single neurons were isolated. We recorded every neuron that we encountered and made no attempt to prescreen neurons for selectivity, ensuring an unbiased sampling of prefrontal activity. The waveforms were digitized and stored for offline manual sorting using principal components analysis (Offline Sorter, Plexon). Neurons were recorded from both the left and right hemispheres.
Data analysis.
Only data from correct trials were used in our analyses. Epochs were defined as follows: first cue, 100–500 ms after first cue presentation; one-object delay, 200–1000 ms after the start of the first delay; second cue, 100–500 ms after second cue presentation; two-object delay, 200–1000 ms after the start of the second delay. These epochs were chosen for simplicity. The results reported here were insensitive to the exact time windows used. Selectivity for the first or the second object was determined using either two-way ANOVA or simple-effects ANOVAs, p < 0.05. “Object-selective” neurons were defined as neurons that showed selectivity for either the first or the second object during at least one of the above epochs in at least one task. All p values were Bonferroni corrected. The difference in the proportion of neurons encoding the first or the second object between tasks was tested with the χ2 test. For this test, we used the last 600 ms of the two-object delay period, because this was when the effects were strongest. When screening for “task-selective” neurons, we first equalized the number of trials of each object sequence. All neural activity histograms were calculated with a resolution of 1 ms and then smoothed with a 50–100 ms boxcar window.
For the analyses in Figure 5, we normalized the data by dividing the firing rate corresponding to an object during an epoch by the average firing rate over all conditions during that epoch. This allowed us to directly compare epochs with very different average firing rates. For these analyses, object-selective neurons were defined as those that were significantly selective for the object of interest in the epochs of interest. We tested whether slopes were significantly different from zero using a linear regression t test.
Variance components.
To calculate the percentage variance explained by each object, we first calculated, for each neuron, an ANOVA to determine σ2, the variance within groups, and σ2 + n0σA2, the variance among groups. n0 is the average sample size, and σA2 is the added variance component due to the presence of the object in memory. Simple effects ANOVAs were used instead of two-way ANOVAs because of the presence of a large amount of interaction between the first and second objects. The percentage variance explained by an object can be expressed as follows: We computed this quantity in 200 ms bins slid every 20 ms across the course of the trial for each neuron, yielding a time course of variance explained by each object for each neuron during each task. We then averaged this across the population of object-selective neurons to produce the final curves. We used a two-sample t test (p < 0.05) to test whether the differenced curves were significantly different between tasks.
Results
We recorded the activity of 222 lateral PFC neurons (area 46) from two monkeys during the recognition (bar-release) task, 177 neurons during the recall (saccade-choice) task, and 248 neurons during the switching task (interleaved blocks of the recognition and recall tasks). As in prior studies, we did not preselect neurons for task-related activity; we recorded from every well isolated neuron that we encountered. More than half of the neurons recorded during each task showed selectivity for either the first or the second object or both during at least one trial epoch [125/222 (56%) in the recognition task; 117/177 (66%) in the recall task; 189/248 (76%) in the switching task, two-way ANOVA, p < 0.05].
Proportion of object selectivity in different tasks
We first asked whether the task being performed had an influence on the strength of the neural representation of the two objects held in memory. For this analysis, we focused on the two-object memory delay period, because in this epoch both objects were held in memory and could be compared. A subset of neurons showed selectivity for the identity of either the first cue object, the second cue object, or both objects during this epoch [88/222 (40%) in the recognition task; 79/177 (45%) in the recall task; 120/248 (48%) in the recognition trials of the switching task; 101/248 (41%) during the recall trials of the switching task, two-way ANOVA, p < 0.05]. In both the recognition and recall versions of the task, approximately one-third of the population of selective neurons reflected the identity of both objects in memory [27/88 (31%) in the recognition task; 24/79 (30%) in the recall task, two-way ANOVA, p < 0.05], while the remainder of the population encoded either only the first object or only the second object (Fig. 2A). This relationship also held during the recognition and recall trials of the switching task (Fig. 2B) [39/120 (33%) during the recognition trials; 35/101 (35%) during the recall trials, two-way ANOVA, p < 0.05].
While the proportion of neurons showing significant selectivity for both cue objects was the same for the two tasks, the proportion of neurons reflecting only the specific cue objects differed. In the recognition task, more neurons reflected the identity of the second cue object (40/88, 45%) than the first cue object (21/88, 24%) (one-proportion Z test, p < 0.01). By contrast, during the recall task, the proportion of neurons reflecting the identity of only the first or second cue object was approximately equal (first object 26/79, 33%, second object 29/79, 37%, one-proportion Z test, p > 0.05) (Fig. 2A). These relationships also held during the recognition and recall trials of the same recording session (switching task) (recognition task: first object 18/120, 15%, second object 63/120, 53%, one-proportion Z test, p < 0.0001; recall task: first object 31/101, 31%, second object 35/101, 35%, one-proportion Z test, p > 0.05) (Fig. 2B). The difference in proportions between tasks was significant (χ2 test, p < 0.01) and was accounted for by a decrease in the number of neurons representing the second object and an increase in the number representing the first object during the recall task.
Changes in object selectivity in individual neurons
We also see this shift in selectivity between tasks in individual neurons. Figure 2, C and D, shows an example of a neuron recorded during the switching task. Differences in object selectivity between the recognition trials (Fig. 2C) and recall trials (Fig. 2D) are apparent. The trials in both of these panels are grouped according to which object was used as the first cue; therefore, any differences in firing rate reflect information about the first object. During the two-object memory delay, this neuron showed little selectivity for the first object during recognition trials (Fig. 2C). However, when the monkey was performing recall trials, this neuron showed strong selectivity for the first cue object (Fig. 2D, light blue line).
Time course of object information
We then quantified the time course of information about the identity of each cue object in the population of neurons by computing the percentage of variance in the neural data explained by the first and second objects in a sliding window. This analysis included all neurons that were significantly selective (simple effects ANOVA, p < 0.05) for either object during any epoch. Figure 3A shows the average information about the identity of each cue object during the recognition task. Information about the first object peaked shortly after its appearance and then decayed, but did not disappear; it was maintained during the one-object memory delay and then through the rest of the trial. Information about the second object also increased shortly after presentation and was maintained across the two-object memory delay. During this delay, when both objects were being held in memory simultaneously, there was more information about the second, most recently presented, cue object than the first cue object.
Figure 3B shows the same analysis during the recall task. The strength of the first cue object in memory is similar to that seen during the recognition task during its presentation and the one-object memory delay. However, when the second object was presented, we observed clear differences between the recognition and recall tasks. Instead of decaying away during the second cue and the two-object delay, information about the first object grew stronger throughout the rest of the trial. By the end of the two-object memory delay, there was more information about the first cue object than the second cue object—the opposite of the effect seen in the recognition task.
Figure 3C plots the difference between the information about the first and second objects for each task by subtracting information about the second cue object from information about the first cue object. The resulting difference curves reflect the relative strengths of the objects and how this depends on the task being performed. Note that during the two-object memory delay, the relative representations of the two objects are significantly different between tasks (two-sample t test, p < 0.05). There is more information about the second cue object than the first during the recognition task (negative difference value), while the opposite is true in the recall task (positive difference value).
Thus, different demands to read out the memory of the two objects resulted in different patterns of PFC delay activity. However, it was a possibility that this change in representation resulted from either the months of training it took to switch the monkeys from the recognition task to the recall task or the fact that two different populations of neurons were being compared. Alternatively, the PFC might be flexible in its representations and have the ability to quickly change them in response to changes in task demands. To test this, we recorded the activity of an additional 248 neurons while monkeys alternated between blocks of 100–250 trials of the recognition and recall trials (the switching task).
Figure 4 shows the same analyses described for Figure 3, but for a single population of neurons recorded while the monkey was performing interleaved blocks of the recognition and recall tasks (the switching task). Again, during the two-object memory delay, there was more information about the second cue object in the recognition task (Fig. 4A), while in the recall task (Fig. 4B), there was more information about the first cue object. This difference was again significant throughout most of the two-object delay period (two-sample t test, p < 0.05) (Fig. 4C). The overall variance in neural activity during the two-object memory delay was similar between the two tasks (Levene's test, p > 0.43), and therefore cannot account for the differences between tasks.
Population changes in object preferences
In previous work (Warden and Miller, 2007) on the recognition task, we examined how selectivity for the first cue object was affected by the addition the second cue object to memory. We found that a neuron's relative preference for different first cue objects was often changed by the addition of the second cue object to memory. Here we extend these effects. We asked whether the same neurons could show different object preferences during recognition versus recall trials. This is a question that is complementary to the population analyses in Figures 3 and 4, which addressed the issue of the amount of information about the objects in general, not which objects were preferred.
We defined, for each neuron, a canonical preference ordering of objects (“best” to “worst” object) based on its response to these objects during first cue presentation. We then used this definition to examine the representation of this first cue during other epochs in the trial: the one-object delay, the second cue, and the two-object delay. Results for the recognition task are shown in Figure 5A–C. We first examined whether object preferences were different in the one-object delay period (Fig. 5A). For this figure, we only used neurons that were significantly selective for the first cue during both the first cue presentation and one-object delay epochs (two-way ANOVA, p < 0.05). We calculated the firing rate of each neuron for each of the four objects during the first cue presentation, normalized this to the average firing rate of that neuron during that epoch, ordered the normalized rates from best to worst, and averaged the neurons together. This produced the red curve, which, by definition, is monotonically decreasing. We then did the same thing for the one-object delay period, using the best–worst ordering defined during the first cue period, to produce the pink curve. Here we see that, although the slope is shallower, the relative best–worst ordering of objects is preserved during this delay period (negative slope, linear regression t test, p < 0.05). When we extend this analysis into the second cue and two-object delay epochs (Fig. 5B,C), we see that the population average response (pink line) has inverted during the second cue presentation (positive slope, linear regression t test, p < 0.05) and flattened during the two-object delay period (no slope, linear regression t test, p > 0.05). This flattening does not mean that selectivity for the first object has disappeared—on the contrary. To create Figure 5C, we used neurons that were significantly selective for the first object during both the first cue and two-object delay epochs (two-way ANOVA, p < 0.05). These neurons have instead changed the way they encode the first object. Some neurons invert their best–worst preferences, some maintain their preferences, and others shuffle them. When these are all averaged together, they produce a flat line. Thus, while individual neurons maintain selectivity for the first cue object after the presentation of the second cue object, the presentation of a second cue can adjust the rank preference order of the first cue object.
We repeated these analyses for the recall task, shown in Figure 5D–F. Here, the effects on object best–worst preferences were the same as in the recognition task for the one-object delay, but differed during the second cue and two-object delay epochs. Figure 5D shows that neurons maintain their object best–worst preferences during the one-object delay period, although, as seen in Figure 5A, there was again a decreased slope (negative slope, linear regression t test, p < 0.05). Figure 5E shows that the average object preference has flattened, instead of the inversion seen in Figure 5B (no slope, linear regression t test, p > 0.05). Again, this should not be misinterpreted as a decrease in object selectivity: only objects that were selective for the first object during both first cue presentation and second cue presentation (two-way ANOVA, p < 0.05) were included in the analysis. In Figure 5F, there is a reemergence of best–worst preference ordering across the population of selective neurons—the previously defined best object is now again the best object for this population of neurons (negative slope, linear regression t test, p < 0.05). This result is complementary to the increased strength of coding of the first object during this epoch.
Task selectivity
A large fraction of neurons in the population showed different levels of activity depending on which task was being performed. These neurons did not always show object selectivity (although most did, discussed below), but did respond preferentially during the performance of one or the other task. Four example neurons are shown in Figure 6A–D. The neurons in Figure 6, A and C, had a higher firing rate during the two-object memory delay during recognition trials, while the neurons in Figure 6, B and D, had a higher firing rate during this period for the recall task. These neurons are typical of those found in the population.
Across the population, 70% (174/248) of neurons showed a significant difference in firing rate between tasks during at least one task epoch (p < 0.05, Bonferroni-corrected one-way ANOVA) (Table 1). A greater number of neurons showed a significant difference between tasks during the two-object delay period than during any other epoch [79/248 (32%) during fixation; 59/248 (24%) during the first cue; 56/248 (23%) during the one-object delay; 60/248 (24%) during the second cue; 101/248 (41%) during the two-object delay]. We looked at the difference in firing rates between the recognition and recall trials of the switching task (Fig. 7A) as a function of time during the trial, averaged across the population of neurons showing significant effects of task during any epoch. We found that while all trial epochs showed a significant difference between tasks, the greatest difference was seen during the two-object delay period. In fact, the effect of task climbed steadily during the two-object delay, reaching its apex near the end.
We then looked at the proportion of neurons that fired more strongly during the recognition or recall task (recognition- or recall-selective neurons) (Fig. 7B). Across the population of task-selective neurons, there was no significant difference in the proportion of neurons representing either the recognition task or the recall task more strongly during the first cue, one-object delay, or two-object delay, but the recognition task was represented more strongly during the second cue period (p < 0.01, one-proportion Z test). A scatter plot of differences in firing rate between tasks is shown in Figure 7C. Each point in this figure represents a single neuron during the second cue epoch. The firing rate during the recognition task is plotted against the firing rate during the recall task.
Task and object coding in individual neurons
We found that the vast majority (146/174, 84%) of task-selective neurons were also object-selective at some point during the trial (Table 1, “Task-selective neurons that are also object-selective”), and that this object selectivity in task-selective neurons peaked on average during the second cue and two-object delay epochs (Table 1). Task and object selectivity were both widely distributed across the prefrontal cortex, and we found examples of both types of selectivity at most recording sites (Fig. 8). An example of a neuron with both task and object selectivity is shown in Figure 9. This neuron is more strongly activated during the two-object delay period when the monkey is performing recall trials than during recognition trials (Fig. 9A). The activity of this neuron also reflects the objects that the monkey is remembering during the two-object delay period, but the strength of this object coding is dependent on which version of the task the monkey is performing. During recognition trials (Fig. 9B), the representation of the first object is weak during the second delay period (4% variance explained by the first object). However, during recall trials (Fig. 9C), the coding of the first object is much stronger (24% variance explained by the first object). The effect of task was less pronounced on the representation of the second object; 14% of the variance was explained by the second object during the recognition task (Fig. 9D), while 10% of the variance was explained by the second object during the recall task (Fig. 9E). Overall, approximately half of task- and object-selective neurons were object selective during only one task in any given epoch, while the other half was object selective during both tasks in that epoch (Table 1).
Discussion
We found that differences in how monkeys reported short-term memories for two-object sequences changed how that information was reflected in the memory delay activity of PFC neurons. When the monkeys had to recognize a sequence (by releasing a bar for the matching sequence), the most recently seen (second) object was more strongly reflected in PFC activity than the first object. This suggests a passive, automatic, buffer-like memory mechanism in which the strength of a memory trace is a function of how recently the object was seen (Brown, 1958; Peterson and Peterson, 1959; Murdock, 1961). By contrast, when monkeys had to “recall” (reproduce) an object sequence by choosing and saccading to the objects from a stimulus array, the earlier (first) object was more strongly reflected in the neural activity immediately preceding the behavioral choice. Furthermore, the “original” pattern of object selectivity representing the first cue when it was first presented seems to be at least partly reactivated in the second memory delay of the recall task. This is consistent with a more volitional and proactive memory mechanism that prospectively encodes anticipated events (Rainer et al., 1999; Tomita et al., 1999; Roesch and Olson, 2005). It seems to reflect monkeys recalling the first object in anticipation of the saccade to it after the delay. This is similar to effects seen in monkeys trained to copy geometric shapes (Averbeck et al., 2002) in which PFC activity reflects a shape segment immediately before monkeys began to draw it. Recall could also support the choice of the second object, although it is possible that it could have been chosen because it was more familiar than the remaining object that was not part of the sequence.
We also observed task effects, differences in activity between recognition and recall trials. These were also strongest near the end of the second memory delay when the time for the different types of motor responses was drawing near. This could reflect a general (eye vs arm) premotor signal, but it cannot reflect the specific motor plan—this was not known until after the delay when the monkeys were presented with the test stimuli. This premotor signal could be linked to the differences in object selectivity between tasks, as discussed below. Task effects and differences in object selectivity between tasks overlapped in many neurons and both types of effects were strongest at around the same time (near the end of the second memory delay). Task-dependent PFC neuronal activity has been widely reported (Hoshi et al., 1998; White and Wise, 1999; Wallis et al., 2001), and a few investigators have also shown examples of individual neurons with task-dependent object selectivity (Asaad et al., 2000; Johnston and Everling, 2006). We too find these effects in our data and additionally demonstrate, over the neural population, task influence on the relative strengths of object representations and on object rank ordering in PFC neural activity. It is not surprising that these effects have not been seen before. In our data, they only become apparent because two items were simultaneously held in memory. There was no difference in object selectivity between tasks until the second object was seen.
These data support the idea that representations in the PFC are highly dependent on context. Strength of representation and rank ordering of neural responses to objects in memory were affected by which task the monkey was performing as well as what other objects were in memory. In addition, single neurons rarely encoded a single object, and instead also usually reflected both objects held in memory as well as the task being performed. Recent computational models suggest that this type of “mixed, dynamic selectivity” is an essential feature of the PFC. It endows the ability to quickly learn and flexibly implement new rules because it enables the storage of relations between many items (stimuli, rules, responses, etc.) without the need for a large number of neurons (Rigotti et al., 2010; Salzman and Fusi, 2010). This, however, makes the readout of these neurons less straightforward, because they do not have a canonical representation that is constant across contexts. Nonetheless, it could be accomplished because information about task and epoch is embedded along with information about the items. This produces separable activity patterns that can be decoded by neurons that take the context into account (Rigotti et al., 2010). Mixed coding is also consistent with the observation that stimulus selectivity in the PFC is widely distributed across many neurons that, individually, show weaker selectivity than typical sensory or motor neurons (Duncan and Miller, 2002; Rigotti et al., 2010). It is becoming clear that this is the predominant form of neural coding in the PFC (Asaad et al., 2000; Miller and Cohen, 2001; Johnston and Everling, 2006; Sigala et al., 2008).
Our previous work (Siegel et al., 2009) on the recall task demonstrated that PFC population activity was rhythmically synchronized at frequencies around 32 Hz, and that information about the two objects was selectively enhanced at specific phases of the local field potential. As the results that we present in this paper are complementary to those already published, the relationship between the two studies should be explicitly considered. The current paper places a greater emphasis on information encoded in the average firing rates of neurons in the PFC population, while our previous work discussed instead the additional information gained by taking oscillatory phase into account. While the bulk of the information about the two objects can be decoded with average firing rates, using phase enables an extraction by an additional 12–16% of object information (Siegel et al., 2009) and may help explicitly code object order. This experiment instead focused on the effects of multiple objects and task on object selectivity per se.
In sum, our results indicate that changes in task demands can change object strength and selectivity in the prefrontal cortex. The differences between the tasks (i.e., how information is read out from memory) do not change the behavioral requirements of the task during the memory delay (i.e., briefly remember a sequence of two objects). Nonetheless, it changed the way the objects were represented in PFC delay activity. This suggests that PFC delay activity does not simply buffer sensory information. That information is embedded in the context of information about the task at hand.
Footnotes
This work was supported by the Center of Excellence for Learning in Education, Science, and Technology, a National Science Foundation Science of Learning Center. We are grateful to A. Diogo, M. Histed, and A. Pasupathy for discussions and comments and to K. MacCully for technical assistance.
- Correspondence should be addressed to Earl K. Miller, The Picower Institute for Learning and Memory and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Building 46-6241, 77 Massachusetts Avenue, Cambridge, MA 02139. ekmiller{at}mit.edu