Abstract
We compared neuronal activity in the dorsolateral (PFdl), orbital (PFo), and polar (PFp) prefrontal cortex as monkeys performed three tasks. In two tasks, a cue instructed one of two strategies: stay with the previous response or shift to the alternative. Visual stimuli served as cues in one of these tasks; in the other, fluid rewards did so. In the third task, visuospatial cues instructed each response. A delay period followed each cue. As reported previously, PFdl encoded strategies (stay or shift) and responses (left or right) during the cue and delay periods, while PFo encoded strategies and PFp encoded neither strategies nor responses; during the feedback period, all three areas encoded responses, but not strategies. Four novel findings emerged from the present analysis. (1) The strategy encoded by PFdl and PFo cells during the cue and delay periods was modality specific. (2) The response encoded by PFdl cells was task and modality specific during the cue period, but during the delay and feedback periods it became task and modality general. (3) Although some PFdl and PFo cells responded to or anticipated rewards, we could rule out reward effects for most strategy- and response-related activity. (4) Immediately before feedback, only PFp signaled responses that were correct according to the cued strategy; after feedback, only PFo signaled the response that had been made, whether correct or incorrect. These signals support a role in generating responses by PFdl, assigning outcomes to choices by PFo, and assigning outcomes to cognitive processes by PFp.
Introduction
Abstract rules and strategies allow animals to make beneficial choices before learning concrete stimulus-response associations, and they provide an advantage by reducing errors during learning (Genovesio and Wise, 2008; Mushiake et al., 2009). Although strategies differ from rules in certain ways, for the present purposes they can be considered together. Several prefrontal (PF) areas contribute to these cognitive capabilities, including the dorsolateral PF cortex (PFdl, area 46) (Milner, 1963; Collins et al., 1998; White and Wise, 1999; Hoshi et al., 2000; Stuss et al., 2000; Monchi et al., 2001; Wallis et al., 2001; Brass et al., 2003; Bunge et al., 2003; Genovesio et al., 2005; Mansouri et al., 2006; Buckley et al., 2009) and the orbital PF cortex (PFo, areas 11, 13 and 14) (Wallis et al., 2001; Wallis and Miller, 2003; Buckley et al., 2009; Yamada et al., 2010; Tsujimoto et al., 2011a).
To explore the mechanisms of abstract rules and strategies, we studied neuronal activity as macaque monkeys performed three tasks: a visually cued strategy task, a fluid-reward cued strategy task, and an oculomotor delayed-response (DR) task. In addition to PFdl and PFo, we also sampled activity in the polar PF cortex (PFp, area 10). For this dataset, we have previously contrasted response coding in PFdl and PFo at feedback time (Tsujimoto et al., 2009), reported results from PFp (Tsujimoto et al., 2010), and compared strategy coding in PFdl and PFo during the cue period (Tsujimoto et al., 2011a). We showed that: (1) cells in PFo and PFdl (but not PFp) encoded strategies during the cue period; (2) the strategy signal in PFo preceded that in PFdl; (3) on error trials, PFo encoded the strategy that was cued and not the one that the monkey used, whereas PFdl lost strategy selectivity on error trials (see also Genovesio et al., 2008); (4) cells in PFdl (but not PFo or PFp) encoded responses during the delay period; (5) cells in all three areas encoded the response (but not the strategy) at feedback time; and (6) in PFp this signal encoded correct (but not incorrect) responses.
Our previous reports have deferred several analyses, however. They did not contrast activity during all three tasks for PFo and PFdl, as Tsujimoto et al. (2010) did for PFp. A contrast of that kind assesses whether the strategy and response signals observed in PFo and PFdl show task or modality generality. Furthermore, our previous papers did not report results from the fluid-reward cued task for either PFo or PFdl. Results from this task allow us to rule out reward effects on strategy and response coding. And, finally, our previous reports have not contrasted error-trial activity before versus after feedback for PFo or PFdl, as we did for PFp. Dividing the feedback period in this way allows us to assess additional differences among the three areas.
Materials and Methods
Subjects.
Two male rhesus monkeys (Macaca mulatta), 10–11 kg, were used in this study. Before recordings began, the monkeys were operantly conditioned to perform three saccade tasks (Fig. 1): a visually cued strategy task, a fluid-reward cued strategy task, and an oculomotor delayed-response (DR) task. All procedures conformed to the Guide for the Care and Use of Laboratory Animals and were approved by the appropriate Institutional Animal Care and Use Committee.
Behavioral task and recording locations. A, Example sequence of task events for the visually cued strategy task, from top to bottom. Each dark gray rectangle represents the video monitor as viewed by the monkey. The red arrows indicate the target of gaze. The blue drop-like shape at the lower left represents fluid delivery as a reward. In this task, the visual stimuli shown in D instructed which strategy should be applied. B, Example sequence for the fluid-reward cued strategy task. The blue shapes illustrate a single drop of fluid or two drops half that size. In this task, two half-drops of fluid instructed the shift strategy and a single drop instructed the stay strategy. The example trial involves two half-drops. C, Example sequence for the DR task. D, Cues used in the visually cued strategy task. E–G, Recording zones for PFp (E), PFdl (F), and PFo (G). AS, Arcuate sulcus; LOS, lateral orbital sulcus; MOS, medial orbital sulcus; PS, principal sulcus.
Behavioral tasks.
The three tasks were presented to the monkeys in separate blocks of trials. Blocks in the visually cued strategy task consisted of a mean of 94 ± 19 (SD) trials, blocks in the fluid-reward cued strategy task averaged 55 ± 12 trials, and those in the DR task averaged 49 ± 10 trials. As they performed the tasks, each monkey sat in a primate chair, with its head stabilized, and oriented toward a video monitor 32 cm away.
Each trial began when the fixation point, a 0.6° filled white circle, appeared at the center of the video monitor (Fig. 1). At the same time, two potential saccade targets appeared. They were 2.0° unfilled white squares, located 11.6° to the left and right of the fixation point.
To continue a trial, the monkeys needed to attain and maintain fixation on the central point for 1.5 s. An infrared oculometer (Arrington Research) recorded eye position. The monkeys were required to maintain fixation continuously within a ±3° square window centered on the fixation point. In practice, however, both monkeys maintained fixation much more accurately than that requirement and only rarely made a saccade within the fixation window (Tsujimoto et al., 2009).
After the 1.5 s precue fixation period, a cue was presented, which differed by task. In the visually cued strategy task (Fig. 1A), the cue consisted of a square or rectangle that appeared at the fixation point for 0.5 s. On each trial, one cue was selected pseudorandomly from a set of four: a vertical rectangle (light gray, 1.0° × 4.9°), a horizontal rectangle of the same dimensions and brightness, a yellow square (2.0° × 2.0°), and a purple square of the same size (Fig. 1D). The vertical rectangle and yellow square instructed the monkeys to select the same response on the current trial as they had on the previous trial. These cues thus instructed the stay strategy and were called stay cues. The horizontal rectangle and purple square instructed the monkeys to shift from their previous response. They thus instructed the shift strategy and were called shift cues. Given that there were only two potential responses, a left or a right saccade, a shift cue determined the response and reward probability in the same way as a stay cue. In both cases, the reference for the current response was the most recent correct and rewarded one. A given block of trials in the visually cued task included approximately equal numbers of stay cues, shift cues, colored squares, oriented rectangles, left responses, and right responses.
In the fluid-reward cued strategy task (Fig. 1B), one drop of fluid (0.2 ml) served as the stay cue and two half-drops of fluid (0.1 ml each) served as the shift cue. Figure 1B illustrates a cue consisting of two half-drops (blue shapes). Fluid delivery began at the start of the cue period, at the same time as the visual cue appeared during the visually cued task. In blocks of this task, stay and shift cues were intermixed pseudorandomly and both occurred on approximately half of the trials.
In the DR task (Fig. 1C), a 0.6° filled white circle was presented inside either the left or right target square. This visuospatial cue indicated that a saccade to its location would be rewarded on that trial. The left and right trials were intermixed pseudorandomly with equal probability.
For all three tasks, the monkeys had to maintain fixation on the center of the screen for the entire cue period (0.5 s), as well as for a subsequent 1.0 s, 1.25 s, or 1.5 s delay period (selected pseudorandomly). Fixation breaks during the cue or delay periods caused a trial to be aborted. The fixation spot and the two potential saccade targets remained on the screen during the delay period, after which the fixation spot disappeared as the go signal for a saccadic eye movement.
Once the monkey made a saccade to one of the two potential targets, defined by a centered square window of ±3.75°, both targets became solid white on every trial, whether correctly or incorrectly performed. Entry of gaze into the response window was designated as target acquisition. After 0.5 s of continuous target fixation, feedback was delivered. Fixation breaks during the 0.5 s prereward period aborted the trial. If fixation had been maintained, a single drop of fluid (0.2 ml) usually served as reward feedback after correct responses; the presentation of red squares over both targets always served as negative feedback after incorrect responses. On some blocks of the fluid-reward cued task, reward feedback was, like the cues for that task, either a single drop of fluid (0.2 ml) or two half drops (0.1 ml each). Because the number of drops was pseudorandomly determined for both cue and feedback, all combinations of drop number occurred with approximately equal frequency.
After errors, the cue from that trial was repeated on a correction trial. Correction trials were presented until the monkey obtained a reward by responding correctly. In practice, the monkeys rarely required more than one correction trial after an error (Tsujimoto et al., 2009).
Data collection.
After the monkeys had learned the tasks, a recording chamber (10.65 mm inner diameter) was implanted over the exposed dura mater of PFp in the right hemisphere. The procedures used for PFp recordings, including chamber design and surgical methods, have been described previously (Mitz et al., 2009). After we completed data collection from PFp, another chamber (18 mm inner diameter) was implanted over a more caudal part of the frontal lobe in the same hemisphere. The position and angle of the caudal chamber was adjusted based on magnetic resonance images (MRI), so that both PFdl and PFo were accessible through the same port.
Single-cell activity was recorded by using up to 16 platinum-iridium electrodes (0.5–1.5 MΩ at 1 kHz; Thomas Recording) simultaneously. These electrodes were individually inserted into the cortex with a multielectrode drive (Thomas Recording) that enabled independent control of each electrode. In typical recording sessions for the caudal port, about a half of the electrodes were advanced into PFo and the others were maintained more superficially in PFdl.
Signals from each electrode were recorded using a Multichannel Acquisition Processor (Plexon). Single-cells potentials were isolated offline using a cluster cutting technique (Off Line Sorter, Plexon). Spike sorting was based on the multiple criteria, including principal component analysis, the minimal interspike intervals, and close visual inspection of the entire waveforms for each cell.
Histology.
The recording sites were reconstructed by histological analysis, supplemented by structural MRI. Near the end of data collection, electrolytic lesions (20 μA for 20 s, anodal current) were placed in selected locations at two depths per penetration in the caudal chamber. After 10 d, the animal was deeply anesthetized and then perfused through the heart with 10% (v/v) formol saline. Immediately before and during the perfusion, a pin was inserted through the center of both the rostral and caudal chambers. The penetration sites and tracks were reconstructed in Nissl-strained sections by reference to recovered electrolytic lesions and to the marking pins inserted at the time of the perfusion. Recoding sites in PFo were confirmed by cytoarchitectonic analysis to come from granular prefrontal (i.e., homotypical) cortex. The locations of recording sites have been described previously (Tsujimoto et al., 2010, 2011a).
Data analysis.
Correction trials were excluded from all analyses reported here, as were error trials except where noted. To perform the analyses, we used SPSS (SPSS) and MatLab (MathWorks).
To select task-related neurons for further analysis, we used the Kruskal–Wallis test (α = 0.05) to compare mean activity rates among four task periods. The task periods were defined on the basis of time widows relative to task events: a fixation period (0.5–1.0 s after fixation onset), a cue period (0.0–0.5 s after cue onset), a delay period (0.0–1.0 s after cue offset), and a feedback period. The feedback period for this general analysis was defined as the interval from 0.3 s before feedback onset until 0.2 s afterward, in accord with our previous reports (Tsujimoto et al., 2009, 2010, 2011a). (For specialized analyses, we adjusted and divided the feedback period as described below.) If activity during any task period significantly differed from that during another task period, a neuron was classified as task related, as in the previous papers in this series (Tsujimoto et al., 2009, 2010, 2011a).
For all task-related neurons, we performed a two-factor ANOVA (α = 0.05) separately for cue-, delay-, and feedback-period activity. The factors were response (left or right) and strategy (stay or shift). We also applied two-factor ANOVA to fixation-period activity, using the response and strategy on the previous trial.
To examine how the coding properties of individual cells evolved during a trial, we computed a sliding two-factor ANOVA. For all task-related cells, the ANOVA was performed for a 200 ms centered window, stepped in increments of 20 ms for the period beginning 1.0 s before and ending 1.5 s after cue onset.
For cells recorded in the fluid-reward cued task, we compared activity during the 500 ms interval before cue onset (the fixation period) with activity levels after the cue or after feedback (80–500 ms after the onset of fluid delivery), using the Mann–Whitney U test (α = 0.05). Cells were then classified as showing significant modulation during the cue period, the feedback period, or both.
As a measure of response and strategy selectivity, we computed two indices: the response and strategy selectivity indices (RSI and SSI, respectively). These indices were defined as follows: RSI = (L − R)/(L + R), where L and R represent mean firing rate when the correct response was left and right, respectively; SSI = (St − Sh)/(St + Sh), where St and Sh indicate mean firing rates during stay and shift trials, respectively. These indices can vary from +1 to −1, with positive values indicating selectivity for left responses or the stay strategy, depending on the index, and negative values indicating selectivity for right responses or the shift strategy.
As another measure of response and strategy selectivity, we computed the area under the receiver operating characteristic (ROC) curve. To examine the time course of selectivity, we computed the area under ROC curve in a 200 ms centered window that stepped across the trial in increments of 20 ms. (Smaller time windows yielded unstable results.) ROC values measure the ability to detect a signal based on single-trial activity, independent of a cell's mean discharge rate or its dynamic range. Ideally, an ROC value of 0.5 would indicate no selectivity and a value of 1.0 would reflect absolute selectivity. However, because we computed the ROC curves based on preferred and antipreferred strategies or responses, a positive bias was introduced. Accordingly, we evaluated postcue ROC values against the upper bound of the 95% confidence limit, calculated from ROC values during the fixation period.
We also compared response selectivity during correct versus error trials. Because the low number of error trials precluded a cell-by-cell analysis, we used a bootstrap procedure at the population level (Tsujimoto et al., 2010, 2011a). This analysis was based on the difference between preferred (P) and anti-preferred (A) responses. First, we computed the activity differences (P − A) observed in population averages for both correct trials and error trials. Each cell's preference for the correct trials was used for the error trials, regardless of activity levels on the error trials. Take, for example, a cell that preferred left responses on correctly executed trials. For error trials, left responses were designated P and right responses were designated A, regardless of the cell's activity. As a result of this convention, if the cells in a population encoded the incorrect response on error trials, the response that had actually been made, then the value was positive. In our example, this would result when activity for left responses (P) exceeded that for right responses (A) on error trials, as it did on correct trials. However, if the cells encoded the correct response on error trials, the response that should have been made, then the value was negative. In our example, such a cell would have had greater activity for left responses on correct trials and for right responses on error trials, and so P − A < 0 for the error trials. This procedure led to the points marked with arrows in Figure 7. Next, the response designation was shuffled randomly across all trials, correct and incorrect. P − A activity differences were computed separately for correct and error trials, using the same convention as for the observed data. This shuffling procedure was repeated 1000 times, which yielded a distribution of 1000 sets of P − A activity differences, to which the observed set of P − A activity differences could be compared statistically.
Results
Behavior
Tables 1 and 2 summarize the monkeys' behavior during the recording sessions. Table 1 shows that both monkeys performed the visually cued strategy task at >90% correct, and nearly did so for the DR task, as well. In the fluid-reward cued strategy task, the performance of the first monkey nearly matched this level. The second monkey made many more errors on this task, but still performed above chance level. These aspects of behavior remained stable throughout the recording sessions, including the early sessions in which we recorded from PFp and the later ones in which we recorded from PFdl and PFo (Table 1).
Percentage of correct responses during the recording sessions by task, area, monkey, and response
Reaction times (ms ± SD in parenthesis) by task and strategy during the recording sessions
Oculomotor reaction times were 290–330 ms for both monkeys. They were similar for the two strategies and for early and late recordings sessions (Table 2). Both monkeys maintained stable and accurate fixation throughout the fixation and cue periods, within ±1° on >90% of the trials (Tsujimoto et al., 2009).
Neuronal database
As the monkeys performed the visually cued strategy task, 1463 neurons were isolated and recorded. Of these, 577 cells were located in PFp, 537 in PFdl, and 349 in PFo. For the fluid-reward cued strategy task, 746 cells were recorded: 302 from PFp, 241 from PFdl, and 203 from PFo. For the DR task, we collected adequate data from 398 cells: 221 from PFp, 146 from PFdl, and 31 from PFo. Table 3 gives the breakdown of the number of isolated and task-related neurons by monkey and area. Note that we did not record from PFo in monkey 1 during the DR task.
Number of isolated and task-related neurons by task, area, and monkey
Visually cued strategy task
Of the cells with significant task-related activity, many showed significant strategy and/or response effects in the visually cued strategy task. Figure 2 illustrates the activity of four such neurons, all from PFdl. Figure 2A shows a cell that had significantly higher delay-period activity for responses to the left than for responses to the right (F(1, 135) = 197.8, p < 0.001). Figure 2B shows a cell with cue-period activity that showed significant main effects of both strategy (F(1, 66) = 8.3, p = 0.005) and response (F(1, 66) = 52.3, p < 0.001), as well as their interaction (F(1, 66) = 10.3, p = 0.002), with a preference for stay–right trials. The neuron in Figure 2C exemplifies cells that encoded the previous response: to the left in this case (F(1, 105) = 13.8, p < 0.001). And Figure 2D illustrates a cell that encoded a strategy early in the trial, but a response later. Its cue-period activity was significantly higher for shift cues than for stay cues (F(1, 103) = 55.1, p < 0.001), and its delay-period activity was higher for rightward responses than for leftward responses (F(1, 103) = 38.5, p < 0.001).
Activity of four PFdl neurons during the visually cued strategy task. In each panel, activity is aligned on cue onset (left) and the go signal (right), with saccade onset (sac) and the reward feedback indicated by marks on each raster line. Raster displays show spike times with spike-density averages above. Dark and light gray background indicate cue and delay periods, respectively. Numbers in circles to the right of the key correspond to those to the left of the raster displays. A, A neuron with a preference for left future responses during the delay period, regardless of strategy. B, A neuron that encodes the conjunction of response and strategy during the cue period. C, A neuron that encoded the previous correct response during the fixation period. D, A neuron that encoded the cued strategy during the cue period, then encoded the future response during the delay period.
Figure 3 summarizes the results of the two-way ANOVA, which revealed significant strategy and response effects. This section focuses on the left and right columns of Figure 3. Although the results in the left column repeat those presented in previous papers (Tsujimoto et al., 2009, 2010, 2011a), they establish the basis for the comparison among signals, tasks, and areas to follow. In this analysis, the feedback (Fb) period was defined as the 0.5 s interval beginning 0.3 s before the onset of reward delivery. Previous reports provided results for the periods before and after feedback separately (Tsujimoto et al., 2009, 2010).
Results of two-way ANOVA, with factors strategy (stay or shift) and response (left or right), for the three tasks: the visually cued strategy task (far left column), the fluid-reward cued strategy task (middle left), and the DR task (middle right). Dashed horizontal line: percentage expected by chance. Right column: Strategy- and response-selectivity indices for the visually cued strategy task, plotted as the absolute value of the index, means, and SEM. These indices were computed in separate 250 ms bins relative to the cue onset, go, and feedback onset. Shading: SEM for the baseline period, measured for the 500 ms interval immediately before cue onset. A, PFp. B, PFdl. C, PFo.
Like many of the cells in Figure 2, a significant proportion of PFdl neurons showed main effects of strategy and/or response during both the cue and delay periods (Fig. 3B, left). PFo neurons also showed a significant strategy effect during the cue and delay periods, but they did not show an above-chance percentage of response effects (Fig. 3C, left). PFp neurons did not show above-chance levels of either strategy or response effects during the cue or delay period (Fig. 3A, left). The proportion of strategy-selective cells differed significantly among the three areas for both the cue (χ2 = 23.9, df = 2, p < 0.001) and delay (χ2 = 14.0, df = 2, p < 0.001) periods, as did the proportion of response-selective cells (cue period, χ2 = 26.7, df = 2, p < 0.001; delay period, χ2 = 20.2, df = 2, p < 0.001).
During the feedback period, by which time the response had been completed, a large proportion of neurons in all three areas showed response effects. There was no significant encoding of the strategy during the feedback period in any of the three areas. The proportion of response-selective cells did not significantly differ among the three areas (χ2 = 2.6, df = 2, p = 0.28).
During the period between trials, some PFdl cells, like the one illustrated in Figure 2C, encoded the previous response, as measured during the fixation (fix) period (Fig. 3B, left). Neurons in the other two areas did not encode the previous response in numbers above chance levels (Fig. 3A,C, left). The proportion therefore differed significantly among the three areas (χ2 = 21.7, df = 2, p < 0.001).
Two additional methods confirmed the results from ANOVA: (1) SSI and RSI and (2) ROC analysis. In PFdl, both the SSI and the RSI were elevated during the cue period, and the RSI remained high during the delay period (Fig. 3B, right). PFo also showed an increased SSI during the cue period (Fig. 3C, right). All three areas showed an increased RSI during the feedback period (Fig. 3A–C, right). The results from the two monkeys differed little (Fig. 4A), and so they are combined for the remainder of the analysis.
A, The results depicted in Figure 3, right, divided by monkey. B, ROC analysis. Area under the ROC curves for strategy signals (solid lines) and response signals (dashed lines). Horizontal lines show the upper limit of 95% confidence interval for the baseline. Shading: SEM. Acq, Target acquisition time, the time when monkey's gaze angle entered the target window; Fb, feedback time, the time on correct trials when reward delivery began.
ROC analysis yielded very similar results, which Figure 4B shows sorted by area and Figure 5 shows sorted by task and signal (strategy and response). During the cue period, a strategy signal could be detected in PFdl and PFo, but not in PFp (Figs. 4B, 5A, left). Only PFdl had a response signal during the delay period, but this signal could be detected in all three areas around feedback time (Figs. 4B, 5A, right).
A–C, Sliding ROC plots for neurons with task-related activity. Area under the ROC curves for strategy (left column) and response (right column) signals, calculated using the same population of neurons for both. Shading shows the upper and lower limit of 95% confidence interval for the baseline. Format as in Figure 4B. A, Visually cued strategy task. B, Fluid-cued strategy task. C, DR task.
Figures 3⇑–5 compare signal strength among the three tasks and among the three areas in the three ways mentioned: the number of cells with significant coding (Fig. 3, left), the selectivity indices (Figs. 3, right; 4A), and ROC analysis (Figs. 4B, 5). Response coding exceeded strategy coding during the delay period, both in terms of the number of neurons encoding this variable (Fig. 3, left) and the strength of the signal (Figs. 3, right; 4, 5A). Later, during the feedback period, the response signal reached levels that the strategy signal never attained at any time (Figs. 3A–C, right; 5A).
As reported previously (Tsujimoto et al., 2011a), the strategy signal appeared earlier and reached a higher level in PFo (Fig. 5A, green) than in PFdl (orange), and a strategy signal could not be detected in PFp (magenta) during any task period (Fig. 5A, left). Note that the present analysis differed from our previous reports in that it included all task-related cells; in the previous papers we used the cells that showed significant strategy effects to analyze strategy selectivity and cells with significant response effects to analyze response selectivity. Close inspection of Figure 5A (left) might suggest that PFo encodes strategies during the feedback period. This signal fluctuation, however, is an artifact that has no convincing counterpart in the ANOVA (Fig. 3C, left) or in the strategy selectivity index of either monkey (Fig. 4A, bottom). It occurs when right and left responses are distributed unequally, a conclusion that we confirmed by balancing the numbers of each response (data not shown). Thus, we conclude that no strategy coding was observed during the feedback period in any area (Figs. 3⇑–5).
None of the results presented in Figures 3⇑–5 address coding on a cell-by-cell basis, as it evolves during a trial. For example, cells carrying a strategy signal during the cue period might have a response signal during the delay period, as in the example neuron illustrated in Figure 2D. To examine systematically how coding changed during a trial, we used a sliding version of the two-way ANOVA used for the previous analysis. Figure 6, A and B, shows significant bins in white, exclusively for PFdl because only cells in this area had both strategy- and response-coding during the cue and delay periods. Figure 6A illustrates main effects of strategy, with one line for each neuron, sorted according to the number of significant bins during the cue period. Figure 6B shows results for the same neurons in the same order, but for significant main effects of response instead of strategy. Visual inspection did not reveal any strong relationship between the cue and delay periods. Statistically, however, cells with many strategy-coding bins during the cue period tended to have many strategy-coding bins (r = 0.24, p < 0.001) and response-coding bins (r = 0.18, p < 0001) during the delay period.
Sliding ANOVA. A, Main effect of strategy for each PFdl cell during the visually cued strategy task, with each line corresponding to a cell and bins with a significant main effect of strategy in white. Bin width 200 ms, stepped in increments of 20 ms. The order of neurons was ranked according to the number of significant bins during the cue period. All task-related neurons were included in this analysis (N = 405). B, Results for the main effect of response. Each line corresponds to the same neuron as in A. C, Percentage of neurons with significant main effects of either strategy (blue), response (red), or both (green), as a function of time from cue onset, for the visually cued strategy task. Solid lines, PFdl; dashed line, PFo. The horizontal red-blue dashed line shows chance level for the red and blue lines; the horizontal green line shows the chance percentage for both effects (5% × 5% = 0.25%). D, Fluid-reward cued strategy task, in the format of C. E, Visually cued strategy task, in the format of C, aligned on the onset of feedback. Thin solid line, PFp; thick solid lines, PFdl; dashed line, PFo. F, Fluid-reward cued strategy task, in the format of D, aligned on the onset of feedback. Note the change in scale for E and F compared with C and D.
We performed a similar analysis on the feedback period and confirmed that the cells with significant response-coding immediately before feedback (the 200 ms interval immediately before the onset of feedback) were also likely to show such coding shortly after the feedback arrived (the 200 ms interval from 100 to 300 ms after the onset of feedback) (r = 0.47, p < 0.001; not illustrated).
Figure 6C shows the percentage of cells with response selectivity only, strategy selectivity only, or both, as a function of time during the trial, for the visually cued strategy task. The time course of cell recruitment was similar for all three populations in PFdl. Figure 6E presents the same populations time-locked to the onset of feedback. Among 49 PFo cells that encoded a strategy during the cue period, 18 (37%) cells encoded a response during the feedback period. For PFdl, of the 52 cells that encoded strategy during the cue period, 19 (35%) encoded the response during the feedback period.
Error versus correct trials
The results presented so far come entirely from correctly executed trials. An examination of error trials reveals some additional differences among the three areas. For the visually cued strategy task, we examined response selectivity using a bootstrap procedure (see Materials and Methods). As mentioned there, the small proportion of errors precluded a cell-by-cell analysis, so we studied the properties of neurons in each area at the population level. For the calculation of error data, we used neurons that showed a significant main effect of response by ANOVA during the feedback period and had at least one error trial for each response.
Figure 7 shows that—for correctly executed trials—the observed difference between preferred and anti-preferred responses (arrows) vastly exceeded all 1000 sets of shuffled data (p < 0.001). This result was to be expected in view of the strong response signal observed in the feedback period (Figs. 3⇑–5). The critical test involved the comparison of observed and shuffled data for error trials. To this end, Figure 7 divides the feedback period into two parts. The pre-feedback period was the 0.2 s interval immediately before the onset of feedback, and the post-feedback period was the 0.2 s interval from 0.1–0.3 s after the onset of feedback.
Activity difference between preferred and anti-preferred responses during the feedback period for the observed (arrow) and shuffled data. The data on correct and error trials are shown on the abscissa and ordinate, respectively, with the distribution shown above and to the right of the plots. For the calculation of error data, we used neurons that showed a significant main effect of response by ANOVA during the feedback period and had at least one error trial for each response. Red points significantly encode response on both correct and error trials. The neurons selected for this analysis showed significant response selectivity for the feedback period, as a whole; N = 99, 139, and 79 for PFp, PFdl, and PFo, respectively. The oblique line has unity slope, showing that for the red points, activity difference was similar on correct and error trials. Background shadings show the range of shuffled data. Pre-feedback period: the 200 ms interval immediately before the onset of feedback; post-feedback period, the 200 ms interval from 100 to 300 ms after the onset of feedback. A, PFp. B, PFdl. C, PFo.
For PFp (Fig. 7A), the observed value did not significantly differ from the shuffled data in either the pre-feedback or post-feedback period on error trials (p = 0.07 before and p = 0.11 after feedback). For PFdl (Fig. 7B), the observed value significantly differed from the shuffled data before feedback (p < 0.001) but not afterward (p = 0.08). For PFo (Fig. 7C), the observed value differed from the shuffled data in both periods (p < 0.001). The unity slope lines in Figure 7 emphasize that when the observed response signal differed significantly from the shuffled data (red points), response selectivity was comparable on correct and error trials.
These results indicate that the response signal was weak or absent on error trials in PFp (Fig. 7A), that it was similar on correct and error trials in PFo (Fig. 7C), and that in PFdl it depended on whether feedback had yet arrived (Fig. 7B). Before feedback, PFdl activity encoded the response; after feedback the response signal was weak or absent. Note that, because of the convention that we used, the response encoded by PFo and PFdl neurons on error trials was the one that had been made, and not the one that should have been made according to the cued strategy. For example, neurons that preferred left responses when the monkey performed correctly also did so when the monkey performed incorrectly.
Fluid-reward cued strategy task
The middle left column of Figure 3 summarizes the results from ANOVA for the fluid-reward cued strategy task. In general, few differences were observed between the two strategy tasks. For both, PFdl showed main effects of both strategy and response during the delay period (Fig. 3B, middle left); PFo neurons showed strategy effects, but not response effects during the cue and delay periods (Fig. 3C, middle left); and PFp neurons showed neither strategy nor response effects during these periods (Fig. 3A, middle left). One difference between the two strategy tasks involved response coding during the cue period. Unlike the visually cued task, for the fluid-reward cued task the number of cells with significant response effects in PFdl did not exceed chance level during the cue period.
The proportion of strategy-selective cells was significantly different among the three areas for both the cue (χ2 = 19.8, df = 2, p < 0.001) and delay (χ2 = 15.4, df = 2, p < 0.001) periods. The proportion of response-selective cells was also significantly different among the three areas for the delay period (χ2 = 16.6, df = 2, p < 0.001), but not for the cue period (χ2 = 0.6, df = 2, p = 0.74) or the feedback period (χ2 = 2.4, df = 2, p = 0.30).
Figure 5B compares the strategy (left) and response (right) signals during the fluid-reward cued strategy task. The results for the fluid-reward cued task closely resembled those for the visually cued task (Fig. 5A). As in the visually cued strategy task (Fig. 5A, left), the strategy signal in the fluid-reward cued strategy task was greater in PFo (green) than in PFdl (orange), and the signal reached its peak earlier (Fig. 5B, left). PFp (magenta) showed little, if any, change in strategy selectivity at any time during the fluid-reward cued strategy task. And like the visually cued task, only PFdl showed a response signal during most of the delay period. There were some differences between the two strategy tasks, however. For example, in the fluid-reward cued task, the response signal of PFdl was weak or absent during the cue period and gradually increased during the delay period, whereas in the visually cued task it increased during the cue period and remained fairly stable through the delay period. By the end of the delay period, however, the response signal was comparable in the two strategy tasks. The sliding ANOVA confirmed these observations (Fig. 6C–F).
Because this task used fluid delivery both as a cue and as a feedback, we could rule out reward effects for many strategy- and response-encoding cells. This was especially important in interpreting feedback-period activity because positive feedback always occurred in the form of a fluid reward. Figure 8A–C shows three examples of cell activity. Figure 8A shows a neuron that increased its activity after the fluid was delivered as a cue, but showed no clear activity modulation when the fluid was delivered as feedback. This finding supports the conclusion that the activity of the cell encoded the strategy (“shift” in this example) and not the occurrence of a reward, even when rewards provided the strategy cue. Figure 8B shows the activity of a neuron that had increased activity after the fluid delivery regardless of whether it had been a cue or feedback. Such cells probably encoded the reward per se. Figure 8C illustrates a neuron that showed activity modulation in anticipation of fluid as feedback, but not for fluid cues. This property rules out reward anticipation; the cell instead encoded the response that had been made on a given trial.
A, Example activity during the fluid-reward cued strategy task, in the format of Figure 2; a PFdl neuron that had no activity related to fluid delivered as rewarding feedback but encoded the strategy cued by a fluid reward. B, A PFo neuron that showed an increase in activity after the delivery of fluid, whether as a cue or as feedback. C, A PFp neuron that encoded a response before fluid reward delivered as feedback, but showed no modulation for the same reward as a cue. D, Percentage of cells that showed significantly different activity after fluid-reward cue and/or after fluid-reward feedback, contrasted with activity during the fixation period (Mann–Whitney U test). Color code as in Figure 5.
As summarized in Figure 8D, all three areas had a considerable proportion of neurons for which the anticipation of reward or response to reward could be ruled out as factors contributing to strategy or response coding. Other cells in PFdl and PFo (but not PFp) also encoded reward per se (Fig. 8D, labeled “both”).
The DR task
The middle right column of Figure 3 shows results of ANOVA for the DR task. In this task, many cells in all three areas also showed response-selectivity during the feedback period, with similar proportions (χ2 = 1.3, df = 2, p = 0.52). In addition, PFdl also had large proportion of response-selective cells during the cue and delay periods. Thus, the proportion of response-selective neurons was significantly different across areas in the delay period (χ2 = 9.1, df = 2, p = 0.01), but not in the cue period (χ2 = 4.4, df = 2, p = 0.11).
Even though it was irrelevant to the DR task, an above-chance proportion of PFdl cells encoded the response on the previous trial, as measured during the precue fixation period. There was no strategy cue for the DR task, but we could evaluate whether the cells encoded the fact that a visuospatial cue dictated a shift from the previous response or a repeated response, as the strategy cues did. This potential signal is plotted in lieu of the strategy signal in Figure 3 (middle right) and Figure 5C. No clear selectivity was observed for whether the visuospatial cue instructed the same response as on the previous trial or a different one (for cue period, χ2 = 2.9, df = 2, p = 0.24; for delay period, χ2 = 2.8, df = 2, p = 0.24; for feedback period, χ2 = 2.2, df = 2, p = 0.33).
Figure 5C (right) shows that—in PFdl and PFo, but not in PFp—a response signal could be detected during the cue period for the DR task, although it dissipated during the delay period. A response signal could be detected in all three areas around the time of feedback, as in the other two tasks. A previous report presented a detailed comparison of activity in the DR and visually cued tasks for PFp (Tsujimoto et al., 2010). We could not perform the same analyses for PFo and PFdl because the necessary conditions were not used while recording activity in those two areas.
Correlations across tasks
To examine whether strategy and response coding was task or modality specific, we performed a correlational analysis on the selectivity indices for cells recorded in specific pairs of tasks.
First, we examined the RSI for cells recorded in both the visually cued and fluid-reward cued strategy tasks. We consider this to be a test of modality specificity or generality. RSI > 0 indicates a preference for left responses; RSI < 0 shows a preference for right responses. Figure 9A–D plots the data for PFdl cells in four task periods: the cue period, the initial 500 ms of the delay period, the last 500 ms of the delay period, and the feedback period, respectively. During the cue period (Fig. 9A), selectivity for the left versus right response was not significantly correlated between the two strategy tasks, but it became so during the early delay (Fig. 9B), late delay (Fig. 9C), and feedback (Fig. 9D) periods. Neurons in both PFo (Fig. 9E) and PFp (Fig. 9F) showed similar correlations during the feedback period. (Plots corresponding to Fig. 9A–C were not possible for PFo and PFp because they did not have a response signal during the cue or delay periods.) Figure 9G plots the correlation coefficients for each period, and Table 4 gives the correlation coefficients and significance levels.
Comparison of RSI between visually cued and fluid-reward cued strategy tasks. Each panel plots each neuron's index value for fluid-cued task against visually cued task. Black squares: neurons that showed a significant main effect of response in at least one task; black crosses: other neurons. A–D, PFdl during the cue (A), early delay (B, from 0 to 500 ms after cue offset), late delay (C, from 500 to 0 ms before the go signal), and feedback (D) periods. E, PFo during the feedback period. F, PFp during the feedback period. G, Plot of correlation coefficient between RSIs of visually and fluid-cued strategy tasks for each task period. Squares indicate the values for PFdl neurons for all five task periods. Data from PFo (triangle) and PFp (circle) are also shown for the feedback period. ns, nonsignificant correlations.
Correlations of RSI between pairs of tasks by index, task, area, and task period
Figure 10A–D shows a similar analysis for the PFdl cells recorded in the visually cued strategy task and the DR task. Because DR is an imperatively cued task, in contrast to the contingently cued strategy tasks, we consider this to be a test of task specificity or generality. The results closely resemble the RSI correlations presented in Figure 9. There was no significant correlation during the cue period (Fig. 10A), but it became significant in the early delay (Fig. 10B), late delay (Fig. 10C), and feedback (Fig. 10D) periods. PFp had a significant correlation during the feedback period (Fig. 10F). PFo (Fig. 10E) showed a correlation of r = 0.44–0.48, but this value did not reach significance, in part due to the low number of neurons recorded in this pair of tasks. Figure 10G and Table 4 summarize the correlation coefficients for each period.
Comparison between visually cued strategy task and DR task. Format as in Figure 9. A–D, PFdl during the cue (A), early delay (B), late delay (C), and feedback (D) periods. E, PFo during the feedback period. F, PFp during the feedback period. G, Plot of correlation coefficient between RSIs of visually cued and DR tasks for each task period.
Analysis of the SSI yielded a different result (Table 5). Neither PFdl (Fig. 11A–C) nor PFo (Fig. 11D–F) neurons showed a significant correlation in selectivity for the stay versus shift strategies in the two strategy tasks (Fig. 11G), except in one instance. In that case, PFo data for the early delay period did reach statistical significance, but the effect size was small (r = 0.21) and its statistical significance depended on an outlier artifact (Fig. 11E, arrow).
Correlations of SSI between pairs of tasks by index, task, area, and task period
Comparison of SSI between visually cued and fluid-reward cued strategy tasks. Format as in Figure 10. A–C, PFdl during the cue (A), early delay (B, from 0 to 500 ms after cue offset), and late delay (C, from −500 to 0 ms before the go signal) periods. D–F, PFo during the same periods. G, Plot of correlation coefficient between SSIs of visually and fluid-reward cued strategy tasks for each task period.
In contrast to the modality-specific strategy coding revealed by comparing results from the visually cued and fluid-reward cued strategy tasks (Table 5, top), a comparison of strategy coding for the two kinds of visual cues, colored squares (yellow for stay, purple for shift) versus oriented bars (vertical for stay, horizontal for shift), showed a strong and highly significant correlation (Table 5, bottom).
Discussion
In previous reports on this dataset, we analyzed activity in selected populations, periods, tasks, and areas. The reports on PFo and PFdl compared areas for the visually cued task, but not for the other two tasks, and they did not distinguish the pre-feedback and post-feedback periods (Tsujimoto et al., 2009, 2011a). The report on PFp compared all of the tasks but not the other two areas (Tsujimoto et al., 2010). Here, we compare results for all task-related neurons, in all three areas and tasks. Novel results bear on strategy and response generation, error-trial activity, and reward coding, topics taken up, in turn.
Strategy generation
For the visually cued strategy task, Tsujimoto et al. (2011a) showed that both PFdl and PFo encoded strategies during the cue period. The present results extend these findings to strategies cued by fluid rewards. The capacity of PFo to generate strategies from both visual and reward cues probably reflects convergent inputs from visual cortex and from areas conveying the sensory information associated with rewards, such as gustatory cortex and the mouth representation of the somatosensory cortex (Carmichael and Price, 1996; Walton et al., 2011).
Comparison of the visually cued and fluid-reward cued strategy tasks revealed a lack of correlation in preferred strategies at the single-cell level (Fig. 11). A cell might prefer the stay strategy in one task and the shift strategy in the other. Thus, PFo and PFdl encoded the stay and shift strategies in a modality-specific manner, at least for the present tasks. We know that for the matching and nonmatching rules, PFo and PFdl cells appear to encode the rule in a more modality-general manner than observed here (Wallis et al., 2001). We doubt that this reflects the difference between rules and strategies because, in the present experiment, what we call stay and shift strategies could just as well be considered stay and shift rules. We also know that PFdl encodes perceptual decisions in a domain-specific manner during temporal and spatial discrimination tasks (Genovesio et al., 2012). The degree of domain or modality specificity might depend on details of task design, such as whether cues of the different modalities are interleaved, as in the experiment of Wallis and Miller (2003), or blocked, as here.
A separate point about strategy generation emerges from cue period activity in the DR task. No signal in any area reflected shifting from or staying with the previous response in this task (Fig. 5C, left). This finding supports the conclusion that what we have called strategy signals specifically reflect a strategy generated on the basis of a sensory cue, rather than the consequences of applying that strategy.
Response generation
In accord with previous findings, PFdl—but neither PFo nor PFp—encoded future responses during the cue and delay period (Wallis and Miller, 2003; Padoa-Schioppa and Assad, 2006; Tsujimoto et al., 2011a). Comparison among tasks revealed that response coding during the cue period was task and modality specific, as reflected in the lack of correlation in response preferences. During the delay and feedback periods, the signal transformed to a task- and modality-general response code, as shown by the strong correlation among response preferences in the tasks (Figs. 9G, 10G).
Results from the DR task showed that cells in PFo encoded the response during the cue period (Fig. 5C, right), as reported previously (Ichihara-Takeda and Funahashi, 2007). Our tasks could not distinguish visuospatial from response coding. In either case, however, this result indicates that PFo functions extend beyond the valuation of foods and objects.
Error-trial activity
Although, on correct trials, all three areas showed response selectivity during the feedback period (Tsujimoto et al., 2009, 2010), significant differences emerged on error trials (Fig. 7). PFo neurons encoded the response that had been made, correct or incorrect, both before and after feedback (Fig. 7C). In contrast, PFdl neurons did so only before feedback; after error feedback arrived, PFdl neurons encoded responses much more weakly, if at all (Fig. 7B). The response signal in PFp was weak or absent on error trials throughout the feedback period (Fig. 7A) (Tsujimoto et al., 2010).
Previous results showed that cells in PF monitor or maintain a response in memory during the feedback period (Barraclough et al., 2004; Seo et al., 2007; Tsujimoto et al., 2009), an idea that has been linked to output monitoring and credit assignment (Walton et al., 2009; Tsujimoto et al., 2010). The present results allow us to expand on this idea. Compared with PFdl and PFo, the pre-feedback response signal in PFp permitted it—but not PFo or PFdl—to monitoring the response that was correct according to the cued strategy. Such coding could promote learning about which kind of response-generating cognitive processes produced a given outcome (Tsujimoto et al., 2011b).
This idea might explain why response coding also occurred during the fixation period in the DR task, which did not require response monitoring or maintenance for selecting a response. Because both monkeys had extensive experience with strategy tasks before we introduced the DR task, this property could reflect the habit of monitoring previous responses even when it was unnecessary to do so. But another possibility is that all three tasks involve monitoring responses according to some abstract rule. In DR, the rule is to use the most recent cue to guide the next response, as opposed to cues that occurred on earlier trials (Passingham and Wise, 2012).
Note that pre-feedback coding in PFp must reflect some internal signal about whether a response was correct because no external stimulus provided this information until feedback arrived. Neuroimaging results support the idea that “internal” or self-referential signals occur in the human PFp, especially its medial parts (Gilbert et al., 2006; Burgess et al., 2007; Hassabis et al., 2007; Mason et al., 2007; Raichle and Snyder, 2007; Boorman et al., 2009).
PFo cells, in contrast to those in PFp, encoded the response actually made, and they did so both before and after feedback. Such information is important for the assignment of response choices to outcomes, which requires monitoring the response that has been made on a particular trial, not the one cued. The present findings support the idea that feedback-period activity in PFo subserves this function (Walton et al., 2010; Wallis, 2012).
PFdl has more complex properties. PFdl ceased encoding erroneous responses after feedback arrived (Fig. 7B). It seemed therefore to encode something about the response–outcome conjunction, which is the information needed to respond correctly on the next trial.
Tsujimoto et al. (2011a) reported another feature of error-trial activity. During the cue period, PFo encoded the correct strategy, not the one used, whereas during the feedback period PFo encoded the response made, not the correct one (see also Genovesio et al., 2008). Thus, during the cue period the correct strategy is available for predicting future events; after feedback the response actually made is available for linkage with outcomes. PFdl has similar properties, until the post-feedback period.
Reward coding
Some cells in PFo and PFdl had activity related to rewards both when they served as cues and when they provided feedback (Fig. 8D). This property is likely to reflect the motivational aspects of reward. But for many other cells reward effects could be ruled out. This result lends further support to the idea that both PFdl and PFo play a role in cognitive control of behavior, exemplified by abstract rules and strategies (Buckley et al., 2009; Walton et al., 2010; Rushworth et al., 2011), as well as in behaviors guided by concrete associations among stimuli, responses, and outcomes.
Proposal
Figure 12 summarizes the present results in the form of a proposal that has three key features. First, PFo sends modality-specific strategy information to PFdl, which generates a response in the context of its representation of the previous response (Genovesio et al., 2006). Second, PFdl converts a modality- and task-specific response signal during the cue period into a modality- and task-general response signal during the delay period. Likewise, Genovesio et al. (2012) have reported that PFdl cells encode temporal and spatial discriminations in a domain-specific manner, but encode response choices in a domain-general way. And third, all three areas monitor output, but in different ways. Just before feedback arrives, only PFp encodes that a response is correct according to the cued strategy: a signal well that is suited to associating outcomes with responses generated by specific cognitive processes (Tsujimoto et al., 2011b). After feedback, only PFo encodes the response actually made: a signal well that is suited to associating outcomes with responses (Walton et al., 2010). And PFdl signals the chosen response before feedback, but only correct responses after feedback: a response–outcome conjunction required for the next trial (Barraclough et al., 2004; Tsujimoto and Sawaguchi, 2004, 2005; Seo et al., 2007).
Proposal. DR, Oculomotor delayed-response task; L, left response; R, right response. Superscripts refer to figure numbers documenting various points.
As a result of interactions among these areas, the PF cortex as a whole can generate responses adaptively, maintain previous successes in memory, and assign outcomes to both a response and the cognitive process that generated it.
Footnotes
This work was supported by the Division of Intramural Research of the National Institute of Mental Health and in part by Grants from Ministry of Education, Culture, Sports, Science and Technology of Japan (23119714), Japan Society for the Promotion of Science (22700340), and National Center for Neurology and Psychiatry (22-6) in Japan. We thank Dr. Andrew R. Mitz, Mr. James Fellows, and Ms. Ping-Yu Chen for technical support.
- Correspondence should be addressed to Satoshi Tsujimoto, Developmental Cognitive Neuroscience Laboratory, Graduate School of Human Development and Environment, Kobe University, 3-11 Tsurukabuto, Nada-Ku, Kobe 657-8501, Japan. tsujimoto{at}ruby.kobe-u.ac.jp
References
In this issue
